* startup: add sum testcase

* mean, max, min, sum

* refine test mul

* refine test mul case

* refine div, mul, sub test case

* refine masked_fill test case

Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine gather test

* add masked_fill backward test

* add mean module backward test

* rewrite module to test

* refien

* add layernorm backward test

* refine math_ops tests

Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>

* align mean module

* allow negative dim param

* support tuple of negative dim param

* refine

* format

Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add experimental apis

* merge master fix conflict

* revert flow._oneflow_internal.dtype to flow.dtype

* refine

* fix test optimizer

* update module docs

* fix unit tests

* fix matmul module test

* fix adamw and rmsprop tests

* fix crossentropy loss grad

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add modules:

- sum
- mean
- add
- sub
- mul
- div
- reciprocal

* add +, -, *, / for tensor

* add test case for add,div,mean,mul,reciprocal,sub,sum

* refine test cases

* refine sum module

* remove useless oneflow_export for mean and sum

* add tensor op test for mean and sum

* add CrossEntropyLoss module

* refine Sum

* sum axis attr not found

* refine sum

* refine math ops

* refien cross entropy loss

* refine (not finished)

* fix scalar div, sub

* rename test Module name and refine test case

* rename test Module name and refine test case

* add neg method for Tensor

* remove useless improt items

* refine

* reformat

* remove useless dynamic attr

* fix math_ops and loss

* rename test

* fix format

* delete name

* refine codes and format

* style(*): format codes

* fix(MathOp): fix merge bug

Co-authored-by: simonJJJ <821898965@qq.com>
Co-authored-by: wyg1997 <wyg19970408@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add exp_tanh_gelu module

* fix comment

* fix comment

* fix comment

* change docs location

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add exp_tanh_gelu module

* fix comment

* fix comment

* fix comment

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* integrate module and op builder

Signed-off-by: daquexian <daquexian566@gmail.com>

* update test

Signed-off-by: daquexian <daquexian566@gmail.com>

* format

Signed-off-by: daquexian <daquexian566@gmail.com>

* skip test in lazy mode

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* pytorch-style module

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix typo, update parameter

Signed-off-by: daquexian <daquexian566@gmail.com>

* add TODO, remove unused import, set consistent to True in parameter

Signed-off-by: daquexian <daquexian566@gmail.com>

* set placement of parameter

Signed-off-by: daquexian <daquexian566@gmail.com>

* reformat

Signed-off-by: daquexian <daquexian566@gmail.com>

* add __init__.py in oneflow.python.nn.modules

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix deprecated module test

Signed-off-by: daquexian <daquexian566@gmail.com>

* align with latest checkpoint api

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix missing scope_symbol_id

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add classes for different test base class

* add export

* rm old main files

* fix

* fix class name

* check in changes of tests

* fix else

* fix bugs

* fix order

* better naming

* update ci script

* fix import unittest

* add --failfast

* filter by card num

* fix typo

* update ci script

* dont set CUDA_VISIBLE_DEVICES

* use skip unless

* add OneGpuTestCase

* fix typo

* shorter reason

* fix class

* fix typo

* skip test_train_consistent for now

* add tearDown

* fix setup

* rm todo

* skip TestImageResize

* fix for pure cpu

* skip TestImageTargetResize

* fix typo

* rename

* mv demo

* rm readme

* rename

* rename cases

* add --verbose

* rename

* address reviews

* fix export

* minor fix

* fix export

* fix skips

* more renames

* rename

* rename class

* rm gpu

* rename num in scripts

* skip resize img

* fix sys env

* fix dynamic reshape

* fix dynamic reshape

* fix skip msg

* rm num_gpus

* address review

* address reviews

* rm class

* update decorator

* fix inden

* fix typo

* replace class with decorator

* fix

* refine

* fix decorator

* mv code

* add more decorators

* add ONEFLOW_TEST_DEVICE_NUM=1

* fix typo

* fix typo

* fix name

* fix skip msg

* use decorators in scripts

* larger rol for xla ln

Co-authored-by: tsai <caishenghang@oneflow.org>
Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* add license at root dir

* check in empty files

* rm space

* check in script

* update script

* fix bug

* add print

* fix

* add exit

* add to of_format

* add CI task

* fix license

* Revert "fix license"

This reverts commit 818b6d7691d3a8b4a25dd41a47ff2c5922b8ec57.

* only add once

* quick fix

* fix script

* dont fmt empty file

* fix

* quick fix

* fix py

* add license

* fix exit

* add license for hpp

* add license

* license new vm files

Co-authored-by: tsai <caishenghang@oneflow.org>

* Add py format and ci check format , by black and isort

* py import sort

* add .isort.cfg

* Merge develop

* Add ci docker check format

* Fix undefined name oneflow

* remove isort

* Add docker hash tag

* Add py check format verbose

* Prompt install black

* refactor oneflow_export with OneflowApi

* runtime_mode