* init instancenorm.

* fix bert backward bug

* set time limit

* add time limit

* add instancenorm and unittests

* add to rst

* clean code.

* add references to pytorch

* add tests

* auto format by CI

Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: jackalcooper <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add flag -lrt for cares

* Don't build cares exes so it won't fail at linkage

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add CMake arg PIP_INDEX_MIRROR

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check in files

* fix indentation

* refine

* first commit of op floor

* first commit of op acos

* split acos from math_ops and add doctest

* resolve conflicting files

* fix format bug

* auto format by CI

* add check license

* synchronize local with remote

* auto format by CI

* first commit of op acos

* Update tensor.cpp

* solve conflicts

* add of_softmax_use_fast_math (#4979)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* auto format by CI

* ci check license

Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: JiaojiaoYe1994 <jiaojiaoye@tum.de>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add min/max modules

* Add backward test

* Fix bug

* Add backward test

* Revert

* fix min module backward error

* Add max test

* delete unnecessary code

* Revert sum module

* Update doctest

* Fix experimental api

* fix loss test

* Fix Std op

* Rename keepdims to keepdim

* Fix l1loss import bug

Co-authored-by: Ldpe2G <liangdepeng@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

* added flatten backward

* flatten and softmax backward

* auto format by CI

* rename test functions, deleted useless attributes

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* init commit

* init commit

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine imports

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* module l1_loss

* modify l1_loss

* add l1loss_backward_testCase

* modify doctest format

* set raise_on_error=True

* add empty line

* modify empty line

* auto format by CI

* modify note

* modify tolerance

* modify testCase

* modify and add more testCase

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

* Update generated functional files only when different.

* Use default target other than custom command.

* fix typo

Co-authored-by: jackalcooper <jackalcooper@gmail.com>

* update nightly version 0.5.0

* list cu112

* fix conflict

* add topk op and testcases

* add old top_k testcase

* fix doc_test

* fix testcase

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine and add test case

* support scalar input

* ../oneflow/python/nn/modules/eq.py

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Dev functional interface.

* Remove repeated function_traits

* Implement add and add_scalar static functional op.

* Remove unused code

* Refactor

* Refine

* Generate and export functional apis.

* Refine

* Refine

* Refine

* Generate functional api and pybind cpp when building the project.

* Refine code style and implement normalization functor.

* Fix cmake

* Add PyYAML requirement.

* Add JUST

* Fix scalar IsIntegral and IsSigned

* Add scalar add grad func.

* Fix norm grad func to support dynamic attrs.

* Support DataType.

* Recursive determine input tensors.

* Update generate_functional_api.py

* Fix conversion from python object to dtype.

* Fix crash since function has been cast with wrong function signature, the converted function behavior is undefined.

* Check and throw error.

* Refine

* Use Maybe instead of throwing exception directly.

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix bert backward bug

* fix argwhere module bug

* align dropout module with torch

* align dropout module with torch

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* feat(*): export Optimizer and _LRScheduler

* fix(*): import module

* fix

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix bert backward bug

* fix argwhere module bug

* auto format by CI

* fix argwhere module bug

* fix license

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* dynamic link cudnn

* reffine

* refine

* fix

* update dockerfile and build wheel

* refine

* force cmake to always find cudnn again

* build wheel enforce cudnn static off if cu11

* refine find cudnn cmake

Co-authored-by: guo-ran <360112263@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* feat(RecordingReplay): add RecordingReplay

* refine code

* delete instruction_replay interface in flow.experimental

* refine codes

* test(InstructionReplay): add test

* style(InstrReplay): rename interface

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* align squeeze module to torch

* add squeeze for slice

* refine as comments

* refine as comments

* refine

* refine

* redfine

* refine test case

* refine

* refine conv

* fix conv slice

* refine

Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add eager experimental stack

* complete ops and test program

* complete ops and test program

* add copyright and format

* add copyright and format

* fix docstring

* fix bug and doc

* fix docs

* add multi_input test

* add doctest

* add doctest

* fix docs

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix bert backward bug

* set time limit

* add time limit

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: jackalcooper <jackalcooper@gmail.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* support expand and repeat module int8

* support expand and repeat module int8

* auto format by CI

* timeout-minutes: 45

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: jackalcooper <jackalcooper@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check in changes

* refine

* rename

* refine

* revert

* Delete CMakeLists.txt

* Delete CMakeLists.txt

* Delete CMakeLists.txt

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* ceil&expm1 updated

* add doctest

* fix conflicts

* fix test cases

* add test cases into test_tensor.py

* fix test cases

* auto format by CI

* fix test

* auto format by CI

* resolve conflicts

* fix doctest

* auto format by CI

* Update math_ops.py

* auto format by CI

Co-authored-by: yy1a <yyangg53@gmail.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* add doctest for math modules

* add doctest for math modules

* add doctest for math modules

* fix conflict

* fix conflict

* refine docs

* refine docs

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add pixel_shuffle module

* add pixel_shuffle testcase

* simplify implement

* simplify implement

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add convtranspose module

* add deconv2d module

* fix convtranspose2d build bug

* fix convtranspose module backward bug

* add deconv samples

* refine convtranspose2d docs

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* implement tensor.register_hook

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove SetAccTensor and misc changes

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove unnecessary dynamic cast

Signed-off-by: daquexian <daquexian566@gmail.com>

* add requires_grad check, remove local check

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add mseloss module

* add mseloss testcase

* delete debug code

* add mseloss testcase

* rename mseloss testcase

* fix docstring warning

* format docstring

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>