Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Test 'nn.Linear' and 'nn.Identity' with random data

* Test 'nn.PixelShuffle' with random data

* Test 'greater' and 'less' with random data

* Test 'flow.Tensor.flatten' with random data

* Test 'greater' and 'less' with random-sized data

* Test 'nn.CrossEntropyLoss' with random data

* Replace multiple return entries with the 'oneof' generator

* Add 'nothing()' test cases for parameters with default values

* Remove the hard-coded test cases of `nn.CrossEntropyLoss`

* auto format by CI

* Skip the test of 'nn.CrossEntropyLoss' temporarily

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* fix-abs

* fmod-init

* broadcastfloormod_back

* vstorch

* modify_test

* fix_doctest_flag

* add-fmod-kernel

* fix-marco4binaryop

* split-marco

* fix-format

* add-autotest

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* add new autotest case

* add more tests

* fix bug

* auto format by CI

* add rtol and atol

* add rtol and atol

* add nothing

* auto format by CI

* fix export bug

* fix bug

* fix bug

* fix bug

* fix bug

* auto format by CI

* fix bug

* fix bug

* fix ci error

* fix bug

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* Generalize 'AdaptiveAvgPool' for 1~3 dims usage

* Implement GPU version

* Enable pooling with larger 'output_size' which is supported by PyTorch

* Add docs for 'AdaptiveAvgPool1d' and 'AdaptiveAvgPool3d'

* Add functional API for 'AdaptiveAvgPoolXd'

* Add 'flow.adaptive_avg_poolxd'

* Add test cases for 'flow.adaptive_avg_poolxd'

* auto format by CI

* Avoid using 'Shape::Count' in for loop

* Change names of compute functions

* Change 'AdaptivePoolNdGradOp' to functor

* Register integer types for CUDA 'adaptive_avg_poolxd' kernel

* Integrate 'adaptive_avg_poolxd' into 'nn.functional.interpolate'

* Correct wrong 'AdaptiveAvgPoolXd' entries

* Add missing 'not None' assertions for 'output_size'

* Support tuple input for 'AdaptiveAvgPool1d'

* Mark TODO for auto testing 'AdaptiveAvgPoolXd'

* Denote return types for 'BackwardOpConfGenFn'

* Combine test classes of 'AdaptiveAvgPoolXd' into one

* Rename 'AvgXXXCompute'

* auto format by CI

* Simplify output shape inference

* Delete the wrong 'if' block

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* update automated test framework

Signed-off-by: daquexian <daquexian566@gmail.com>

* rename func -> generator

Signed-off-by: daquexian <daquexian566@gmail.com>

* reformat

Signed-off-by: daquexian <daquexian566@gmail.com>

* update more powerful conv test

Signed-off-by: daquexian <daquexian566@gmail.com>

* new imperative autotest api

Signed-off-by: daquexian <daquexian566@gmail.com>

* update automated test framework

Signed-off-by: daquexian <daquexian566@gmail.com>

* add some tests and refine code

Signed-off-by: daquexian <daquexian566@gmail.com>

* update test_matmul.py

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix test_matmul.py

Signed-off-by: daquexian <daquexian566@gmail.com>

* add atol and rtol, fix test_matmul.py

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* reformat

Signed-off-by: daquexian <daquexian566@gmail.com>

* tricky export

* tricky export

* fix comments

* auto format by CI

* fix comments

* auto format by CI

* fix code

* dirtyfix

* fixname

* fix

* fix bug

* fix comments

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: tsai <jackalcooper@gmail.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* init flip op

* add flip op register

* add flip cpu kernel forward

* add flip kernel impl

* add flip op functor and gradient_funcs

* add test, still need fix bug

* fix segmentfault bug

* add docs

* fix comments

* fix comments

* fix comments

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* implementation of constantpad-3d op

* format

* refine

* rename files

* refine

* refine test case

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Multi-Client LogicalRun degenerate to PhysicalRun

* Add Logical->Physical in Multi-Client for IdGenerator and InstructionsBuilder

* multi client test

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* fix wrong std::shared_ptr

Signed-off-by: daquexian <daquexian566@gmail.com>

* run module api test by test_multi_client/generic_test.sh

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix test script

Signed-off-by: daquexian <daquexian566@gmail.com>

* use oneflow.distributed.launch -m unittest discover instead of bash for loop to speed up the test

Signed-off-by: daquexian <daquexian566@gmail.com>

* update ci/test_multi_client/generic_test.sh

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* mkdir compatible/single_client

* mv to compatible/singe_client/python

* mv to ./python/compatible/

* use oneflow.compatible.single_client

* back up

* backup

* refactor oneflow.xxx to oneflow.compatible.single_client.xxx

* mv to single_client

* refactor import oneflow to oneflow.compatible.single_client

* backup

* refine

* refine

* make single_clint ci test copy from correct path

* refine

* refine

* delete ioconf and split single-clint ci test

* auto format by CI

* delete default session

* fix method of erase sess id

* delete string multi client in test.yml

* fix spell error

* delete space

* refine

* mv single client python file to compatible_single_client_python

* refine

* fix ci error

* fix bug

* fix bug

* refine

* refine

* delete skip if in test/(modules/tensor)/* because of default eager env

* fix bug

* synchronization oneflow/compatible_single_client and oneflow/python

* fix distributed_run bug

* fix test_multi_process dead lock

* fix test_multi_process dead lock

* synchronization modify between oneflow/python and oneflow/compatible_single_client

* refine

* minor fix

* fix bug

* fix doctest.sh

* rm files

* remove files and add checks

* delete moduels

* Revert "delete moduels"

This reverts commit f95ab369f556b71fd0663b67c6dd51c9b6d5ee81.

Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: liufengwei <2472937968@qq.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: tsai <jackalcooper@gmail.com>

* fix param

* fix maxpool1d params error

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add scale and new_size

* add and refine test_case

* fix testcase

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* startup: add sum testcase

* mean, max, min, sum

* refine test mul

* refine test mul case

* refine div, mul, sub test case

* refine masked_fill test case

Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix upsample bilinear bug

* recover code

* align with pytorch

* redesign upsample bilinear

* fix align corner bug

* align bilinear with pytorch

* fix conflict

* fix bug

* code reuse

* fix commets

* add upsample_linear_1d_user_op

* add upsample_linear_1d_user_op

* add linear1d interp cpu kernel

* add linear1d interp gpu kernel

* rename kernel name

* add upsample nearest_1d kernel

* add upsample nearest_1d cuda kernel

* add upsample nearest_1d op

* fix complie bug

* add bicubic interp cpu kernel forward

* add bicubic2d interp cpu kernel backward impl

* add bicubic2d gpu kernel impl

* fix bicubic kernel compile error

* add bicubic2d kernel register

* fix compile bug

* nearest3d interp cpu kernel

* add nearest3d gpu kernel

* add nearest3d kernel registry

* add nearest3d cpu kernel

* add nearest3d cpu kernel

* add nearest3d cpu kernel

* fix upsample nearest interp bug

* fix upsample nearest interp bug

* add trilinear3d cpu kernel forward

* add trilinear3d cpu kernel backward

* add trilinear gpu kernel impl

* add trilinear op registry

* init interpolate nn function

* add upsample functors

* add interolate module

* add interpolate test_case

* restruct upsample_nearest2d and upsample_bilinear2d

* add upsample nearest2d and bilinear2d op registry

* add upsample nearest2d and bilinear2d op registry

* add upsample kernel grad functor

* add upsample_nearest_2d backward

* add upsample_linear_1d backward

* add upsample_nearest_1d backward

* add upsample_bicubic_2d backward

* fix backward impl bug

* fix nearest1d backward bug

* try to fix backward segmentfault

* add data_format parammeter

* fix upsample kernel backward bug

* fix cuda kernel impl bug

* fix upsample nearest bug

* fix align_corner attr error

* align linear1d impl with pytorch

* fix trilinear register bug

* fix trilinear bug

* add trilinear aligncorners test

* add test for bicubic, still has bug

* reimpl bicubic2d kernel

* align cubic interpolate with pytorch

* add bicubic cuda kernel

* fix bug

* align api with pytorch

* add doctest for interpolate

* update interpolate docs

* fix comments

* fix check_of_return comment

* fix comment

* fix comments

* fix commnet

* fix comment

* fix comments

* fix comments

* fix comments

* fix comments

* fix comment

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Luyang <flowingsun007@163.com>

* add gelu test

* add test for activation module

* add acos test

* add acosh test

* add tensor sigmoid test

* remove wrong acos

* add arccosh test

* remove acosh arccosh atan2 unittest

* remove arccosh acosh atan2 unittest to test_math_ops.py

Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* fix default_value_not_set_bug

* fix bug

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add gather_nd and scatter_nd

* add testcase

* stable_api

* add docs and doctest

* add scatter_nd grad

* gradfunc

* delete some code comments

* add grad test

* add docs

* update words usage

* delete some import

* update

* update

* auto format by CI

Co-authored-by: Yao Chi <later@usopp.net>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refine

* refine

* refine

* add inplace testcase

* check inplace valid

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

* refine

* refine inplace name

* reformat

* auto format by CI

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* add asinh and arcsinh test

* add asinh and arcsinh test

* add sinh test

* add atan2 module test

* add softplus test, softplus module has bug

* comment softplus

* add elu module autotest

* refine

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* init nn.image.flip

* clean code

* add unit test.

* revert python.cmake

* auto format by CI

* resolve comments

* add functor

* fix image_flip

* fix testcase

* fix warning:  Unexpected indentation

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: simonJJJ <821898965@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* init of op diag

* init of op diag

* modify  op diag

* merge testcase

* delete no use tensor diag

* amend diag api docs

* resolve conficts

* resolve confict

* resolve confict

* resolve confict

* auto format by CI

* add tensor.diag

* auto format by CI

* amend API description

* add non_square test

* amend diag API description

* amend diag API description

* compact test code

* compact code

* auto format by CI

* amend API description

* amend API description

* motify functional API

* motify test_diag

* amend diag functional API

* amend diag docstring

* auto format by CI

* amend code standards

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* add autotest part1

* add asin and arcsin module test

* auto format by CI

* try to fix export oneflow.python.test ci error

* delete __init__.py

* delete autotest tensor api

* fix ci export test_tensor_against_pytorch bug

* recover

* add tensor test

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add in_top_k function

* auto format by CI

* fix bug

* Update oneflow/python/ops/math_ops.py

Co-authored-by: Yao Chi <later@usopp.net>

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Yao Chi <later@usopp.net>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* startup: use python, not C extension

* rst changes test

* refine

* add F.rst

* add bernoulli add docstr example

* add doctest for functional method

* del docstr utils immediately in init.py

* refine test case

* use sin, cos instead of add, mul

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add torch maxpool2d cpu kernel implementation

* refine cpu implementation

* op gpu implemetation

* bug-fix

* refine

* add pooling kernel util

* reimplementation of cpu kernel

* gpu kernel implementation

* gpu kernel implementation

* bug fix

* change indice index

* change indice index of grad compute

* add flag: return_indices

* format code

* update inteface param

* add param dilation

* bug-fix

* update padding param

* add maxpool2d test case

* format code

* bug fix

* update tese case

* update sbp of indice

* add code sample

* add new line

* use oneflow_api.BlobDesc

* add op torch maxpool1d

* add sample codes

* change dtype of indice

* add torch maxpool3d interface

* add implementation of maxpool3d

* reformat codes

* add atomic add

* bug-fix

* backward bug fix

* update search index

* format code

* update imterface params

* add sample code of maxpool3d

* refine and add test case

* support ellipsis type slice

* refine

* refine

* support slice assign ellipsis type

* refine

* register fn to localtensor

* merge master

* update maxpooling ops

* refactor maxpool2d op

* add ceil mode testcase

* rm useless code

* refine

* refactor maxpool3d op and test cases

* refine

* add maxpool1d implementation

* simplify params using struct

* fix sbp fn

* del notes

* refine

* refine

* add pooling functional api impl

* refine

* update sbp fn

* refin thread num

* refine

* auto format by CI

* rm bn test for debug

* limit registers num

* remove double dtype

* .

* auto format by CI

* update data type

* refine

* add cuda synchronize and error string capture

* move synchronize placement

* debug all kernel compute

* debug

* debug

* debug

* debug

* auto format by CI

* debug std infinity

* debug

* add numeric utils

* format

* refine

* add reference

* remove useless header

* move namespace to detail

* fix doctest error

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* feat(add_): support inplace add

* fix(add_): fix push bug

* Add InplaceableBinaryFunctor

* feat(add_): add tensor += other

* docs(add_): fix docs

* fix relu module bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix -1 index not support bug

* add test case

* fix -1 index not support bug

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* fix concat backward bug

* auto format by CI

* format

* Add autograd engine warning (#5444)

* add autograd engine warning

* fix bug

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* implement smoothl1loss

* add smoothl1loss  test

* fix bug

* fix bug

* remove swp file

* fix bug

* refine

* refine_grpc_cmake_for_conda

* refine

* refine

* refine

* rm

* refine

* refine

* refine

* refine

* also fix opencv

* refine

* reorder

* refine gflags and glog

* rm useless

* rm useless

* rm useless

* refine

* rm var only available in protobuf

* add not-suppported check for reduce and size_average

* add test for datatype of int32

* auto format by CI

* implement backward and rebuild Factor

* rebuild Factor for smooth_l1_loss_grad

* auto format by CI

* fix bug

Co-authored-by: jackalcooper <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add pixelshufflev2 module

* update pixel shuffle module to v2

* handle pixelshufflev1

* handle pixelshufflev1

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add bernoulli module

* fix doc test

* add bernoulli functor

* make changes according to review

* refine (#5415)

* refine rng related tests; make generator.seed() as api

Co-authored-by: Ldpe2G <liangdepeng@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* support inplace forward

* support inplace backward

* add test case

* add test case for clone

* inplace is not support for leaf nodes

* refine clone

* add checks

* refine

* forbid clone with no grad

* Separate autograd meta to tensor (#5267)

* separate autograd meta

* minor fix

* fix acc_grad interface

* fix acc_grad with null

* minor fix

* inplace without clone

* refine

* minor fix

* remove maybe from constructor

* change from create to set

* fix merge bugs

* fix merge bug

* remove inplace flag in local_call_opkernel_phy_instr_operand

* remove out-date codes

* refine code

* add JUST

* fix merge master bug

* revert autograd engine input_grad check

* fix bug in tensor_hook

Co-authored-by: wyg1997 <wyg19970408@gmail.com>
Co-authored-by: Houjiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add bernoulli module

* fix doc test

* add bernoulli functor

* make changes according to review

* refine (#5415)

* fix

Co-authored-by: Bowen Chen <bob2420083992@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* add test_flow_tensor_xxx_against_pytorch func

* add test

* fix first comments

* code reuse

* fix comments

* fix comments

* fix conv test bug

* fix conv test bug

* fix comments

* fix comment

* fix comment

* fix comment

* fix comment

* fix bug

* fix comment

* fix comment

* fix comments

* fix comments

* fix comments

* fix comments

* fix comments

* code format

* refine autotest

* fix comments

* fix comments

* fix comment

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* delete maxpool failed test

* delete maxpool failed test

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* chunk forward

* chunk forward

* forward nn.chunk

* chunk forward

* chunk forward

* complete module flow.chunk

* auto format by CI

* flow.chunk/tensor.chunk

* resolve confict

* auto format by CI

* functional api

* auto format by CI

* doctest.testmod(raise_on_error=True)

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>