* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* fix concat backward bug

* auto format by CI

* format

* Add autograd engine warning (#5444)

* add autograd engine warning

* fix bug

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* format

Signed-off-by: daquexian <daquexian566@gmail.com>

* only call getAsString() when isClassType() is true

Signed-off-by: daquexian <daquexian566@gmail.com>

* only call getAsString() when not null and isClassType() is true

Signed-off-by: daquexian <daquexian566@gmail.com>

* unset env vars, only set plugin when building oneflow itself

Signed-off-by: daquexian <daquexian566@gmail.com>

* reformat to llvm style

Signed-off-by: daquexian <daquexian566@gmail.com>

* Debug -> Release

Signed-off-by: daquexian <daquexian566@gmail.com>

* add missing JUST in master

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* MultiClientSessionContext

* export multi-client session context

* add NOTE

* fix for new IsMultiClient

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* graph api

* add graph dummy test

* add test

* add recursive module mode

* graph.build test pass

* add detail check on graph inner node

* support config and train

* add repr for debug

* test buffer

* test buffer add

* refine test

* add comment

* refine test

* refactor Node to Block

* add named_state

* refine Graph.named_state()

* add state_tensortuple

* graph._compile()

* add mc session 0

* nn.graph: state tuple to private var; add BlockType; add simple multi client session

* NNGraphIf

* rm old graph.cpp

* nn.graph: add cpp NNGraph; export and call NNGraph

* add comment

* nn.Graph: rm prototype MultiClientSession

* nn.Graph: rm prototype MultiClientSession test

* nn.Graph: add TODO

* nn.Graph: hack to get Graph object name

* nn.Graph: get obj name

* nn.Graph: get obj name 2

* nn.Graph: format for review

* nn.Graph: format

* nn.Graph: format

* nn.Graph: pass flake8 check

* Update graph.py

* name with init count

* name with init count 2

Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* implement smoothl1loss

* add smoothl1loss  test

* fix bug

* fix bug

* remove swp file

* fix bug

* refine

* refine_grpc_cmake_for_conda

* refine

* refine

* refine

* rm

* refine

* refine

* refine

* refine

* also fix opencv

* refine

* reorder

* refine gflags and glog

* rm useless

* rm useless

* rm useless

* refine

* rm var only available in protobuf

* add not-suppported check for reduce and size_average

* add test for datatype of int32

* auto format by CI

* implement backward and rebuild Factor

* rebuild Factor for smooth_l1_loss_grad

* auto format by CI

* fix bug

Co-authored-by: jackalcooper <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* store is_multi_client in a separate global obj, add launcher

Signed-off-by: daquexian <daquexian566@gmail.com>

* only call MasterSendAbort in single client

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove is_multi_client in ProcessCtx proto

Signed-off-by: daquexian <daquexian566@gmail.com>

* ClusterInstruction::HaltBarrier -> OF_ENV_BARRIER

Signed-off-by: daquexian <daquexian566@gmail.com>

* use Maybe<bool> to store is_multi_client flag, restore init.py changes

Signed-off-by: daquexian <daquexian566@gmail.com>

* set an Error as initial value of Global<Maybe<bool>, MultiClient>

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* remove --user

Remove --user to fix the problem that oneflow cannot be installed to anaconda envs

* Update README.md

* Update README.md

* Update README.md

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* rm nn.Graph.train

* Update graph.py

GraphCofig default predict

* Update test_graph.py

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add pixelshufflev2 module

* update pixel shuffle module to v2

* handle pixelshufflev1

* handle pixelshufflev1

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refactor_build_wheel_to_support_oneflowinc_images

* address review

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add bernoulli module

* fix doc test

* add bernoulli functor

* make changes according to review

* refine (#5415)

* refine rng related tests; make generator.seed() as api

Co-authored-by: Ldpe2G <liangdepeng@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add arg skip_audit

* refine

* support inplace forward

* support inplace backward

* add test case

* add test case for clone

* inplace is not support for leaf nodes

* refine clone

* add checks

* refine

* forbid clone with no grad

* Separate autograd meta to tensor (#5267)

* separate autograd meta

* minor fix

* fix acc_grad interface

* fix acc_grad with null

* minor fix

* inplace without clone

* refine

* minor fix

* remove maybe from constructor

* change from create to set

* fix merge bugs

* fix merge bug

* remove inplace flag in local_call_opkernel_phy_instr_operand

* remove out-date codes

* refine code

* add JUST

* fix merge master bug

* revert autograd engine input_grad check

* fix bug in tensor_hook

Co-authored-by: wyg1997 <wyg19970408@gmail.com>
Co-authored-by: Houjiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* graph api

* add graph dummy test

* add test

* add recursive module mode

* graph.build test pass

* add detail check on graph inner node

* support config and train

* add repr for debug

* test buffer

* test buffer add

* refine test

* add comment

* refine test

* refactor Node to Block

* add named_state

* refine Graph.named_state()

* add state_tensortuple

* graph._compile()

* add mc session 0

* nn.graph: state tuple to private var; add BlockType; add simple multi client session

* NNGraphIf

* rm old graph.cpp

* nn.graph: add cpp NNGraph; export and call NNGraph

* add comment

* nn.Graph: rm prototype MultiClientSession

* nn.Graph: rm prototype MultiClientSession test

* nn.Graph: add TODO

* nn.Graph: format for review

* nn.Graph: format

* nn.Graph: format

* nn.Graph: pass flake8 check

Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: cheng cheng <472491134@qq.com>

* add bernoulli module

* fix doc test

* add bernoulli functor

* make changes according to review

* refine (#5415)

* fix

Co-authored-by: Bowen Chen <bob2420083992@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* add test_flow_tensor_xxx_against_pytorch func

* add test

* fix first comments

* code reuse

* fix comments

* fix comments

* fix conv test bug

* fix conv test bug

* fix comments

* fix comment

* fix comment

* fix comment

* fix comment

* fix bug

* fix comment

* fix comment

* fix comments

* fix comments

* fix comments

* fix comments

* fix comments

* code format

* refine autotest

* fix comments

* fix comments

* fix comment

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add random generator

* reformat

* refactor: allow auto generator

* refactor: remove kAUTO, update python api and test

* refactor: use member initializer lists, fix build issue when cpu only

* handle exeception given invalid device

* add dropout functor; add OpExprInterpContext; refactor random_mask_like based on random_generator

* refactor random generator

* Cast explicitily in unpack call to avoid confilt with Optional.

* Refactor generator

* Fix bad merge

* Refine

* Private default generator for each cuda device.

* Fix

* Refactor generator and refine unittest

* add lock; refine testcase

Co-authored-by: VertexC <bob2420083992@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* delete maxpool failed test

* delete maxpool failed test

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* Enhance the difficulty of test samples

* fix

* fix

* fix

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix upsample bilinear bug

* fix test

* recover code

* align with pytorch

* redesign upsample bilinear

* fix align corner bug

* align bilinear with pytorch

* enchaned 4-dim samples

* fix conflict

* add backward test

* fix bug

* complete grad test

* fix merge conflict

* delete handwritten interpolate

* bypass pytorch interpolate bug in test

* add tensor grad assignment

* add test

* revert upsample changes

* fix TypeError: set_grad(): incompatible function arguments.  bug

* add zero_grad() docstring

* formmat comment

* fix docstring

* fix suggestions

* Update oneflow/python/test/tensor/test_tensor.py

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

* Update oneflow/api/python/framework/tensor.cpp

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

* enhance grad setter

* refine zero_grad() docstring and add slice docstring declaration to rst

* adjust optim.rst order

* refine comment

* refine zero_grad() docstring

* fix bug

Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

* flow.S/B/P

* optimize

* optimize

* fix according comment

* add attr in experimental namespace

* optimize

* oneflow_export_value

* refine

* refine

* refine

* customized_symbol module

* refine

* fix bug

* Add new placement init func (#5408)

* add_new_placement_init_func

* flow.env.all_device_placement

* refine

* refine

* refine

* refine

* refine

* refine

* refine

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* chunk forward

* chunk forward

* forward nn.chunk

* chunk forward

* chunk forward

* complete module flow.chunk

* auto format by CI

* flow.chunk/tensor.chunk

* resolve confict

* auto format by CI

* functional api

* auto format by CI

* doctest.testmod(raise_on_error=True)

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add image_batch_align module

* add testcase

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add instruction

* refine

* address review

* fix

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix-abs

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add coco reader module

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refine and add test case

* support ellipsis type slice

* refine

* refine

* support slice assign ellipsis type

* refine

* register fn to localtensor

* support bn track_running_stats=False

* remove useless import

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine and add test case

* support ellipsis type slice

* refine

* refine

* support slice assign ellipsis type

* refine

* register fn to localtensor

* register uint8 dtype of constantop

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* hwloc disable shared

* fix

* add conda env to simple ci

* refine

* refine

* fix

* refine

* refine

* fix

Co-authored-by: liujuncheng <liujuncheng1022@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* init

* first restruct version

* remove useless code

* add xpu_kernel class for elementwise ops

* xpu_kernel class for elementwise finished

* remove useless code

* Add Binary template and elu backward

* rename macro

* fix params order

* Add Ternary elemwise class

* add simple ternary op to test. will be deleted after restructed

* remove useless code

* restruct elu

* remove useless code

* refine

* restruct hardswish

* restruct activation

* remove useless code

* rename activation op

* remove elu useless code

* remove hardswish useless code

* fix file dir

* restruct hardsigmoid

* remove useless hardtanh code

* fix double to float

* restruct hardtanh

* fix elu params

* add space

* refine

* refine format

* test variable args macro

* refine header and macro

* rename code file

* remove code

* rename launcher

* rename macro

* change header file name

* restruct macro

Co-authored-by: doombeaker <later@usopp.net>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* delete useless code

* delete useless code

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine

* Update test.yml

* Update test.yml

* Update test.yml

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* hwloc disable shared

* fix

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Bowen Chen <bob2420083992@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add random generator

* reformat

* refactor: allow auto generator

* refactor: remove kAUTO, update python api and test

* refactor: use member initializer lists, fix build issue when cpu only

* handle exeception given invalid device

* add dropout functor; add OpExprInterpContext; refactor random_mask_like based on random_generator

* refactor random generator

* disable generator's copyandmove constructor

* reformat

* fix bad merge

* refine

* fix cpu only build

* auto format by CI

* refactor

* use global generator when no generator specified in functional api

* refine

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Houjiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>