* fix-abs

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add coco reader module

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refine and add test case

* support ellipsis type slice

* refine

* refine

* support slice assign ellipsis type

* refine

* register fn to localtensor

* support bn track_running_stats=False

* remove useless import

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine and add test case

* support ellipsis type slice

* refine

* refine

* support slice assign ellipsis type

* refine

* register fn to localtensor

* register uint8 dtype of constantop

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* hwloc disable shared

* fix

* add conda env to simple ci

* refine

* refine

* fix

* refine

* refine

* fix

Co-authored-by: liujuncheng <liujuncheng1022@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* init

* first restruct version

* remove useless code

* add xpu_kernel class for elementwise ops

* xpu_kernel class for elementwise finished

* remove useless code

* Add Binary template and elu backward

* rename macro

* fix params order

* Add Ternary elemwise class

* add simple ternary op to test. will be deleted after restructed

* remove useless code

* restruct elu

* remove useless code

* refine

* restruct hardswish

* restruct activation

* remove useless code

* rename activation op

* remove elu useless code

* remove hardswish useless code

* fix file dir

* restruct hardsigmoid

* remove useless hardtanh code

* fix double to float

* restruct hardtanh

* fix elu params

* add space

* refine

* refine format

* test variable args macro

* refine header and macro

* rename code file

* remove code

* rename launcher

* rename macro

* change header file name

* restruct macro

Co-authored-by: doombeaker <later@usopp.net>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* delete useless code

* delete useless code

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine

* Update test.yml

* Update test.yml

* Update test.yml

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* hwloc disable shared

* fix

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Bowen Chen <bob2420083992@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add random generator

* reformat

* refactor: allow auto generator

* refactor: remove kAUTO, update python api and test

* refactor: use member initializer lists, fix build issue when cpu only

* handle exeception given invalid device

* add dropout functor; add OpExprInterpContext; refactor random_mask_like based on random_generator

* refactor random generator

* disable generator's copyandmove constructor

* reformat

* fix bad merge

* refine

* fix cpu only build

* auto format by CI

* refactor

* use global generator when no generator specified in functional api

* refine

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Houjiang Chen <chenhoujiangcug@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add changes for multi dev demo

Signed-off-by: daquexian <daquexian566@gmail.com>

* add part of backward hook

Signed-off-by: daquexian <daquexian566@gmail.com>

* update

Signed-off-by: daquexian <daquexian566@gmail.com>

* add naive init_with_env

Signed-off-by: daquexian <daquexian566@gmail.com>

* update

Signed-off-by: daquexian <daquexian566@gmail.com>

* update

Signed-off-by: daquexian <daquexian566@gmail.com>

* support_multi_client

* update

Signed-off-by: daquexian <daquexian566@gmail.com>

* Remove unused code

* Fix multi client launch

* fix __main__ bug

* update abcd op

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix multi client sync, make nccl instr ordered

Signed-off-by: daquexian <daquexian566@gmail.com>

* temp changes

Signed-off-by: daquexian <daquexian566@gmail.com>

* Use functional api instead of op_expr_helper::XXXOp.

* align with latest master, remove unused code

Signed-off-by: daquexian <daquexian566@gmail.com>

* local rank returns 0 when no env var, save is_multi_client in EnvDesc

Signed-off-by: daquexian <daquexian566@gmail.com>

* move is_multi_client to ProcessCtx, rename cuda_d2d device to nccl, remove unused code

Signed-off-by: daquexian <daquexian566@gmail.com>

* abcd -> return_first_input op

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove launch.py for now

Signed-off-by: daquexian <daquexian566@gmail.com>

* refine

Signed-off-by: daquexian <daquexian566@gmail.com>

* update IsMultiClient in env_util.py

Signed-off-by: daquexian <daquexian566@gmail.com>

* rm multi_dev_demo.py

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove exported functions in env_util.py

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove unused op expr helper func

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix bug

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove ddp code

Signed-off-by: daquexian <daquexian566@gmail.com>

* refine env.init. only call env.init in init.py when multi client

Signed-off-by: daquexian <daquexian566@gmail.com>

* revert unrelated changes

Signed-off-by: daquexian <daquexian566@gmail.com>

* add missing parameter

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix python api bug

Signed-off-by: daquexian <daquexian566@gmail.com>

* address comments

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* upload clang plugin log

Signed-off-by: daquexian <daquexian566@gmail.com>

* set build type to RelWithDebInfo

Signed-off-by: daquexian <daquexian566@gmail.com>

* add flow.view

* clean code

* clean code

* clean code

* clean code

* clean code

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add random generator

* reformat

* refactor: allow auto generator

* refactor: remove kAUTO, update python api and test

* refactor: use member initializer lists, fix build issue when cpu only

* handle exeception given invalid device

* Refine code style (#5375)

* fix unordered map: enum class not hashable

* roll back to unordered map

Co-authored-by: Houjiang Chen <chenhoujiangcug@gmail.com>

* add masked_select module impl

* add masked_select module test

* add masked_select module test

* add masked_select module test

* fix bug

* fix comment

* fix comment

* fix comment

* init gather_nd backward

* fix backward functor bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Use gh-action-scheduler-v2

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix upsample bilinear bug

* recover code

* align with pytorch

* redesign upsample bilinear

* fix align corner bug

* align bilinear with pytorch

* fix conflict

* fix bug

* code reuse

* fix commets

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add partial unary and math functional apis.

* Revert elementwise pow.

* auto format by CI

* Support add with large number of inputs.

* Update oneflow/python/nn/modules/math_ops.py

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

* Refine

* Migrate binary and activation ops.

* Migrate array ops.

* Add or refactor activation grad funcs.

* Add or refactor activation grad funcs.

* Revert unpack all

* Fix masked fill

* Refine

* Add nn ops.

* Refine

* Refine

* Migrate conv op

* Fix functional normalization.

* auto format by CI

* unfinished

* Refine code style

* align Torch params

* align Torch params

* develop unfinish

* add conv1d

* add conv1d docs rst

* add conv1d module and docs

* fix bias add error

* fix groups bug

* add test case

* Support optional parameter.

* fix group bug

* add new test case

* add more test case

* small fix

* add torch reference

* add conv base functor

* remove useless print

* reorganize code structure

* fix name and vector size

* fix pushback to at

* small fix for deconv docs

Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add hwloc for topology detection

* refine

* fix

* SetAffinityByDevice

* fix ninja build

* fix

* fix

Co-authored-by: Luyang <flowingsun007@163.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix(NotContiguousInput): fix not contiguous ndarray to tensor

* auto format by CI

* fix(*): fix old eager not contiguous ndarray bug

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add conv-with-bias test

* fix conv bias segmentfault bug

* fix compare

* hotfix for conv bias tmp_buffer

* refine conv2d-with-bias test

* refine test

* fix layers.conv2d

* fix test with nn.conv2d

* auto format by CI

Co-authored-by: BBuf <1182563586@qq.com>
Co-authored-by: simonJJJ <821898965@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* implement automated test

Signed-off-by: daquexian <daquexian566@gmail.com>

* add comments

Signed-off-by: daquexian <daquexian566@gmail.com>

* support backward

Signed-off-by: daquexian <daquexian566@gmail.com>

* add flatten example, handle args with default value, add comments

Signed-off-by: daquexian <daquexian566@gmail.com>

* raise exception on error

Signed-off-by: daquexian <daquexian566@gmail.com>

* use assertTrue instead of raise exception, add bn example

Signed-off-by: daquexian <daquexian566@gmail.com>

* set default n to 20

Signed-off-by: daquexian <daquexian566@gmail.com>

* use random eps and momentum in bn test example

Signed-off-by: daquexian <daquexian566@gmail.com>

* implement random_nd_tensor

* enhance random_nd_tensor, restore mis-commented code

Signed-off-by: daquexian <daquexian566@gmail.com>

* rename func -> module_class, add pytorch_module_class_name

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove enable_eager_execution()

Signed-off-by: daquexian <daquexian566@gmail.com>

* use dimX as arg name

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* skip conv test

Signed-off-by: daquexian <daquexian566@gmail.com>

* refine

* fix

* refien

* refine

* refine

* fix

* refine

* clear unknown user

* refine

* update tag and url

* refine

* refine

* refine

* refine

* refine

* refine

* add pip index

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: tsai <jackalcooper@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add ctc_greedy_decoder op

* Update gpu kernel

* Update example

* Performance optimizing

* Add OF_CUDA_CHECK

* Code optimization

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refien simple ci

* refine simple ci

* refine to see err easier

* refine

* refine

* refine

* refine

* refine

* refine

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add not_equal, greater_equal and less_equal module

* fix docs

* add hash method to Tensor

* remove __eq__ and __ne__ override

* fix doc

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix upsample bilinear bug

* recover code

* align with pytorch

* redesign upsample bilinear

* fix align corner bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check in files

* refine

* fix yml

* fix path

* refine

* refine

* refine

* refine

* refine cache

* refine

* refine

* check out oneflow first

* refine

* setup clang as compiler

* add ONEFLOW_MAYBE_CHECK_ONLY_FN

* add llvm libs to LD_LIBRARY_PATH

* refine

* refine

* refine

* refine

* update yml to use apt to install clang

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix wrong --target

Signed-off-by: daquexian <daquexian566@gmail.com>

* add oneflow deps

* refine

* use debug to run faster

* add note

* easy to see

* refine

* fix

* refine

* refine

* refine

* add missing JUST

Signed-off-by: daquexian <daquexian566@gmail.com>

* skip .cfg.cpp

Signed-off-by: daquexian <daquexian566@gmail.com>

* improve ONEFLOW_MAYBE_CHECK_ONLY_FN

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix cmake wrong message and fix env var

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove temp log

Signed-off-by: daquexian <daquexian566@gmail.com>

* add missing JUST in master

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Register no gradient user op, and check failed if the gradient func was missing for the op that needs gradient.

* Remove the default gradient func.

* Make broadcast logical ops no grad.

* Add identity op grad func

Co-authored-by: cheng cheng <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Device::compute_dep_object_

* sequantialize instructions in the same stream.

* refactor AttrMap

* refactor Tensor

* Export ConsistentTensor::is_cuda

* remove ConsistentTensor::blob_object

* refactor TensorImpl

* minor fix

* fix compiler' complains

* Implements EagerConsistentTensorImpl::New

* minor fix

* fix compiler complains

* remove unused code

* skip test_creating_consistent_tensor

* backup code

* Symbol::shared_from_symbol

* remove redundant header file includes

* fix bug in Symbol::shared_from_symbol

* symbolize ParallelDesc and ParallelDistribution

* symbolize Scope::GetParallelDesc()

* IsScalarType

* fix compiler complains

* InputConsistentTensorMeta

* refactor Scope with PlacementScope

* fix bug in exporting Scope to python

* backup code

* refactor DType

* fix compiler complains

* backup code

* DType is only allowed to be used in python code

* backup code

* dtype api bugfix

* fix error on exiting

Signed-off-by: daquexian <daquexian566@gmail.com>

* lazily get rank

Signed-off-by: daquexian <daquexian566@gmail.com>

* Export const DType* into python

* minor fix

* fix bug

* refine

* refactor signature of OpExpr::InferLogicalShapeAndDtype

* fix bug

* backup_code

* fix bug

* refactor SbpXXX to cfg::SbpXXX

* merge refactor_sbp_to_cfg_sbp

* fix bug

* Infer ConsistentTensorMeta

* Implement EagerConsistentInterpret::ApplyImpl

* 1) move XXXTensorMeta into the new file tensor_meta.h; 2) add new Class ConsistentTensorInferCache

* add class ConsistentTensorInferResult

* remove unused OpArgMutConsistentTensorMeta::parallel_distribution_

* fix stack-overflow bug in Tensor::mut_eager_mirrored_tensor_impl

* ignore empty parallel distribution constaint

* fix bug

* add explicit of cfg

* fix xla compile bug

* auto format by CI

* fix according comment

* fix bug

* refine

* refine

Co-authored-by: lixinqi <lixinqi0703106@163.com>
Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add changes

* fix

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>