* pass test on linear with training

* Refactor RuntimeCtx for multi-runtime

* refactor inplace to support nn graph

* block support iterator

* block iter add check

* fix scalar_mul op conf build

* deal with inplace after merge master

* add alexnet graph test

* add cpu test and format

* cout to glog

* deal with Job run finish bug

* refactor lazy deal with inplace

* deal with 0D tensor

* update data path

* address review

* deal with lazy default attr

* mv according to ci

* merge master

* fix for ci

* fix for ci limit

* block proxy func

* support module custom func and refacotr get attr of block

* auto format by CI

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* LazyInterpret::ApplyImpl for SourceUserOp and Copy

* Fix bug of CropMirrorNormalize ops; output ops check

* add test

* Add test case for tensor.to in nn.Graph

* remove useless code and refine error catch

* refine note

* Support CoinFlip module with consistent and device

* using const&

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Dev logical_and/or modules

* Add docs

* Add autotest

* Add single_client api

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add changes for multi dev demo

Signed-off-by: daquexian <daquexian566@gmail.com>

* add part of backward hook

Signed-off-by: daquexian <daquexian566@gmail.com>

* update

Signed-off-by: daquexian <daquexian566@gmail.com>

* add naive init_with_env

Signed-off-by: daquexian <daquexian566@gmail.com>

* update

Signed-off-by: daquexian <daquexian566@gmail.com>

* update

Signed-off-by: daquexian <daquexian566@gmail.com>

* support_multi_client

* update

Signed-off-by: daquexian <daquexian566@gmail.com>

* Remove unused code

* Fix multi client launch

* fix __main__ bug

* update abcd op

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix multi client sync, make nccl instr ordered

Signed-off-by: daquexian <daquexian566@gmail.com>

* temp changes

Signed-off-by: daquexian <daquexian566@gmail.com>

* Use functional api instead of op_expr_helper::XXXOp.

* align with latest master, remove unused code

Signed-off-by: daquexian <daquexian566@gmail.com>

* local rank returns 0 when no env v...

* pass test on linear with training

* Refactor RuntimeCtx for multi-runtime

* refactor inplace to support nn graph

* block support iterator

* block iter add check

* fix scalar_mul op conf build

* deal with inplace after merge master

* add alexnet graph test

* add cpu test and format

* cout to glog

* deal with Job run finish bug

* refactor lazy deal with inplace

* deal with 0D tensor

* update data path

* address review

* deal with lazy default attr

* mv according to ci

* merge master

* fix for ci

* fix for ci limit

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* export datasets interface

* auto format by CI

* fix docs

* add fix_datasets_export

* refine

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add RecordBytesDecoder

* add docs for OFRecordBytesDecoder

* update

* reformat the code

* resovlved conflits

* resolve conflicts

* resolving conflicts

* format the code

* Delete randperm.py

* format

* auto format by CI

* Update test_dataset.py

* auto format by CI

* The path in doctest fixed?

* Update test_dataset.py

* auto format by CI

* refine doctest

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* remove experimental

* add test num

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add multi client multi machine test

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove copy cores

Signed-off-by: daquexian <daquexian566@gmail.com>

* use discover in bash

Signed-off-by: daquexian <daquexian566@gmail.com>

* add tests in test.yml and refine

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove multi_client test files into test dir to reuse code

Signed-off-by: daquexian <daquexian566@gmail.com>

* delete distributed_run_multi_client.py and move impl in distributed_run.py

Signed-off-by: daquexian <daquexian566@gmail.com>

* if -> elif

Signed-off-by: daquexian <daquexian566@gmail.com>

* try three times and upload log

Signed-off-by: daquexian <daquexian566@gmail.com>

* add 'mode' arg in py

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* remove --multi_client in yml

Signed-off-by: daquexian <daquexian566@gmail.com>

* skip distributed test in cpu

Signed-off-by: daquexian <daquexian566@gmail.com>

* use new test container

Signed-off-by: daquexian <daquexian566@gmail.com>

* add host key to all machines

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* fix python version

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix python version

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Multi-Runtime for multi nn.Graph

* fix import os

* Fix BUG of IsMultiClient

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* 0-dim tensor support

* test case

* add more test

* refine

* update

* update default constructor

* reconstuct

* merge master

* remove notes

* remove useless codes

* fix comments

* fix comment

* add test case

* format

* refine

* refine

* refine

* refine

* MirroredTensorMeta::MirroredTensorMeta()

* support 0-dim slice

* support 0-dim slice grad

* refine

* auto format by CI

* refine

* refine

* auto format by CI

* refine

* fix slice bug

* auto format by CI

* fix resnet50 0-im loss uasge

* fix 0-dim tensor usage in test cases

* add skip test

* auto format by CI

* fix test_dataset

* check blobdesc.shape init

* auto format by CI

* remove useless empty shape init

* fix l1loss 0-dim error

* auto format by CI

* fix argmax op test

* fix add_n op test

* auto format by CI

* fix bce loss op test

* auto format by CI

* fix squeeze op test

* fix conv2d op test

* fix xpu_shape for clip_grad_norm

* auto format by CI

* resolve confilct

* fix multi-cpu slice_copier 0-dim bug

* auto format by CI

* add memory copy for 0-dim

* auto format by CI

* support copy0dim

* refine

* auto format by CI

* remove unuse codes

* fix check for kldivloss

* gpu 0-dim copy

* auto format by CI

* fix clip_grad_norm doctest

* fix reduce_ops doctest

* fix argmax doctest

* fix loss module doctests

* fix math_ops doctests

* fix norm modules doctest

Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add conv3d module

* add simple test case

* use conv base class to write conv3d

* still test error

* add torch style conv3d unit test

* fix format

* add assert

* unittest still error

* auto format by CI

* fix format and autotest

* remove dir

* remove useless file

* add extra expr

* auto format by CI

* fix import

* fix doc

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add batchnorm3d module

* add testing flag, add more log

Signed-off-by: daquexian <daquexian566@gmail.com>

* only test cpu device

* auto format by CI

Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* feat(Tensor): support 0shape tensor

* math binary broadcast support emoty tensor input

* slice support empty tensor input and output

* fix check in slice

* test(Cat): add 0shape cat module test

* fix return type error on gcc 4.8.5

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* add module op test for empty tensor, cuda kernel support empty tensor

* format

* feat(ReduceOp): reduce op kernels support 0shape tensor

* delete files added by mistake

* refine if

* refine if

* feat(ConstantOp): constant ops support 0shape tensor

* feat(ReshapeOp): reshape kernel support 0shape tensor

* math binary and unary backward skip when elem equal to zeros

* fix(ReduceOp): fix reduce not memset bug

* support getitem output empty tensor

* fix comment

* getitem support input is empty

* reduce_like kernel support empty

* fix op test bug

* feat(ReduceOp): refine reduce ops initialize value

* format code

* fix triu bug when input is empty

* test(AbsOp): fix test bug

* test(DivOp): fix test bug

* fix clamp bug

* fix test_sub bug

* fix(ReduceOp): fix reduce op memset bug

* auto format by CI

* fix random

Co-authored-by: liufengwei <2472937968@qq.com>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* rewrite slice_update backward

* use functional api

* fix comments

* refine

* refine

* refine

* add slice_grad functor

* refine

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* startup of dev scatter ops

* use dim scatter base class

* refine(using binop to abstract scatter update and add

* refine (use macros to implement kerenl class and functors)

* refine(description for register scatter ops/kernels)

* refine

* add inplace ops

* python wraper scatter_add inplace

* dev inplace ops

* refine dim_gather (using macros register mechanism)

* add grad of scatter_add_like

* refine (add src, like versions for scatter)

* refine src/like tensor

* gather refine(no need outplace/inplace versions)

* reformat

* refine

* test case of dim scatter

* test case for dim_scatter_add_like

* 1n2d test case for dim_scatter_add_like

* refine scatter sbp

* fail to sccater_add_like on 1n2d

* refing sbp

* refine test case, unify add and update like ops

* test case for scatter_add/update like ops finished

* test cases for scatter ops

* refine, merge test class

* startup of api docs

* add scatter api docs and assertion in python

* fix make error but still segment fault

* annotate sbp infer

* rewrite scatter kernel logic

* remove inplace proposal and fix macro name

* remove outdated atomic add

* move sbp infer

* add const and throw error

* add check

* set grad op

* add scatter scalar

* add scatter scalar gpu kernel

* add torch style backprop

* add torch style backprop check

* align with master

* remove redundant sbp check

* add test

* add float16n register

* fix sbp

* fix sbp

* add api doc

* make format

* add new line

* refine

* revert dim gather

* extract dim_scatter_add

* extracat scatter update ops

* add add/update functor

* rewrting by functors

* refine

* remove dim_gather_scatter_uitl.h

* add blank line

* refine macros for registering kerenls

* refine dim_scatter_scalar files name

* refine

* refine register ops

* refine

* add F.dim_scatter_scalar

* add scatter op

* refine docstr

* add scatter reduce arg

* finally(!): a draft for scatter constitent with pytroch

* change import package name

* remmove lazy test and add scatter_add and scatter_mul

* startup of scatter backward op

* add backward for scatter

* scatter ops backward finished

* add scatter, scatter_add test cases

* remove useless scatter_update_like

* reformat

* refine test cases

* refine according to comments

* revert op_exprt_helper

* fixed index element

* fix scatter update like expr for dim gather backward

Co-authored-by: doombeaker <later@usopp.net>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add autotest tan tanh floor arctanh

* add autotest tan floor tan

* Add autotest  for log1p

* Code format

* delete no use import

* add eye op alignment

* amend pytorch test

* delete merge error branch

* Delete log1p.py

delete merge error branch

* amend code because the list of master change

* amend eye test code

* amend eye docsting

* amend eye docsting

* amend eye docstring

* autotest test_eye

* auto format by CI

* amend param of eye

Co-authored-by: Zhenhua <huangzhenhua@zhejianglab.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add minimum and maximum

* add testcase

* fix docs

* move to python

* add elementwise min max grad func

* add autotest

* add broadcast min max grad func

* add broadcast min max testcase

* add broadcast_binary related functional

* convert gradient func to functional

* delete elementwise op_expr

* delete grad_op

* bind python

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* align view input style with torch

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* rewrite arange

* rewrite argwhere

* argwhere tensor testcase

* rewrite argmax test

* fix replicationpad2d padding size check

* rewrite replicationpad2d test

* rewrite layernorm test

* rewrite instantnorm123d test

* refine params to size_t

* fix comments

* refine test case

* auto format by CI

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add autotest for some math ops

* add autotest for some activation ops

* clean activation tests

* clean math op tests

* fix a bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refein

* refein

* refein

* port more changrs

* dirty workaround in autotest

* use cpu pkg to build doc

* fix

* fix

* dirty fix
cpu0_device

* refein

* refien

* refein

* refien

* reffine

* refein

* fix

* fix dataload api

* fix

* fix

* refein

* refein

* refein

* refei

* fix

* use a different path for lenet

* fxi

* add @flow.unittest.skip_unless_1n1d() for graph tests

* fxi

* refein

* refien

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* rebase

* check in gen py

* merge master and fix bugs

* address pr comments

* address pr comments

* auto format by CI

* functional python_arg

* auto format by CI

* remove unused files

* fix return type error on gcc 4.8.5

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* fix return type error in xrt

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix tick ibn sbp signature

* auto format by CI

Co-authored-by: tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: daquexian <daquexian566@gmail.com>

* Use function interface

* Delete blank lines

* modefied scalar add

* elementwise add

* broadcase add

* modified  ScalarAddByTensor

* modified Addmm

* Delete the old implementation

* fix _sub errot

* auto format by CI

* modified  _inplaceable

Co-authored-by: Yao Chi <later@usopp.net>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* add conv autotest

* add dead loop error raise

* add floor auto_test

* fix random tensor generator bug

* restruct math ops test

* add atan autotest

* restruct math_ops

* fix bug

* fix conv test bug

* fix commnets

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add activation

* rename swish to silu

* add selu

* add four activation op

* add softsign test

* add silu mish selu softsign

* Add softsign docs

* Add functional impl

* small fix for softsign backward

* remove flow.mish test

* add silu module test

* add selu test

* fix docs

* fix softsign docs

* fix format

* fix static cast

* merge master functional api yaml

* add torch style unittest

* Remove assert and add torch unittest

* add tensor def

* remove softsign test temporary

* add return maybe ok

* migrate nn ops to single_client dir

* migrate unittest

* remove lazy unittest

* add unittest

* fix to new directory

* Remove useless docs and single client test

* add doc

* fix docs

* add docs in oneflow and tensor namespace

* add torch autotest

* fix to new autotest

* remove outdated python code

* remove useless docs

* enlarge unittest tolerance

* Add static cast for const value

* skip softsign unittest

* skip tensor softsign

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* fix flip bug

* fix clip bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Feat: memory shared between EagerTensor with VariableRegst

* Fix BUG of Maybe<void> no return

* Add args of Runtime; Add Test Scripts and Runnable

* refine note

* refine test scripts

* refine code for review

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* support expand flow.Size and tuple input

* delete tmp file

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add valid optimizer case to pass job complete

* refactor and fix optimizer

* support l2 in variable op

* merge master

* format

* fix merge

* rm useless include

* address review

* auto format by CI

* rm internal tensor check after merge master

* restore add loss after merge master

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* support flow.Size

* support flow.Size

* format

* auto format by CI

* fix repeat module bug

* fix repeat module bug

* fix repeat module bug

* fix repeat module bug

* auto format by CI

* fix repeat module bug

* fix conflict

* fix conflict

* auto format by CI

* support tuple parameter

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* Adapt to new structure

* auto format by CI

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* move tensor to c++

Signed-off-by: daquexian <daquexian566@gmail.com>

* align with master and fix export error

Signed-off-by: daquexian <daquexian566@gmail.com>

* reformat

Signed-off-by: daquexian <daquexian566@gmail.com>

* rename some functions

Signed-off-by: daquexian <daquexian566@gmail.com>

* refine

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove unused header

Signed-off-by: daquexian <daquexian566@gmail.com>

* restore device repr and str

Signed-off-by: daquexian <daquexian566@gmail.com>

* upgrade pybind11 to 2.7.0 for https://github.com/pybind/pybind11/pull/2956



Signed-off-by: daquexian <daquexian566@gmail.com>

* impl parameter in c++, remove determine() call

Signed-off-by: daquexian <daquexian566@gmail.com>

* let parameter derive from TensorIf to own a separate grad_fn_node, add missing get_device method, remove out-dated methods in docs

Signed-off-by: daquexian <daquexian566@gmail.com>

* add test_parameter.py, fix requires_grad_ and get_de...

* LazyInterpret gen Variable ParallelDistribution

* LazyInterpret NewScopeWithParallelDescByTensor

* Debug and pass test for  input/output op expr lazy tensor infer device

* Add normal infer user op scope impl; add test user op expr for cuda

* fix for review

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* nn.Graph call and launch impl

* Fix bug of 1.pybind11 list->vector; 2.GlobalJobDesc in NNGraph Compile

* Fix bug: AutoTick Multi-Client callback notifier lbn

* Fix bug: callback notifier op check in multi-client

* Fix bug: CheckOpGraph in multi-client

* Fix bug of MultiClientSessionContext new ProfileConf

* Fix bug of create NONE type kernel; add note; refine PlanUtil::PopulateOpAttibute

* Fix bug: CallbackNotifier kernel dtype with uint8; change tick dtype; Feat: add Kernel create fault-tolerant

* inputs tensor list

* rollback of hack

* Using TensorTuple for RunLazyNNGraph

* Fix bug: WaitAndSendIdsKernel handle for Multi-Client

* skip graph runnable test

* refine

* skip test graph optimizer for nn.Graph train job completer error

* debug code

* rm InputKernel Forward overwrite

* rm debug code

* revert useless change

* revert debug code in graph

* rm skip in test

* revert source tick op

* rm useless import

* mv test_graph_relu.py

* correct break

* change the code of finding tick

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* fix conflict

* auto format by CI

* fix license

* fix ci error

* Delete version.py

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refine and add test case

* support ellipsis type slice

* refine

* refine

* support slice assign ellipsis type

* refine

* register fn to localtensor

* basic implemetation of dataloader

* format

* refine as comments

* fix comments

* remove useless code

* add test case into unnitest

* refine ascomments

* refine as comments

* register uint8 dtypeto support dataloader

* add cifer mnist dataset support

* format

* refine

* refine

* use F.interpolation

* export dataloader dataset

* export more api

* refine

* refine

* add oneflow_test_cache_dir

* refien

* refine

* refine

* refine

* update datasets links

* refine

* fix comment

* unset export

* auto format by CI

* add requirement

* always reisntall test deps

* update requirement

* add pillow as oneflow dependency

* auto format by CI

* add sessiong context debug str

* remove debug str

* remove experimental

* fix export

* update

* fixbug

* auto format by CI

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* Support numpy advanced indexing.

* Finally adjust if continuos subspace.

* Finish integer array indexing and combining indexing.

* Finish index array parsing.

* Add tensor indexing unittest

* Add check

* Refine

* Refine tensor getitem unittest

* Support int64_t cuda atomicAdd.

* Not use PyObjectPtr since PySequence_Fast_GET_ITEM returns borrowed reference.

* Remove experimental in indexing unittest.

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine

* change abs.py for test

* refine

* refine

* refine

* refine

* refine

* refine

* add

* refeom

* refine

* refine

* refien

* refine

* refine

* refine

* refien

* add check

* refine

* refine

* refine

* skip license

* refine

* refien

* refine

* refien

* refien

* refine

* refine

* refine acos

* refien

* remove enable_eager_execution() in oneflow/python

remove @experimental_api decorator for apis, except that ones
having same name with single client api.

* remove all import oneflow.experimental in doctest

* replace oneflow.experimental with oneflow in docstr

* reformat

* auto format by CI

* refine

* refien

* refine

* refien

* refine

* refine

* refine

* refein

* refine

* remove signle client twice api

* refine

* refine

* refine

* refien

* refien

* refine

* refien

* refien

* refien

* refine

* refein

* refein

* refein

* refeine

* refein

* refien

* fix

* remove experimental of nn

* refine

* rm oneflow.experimental.F

* refein

* refine

* refien

* refein

* okokok

* remove experimental in test/modules and test/tensor

* refein

* oneflow.python => oneflow

* rm absolute_import

* remove experimental in test/dataloader

* remove experimental call in nn/modules

* remove experimental from nn/ framework/ test_utils

* remove experimental in functionalpy

* refein

* refein

* refein

* refein

* fix python reserved words

* refein

* fix alias

* refein

* refien

* refein

* refein

* refein

* refein

* refine

* refein

* refein

* refein

* add

* refein

* refein

* remove experimental of new merge

* fix

* rm

* refien

* mv setuppy

* refien

* skip fmod grad test

* remove experimental of fmod

* refein

* setup py proj

* refien

* fix error when do multi case test on graph

* remove outdated rst files

* refine and add new rst files

* refein

* check in src

* format

* revert test_fmod.py

* refien

* remove all experimental in python/nn dir (done!)

* remove experimental in test/graph

* refein

* refein

* format

* add ignore

* refein

* add linagl

* refei

* refine rst

* update build whl

* fix

* refein

* refein

* refine

* refine

* auto format by CI

* add license

* add step

* refine

* refien

* refein

* update paths

* updatte paths

* refine

* reifne

* refein

* refein

* refein

* rrefien

* refien

* refein

* refine scatter_nd

* refine oneflow.rst

* refein

* from . import linalg
from . import autograd
from . import optim

* refein

* refine load_minst's docstring

add image.rst module.rst

refine rst files

* reformat

* remove Graph api

* fix file name

* refien

* fix oneflow.nn.image

* import oneflow.nn.image

* refein

* fix from . import lr_scheduler

* fix

* refeine

* refein

* refein

* rm todo

* rm unused box

* refein

* refien

* refein

* refien

* add ignore

* fix distributed api

* refein

* refein

* refine

* refine

* fix doctest

* refein

* update

* refien

* refein

* refein

* refein

* refein

* refein

* fix colition

* refien

* refien

* fix doc

* fix module doc

* fmt

* auto format by CI

* create moduels

* fix

* fix

* auto format by CI

* fix proto problem

* update cmake

* refien

* refein

* refienrefein

* refien

* refein

* add ignore

* refein

* refein

* refien

* auto format by CI

* refein

* refei

* refein

* refien

* refein

* udpate commit

* add ignore

* fix

* refein

* refein

* fix saved_model

* rjefein

* refein

Co-authored-by: GehangZhang <649975329@qq.com>
Co-authored-by: YaoChi <later@usopp.net>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: strint <xiaoyulink@gmail.com>