* align view input style with torch

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* rewrite arange

* rewrite argwhere

* argwhere tensor testcase

* rewrite argmax test

* fix replicationpad2d padding size check

* rewrite replicationpad2d test

* rewrite layernorm test

* rewrite instantnorm123d test

* refine params to size_t

* fix comments

* refine test case

* auto format by CI

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add autotest for some math ops

* add autotest for some activation ops

* clean activation tests

* clean math op tests

* fix a bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refein

* refein

* refein

* port more changrs

* dirty workaround in autotest

* use cpu pkg to build doc

* fix

* fix

* dirty fix
cpu0_device

* refein

* refien

* refein

* refien

* reffine

* refein

* fix

* fix dataload api

* fix

* fix

* refein

* refein

* refein

* refei

* fix

* use a different path for lenet

* fxi

* add @flow.unittest.skip_unless_1n1d() for graph tests

* fxi

* refein

* refien

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* rebase

* check in gen py

* merge master and fix bugs

* address pr comments

* address pr comments

* auto format by CI

* functional python_arg

* auto format by CI

* remove unused files

* fix return type error on gcc 4.8.5

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* fix return type error in xrt

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix tick ibn sbp signature

* auto format by CI

Co-authored-by: tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: daquexian <daquexian566@gmail.com>

* Use function interface

* Delete blank lines

* modefied scalar add

* elementwise add

* broadcase add

* modified  ScalarAddByTensor

* modified Addmm

* Delete the old implementation

* fix _sub errot

* auto format by CI

* modified  _inplaceable

Co-authored-by: Yao Chi <later@usopp.net>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* add conv autotest

* add dead loop error raise

* add floor auto_test

* fix random tensor generator bug

* restruct math ops test

* add atan autotest

* restruct math_ops

* fix bug

* fix conv test bug

* fix commnets

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add activation

* rename swish to silu

* add selu

* add four activation op

* add softsign test

* add silu mish selu softsign

* Add softsign docs

* Add functional impl

* small fix for softsign backward

* remove flow.mish test

* add silu module test

* add selu test

* fix docs

* fix softsign docs

* fix format

* fix static cast

* merge master functional api yaml

* add torch style unittest

* Remove assert and add torch unittest

* add tensor def

* remove softsign test temporary

* add return maybe ok

* migrate nn ops to single_client dir

* migrate unittest

* remove lazy unittest

* add unittest

* fix to new directory

* Remove useless docs and single client test

* add doc

* fix docs

* add docs in oneflow and tensor namespace

* add torch autotest

* fix to new autotest

* remove outdated python code

* remove useless docs

* enlarge unittest tolerance

* Add static cast for const value

* skip softsign unittest

* skip tensor softsign

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* fix flip bug

* fix clip bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Feat: memory shared between EagerTensor with VariableRegst

* Fix BUG of Maybe<void> no return

* Add args of Runtime; Add Test Scripts and Runnable

* refine note

* refine test scripts

* refine code for review

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* support expand flow.Size and tuple input

* delete tmp file

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* add valid optimizer case to pass job complete

* refactor and fix optimizer

* support l2 in variable op

* merge master

* format

* fix merge

* rm useless include

* address review

* auto format by CI

* rm internal tensor check after merge master

* restore add loss after merge master

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* support flow.Size

* support flow.Size

* format

* auto format by CI

* fix repeat module bug

* fix repeat module bug

* fix repeat module bug

* fix repeat module bug

* auto format by CI

* fix repeat module bug

* fix conflict

* fix conflict

* auto format by CI

* support tuple parameter

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* Adapt to new structure

* auto format by CI

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* IBVerbsCommNet add knobs

* clean

* move tensor to c++

Signed-off-by: daquexian <daquexian566@gmail.com>

* align with master and fix export error

Signed-off-by: daquexian <daquexian566@gmail.com>

* reformat

Signed-off-by: daquexian <daquexian566@gmail.com>

* rename some functions

Signed-off-by: daquexian <daquexian566@gmail.com>

* refine

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove unused header

Signed-off-by: daquexian <daquexian566@gmail.com>

* restore device repr and str

Signed-off-by: daquexian <daquexian566@gmail.com>

* upgrade pybind11 to 2.7.0 for https://github.com/pybind/pybind11/pull/2956



Signed-off-by: daquexian <daquexian566@gmail.com>

* impl parameter in c++, remove determine() call

Signed-off-by: daquexian <daquexian566@gmail.com>

* let parameter derive from TensorIf to own a separate grad_fn_node, add missing get_device method, remove out-dated methods in docs

Signed-off-by: daquexian <daquexian566@gmail.com>

* add test_parameter.py, fix requires_grad_ and get_device

Signed-off-by: daquexian <daquexian566@gmail.com>

* use oneflow cast instead of numpy cast

Signed-off-by: daquexian <daquexian566@gmail.com>

* restore format changes

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* fix tests involving _local_or_consistent_tensor

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* LazyInterpret gen Variable ParallelDistribution

* LazyInterpret NewScopeWithParallelDescByTensor

* Debug and pass test for  input/output op expr lazy tensor infer device

* Add normal infer user op scope impl; add test user op expr for cuda

* fix for review

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* nn.Graph call and launch impl

* Fix bug of 1.pybind11 list->vector; 2.GlobalJobDesc in NNGraph Compile

* Fix bug: AutoTick Multi-Client callback notifier lbn

* Fix bug: callback notifier op check in multi-client

* Fix bug: CheckOpGraph in multi-client

* Fix bug of MultiClientSessionContext new ProfileConf

* Fix bug of create NONE type kernel; add note; refine PlanUtil::PopulateOpAttibute

* Fix bug: CallbackNotifier kernel dtype with uint8; change tick dtype; Feat: add Kernel create fault-tolerant

* inputs tensor list

* rollback of hack

* Using TensorTuple for RunLazyNNGraph

* Fix bug: WaitAndSendIdsKernel handle for Multi-Client

* skip graph runnable test

* refine

* skip test graph optimizer for nn.Graph train job completer error

* debug code

* rm InputKernel Forward overwrite

* rm debug code

* revert useless change

* revert debug code in graph

* rm skip in test

* revert source tick op

* rm useless import

* mv test_graph_relu.py

* correct break

* change the code of finding tick

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Support symbol placement type in functional.

* add sbp and sbp list arg

* refine

Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* fix conflict

* auto format by CI

* fix license

* fix ci error

* Delete version.py

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refine and add test case

* support ellipsis type slice

* refine

* refine

* support slice assign ellipsis type

* refine

* register fn to localtensor

* basic implemetation of dataloader

* format

* refine as comments

* fix comments

* remove useless code

* add test case into unnitest

* refine ascomments

* refine as comments

* register uint8 dtypeto support dataloader

* add cifer mnist dataset support

* format

* refine

* refine

* use F.interpolation

* export dataloader dataset

* export more api

* refine

* refine

* add oneflow_test_cache_dir

* refien

* refine

* refine

* refine

* update datasets links

* refine

* fix comment

* unset export

* auto format by CI

* add requirement

* always reisntall test deps

* update requirement

* add pillow as oneflow dependency

* auto format by CI

* add sessiong context debug str

* remove debug str

* remove experimental

* fix export

* update

* fixbug

* auto format by CI

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* Support numpy advanced indexing.

* Finally adjust if continuos subspace.

* Finish integer array indexing and combining indexing.

* Finish index array parsing.

* Add tensor indexing unittest

* Add check

* Refine

* Refine tensor getitem unittest

* Support int64_t cuda atomicAdd.

* Not use PyObjectPtr since PySequence_Fast_GET_ITEM returns borrowed reference.

* Remove experimental in indexing unittest.

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine

* change abs.py for test

* refine

* refine

* refine

* refine

* refine

* refine

* add

* refeom

* refine

* refine

* refien

* refine

* refine

* refine

* refien

* add check

* refine

* refine

* refine

* skip license

* refine

* refien

* refine

* refien

* refien

* refine

* refine

* refine acos

* refien

* remove enable_eager_execution() in oneflow/python

remove @experimental_api decorator for apis, except that ones
having same name with single client api.

* remove all import oneflow.experimental in doctest

* replace oneflow.experimental with oneflow in docstr

* reformat

* auto format by CI

* refine

* refien

* refine

* refien

* refine

* refine

* refine

* refein

* refine

* remove signle client twice api

* refine

* refine

* refine

* refien

* refien

* refine

* refien

* refien

* refien

* refine

* r...