* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* support flow.Size

* support flow.Size

* format

* auto format by CI

* fix repeat module bug

* fix repeat module bug

* fix repeat module bug

* fix repeat module bug

* auto format by CI

* fix repeat module bug

* fix conflict

* fix conflict

* auto format by CI

* support tuple parameter

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* Adapt to new structure

* auto format by CI

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* IBVerbsCommNet add knobs

* clean

* move tensor to c++

Signed-off-by: daquexian <daquexian566@gmail.com>

* align with master and fix export error

Signed-off-by: daquexian <daquexian566@gmail.com>

* reformat

Signed-off-by: daquexian <daquexian566@gmail.com>

* rename some functions

Signed-off-by: daquexian <daquexian566@gmail.com>

* refine

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove unused header

Signed-off-by: daquexian <daquexian566@gmail.com>

* restore device repr and str

Signed-off-by: daquexian <daquexian566@gmail.com>

* upgrade pybind11 to 2.7.0 for https://github.com/pybind/pybind11/pull/2956



Signed-off-by: daquexian <daquexian566@gmail.com>

* impl parameter in c++, remove determine() call

Signed-off-by: daquexian <daquexian566@gmail.com>

* let parameter derive from TensorIf to own a separate grad_fn_node, add missing get_device method, remove out-dated methods in docs

Signed-off-by: daquexian <daquexian566@gmail.com>

* add test_parameter.py, fix requires_grad_ and get_device

Signed-off-by: daquexian <daquexian566@gmail.com>

* use oneflow cast instead of numpy cast

Signed-off-by: daquexian <daquexian566@gmail.com>

* restore format changes

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* fix tests involving _local_or_consistent_tensor

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* LazyInterpret gen Variable ParallelDistribution

* LazyInterpret NewScopeWithParallelDescByTensor

* Debug and pass test for  input/output op expr lazy tensor infer device

* Add normal infer user op scope impl; add test user op expr for cuda

* fix for review

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* nn.Graph call and launch impl

* Fix bug of 1.pybind11 list->vector; 2.GlobalJobDesc in NNGraph Compile

* Fix bug: AutoTick Multi-Client callback notifier lbn

* Fix bug: callback notifier op check in multi-client

* Fix bug: CheckOpGraph in multi-client

* Fix bug of MultiClientSessionContext new ProfileConf

* Fix bug of create NONE type kernel; add note; refine PlanUtil::PopulateOpAttibute

* Fix bug: CallbackNotifier kernel dtype with uint8; change tick dtype; Feat: add Kernel create fault-tolerant

* inputs tensor list

* rollback of hack

* Using TensorTuple for RunLazyNNGraph

* Fix bug: WaitAndSendIdsKernel handle for Multi-Client

* skip graph runnable test

* refine

* skip test graph optimizer for nn.Graph train job completer error

* debug code

* rm InputKernel Forward overwrite

* rm debug code

* revert useless change

* revert debug code in graph

* rm skip in test

* revert source tick op

* rm useless import

* mv test_graph_relu.py

* correct break

* change the code of finding tick

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Support symbol placement type in functional.

* add sbp and sbp list arg

* refine

Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* fix conflict

* auto format by CI

* fix license

* fix ci error

* Delete version.py

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine and add test case

* support ellipsis type slice

* refine

* refine

* support slice assign ellipsis type

* refine

* register fn to localtensor

* basic implemetation of dataloader

* format

* refine as comments

* fix comments

* remove useless code

* add test case into unnitest

* refine ascomments

* refine as comments

* register uint8 dtypeto support dataloader

* add cifer mnist dataset support

* format

* refine

* refine

* use F.interpolation

* export dataloader dataset

* export more api

* refine

* refine

* add oneflow_test_cache_dir

* refien

* refine

* refine

* refine

* update datasets links

* refine

* fix comment

* unset export

* auto format by CI

* add requirement

* always reisntall test deps

* update requirement

* add pillow as oneflow dependency

* auto format by CI

* add sessiong context debug str

* remove debug str

* remove experimental

* fix export

* update

* fixbug

* auto format by CI

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refein

* rm

* fix

* refein

* log

* add ignore

* add ignore

* refein

* refein

* refein

* Support numpy advanced indexing.

* Finally adjust if continuos subspace.

* Finish integer array indexing and combining indexing.

* Finish index array parsing.

* Add tensor indexing unittest

* Add check

* Refine

* Refine tensor getitem unittest

* Support int64_t cuda atomicAdd.

* Not use PyObjectPtr since PySequence_Fast_GET_ITEM returns borrowed reference.

* Remove experimental in indexing unittest.

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* fix bn autograd bug

* fix bn autograd bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine

* change abs.py for test

* refine

* refine

* refine

* refine

* refine

* refine

* add

* refeom

* refine

* refine

* refien

* refine

* refine

* refine

* refien

* add check

* refine

* refine

* refine

* skip license

* refine

* refien

* refine

* refien

* refien

* refine

* refine

* refine acos

* refien

* remove enable_eager_execution() in oneflow/python

remove @experimental_api decorator for apis, except that ones
having same name with single client api.

* remove all import oneflow.experimental in doctest

* replace oneflow.experimental with oneflow in docstr

* reformat

* auto format by CI

* refine

* refien

* refine

* refien

* refine

* refine

* refine

* refein

* refine

* remove signle client twice api

* refine

* refine

* refine

* refien

* refien

* refine

* refien

* refien

* refien

* refine

* refein

* refein

* refein

* refeine

* refein

* refien

* fix

* remove experimental of nn

* refine

* rm oneflow.experimental.F

* refein

* refine

* refien

* refein

* okokok

* remove experimental in test/modules and test/tensor

* refein

* oneflow.python => oneflow

* rm absolute_import

* remove experimental in test/dataloader

* remove experimental call in nn/modules

* remove experimental from nn/ framework/ test_utils

* remove experimental in functionalpy

* refein

* refein

* refein

* refein

* fix python reserved words

* refein

* fix alias

* refein

* refien

* refein

* refein

* refein

* refein

* refine

* refein

* refein

* refein

* add

* refein

* refein

* remove experimental of new merge

* fix

* rm

* refien

* mv setuppy

* refien

* skip fmod grad test

* remove experimental of fmod

* refein

* setup py proj

* refien

* fix error when do multi case test on graph

* remove outdated rst files

* refine and add new rst files

* refein

* check in src

* format

* revert test_fmod.py

* refien

* remove all experimental in python/nn dir (done!)

* remove experimental in test/graph

* refein

* refein

* format

* add ignore

* refein

* add linagl

* refei

* refine rst

* update build whl

* fix

* refein

* refein

* refine

* refine

* auto format by CI

* add license

* add step

* refine

* refien

* refein

* update paths

* updatte paths

* refine

* reifne

* refein

* refein

* refein

* rrefien

* refien

* refein

* refine scatter_nd

* refine oneflow.rst

* refein

* from . import linalg
from . import autograd
from . import optim

* refein

* refine load_minst's docstring

add image.rst module.rst

refine rst files

* reformat

* remove Graph api

* fix file name

* refien

* fix oneflow.nn.image

* import oneflow.nn.image

* refein

* fix from . import lr_scheduler

* fix

* refeine

* refein

* refein

* rm todo

* rm unused box

* refein

* refien

* refein

* refien

* add ignore

* fix distributed api

* refein

* refein

* refine

* refine

* fix doctest

* refein

* update

* refien

* refein

* refein

* refein

* refein

* refein

* fix colition

* refien

* refien

* fix doc

* fix module doc

* fmt

* auto format by CI

* create moduels

* fix

* fix

* auto format by CI

* fix proto problem

* update cmake

* refien

* refein

* refienrefein

* refien

* refein

* add ignore

* refein

* refein

* refien

* auto format by CI

* refein

* refei

* refein

* refien

* refein

* udpate commit

* add ignore

* fix

* refein

* refein

* fix saved_model

* rjefein

* refein

Co-authored-by: GehangZhang <649975329@qq.com>
Co-authored-by: YaoChi <later@usopp.net>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: strint <xiaoyulink@gmail.com>

* refine oneflow/python/ops/array_ops.py

	modified:   ../oneflow/python/nn/modules/masked_select.py
	modified:   ../oneflow/python/ops/array_ops.py

* refine

* refine

* rm reduce_ops.py and summary files

* refine layers etc.

Co-authored-by: binbinHan <han_binbin@163.com>

* refine and add test case

* support ellipsis type slice

* refine

* refine

* support slice assign ellipsis type

* refine

* register fn to localtensor

* add constantpad1d kernel

* add constantpad1d kernel

* implementation of fuctional api/gradients/test case

* refine test case

* refine

* format docstring

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* nn.Graph call and launch impl

* Fix bug of 1.pybind11 list->vector; 2.GlobalJobDesc in NNGraph Compile

* Fix bug: AutoTick Multi-Client callback notifier lbn

* Fix bug: callback notifier op check in multi-client

* Fix bug: CheckOpGraph in multi-client

* Fix bug of MultiClientSessionContext new ProfileConf

* Fix bug of create NONE type kernel; add note; refine PlanUtil::PopulateOpAttibute

* Fix bug: CallbackNotifier kernel dtype with uint8; change tick dtype; Feat: add Kernel create fault-tolerant

* inputs tensor list

* rollback of hack

* Using TensorTuple for RunLazyNNGraph

* Fix bug: WaitAndSendIdsKernel handle for Multi-Client

* skip graph runnable test

* refine

* skip test graph optimizer for nn.Graph train job completer error

* Skip SetDefaultVariableConf JobPass in Multi-Client

* Fix bug: register_variable_op_names_and_tensors args with TensorTuple

* refine test graphs

* rm experimental in new scripts

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix export bug

* add new_autotest_part1

* add new_autotest_part1

* add squeeze test

* add permute test

* add hardswish test

* add prelu

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* change doctest code in abs.py

* del >>> flow.enable_eager_execution() in  modules/abs.py

* del @experimental_api

* del @experimental_api

* slove doctest not pass

* change abs.py for test

* refine acos

* remove enable_eager_execution() in oneflow/python

remove @experimental_api decorator for apis, except that ones
having same name with single client api.

* remove all import oneflow.experimental in doctest

* replace oneflow.experimental with oneflow in docstr

* reformat

* auto format by CI

* remove signle client twice api

* refine

* remove experimental of nn

* okokok

* remove experimental in test/modules and test/tensor

* remove experimental in test/dataloader

* remove experimental call in nn/modules

* remove experimental from nn/ framework/ test_utils

* remove experimental in functionalpy

* remove experimental of new merge

* skip fmod grad test

* remove experimental of fmod

* fix error when do multi case test on graph

* remove outdated rst files

* refine and add new rst files

* format

* revert test_fmod.py

* remove all experimental in python/nn dir (done!)

* remove experimental in test/graph

* add linagl

* refine rst

* refine scatter_nd

* refine oneflow.rst

* refine load_minst's docstring

add image.rst module.rst

refine rst files

* reformat

* remove Graph api

* fix file name

* fix index

Co-authored-by: YaoChi <later@usopp.net>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: strint <xiaoyulink@gmail.com>
Co-authored-by: tsai <jackalcooper@gmail.com>

* refine oneflow/python/ops/array_ops.py

	modified:   ../oneflow/python/nn/modules/masked_select.py
	modified:   ../oneflow/python/ops/array_ops.py

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix error when do multi case test on graph

* format

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Test 'nn.Linear' and 'nn.Identity' with random data

* Test 'nn.PixelShuffle' with random data

* Test 'greater' and 'less' with random data

* Test 'flow.Tensor.flatten' with random data

* Test 'greater' and 'less' with random-sized data

* Test 'nn.CrossEntropyLoss' with random data

* Replace multiple return entries with the 'oneof' generator

* Add 'nothing()' test cases for parameters with default values

* Remove the hard-coded test cases of `nn.CrossEntropyLoss`

* auto format by CI

* Skip the test of 'nn.CrossEntropyLoss' temporarily

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* fix-abs

* fmod-init

* broadcastfloormod_back

* vstorch

* modify_test

* fix_doctest_flag

* add-fmod-kernel

* fix-marco4binaryop

* split-marco

* fix-format

* add-autotest

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* LazyInterpret build LocalTensor if input is local

* refine graph default scope

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine

* allow post result to fail

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* add new autotest case

* add more tests

* fix bug

* auto format by CI

* add rtol and atol

* add rtol and atol

* add nothing

* auto format by CI

* fix export bug

* fix bug

* fix bug

* fix bug

* fix bug

* auto format by CI

* fix bug

* fix bug

* fix ci error

* fix bug

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* refactor job_pass by maybe_system

* refactor AutoSourceAndSinkTick to SingleClientAutoSourceAndSinkTick

* remove useless files

* add job_pass MultiClientAutoSourceAndSinkTick

* address review

* fix check

* fix OpConf4OpName return

Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add test on add input to graph

* add var into graph

* LazyInterpreter for FetchOutputOpExpr and set op parallel_distribution

* refine input var build

* split file

* rename

* mini refine

* add complete_graph_config

* add sgd

* fix style

* Add note

* LazyInterpret::ApplyImpl for UserOpExpr

* refine test scripts

* add output to graph

* format

* tensor.backward to lazy_add_loss(tensor)

* Fix bug of LazyInterpret UserOpExpr for change output lbns

* Add test user op expr test

* fix note mistake

* add userop and test

* rename

* add idea on l2 to variable op

* add scale in sgd

* refactor add loss

* revert change on tensor_name_scope

* fix typo

* check is lazy in tensor.backward

* rm useless state_tensortuple in nn.Graph

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* NNGraph interface and implement for CompileAndRuntime

* fix compile

* RunLazyNNGraph

* New/CloseRuntimeBuffers and RunLazyJob impl

* using job_conf.concurrency_width for Source/Sink BufferSize

* Add note for buffer size

* NNGraph interface and implement for CompileAndRuntime

* fix compile

* RunLazyNNGraph

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* prevent upload logs

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Generalize 'AdaptiveAvgPool' for 1~3 dims usage

* Implement GPU version

* Enable pooling with larger 'output_size' which is supported by PyTorch

* Add docs for 'AdaptiveAvgPool1d' and 'AdaptiveAvgPool3d'

* Add functional API for 'AdaptiveAvgPoolXd'

* Add 'flow.adaptive_avg_poolxd'

* Add test cases for 'flow.adaptive_avg_poolxd'

* auto format by CI

* Avoid using 'Shape::Count' in for loop

* Change names of compute functions

* Change 'AdaptivePoolNdGradOp' to functor

* Register integer types for CUDA 'adaptive_avg_poolxd' kernel

* Integrate 'adaptive_avg_poolxd' into 'nn.functional.interpolate'

* Correct wrong 'AdaptiveAvgPoolXd' entries

* Add missing 'not None' assertions for 'output_size'

* Support tuple input for 'AdaptiveAvgPool1d'

* Mark TODO for auto testing 'AdaptiveAvgPoolXd'

* Denote return types for 'BackwardOpConfGenFn'

* Combine test classes of 'AdaptiveAvgPoolXd' into one

* Rename 'AvgXXXCompute'

* auto format by CI

* Simplify output shape inference

* Delete the wrong 'if' block

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>