* pass test on linear with training

* Refactor RuntimeCtx for multi-runtime

* refactor inplace to support nn graph

* block support iterator

* block iter add check

* fix scalar_mul op conf build

* deal with inplace after merge master

* add alexnet graph test

* add cpu test and format

* cout to glog

* deal with Job run finish bug

* refactor lazy deal with inplace

* deal with 0D tensor

* update data path

* address review

* deal with lazy default attr

* mv according to ci

* merge master

* fix for ci

* fix for ci limit

* block proxy func

* support module custom func and refacotr get attr of block

* auto format by CI

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* LazyInterpret::ApplyImpl for SourceUserOp and Copy

* Fix bug of CropMirrorNormalize ops; output ops check

* add test

* Add test case for tensor.to in nn.Graph

* remove useless code and refine error catch

* refine note

* Support CoinFlip module with consistent and device

* using const&

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* pass test on linear with training

* Refactor RuntimeCtx for multi-runtime

* refactor inplace to support nn graph

* block support iterator

* block iter add check

* fix scalar_mul op conf build

* deal with inplace after merge master

* add alexnet graph test

* add cpu test and format

* cout to glog

* deal with Job run finish bug

* refactor lazy deal with inplace

* deal with 0D tensor

* update data path

* address review

* deal with lazy default attr

* mv according to ci

* merge master

* fix for ci

* fix for ci limit

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Multi-Runtime for multi nn.Graph

* fix import os

* Fix BUG of IsMultiClient

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refein

* refein

* refein

* port more changrs

* dirty workaround in autotest

* use cpu pkg to build doc

* fix

* fix

* dirty fix
cpu0_device

* refein

* refien

* refein

* refien

* reffine

* refein

* fix

* fix dataload api

* fix

* fix

* refein

* refein

* refein

* refei

* fix

* use a different path for lenet

* fxi

* add @flow.unittest.skip_unless_1n1d() for graph tests

* fxi

* refein

* refien

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Feat: memory shared between EagerTensor with VariableRegst

* Fix BUG of Maybe<void> no return

* Add args of Runtime; Add Test Scripts and Runnable

* refine note

* refine test scripts

* refine code for review

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add valid optimizer case to pass job complete

* refactor and fix optimizer

* support l2 in variable op

* merge master

* format

* fix merge

* rm useless include

* address review

* auto format by CI

* rm internal tensor check after merge master

* restore add loss after merge master

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* move tensor to c++

Signed-off-by: daquexian <daquexian566@gmail.com>

* align with master and fix export error

Signed-off-by: daquexian <daquexian566@gmail.com>

* reformat

Signed-off-by: daquexian <daquexian566@gmail.com>

* rename some functions

Signed-off-by: daquexian <daquexian566@gmail.com>

* refine

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove unused header

Signed-off-by: daquexian <daquexian566@gmail.com>

* restore device repr and str

Signed-off-by: daquexian <daquexian566@gmail.com>

* upgrade pybind11 to 2.7.0 for https://github.com/pybind/pybind11/pull/2956



Signed-off-by: daquexian <daquexian566@gmail.com>

* impl parameter in c++, remove determine() call

Signed-off-by: daquexian <daquexian566@gmail.com>

* let parameter derive from TensorIf to own a separate grad_fn_node, add missing get_device method, remove out-dated methods in docs

Signed-off-by: daquexian <daquexian566@gmail.com>

* add test_parameter.py, fix requires_grad_ and get_de...

* LazyInterpret gen Variable ParallelDistribution

* LazyInterpret NewScopeWithParallelDescByTensor

* Debug and pass test for  input/output op expr lazy tensor infer device

* Add normal infer user op scope impl; add test user op expr for cuda

* fix for review

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* nn.Graph call and launch impl

* Fix bug of 1.pybind11 list->vector; 2.GlobalJobDesc in NNGraph Compile

* Fix bug: AutoTick Multi-Client callback notifier lbn

* Fix bug: callback notifier op check in multi-client

* Fix bug: CheckOpGraph in multi-client

* Fix bug of MultiClientSessionContext new ProfileConf

* Fix bug of create NONE type kernel; add note; refine PlanUtil::PopulateOpAttibute

* Fix bug: CallbackNotifier kernel dtype with uint8; change tick dtype; Feat: add Kernel create fault-tolerant

* inputs tensor list

* rollback of hack

* Using TensorTuple for RunLazyNNGraph

* Fix bug: WaitAndSendIdsKernel handle for Multi-Client

* skip graph runnable test

* refine

* skip test graph optimizer for nn.Graph train job completer error

* debug code

* rm InputKernel Forward overwrite

* rm debug code

* revert useless change

* revert debug code in graph

* rm skip in test

* revert source tick op

* rm useless import

* mv test_graph_relu.py

* correct break

* change the code of finding tick

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine

* change abs.py for test

* refine

* refine

* refine

* refine

* refine

* refine

* add

* refeom

* refine

* refine

* refien

* refine

* refine

* refine

* refien

* add check

* refine

* refine

* refine

* skip license

* refine

* refien

* refine

* refien

* refien

* refine

* refine

* refine acos

* refien

* remove enable_eager_execution() in oneflow/python

remove @experimental_api decorator for apis, except that ones
having same name with single client api.

* remove all import oneflow.experimental in doctest

* replace oneflow.experimental with oneflow in docstr

* reformat

* auto format by CI

* refine

* refien

* refine

* refien

* refine

* refine

* refine

* refein

* refine

* remove signle client twice api

* refine

* refine

* refine

* refien

* refien

* refine

* refien

* refien

* refien

* refine

* r...