* rm LocalWorkStreamId

* rm AllocateLocalWorkStreamId in TaskNode

* rm local work stream id in task node and commnet task node

* rm local_work_stream_id param in NewTaskId

* fix test

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refactor CtrlClient

* RpcServer

* use Rpc

* Update rpc_client.cpp

add comment

Co-authored-by: lixinqi <lixinqi0703106@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Remove keep_header_only and BlobDesc::is_body_disabled

* Remove InputBlobModifier::use_header_only and UserOps set_use_header_only

* mig parallel_conf_util

* mig BuildInitialScope BuildScopeWithNewParallelDesc BuildScopeWithNewParallelConf

* add test of GetDeviceTagAndMachineDeviceIds

* mig GetOpConfSymbol

* fix BuildScopeWithNewParallelDesc input type error

* use TRY

* use symbol::Storage<OperatorConfSymbol>

* _NewOpKernelObject

* mig OpKernelObject

* mig object_storage

* make of_format

* del comment

* std::function<void(Object*)

* mig NewOpKernelObject and _StatefulCallOpKernel

* mig _StatefulCallOpKernel and GetSharedOpKernelObject4ParallelConfSymbol

* del object_storage.cpp

* use name GLOBAL_PARA_SYM2SHARED_OPKENEL_OBJ_MUTEX

* del comment

* use cfg_op_conf and Object*

* use Object*

* del _

* fix func name error

* Update object_storage.cpp

rename mutex

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Enable insert nccl logical op pass

* FindMaxConnectedSubgraphForGpuExecOrder~

* through order and interface

* implement of insert nccl logical op in pass

* add nccl logical op using UserOp Implement and EagerNcclCommMgr

* add NCCL ReduceScatter op/kernel; refine pass impl of topo order

* add NCCL logical op/kernel AllGather

* fix bug of reduce scatter/ all gather infer shape

* refine log and note

* fix complier err build with CPU ONLY

* support NCCL ALL2ALL and test pass of alexnet model parallel

* rollback of diff in checkpointing_pass.cpp

* rename to nccl_use_compute_stream; ResourceDesc::nccl_use_compute_stream; refine name for review; create nccl_comm_ in KernelCompute;

* refine code for review

* add unittest for nccl use compute stream

* format test scripts

* refine align

* Refactor InferBatchAxis

* refine

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* add tick op for each machine

* TryBindBnWithOneofRegst

* add sink tick op in main_job

* refactor LinkMainJob

* fix typo in task_graph

* refactor AddGlobalCriticalSection

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

* add src_subset_tick for input-output critical section

* refactor AutoSourceTick and AutoSinkTick

* vectorizedly link main job

* resize vectorh identity_tick_op_names then access elements

* SrcSubsetTickCompTaskNode: bind bns and in_regst if bns is valid in current device

* refactor optional input to repeated inputs for SrcSubsetTickOpConf

* fix a bug in CaseCompTaskNode; fix a bug when create identity tick in main_job

* 1) Insert tick between sourc tick and src_subset_tick; 2) Insert tick between dst_subset_tick and sink tick

* stash code

* refactor MakeMainJob by using Range::ForEachSubRange

* refactor MakeMainJob by using Range::ForEachSubRange

* rename ReentrantLockLinkPoint to ReentrantLockBackEdge

* set piece id for regst sent by wait_and_send_ids actor

* callback_notifier_sink_tick

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* add tick op for each machine

* TryBindBnWithOneofRegst

* add sink tick op in main_job

* refactor LinkMainJob

* fix typo in task_graph

* refactor AddGlobalCriticalSection

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

* add src_subset_tick for input-output critical section

* refactor AutoSourceTick and AutoSinkTick

* vectorizedly link main job

* resize vectorh identity_tick_op_names then access elements

* SrcSubsetTickCompTaskNode: bind bns and in_regst if bns is valid in current device

* refactor optional input to repeated inputs for SrcSubsetTickOpConf

* fix a bug in CaseCompTaskNode; fix a bug when create identity tick in main_job

* 1) Insert tick between sourc tick and src_subset_tick; 2) Insert tick between dst_subset_tick and sink tick

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* add tick op for each machine

* TryBindBnWithOneofRegst

* add sink tick op in main_job

* refactor LinkMainJob

* fix typo in task_graph

* refactor AddGlobalCriticalSection

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

* add src_subset_tick for input-output critical section

* refactor AutoSourceTick and AutoSinkTick

* SrcSubsetTickCompTaskNode: bind bns and in_regst if bns is valid in current device

* refactor optional input to repeated inputs for SrcSubsetTickOpConf

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* add tick op for each machine

* TryBindBnWithOneofRegst

* fix typo in task_graph

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

* SrcSubsetTickCompTaskNode: bind bns and in_regst if bns is valid in current device

* refactor optional input to repeated inputs for SrcSubsetTickOpConf

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* mig parallel_conf_util

* mig BuildInitialScope BuildScopeWithNewParallelDesc BuildScopeWithNewParallelConf

* add test of GetDeviceTagAndMachineDeviceIds

* mig GetOpConfSymbol

* fix BuildScopeWithNewParallelDesc input type error

* use TRY

* use symbol::Storage<OperatorConfSymbol>

* _NewOpKernelObject

* mig OpKernelObject

* mig object_storage

* make of_format

* del comment

* del comment

* use cfg_op_conf and Object*

* use Object*

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* mig parallel_conf_util

* mig BuildInitialScope BuildScopeWithNewParallelDesc BuildScopeWithNewParallelConf

* add test of GetDeviceTagAndMachineDeviceIds

* mig GetOpConfSymbol

* fix BuildScopeWithNewParallelDesc input type error

* use TRY

* use symbol::Storage<OperatorConfSymbol>

* _NewOpKernelObject

* del comment

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add CTC Loss

* Add backward kernel

* Remove tf in test

* Update api document

* Add zero_infinity option

* refine

* Add 1n2d test case

* Switch to consistent_view

* Fix Eager mode

* Remove duplicate license

* Add grad check

* Fix bw test

* Fix bugs

* Add op name

* Refine

* of_format

* Expand annotation

* Performance optimizing for cuda

* Check input_length & target_lengths

* Update __syncthreads

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* TryBindBnWithOneofRegst

* fix typo in task_graph

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Print backtrace when distributed CI failed

* fix path

* typo

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add Operator::InferInternalBlobDescs

* XrtLaunchOp::InferOutBlobDescs

* refine

* add weight_decay arguments

* add weight_decay test

* code format

* add weight_decay note

* Packed all separated mem block for allocate memory speed up

* only packed separated mem block used by same thread.

* refine implement; add thrd_id to mem_block

* refine name

* improve implement

* Remove OpContext

* fix

* Add Operator::InferInplaceObn2IbnIf

* remove useless header

* make InferInplaceObn2Ibn protected

* add pad2d ops and kernels

* fix bug

* add python api and unittest

* reformat, change padding_data_type seq, change dim2vector method usage

* fix typo

* delete ShapeViewToDimVector

* rerun make of_format

* save model and load model demo

* fix

* tensor.proto and copy signature

* pass test

* add load_saved_model function for InferenceSession

* wait_for_all_jobs_finished

* test_alexnet_save_and_load

* support change batch_size

* support batch axis

* add ci test

* revert job_build_and_infer_ctx api

* simplify test script

* following update

* improve search function of InferenceSession

* fix break update

* add cv2 to dev-requirements

* Update Dockerfile

* rm 3.5

* fix

* quick workaround

* speed up bazel

* port changes

* revert workaround

* rm batch_axis in JobInputDef and JobOutputDef

* rm export for ImageNetRecordDataset

* refine Complete api for GraphBuilder and SignatureBuilder

* refine check op is moirrored

* fix by review comment ci test

* InferenceSession is not responsible to destroy env

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: Tsai <caishenghang@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Remove useless code

* Remove OpGraph::MakeGetterBlobDesc4ModelLbi

* Remove ExecNode::fw_node_

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* port with cuda

* refactor generated_compile_flags with cuda

* use_cxx11_abi

* refactor

* larger tol

Co-authored-by: Tsai <caishenghang@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refactor boxing_sub_task_builder

* refine

* refine

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Interface alignment

* format code

* update padding param

* update test case

* format

* reduction

* update test case

* format

* fix slice_boxing s2s multi-machine when empty intersection

* refine

Co-authored-by: Juncheng <liujuncheng1022@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* mig parallel_conf_util

* mig BuildInitialScope BuildScopeWithNewParallelDesc BuildScopeWithNewParallelConf

* add test of GetDeviceTagAndMachineDeviceIds

* fix BuildScopeWithNewParallelDesc input type error

* use TRY

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* manylinux docker use pip_args

* optional bazel url

* move args

* fix repo url

* reorder cmd

* fix github case

* update manylinux sha

* http proxy lower case

* rm err msg

* mv msg

* fix case

* add exit 1

* disable centos-sclo-rh

* centos-sclo-rh skip_if_unavailable

* MANYLINUX_SHA

* Update Dockerfile

* Update Dockerfile

* refine

* use ali

* port more changes

* use oneflow url

* it works

* add rsync

* reorder

* refine

* refine

* refine

* use mirrot install cpython

* larger tol

Co-authored-by: Tsai <caishenghang@oneflow.org>