* refactor job_pass by maybe_system

* remove useless files

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Modified the OutputArgModifyFn interface

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* OutputArgModifier return maybe part_1

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* add JUST for hander in ForEachOperator

Co-authored-by: aishangjj <702572275@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add_node_size_config

* add IsThisProcessMaster

* fix error

* add CHECK_EQ

* mutil_process_on_single_client

* fix error

* minor fix

* fix error

* fix bug

* new_ci_about_mutil_process_on_single_clien

* fix unittest.py

* refactor parallel_desc

* fix bug

* backup code

* fix ResourceDesc::ResourceDesc

* make of_format

* fix test bug

* fix bug

* backup

* minor fix

* backup

* minor fix

* del useless log

* optimize code

* add ci test

* Update test.yml

* minor fix

* backup

* run callback_notifier after all critical sections done

* group ticks originated from different critical sections

* minor fix about plan display

* backup

* refine code

* minor fix

* add comment

* fix path error

* add CHECK

* do not use oneflow_worker

* minor fix

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: lixinqi <lixinqi0703106@163.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

* Refine InferTimeShape

* fix xla

* fix xla

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* add tick op for each machine

* TryBindBnWithOneofRegst

* add sink tick op in main_job

* refactor LinkMainJob

* fix typo in task_graph

* refactor AddGlobalCriticalSection

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

* add src_subset_tick for input-output critical section

* refactor AutoSourceTick and AutoSinkTick

* vectorizedly link main job

* resize vectorh identity_tick_op_names then access elements

* SrcSubsetTickCompTaskNode: bind bns and in_regst if bns is valid in current device

* refactor optional input to repeated inputs for SrcSubsetTickOpConf

* fix a bug in CaseCompTaskNode; fix a bug when create identity tick in main_job

* 1) Insert tick between sourc tick and src_subset_tick; 2) Insert tick between dst_subset_tick and sink tick

* stash code

* refactor MakeMainJob by using Range::ForEachSubRange

* refactor MakeMainJob by using Range::ForEachSubRange

* rename ReentrantLockLinkPoint to ReentrantLockBackEdge

* set piece id for regst sent by wait_and_send_ids actor

* callback_notifier_sink_tick

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* add tick op for each machine

* TryBindBnWithOneofRegst

* add sink tick op in main_job

* refactor LinkMainJob

* fix typo in task_graph

* refactor AddGlobalCriticalSection

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

* add src_subset_tick for input-output critical section

* refactor AutoSourceTick and AutoSinkTick

* vectorizedly link main job

* resize vectorh identity_tick_op_names then access elements

* SrcSubsetTickCompTaskNode: bind bns and in_regst if bns is valid in current device

* refactor optional input to repeated inputs for SrcSubsetTickOpConf

* fix a bug in CaseCompTaskNode; fix a bug when create identity tick in main_job

* 1) Insert tick between sourc tick and src_subset_tick; 2) Insert tick between dst_subset_tick and sink tick

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* add tick op for each machine

* TryBindBnWithOneofRegst

* add sink tick op in main_job

* refactor LinkMainJob

* fix typo in task_graph

* refactor AddGlobalCriticalSection

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

* add src_subset_tick for input-output critical section

* refactor AutoSourceTick and AutoSinkTick

* SrcSubsetTickCompTaskNode: bind bns and in_regst if bns is valid in current device

* refactor optional input to repeated inputs for SrcSubsetTickOpConf

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* add tick op for each machine

* TryBindBnWithOneofRegst

* fix typo in task_graph

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

* SrcSubsetTickCompTaskNode: bind bns and in_regst if bns is valid in current device

* refactor optional input to repeated inputs for SrcSubsetTickOpConf

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* source subset tick

* remove useless header files

* insert DstSubsetTickOp

* remove incorrect CHECK

* TryBindBnWithOneofRegst

* fix typo in task_graph

* rename and refactor DstSubsetTick::InferBlobDescs and SrcSubsetTick::InferBlobDescs

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* AutoRegistrationFactory add key type

* AutoRegistrationFactory add creators accessores (#3662)

* AutoRegistrationFactory add creators accessores

* refine

* const

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>

* refactor ParallelConf

* refactor parallel_conf

* fix parallel_conf init

* refactor opkernel_instruction_type init/compute

* fix test_cpu_only_user_op

* remove notes

* fux non_distributed_optimizer_pass.cpp bug

* fix code style

* fix a small bug

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>

* add license at root dir

* check in empty files

* rm space

* check in script

* update script

* fix bug

* add print

* fix

* add exit

* add to of_format

* add CI task

* fix license

* Revert "fix license"

This reverts commit 818b6d7691d3a8b4a25dd41a47ff2c5922b8ec57.

* only add once

* quick fix

* fix script

* dont fmt empty file

* fix

* quick fix

* fix py

* add license

* fix exit

* add license for hpp

* add license

* license new vm files

Co-authored-by: tsai <caishenghang@oneflow.org>

* 1) class Symbol; 2) return const Shape& from Blob::shape() (#2360)

* 1) class Symbol; 2) return const Shape& from Blob::shape()

* const Symbol<Shape> => const Symbol<Shape>&

* refactor Symbol<T>

* CachedCaller

* define Symbol<T>::ptr_ as std::atomic<const T*>

* replace unique_lock with lock_guard

* refactor Blob::shape()

* refactor Blob::shape_sym

* backup

* refactor cached caller

* op_infer_cache

* ThreadLocalCachedCall

* clang-format

* refactor Blob::shape()

* CheckNumAxesIdenticalAndAssign

* refactor symbol

* rename: FindOrInsertPtr => GetOrCreatePtr

* 1) LeftOnesExtendedAssign; 2) LeftOnesStrippedAssign

* fix a stupid bug in Symbol::operator!=

* fixed_vector

* rename: ToDimVec => ToDimVector

* remove parallel policy

* rm FC/rnn/embedding_look_up op/kernel

* add check data parallel for conv/layer_norm op

* bugfix: bias add + use math_add when batch size = 1

* add job_build_and_infer_ctx interface

* lbn_with_split_hint

* fix maybe macro

* fix signature of Maybe<T>::Error()

* job_build_and_infer_if

* add c_api_util wrapper for job_build_and_infer_ctx

* implement python/job_build_and_infer interface

* CurJobBuildAndInferCtx_AddPlacementGroup

* BuildJobAndInferCtx  and  Mgr  c++ implement (#2074)

* job_build_and_infer_ctx_mgr

* refine interface of infer_ctx_mgr

* JobBuildInferCtx set job conf; add and refine error type

* revert job.proto

* half impl of add op in build_infer_ctx

* generate op produced empty logical blob desc ; infer out blob desc interface

* job_build_and_infer_ctx VERSION 1

* add InferOutBlobDesc for conv op; remove record_piece_size in interface op

* maybe return

* job_set hold by job_build_and_infer_ctx_mgr

* check placement when infer ctx mgr leave cur job

* Global New/Delete JobBuildAndInferCtxMgr

* add JUST when ctx add op

* remove unused job_conf.arg_op_name

* fix bugs caused by python new api

* fix bugs caused by lack of Global<JobDesc>

* fix bugs caused by new api

* refactor compiler.Compile

* merge dev_python

* remove unused message proto

* rename api

* check in fixes

* fix by adding boxing method

* register tick op

* move code and add more check

* fix typo

* fix bug when filtering op nodes before adding tick

* sbp for tick input bn

* interface_blob_conf for output_op/switch_output_op

* set sbp conf for tuple identity op

* fix bugs when merge main plan

* delete useless code

* address review

* fmt

* refactor GenerateOpConf4Trainning

* more refactor

* refactor SetCtrlInOpName4VariableOp

* use uniq ptr

* refactor RewriteBoxingWithAllReduce

* refactor MakeAllReduceSequence

* refactor auto_mixed_precision

* refactor DumpLogicalBlobDescAndSbpSignature

* refactor group_boxing_by_dst_parallel

* refactor add_keep_header_only_op_conf

* refactor AutoSourceTick

* refactor AddTickForTimeShape

* refactor AutoSinkTick

* refactor AddGlobalOutputCriticalSections

* refactor SetOpTimeShape7BatchDimLbis

* speedup compile time

* fix keep header only