* Add sbp signatures and inplace guidance for operators

* Fix inplace guidance(input blob name) for cwise operator

* Reorder conflict edge search, fix conflicts to update node and cancle inplace if mem_shared_id is -1

* check in cherry picks

* split relu at every dim

* OpGraph::MakePredicatorIsAllLbiConsumersReachableToOpName

* refactor TaskGraph::EnableInplaceMemSharing

* TaskGraph::GetSafeInplaceOpBlobArgList

* InplaceLbiGraph::ForEachSafeInplaceEdgesInSourceOpSubTree

* fix a typo

* TaskGraph::SetTaskRegstInplaceInfo

* InplaceRegstGraph

* refine

* refine IsLbiOnTaskEdge

* fix a bug in TaskGraph::ForEachDeviceNodes

* ForEachGpuDeviceNodes

* remove wrong CHECK

* fix wrong use of std::unordered_set::erase

* fix a bug in TaskGraph::GetInplaceOpBlobArgList

* fix inplace bugs

* fix error CHECK between inplace in dptr and inplace out dptr

* Graph::FindFirstNonTrivialSCC

* refine Graph::FindFirstNontrivialSCC

* add comment for FindFirstNontrivialSCC

* FfsForEachNode

* Graph::FindFirstNonTrivialSCC

* refine Graph::FindFirstNontrivialSCC

* add comment for FindFirstNontrivialSCC

* aggregate boxing by dst parallel

* refine

refactor(actor.cpp/task.proto/regst_desc.proto): :recycle:remove RegstInplaceInfo, add inplace_consu (#1876)

* naive impl

* rename

* rename

* rename

* add cudnn only impl

* refine kernel

* refine

* address review

* address review

* rm unnecessary variable

* add new sbp

* add naive batch dim infer

* add sbps

* add sbps

* InplaceObnGraph

* more checks in InplaceObnGraph::InitNodes

* framework of InplaceObnGraph::ComputeSafeInplaceObns

* refine InplaceObnGraph::ComputeSafeInplaceObns

* replace InplaceObnGraph with InplaceLbiGraph

* fix three types of mut_ref conflicts

* InplaceLbiGraph::FindFirstConstRefConflictMutRefEdge

* fix bugs in InplaceLbiGraph::ComputeSafeInplaceObns

* InplaceLbiGraph::DisconnectUnReachabeAndDataMutableEdge

* InplaceLbiGraph::FixMutRefConflictsFromSourceOpNode

* InplaceLbiGraph::FixMutRefConflictsFromSourceOpNode

* Graph::FindFirstBackEdgeDstNode

* more CHECK_ISNULL

* fix a bug in Graph::FindFirstBackEdgeDstNode()

a

* fix bugs in Graph<NodeType, EdgeType>::ForEachConnectedComponent

* rename GetIsMutableIbnConsumer => FindSoleMutableIbnConsumer

* refine InplaceLbiGraph::IsConstRefConflictMutRefNode

* there could be no mut_ref node found in InplaceLbiGraph::FindFirstInterOpRefConflictMutRefEdge

* refine InplaceLbiGraph::FindFirstInterOpRefConflictMutRefEdge

* remove unnecessary CHECK in InplaceLbiGraph::GetSafeInplaceObnEdges

* fix a line of comment in InplaceLbiGraph::GetSafeInplaceObnEdges

* shouldn't delete the edge to updt_node

* refine InplaceLbiGraph::FixMutRefConflictsFromSourceOpNode

* refine FindFirstIntraOpRefConflictMutRefEdge

* fix a bug in InplaceLbiGraph::FindFirstIntraOpRefConflictMutRefEdge

* CheckSubGraph

* change some lambdas to functions

* style(actor.h): move customized virtual function declaration together

* feat: update Actor to support inplace regst

* feat: check inplace in/out regst dptr equal

* fix GetSbpSignatures

* fix

* UNIMPLEMENTED

* rm InferIsModel

* call NaiveInferHasBatchDim explicitly

* implement ReshapeOp::InferHasBatchDim() and AllReduceFacadeOp::InferHasBatchDim()

* identical_sbp_oba_pairs

* more identical_sbp_oba_pairs

* BindFwBwOpBlobArgPairs

* 1) fix bugs in CalcFwBwObaPairs; 2) simplify variable_op's sbp_signature_rule

* Operator::GetSbpSignatures

* sbp_signature_conf must be obeyed, while sbp_signature_hint is merely recommended

* replace sbp_sig_hint with CalcOrderValue4SbpSig

* fix sbp_signature bug in LayerNorm/LayerNormGrad/BatchGather

* implements InferSbpSignature for all reduce ops

* remove old implementation of sbp infer

* identical_sbp_oba_pairs

* more identical_sbp_oba_pairs

* BindFwBwOpBlobArgPairs

* 1) fix bugs in CalcFwBwObaPairs; 2) simplify variable_op's sbp_signature_rule

* InferHasBatchDim

* MatmulOp::InferHasBatchDim; GatherGradOp::InferHasBatchDim

* rm SbpInferHint::is_model_blob_

* fix mem cpy

* fix variable

* fix cond

* TieUpChainHeaderUnreachableFromVariableOps

* add dev_total_loss_instance_num_id_conf for boxing sharing

* constant op as total_loss_instance_num

* TieUpChainHeaderUnreachableFromVariableOps

* add dev_total_loss_instance_num_id_conf for boxing sharing

* replace lbn2sbp_parallel_hint with op_conf.sbp_signature_hint

* refine job_completer.cpp

1) AddRegst for ctrl edge; 2) new TaskNode: OptimizerCompTaskNode; 3) disconnect keep_header_only when chain merging (#1838)