* modified SetInputArgModifyFn

* Delete the CHECK changes in the assign_op.cpp file

* Format

* Modified the OutputArgModifyFn interface

* add return

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* OutputArgModifier return maybe part_1

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* input_arg_modifier return maybe

* gen_bw_fn return maybe

* bw_gen_fn return maybe

* registry_callback_fn return maybe

* fix bug after merge master

* fix bug

Co-authored-by: aishangjj <702572275@qq.com>

* modified SetInputArgModifyFn

* Delete the CHECK changes in the assign_op.cpp file

* Format

* Modified the OutputArgModifyFn interface

* add return

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* OutputArgModifier return maybe part_1

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* input_arg_modifier return maybe

* gen_bw_fn return maybe

* add MakeGenBackwardOpConf because ofstatement-expression not allowed outside function resulting in JUST failed in lambda

* add maybe after merge master

* fix bug: JUST in lambda

Co-authored-by: aishangjj <702572275@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Refactor infer ctx input tensordesc and related calls

* Modify the missing content

* Modify the post-merged error

* Add input judgment

* Missing modifications

* Fix merge errors

* format

* Modify omission

* Fix the crash caused by lack of judgment

* Delete comment

* Optimize naming

* Delete comment

* Delete blank lines

Co-authored-by: liufengwei0103 <2472937968@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add interface InferContext::OutputTensorDesc

* auto format by CI

* Implement 'OutputTensorDesc' for 'UserOpExprInferContext'

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refactor DType

* fix compiler complains

* backup code

* DType is only allowed to be used in python code

* backup code

* dtype api bugfix

* fix error on exiting

Signed-off-by: daquexian <daquexian566@gmail.com>

* lazily get rank

Signed-off-by: daquexian <daquexian566@gmail.com>

* Export const DType* into python

* minor fix

* fix bug

* refine

* refactor signature of OpExpr::InferLogicalShapeAndDtype

* fix bug

* fix bug

* fix bug

* fix bug

* override OutputShape

* fix bug

* more checks for EagerMirroredTensorImpl::eager_blob_object and EagerMirroredTensorImpl::tensor_storage

* override OutputDType

* fix strange bug

* fix complie bug

* Lazy construct functor to make sure that the operators has already been registered.

* override OutputIsDynamic4ArgNameAndIndex

* return Maybe<void>::Ok();

Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: daquexian <daquexian566@gmail.com>
Co-authored-by: hjchen2 <chenhoujiangcug@gmail.com>

* Rename InferXXXFn as XXXInferFn

* rename interface_name

* modify infer_xxx_fn to xxx_infer_fn

Co-authored-by: binbinHan <han_binbin@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* remove_user_op_conf_in_kernel_compute_ctx

* remove_user_op_conf_in_kernel_infer_ctx

* remove_user_op_conf_in_kernel_init_ctx

* remove_user_op_conf_in_kernel_create_ctx

* remove user_op_conf in UserKernelOpInferContext

* solve KernelComputeContext in nvtx_range_kernel.c

* remove user_op_conf in InferContext and slove bug after pull master

* del useless code

* remove attrs_ to derived class

* optimize

* del useless code

* use Attr() instead of attr()

* remove attr() in ctx

* minor fix

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add blob object id and eager blob object interface

* use maybe for blob object id

* minor fix

* add registry for data type, modified dropout as example

* revert origin data type infer

* add infer data type entry

* minor fix infer data type

* user op 32~40

* add unchange data type

* user op 26~40

* fix inferDataType from 0-25

* remove infer data type from util no change

* fix is_dynamic attribute

* user op 70~80

* user op 66~70

* remove redundant dtype

* minor fix one hot

* move data infer interface to user op

* review and fix

* Zailiang_Make infer data type away from tensor desc (#4544)

* same_padding_op.cpp

* scalar_add_op.cpp

* scalar_by_tensor_op.cpp

* scalar_mul_op.cpp

* scalar_pow_op.cpp

* Sigmoid_cross_entropy_op.cpp

* slice_op.cpp

* smooth_l1_loss_op.cpp

* softmax_cross_entropy_op.cpp

* Update scalar_by_tensor_op.cpp

* sigmoid_op.cpp

* Dev split infer data type from tensor desc (81~95) (#4545)

* Splite InferDataType form 81 to 95

* Add is_dynamic

* fix relu

* Fix relu_op

* Add Reduce ops InferDataType

* start up (#4539)

* start up

* 121~128 done

* refine

* edit test_ops.cpp

* SetPhysicalTensorDescInferFn

* finished

* reformat

* remove reduntant code

* upsampling

* test_ops

Co-authored-by: poohRui <yuruil@qq.com>
Co-authored-by: Yurui Li <32978179+poohRui@users.noreply.github.com>

* Dev refactor split infer data type (#4547)

* softmax_op.cpp

* 107:sort_op.cpp

* 106-115

* 108-109

* 110

* 111

* 112

* 113

* 115

* 106-115

* modify 106-115

* modify assign

* zzk fix

Co-authored-by: MARD1NO <359521840@qq.com>
Co-authored-by: Yurui Li <32978179+poohRui@users.noreply.github.com>

* fix compile bug

* warning fix

* user op 41-80 (#4540)

* user op 41-50

* user op 51-55

* fix image_object_preprocess ops

* user op 56-65

* update

* update

* update

* fix a bug

* fix bug

* fix bug

* fix devconv bug

* fix multiple bugs

* make tensor buffer to tesor list as todo

* add impl for tensor buffer to tensor list

* remove data type from infer shape in matmul

* add check

* fix transpose bug

* fix test op bug

* add check for add n

* fix cast to static shape op bug

* fix dynamic loss

* remove unnecessary code in flatten op

* fix hierarchical parallel

* fix leaky relu op

* fix math binary broadcast

* remove unnecessary data type infer in math binary

* fix terrible bug in math binary

* format

* fix softmax bug

* remove unnecessary code for softmax cross entropy

* fix conv

* remove unnecessary code in maximum

* add comments

* fix image preprocess

* fix layer norm

* auto to auto&

* fix partial fc sample

* fix partial fc sample

* fix image preprocess ops

* format

* user sigmoid op

* format

* fix partial fc

* fix prelu

* add newer ops

* fix bug in math binary

* fix combined margin loss op

* fix ofrecord reader

* add fused bias add op

* format

Co-authored-by: MARD1NO <359521840@qq.com>
Co-authored-by: Zailiang <zailiangyu@gmail.com>
Co-authored-by: Zhenhua <1209435+hengzi@users.noreply.github.com>
Co-authored-by: doombeaker <later@usopp.net>
Co-authored-by: Hongsheng Wang <31394900+wanghongsheng01@users.noreply.github.com>
Co-authored-by: Mosout <mosout@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: guo ran <360112263@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Remove batch_axis

* rm partial tick

* temp fix bug of flow.math.add

* fix bug of input_blob_def.split_axis init

* [KEY] refine infer sbp order value consider logical shape enable split

* ignore op get sbp sign ERROR.

* filter and check valid sbp sign by logical shape; rm magic num

* fix bug of return

* merge rm sigmoid cross entropy op

* rm sigmoid batch axis fn

* more debug log for check valid in get sbp sign

* rm useless check

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* layer_norm_grad_add_to_output

* add check

Co-authored-by: Juncheng <liujuncheng1022@gmail.com>

* refactor user_op_registry Attr

* Replace the User Op Registry Attr with a new interface (#3714)

* replace user op attr

Co-authored-by: lixinqi <lixinqi0703106@163.com>

* add license at root dir

* check in empty files

* rm space

* check in script

* update script

* fix bug

* add print

* fix

* add exit

* add to of_format

* add CI task

* fix license

* Revert "fix license"

This reverts commit 818b6d7691d3a8b4a25dd41a47ff2c5922b8ec57.

* only add once

* quick fix

* fix script

* dont fmt empty file

* fix

* quick fix

* fix py

* add license

* fix exit

* add license for hpp

* add license

* license new vm files

Co-authored-by: tsai <caishenghang@oneflow.org>

* Unified atter

* code format

* The blank space

* replace GetAttr to Attr

* replace Attr with SetAttr *.py

* code format

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

* add sparse cross entropy op

* sparse_cross_entropy_kernel_util

* add sparse softmax cross entropy op

* prediction_diff->dx

* test pass

* add if ENABLE_USER_OP

* refine code

* fix and add softmax util for build

* Add AlwaysComputeWhenAllOutputsEmpty()

* use TensorDesc and dx->prediction_diff

* address review

* add comment

* update sbp

* fix bug and InferAndTryRun

* nn support labels axes == prediction axes

* fix layer norm

* fix for dynamic shape

Co-authored-by: tsai <caishenghang@oneflow.org>

* refactor user op get sbp sign

* refine interface name

* remove AddSplitSbpSignList interface

* Add interface for OpArg vec

* refine UserOpSbpSignBuilder to lvalue; using sbp_sig_tmp; set default sbp fn B2B

* using SbpSignatuerList

* Dev cc user sbp feature (#2859)

* ut save data draft

* 将数据存入全局dict里

* change sbp to new interface

* rewrite the reshape user op util

* fix conflicts

* details of the repair

* modify the newly merged user op

* fix format check

* update if

* fix blank line and remove clear

Co-authored-by: ouyangyu <64632757+ouyangyu62@users.noreply.github.com>
Co-authored-by: tsai <caishenghang@oneflow.org>

* check in files

* add

* add

* port fw op

* check in grad

* add y

* add grad

* add

* add grad

* add fw gpu

* add bw

* add param diff

* update

* Optional

* naive py if

* update

* fix

* epsilon

* fix

* fix output

* fix output

* add begin_norm_axis

* fix cudnn_bn_scale_ones

* fix

* fix

* fix

* fix cudnn buf

* fix epsilon

* query optional as cc suggested

* minor fix

* fix op name

* fix grad

* fix cond

* fix norm

* add

* fix need_scale_out_diff

* handle optional in batch axis

* fix fw

* check in

* fix name

* fix test case

* smaller epsilon

* reorder

* use tmp buf

* rm debug info

* add dedicate template arg for param type

* add

* rm ref

* address review

* address review

* rm duplicated

* naive half

* update new if

Co-authored-by: tsai <caishenghang@oneflow.org>