* dim_gather support dynamic input and index

* add dynamic test case

* refine test case

Co-authored-by: doombeaker <later@usopp.net>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* parallel desc with symbol_id

* migrate ParallelDescSymbol

* fix code format

* fix bug in oneflow_testexe

* Make oneflow worker docker stay alive for 6 hours

* exception

* except in pybind11 and python

* finetune api

* print traceback

* fix bug

* fix format

* ParallelDesc::cfg_parallel_conf

* remove traceback in test_checkpoint

* fix python codeformat

* del job_build_and_infer_cfg_error.py

* optimize api struct

* refactor JobDesc

* migrate JobConfSymbol

* del useless lines

* del useless lines

* add CompileOptionWrongError

* add CompileOptionWrongError

* replace python ScopeSymbol with cpp Scope

* fix typo

* fix bug

* rename OF_COMPLIE_OPTION_EEEOR

* fix conflict

* fix format

* fix bug

* fix conflict

Co-authored-by: clackhan <han_binbin@163.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* parallel desc with symbol_id

* migrate ParallelDescSymbol

* fix code format

* fix bug in oneflow_testexe

* Make oneflow worker docker stay alive for 6 hours

* exception

* except in pybind11 and python

* finetune api

* print traceback

* fix bug

* fix format

* ParallelDesc::cfg_parallel_conf

* remove traceback in test_checkpoint

* fix python codeformat

* del job_build_and_infer_cfg_error.py

* optimize api struct

* refactor JobDesc

* migrate JobConfSymbol

* del useless lines

* del useless lines

* add CompileOptionWrongError

* add CompileOptionWrongError

* rename OF_COMPLIE_OPTION_EEEOR

* fix conflict

Co-authored-by: lixinqi <lixinqi0703106@163.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>

* add reflection_pad2d

* add kernel of reflection_pad2d

* add backward implementation of reflection_pad2d

* add backward implementation of reflection_pad2d

* Modify the code according to the comment

* passing by reference

* add python interface of reflection_pad2d

* uptate padding boundry

* remove useless codes

* modify op kernel function

* rename op

* del useless code

* del useless function

* add backward of reflection pad2d

* add test case of reflection_pad2d

* update

* add sample to docs

* add blank line

* refine format

* refine docs

* refine docs

* del useless variable

* reformat code

* using NdIndexToOffset

* rename function

* reformat

* modify sbp fn

* reformat

* modify test case

* change variable name

* reformat

* abstract fuction

* Extract the for loop as a function

* extract codes to public methods

* extract core logic codes

* delete useless variables

* modify function implementation

* reimplement function logic

* add ReflectionPad2dFunctor

* refine op function

* add .cu

* .

* add implemetation of cuda kernel

* bug fix

* update test case

* add op util

* del useless op funtion

* add float16 support

* .

* instantion for fp16 functors

* update test case

* atomic add

* update atomic add

* update

* update thread num

* update test case

* reimplement reflection pad op

* update thread num

* bug fix >> add memset

* update functor

* add fp16 test case

* update sbp function

* format code

* refine

* use cuda::atomic::Add

* refine

* update placement of watch_diff

* update test case

Co-authored-by: doombeaker <later@usopp.net>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* parallel desc with symbol_id

* migrate ParallelDescSymbol

* fix code format

* fix bug in oneflow_testexe

* Make oneflow worker docker stay alive for 6 hours

* exception

* except in pybind11 and python

* finetune api

* print traceback

* fix bug

* fix format

* ParallelDesc::cfg_parallel_conf

* remove traceback in test_checkpoint

* fix python codeformat

* del job_build_and_infer_cfg_error.py

* optimize api struct

* add CompileOptionWrongError

* rename OF_COMPLIE_OPTION_EEEOR

Co-authored-by: lixinqi <lixinqi0703106@163.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* slice support empty blob

* update

* update

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add HobNumAxes

* fix OpKernel::InferShape

* migrate argwhere to user op

* update test

* rm argwhere op_conf  and kernel_conf

* format

* default dtype

* update test for multi device

* Make oneflow worker docker stay alive for 6 hours

* fix pass params

* update test

* update macro lock

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* rm old tanh

* fix

* rm useless

Co-authored-by: guoran <guoran@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Refactor gpu_atomic_add

* refactor safe log

* fix

* fix

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Tidy up op_conf.proto

* fix

* fix

* refine

* fix

* fix

* fix typo

* fix error

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: clackhan <han_binbin@163.com>

Co-authored-by: guo ran <360112263@qq.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* remove sigmoid

* add Bce with logits loss

* add test cases

* refine

* change random distribution

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Tsai <caishenghang@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add instance norm op & unit tests

* fix test case name

Co-authored-by: Mardino <42901638+MARD1NO@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Upload log of distributed CI

* if always

* run_id

* fix indent

* refine

* refine

* fix

* +r

* a'd

Co-authored-by: Tsai <caishenghang@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add kldivloss

* refine

* add kldivloss

* add test case

* convert to int8

* add docs

* fix annotation

* fix test

* Add skip

* Make oneflow worker docker stay alive for 6 hours

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

* fix training state saving in eager mode

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove unused include

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: binbinHan <han_binbin@163.com>

* Allow to keep oneflow_worker log in distributed CI

* save docker worker cid

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add ones api

* add ones

* fix format

* fix to constant

* fix format and change to constant

* refine

* refine

* fix format

Co-authored-by: Liang Depeng <liangdepeng@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix_python_cfg_error_bug

* del job_build_and_infer_error.py

* add userop and general pr tempaltes

* update

* fix labels requirement

* refine labels description

* refine requirments of labels

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

* refine requirment of pr title

* add requirements of test cases

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Add flatten/squeeze/expand_dims to auto mixed precision clear list and use reshape instead of reshape_like to do reshape grad computation (#4015)

* add flatten/squeeze/expand_dims to auto mixed precision clear list

* user static reshap grad for static tensor

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add pixel shuffle

* add grad test

* add name

* fix test case

* add pixel shufflev2

* add pixelshufflev2

* fix doc

* remove useless code

* add random grad check

* fix devices

* fix format

* fix test case

* fix reshape

Co-authored-by: Liang Depeng <liangdepeng@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* remove swig config in oneflow

* Update Dockerfile

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

* replace ErrorProto with cfg::ErrorProto

* fix macro name

* optimize error

* optimize error

* fix code style

* Organize the code

* fix code style

* fix code style

* migrate env rel api to pybind11

* migrate session rel api to pybind11

* migrate vm rel api to pybind11

* migrate op rel api to pybind11

* migrate calibration rel api to pybind11

* migrate framework rel api to pybind11

* migrate partial  ofblob rel api to pybind11

* migrate ofblob copy_shape rel api to pybind11

* migrate ofblob copy_data rel api to pybind11

* change RunInstruction's res to cfg::ErrorProto

* move compiler conditions to type_traits.h

* fix macro error

* remove swig in compilation process

* fix format

* remove oneflow_internal in ofblob.py

* remove oneflow_internal in c_api_util.py

* rename macro switch

* move py_kernel_registry.cpp to oneflow/api/python/framework/

* move py_kernel_registry.cpp to oneflow/api/python/framework/

* add inline and  macro lock in head file

* fix bug

* add head file

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add zeros api

* fix format

* fix to constant

* fix format

* refine

* refine

Co-authored-by: Liang Depeng <liangdepeng@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* 优化 rmsprop optimizer.

* lars optimizer

* bug fix

* reimplement the function of rmsprop.

* delete folder CMakeFiles.

* rewrite computeCrossEntropyGpu with cub.

* adjust the precise problem of crossEntropy fp16.

* fix the unit test script.

* adjust format of code

* fix bug according to review advices.

* adjust format of code

* adjust the code of unit test script.

* fix the format of unit test script.

* adjust the format for CI test.

Co-authored-by: guo ran <360112263@qq.com>

* CUDA template for element-wise kernels

* Add empty line

* ternary

* int8_t/int32_t/int64_t

* rename to .cuh

* tpm

* fix Ternary PackSize

* refine

* factory

* GenericLauncher

* UnaryPrimitive


fix


result_of


BinaryPrimitive


TernaryPrimitive


pack_size


ApplyGeneric


ApplyGeneric


fix


GenericLauncher


fix


refine

* aligned_storage

* ApplyPack

* refine

* refine

* XYZ=>ABC

* fix eager bug

* fix code format

* fix bug of op with more than one output under eager mode

* make of_format

* fix bug in InferBlobBackwardSignature

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>