* AutoRegistrationFactory add key type

* AutoRegistrationFactory add creators accessores (#3662)

* AutoRegistrationFactory add creators accessores

* refine

* const

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* replace urls

* add mirror

* add mirror and md5

* add mirror

* add mirror

* add mirror

* add mirror

* add mirror

* add mirror and md5

* add mirror and url

* add mirror

* quick fix url

* rm submodule files

* update readme

* rm submodule setup in ci

* rm setup submodule in ci release

* update readme

* add instructions on where to run mkdir build

Co-authored-by: Tsai <caishenghang@oneflow.org>
Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* fix bug about pad value

* reformat

* update test

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* support reflecting cfg message

* replace python f-string with traditional format

* more comments for cfg::Message methods

* integrate cfg primary

* add depend of of_ccobj

* add dependence of of_cfgobj

* fine-tuning the way of cfg integration

* add proto/cfg convert & DebugString

* finetune cfg

* optimize cfg & add support of nested message

* fix a small bug

* arrange cfg code

* optimize cfg

* fix code format

* add of_proto_python_dir arg to template_convert.py

* fix cfg constructor bug

* remove useless line in cfg.cmake

* optimize cfg.cmake

* delete useless line in cfg.cmake

* fintune cfg.cmake

* fix a small bug of oneof field

* support oneflow_api_registry

* fix code format

* Expand folder range

* optimize of_api_registry

* Update of_api_registry.h

add a blank line between constructor  and member function

* update pybind_module_registry

* fix code format

* optimize oneflow_pybind_api

* optimize pybind registry

* fix code foemat

* optimize cfg

* fix oneflow.cmake

* add a blank line

* optimize cfg

Co-authored-by: lixinqi <lixinqi0703106@163.com>

* refactor Session.job_name2current_scope_ to Session.scope_stack_

* Seperate scope related functions from Session to scope_util.py

* revert change

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>

* a) refactor Operator::GetCustomizedConf; b) refactor ReplaceInputLbnInOpCustomizedConf();

* rm XrtLaunchOp::GetCustomizedConf

* refactor xrt/passes/rebuild_job_pass

* a) remove oneflow.placement.current_scope; b) refactor PlacementScope

* refactor GlobalModePlacementScope

* Add CudnnConvAlgoCache

* refine

* Fused ImageDecoderRandomCropResize

* format

* fix cpu build

* Add comment

* dest=>dst

* Add comment

* Add ImageDecoderRandomCropResizeOp::InferParallelSignature (#3646)

* Add ImageDecoderRandomCropResizeOp::InferParallelSignature

* Add Scope::scope_proto

* refine

* bn_add_relu use bit mask

* refine

* refine

* MaskAndScaleHalf2

* fix

* fix

* refine

* format

* rename OF_BARRIAER

* add eager_2node_test.py

* InitLazyGlobalSession if eager execution not enabled

* remove Global<LbiDiffWatcherInfo>

* add TODO() comments for OF_SESSION_BARRIER under directory core/comm_network/

* remove eager_2node_test.py

* add classes for different test base class

* add export

* rm old main files

* fix

* fix class name

* check in changes of tests

* fix else

* fix bugs

* fix order

* better naming

* update ci script

* fix import unittest

* add --failfast

* filter by card num

* fix typo

* update ci script

* dont set CUDA_VISIBLE_DEVICES

* use skip unless

* add OneGpuTestCase

* fix typo

* shorter reason

* fix class

* fix typo

* skip test_train_consistent for now

* add tearDown

* fix setup

* rm todo

* skip TestImageResize

* fix for pure cpu

* skip TestImageTargetResize

* fix typo

* rename

* mv demo

* rm readme

* rename

* rename cases

* add --verbose

* rename

* address reviews

* fix export

* minor fix

* fix export

* fix skips

* more renames

* rename

* rename class

* rm gpu

* rename num in scripts

* skip resize img

* fix sys env

* fix dynamic reshape

* fix dynamic reshape

* fix skip msg

* rm num_gpus

* address review

* address reviews

* rm class

* update decorator

* fix inden

* fix typo

* replace class with decorator

* fix

* refine

* fix decorator

* mv code

* add more decorators

* add ONEFLOW_TEST_DEVICE_NUM=1

* fix typo

* fix typo

* fix name

* fix skip msg

* use decorators in scripts

* larger rol for xla ln

Co-authored-by: tsai <caishenghang@oneflow.org>
Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* model update fuse scalar_mul_by_tensor

* add test case

* testcase

* fix

* refine

* fix

* refine

* BiasAdd Row Col Half2

* refine

* fused layer norm

* fix

* rm debug info

* refine

* fix

* testcase

* refine

* testcase

* refine

Co-authored-by: guoran <guoran@oneflow.org>
Co-authored-by: Juncheng <liujuncheng1022@gmail.com>

* layer_norm_param_grad

* refine

* use tmp_buffer in fp16, because gpu_atomic_add is slow

* testcase

* refine

* refine

* refine

* fp16 test case

* fix

* rm trainable

* fix

Co-authored-by: guoran <guoran@oneflow.org>
Co-authored-by: Juncheng <liujuncheng1022@gmail.com>

* support oneflow_api_registry

* fix code format

* Expand folder range

* optimize of_api_registry

* Update of_api_registry.h

add a blank line between constructor  and member function

* fix code format

* optimize oneflow_pybind_api

* fix code foemat

* fix oneflow.cmake

* add a blank line

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

Co-authored-by: Juncheng <liujuncheng1022@gmail.com>

* update lib name in link flags

* get lib path from imp module

* reformat and replace % with .format

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* Networker interface

* global epoll comm net

* half implement of Networker

* implement of Networke::Send

* Implement of Networker::Recieve

* Implement of Networker::HandlerRecieveSendMsgFromSrcMachine

* Implement of Networker::HandlerRecieveAckMsgFromDstMachine

* refine iterator in Networker

* add networker test exe

* add log and blocking count

* OF_BARRIER for networker test

* add log for debug

* add more log

* fix bug and add check

* fix bug of wrong delete global in callback

* fix bug of double free

* exchange Netwoker deconstructor

* moving BlockingCount in networker_test_main

* send_before_recv & recv_before_send ; fix lock bug of status access

* Networker -> Transport

* add TODO(chengcheng) and rename interface of Transport handler

* add more test

* fix bug and refine test code

* fix compile err for new change

* Test for correctness

* Test throughput like ibverbs read bandwith

* Fix BUG: All stat change need be set in the block protected by lock

* test 23 data up to 8388608

* note for debug

* OF_BARRIER_ALL

* ctrl_client clear

* not commnet

* test global

* CtrlServer: Clear cq before shutdown

* Fix cq shutdown on loop thread

* add log

* try shutdown grpc server before cq shutdown

* move grpc server to loop thread

* fix rpc loop return condition

* revert change in ctrl server

* fix bug of delete global in runtime

* transport support 1. Receive size > Send size; 2. Local Send/Recv

* add test for Send size < Recv size and local transport

* fix bug when Send size < Recv size

* refine code for review

* Fix bug of Transport UnRegisterMemory

* add Transport user doc

* refine code for review. move memcopy from block protected by mutex

* Add model update user ops

* fix

* extract MakePredicatorIsSafeToDelete/IsUserOpWithTypeName

* Remove useless code

* Dev indexed slices model update user ops (#3561)

* indexed_slices user ops and rewrite pass

* fix lazy adam

* fix

* add testcase

* fix sbp

* rename IndexedSlicesUpdateOpKernelState

* address review

* rename state

* check model_diff dtype

* use TmpBufferManager

* refine

* refine

Co-authored-by: guoran <guoran@oneflow.org>

* Dev adam xla and rm sys op (#3584)

* rm model_update sys op

* mv indexed slice op kernel to model_update op

* adam optimizer

* revert indexed_slices_reduce_sum op

* fix

* refine

* rm mask in lazy_adam testcase

* add cast

Co-authored-by: guoran <guoran@oneflow.org>

* reuse adam optm

* refine

* enable_fuse_model_update_ops

* fix

* pure CPU

* model update ops SetAreaId

* Node is not safe to delete if it has more than one consumer.

* refine

* fix

Co-authored-by: guo ran <360112263@qq.com>
Co-authored-by: guoran <guoran@oneflow.org>

* add convert_url_to_oss_https_url and DCN flag

* add third_party_mirror flag in ci

* fix the reviewer's suggestions

* recover the include order && skip oss2 if not exists

* fix find oss module

* of_format

* fix the reviewer's suggestions and add some md5 checks

* check for non-empty third-party-mirro variable

* add flag introduction in readme

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>