* refactor message OperatorConf, change device_type to device_tag

* subsititute HobDeviceType with HobDeviceTag in user_op kernel registration

* remove c_api_util.DeviceType4DeviceTag

* fix error when buil with cuda off

* fix can not use CHECK_JUST macro in another macro

* rm trainable in if

* add ln cpu and refactor bn

* refactor bn

* do affine in LN nor BN

* fix typo

* fix typo

* fix typo

* add NotImplementedError

* add nn test cpu

* refactor nn bn

* fix test case

* update log

* rename axes

* check in moments

* rm print

* fix range

* add log

* fix typo

* add log

* fix bugs

* Dev bn training cpu (#3445)

* enable trainable_without_training for cpu

* add training

Co-authored-by: tsai <caishenghang@oneflow.org>

* run cpu gan

* Revert "run cpu gan"

This reverts commit e98deecf06f5ad3e50ad42993ba5b7149d4e1642.

* fix order

* optional normalized

* larger tol when cpu

Co-authored-by: tsai <caishenghang@oneflow.org>
Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>

* add oneflow.typing.Collection

* add test script of oneflow.typing.Collection

* fix code style

* simplify code

* remove notes

* fix code style

* remove code useless

* simplified code

* Decoupling container transform precess

* remove brackets

* rename container to bundle

* fix code style

* add docstring

* fix docstring

* add docstring

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* Add OF_CUDA_CHECK/OF_CUDNN_CHECK/OF_CUBLAS_CHECK/OF_CURAND_CHECK

* OF_NCCL_CHECK

* use OF_CUDA_CHECK

* use OF_CUDNN_CHECK

* use OF_CURAND_CHECK

* use OF_NCCL_CHECK

* use OF_CUBLAS_CHECK

* use OF_CUDNN_CHECK

* use OF_CUBLAS_CHECK

* OF_CUDA_CHECK

* OF_CUDA_CHECK

* image.random_crop

* refine code for review

* refine code for review

* ofrecord_image_classification_reader

* refine

* use num_decode_threads_per_machine

* Specify image_feature_name and label_feature_name

* Extract LoadWorker and DecodeWorker

* Remove image_feature_name/label_feature_name default value

* underlaying_=>base

* set decode_buffer_size_per_thread to 32

* rm blank

* add more log

* support bn cpu inference

Co-authored-by: tsai <caishenghang@oneflow.org>
Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* cmake dont panic when build cuda

* naive changes

* fix cudaMemcpyKind

* fix acc actor

* fix actor

* fix gdb

* fix vm

* fix work type

* fix cuda type

* fix cuda type

* fix collective backend

* fix global scope

* amp

* rm PackKernelUtil gpu

* fix log

* fix rand

* fix sync size

* fix allocator

* fix vm

* fix kernel

* fix kernel

* fix kernels

* fix kernel

* fix softmax

* fix kernels

* fix reshape kernels

* add workaround

* try fix symbol not found

* fix vm

* fix vm

* fix jpeg

* fix broadcast gpu

* fix broadcast like

* fix transpose

* fix matmul

* fix CopyElemOnGpu

* fix sigmoid

* fix sigmoid and softmax

* fix relu

* fix sparse cross entropy

* fix kernels

* fix tanh

* fix same padding

* fix softmax

* fix undefined symbol: gzgets

* fix CopyField

* fix scalar add

* fix CopyNDGpuImpl

* copier

* fix slice boxing

* fix mem copier

* fix zero l...

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

1. add link
2. add item in table of contents

Co-authored-by: Yao Chi <yaochi@oneflow.org>

* add conv1d conv3d

* add transpose in conv3d NDHWC

* fix

* fix

* NCHW->NCDHW

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* Add tensor_buffer_to_tensor and tensor_to_tensor_buffer

* const auto& => const auto

* move comp note up

* add sorry

* quick fix readme

* add readme

* CheckPoint.save in eager

* fix minor spell error

* add test

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* use tf protobuf headers

* add readme

* rm patch

* add inc file

* fix rare grpc build bug

* use different headers

* fix file path

* fix cublas conflict with tf

* use cublas dlib for xla

Co-authored-by: tsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>

Co-authored-by: oneflow-bot <69100618+oneflow-bot@users.noreply.github.com>

* Update const std::string param type

* Fix lambda return type

Co-authored-by: tsai <caishenghang@oneflow.org>

This reverts commit e338a5f3.

* Final Cut (#3354)

* bump version

* rm pip install

Co-authored-by: tsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>

* remove windows directory (#3358)

* fix building error and warning in gcc10

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: tsai <caishenghang@1f-dev.kbaeegfb1x0ubnoznzequyxzve.bx.internal.cloudapp.net>
Co-authored-by: OuYang Yu <xuanjiuye@gmail.com>
Co-authored-by: Li Xinqi <lixinqi2010@gmail.com>