* implementation of constantpad-3d op

* format

* refine

* rename files

* refine

* refine test case

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* restruct upsample module

* auto format by CI

* change docs

* fix bug

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* del default env init

* refine

* fix bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* prevent endless loop

* add timeout 2 hours to prevent exhaust machine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* LazyInterpreter for FetchOutputOpExpr and set op parallel_distribution

* Add note

* refine test scripts

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* replace ForeignJobInstance using JobInstance

* LazyJobStreamType

* NNGraphIf

* NNGraph -> NNGraphIf

* fix compile bugs

* add unit tests for instruction RunLazyJob

* GetInputBufferName/GetOutputBufferName

* multi-client push pull

* refactor wait_and_send_ids_kernel to support multi-client mode

* support multi-client mode return_op

Co-authored-by: liufengwei <2472937968@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix single client env reinit

* addnote

* Multi-Client LogicalRun degenerate to PhysicalRun

* Add Logical->Physical in Multi-Client for IdGenerator and InstructionsBuilder

* multi client test

Signed-off-by: daquexian <daquexian566@gmail.com>

* auto format by CI

* fix wrong std::shared_ptr

Signed-off-by: daquexian <daquexian566@gmail.com>

* run module api test by test_multi_client/generic_test.sh

Signed-off-by: daquexian <daquexian566@gmail.com>

* fix test script

Signed-off-by: daquexian <daquexian566@gmail.com>

* use oneflow.distributed.launch -m unittest discover instead of bash for loop to speed up the test

Signed-off-by: daquexian <daquexian566@gmail.com>

* update ci/test_multi_client/generic_test.sh

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: chengtbf <472491134@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* modified SetInputArgModifyFn

* Delete the CHECK changes in the assign_op.cpp file

* Format

* Modified the OutputArgModifyFn interface

* add return

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* OutputArgModifier return maybe part_1

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* input_arg_modifier return maybe

* gen_bw_fn return maybe

* bw_gen_fn return maybe

* registry_callback_fn return maybe

* fix bug after merge master

* fix bug

Co-authored-by: aishangjj <702572275@qq.com>

* edit cmake cache and protobuf

* edit more cmakes

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* modified SetInputArgModifyFn

* Delete the CHECK changes in the assign_op.cpp file

* Format

* Modified the OutputArgModifyFn interface

* add return

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* OutputArgModifier return maybe part_1

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* input_arg_modifier return maybe

* gen_bw_fn return maybe

* bw_gen_fn return maybe

* fix bug: return Maybe without JUST

Co-authored-by: aishangjj <702572275@qq.com>

* fix TryClose

* fix addressing watch callback

* modified SetInputArgModifyFn

* Delete the CHECK changes in the assign_op.cpp file

* Format

* Modified the OutputArgModifyFn interface

* add return

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* OutputArgModifier return maybe part_1

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* input_arg_modifier return maybe

* gen_bw_fn return maybe

* add MakeGenBackwardOpConf because ofstatement-expression not allowed outside function resulting in JUST failed in lambda

* add maybe after merge master

* fix bug: JUST in lambda

Co-authored-by: aishangjj <702572275@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* mkdir compatible/single_client

* mv to compatible/singe_client/python

* mv to ./python/compatible/

* use oneflow.compatible.single_client

* back up

* backup

* refactor oneflow.xxx to oneflow.compatible.single_client.xxx

* mv to single_client

* refactor import oneflow to oneflow.compatible.single_client

* backup

* refine

* refine

* make single_clint ci test copy from correct path

* refine

* refine

* delete ioconf and split single-clint ci test

* auto format by CI

* delete default session

* fix method of erase sess id

* delete string multi client in test.yml

* fix spell error

* delete space

* refine

* mv single client python file to compatible_single_client_python

* refine

* fix ci error

* fix bug

* fix bug

* refine

* refine

* delete skip if in test/(modules/tensor)/* because of default eager env

* fix bug

* synchronization oneflow/compatible_single_client and oneflow/python

* fix distributed_run bug

* fix test_multi_process dead lock

* fix test_multi_process dead lock

* synchronization modify between oneflow/python and oneflow/compatible_single_client

* refine

* minor fix

* fix bug

* fix doctest.sh

* rm files

* remove files and add checks

* delete moduels

* Revert "delete moduels"

This reverts commit f95ab369f556b71fd0663b67c6dd51c9b6d5ee81.

Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: liufengwei <2472937968@qq.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: tsai <jackalcooper@gmail.com>

* feat(Module): support __repr__ function

* feat(Modules): add extra_repr for some modules

* fix(ConvModule): fix conv module extra_repr

* fix(ActivityModule): raise inplace error to warning

* fix(ModuleRepr): fix module print doctest bug

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix param

* fix maxpool1d params error

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add scale and new_size

* add and refine test_case

* fix testcase

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Refactor expand and tensor setitem functional apis.

* Raise IndexInception in tensor getitem.

* Bugfix

* Bugfix

* Fix conflicit with xrt

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* split vector-matrix norm

* fix_vector_norm

* fix_docstring

* fix_matrix_norm

* fix-default

* fix-doctest

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* remove automerge if cuda_new_interface failed

* raise failure on purpose

* refine

* refine

* Update test.yml

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* core: GlobalMultiClientEnv; refine EagerExecution

* move IsMultiClient to EnvDesc

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* prevent adding subdir in python/test

* refine

* add ci

* refactor

* refine

* refine

* add more timeout-minutes: 45

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* check in changes

* refine

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refactor job_pass by maybe_system

* remove useless files

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* startup: add sum testcase

* mean, max, min, sum

* refine test mul

* refine test mul case

* refine div, mul, sub test case

* refine masked_fill test case

Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample nearest bug

* fix upsample nearest bug (#5347)

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix upsample bilinear bug

* fix upsample bilinear bug

* recover code

* align with pytorch

* redesign upsample bilinear

* fix align corner bug

* align bilinear with pytorch

* fix conflict

* fix bug

* code reuse

* fix commets

* add upsample_linear_1d_user_op

* add upsample_linear_1d_user_op

* add linear1d interp cpu kernel

* add linear1d interp gpu kernel

* rename kernel name

* add upsample nearest_1d kernel

* add upsample nearest_1d cuda kernel

* add upsample nearest_1d op

* fix complie bug

* add bicubic interp cpu kernel forward

* add bicubic2d interp cpu kernel backward impl

* add bicubic2d gpu kernel impl

* fix bicubic kernel compile error

* add bicubic2d kernel register

* fix compile bug

* nearest3d interp cpu kernel

* add nearest3d gpu kernel

* add nearest3d kernel registry

* add nearest3d cpu kernel

* add nearest3d cpu kernel

* add nearest3d cpu kernel

* fix upsample nearest interp bug

* fix upsample nearest interp bug

* add trilinear3d cpu kernel forward

* add trilinear3d cpu kernel backward

* add trilinear gpu kernel impl

* add trilinear op registry

* init interpolate nn function

* add upsample functors

* add interolate module

* add interpolate test_case

* restruct upsample_nearest2d and upsample_bilinear2d

* add upsample nearest2d and bilinear2d op registry

* add upsample nearest2d and bilinear2d op registry

* add upsample kernel grad functor

* add upsample_nearest_2d backward

* add upsample_linear_1d backward

* add upsample_nearest_1d backward

* add upsample_bicubic_2d backward

* fix backward impl bug

* fix nearest1d backward bug

* try to fix backward segmentfault

* add data_format parammeter

* fix upsample kernel backward bug

* fix cuda kernel impl bug

* fix upsample nearest bug

* fix align_corner attr error

* align linear1d impl with pytorch

* fix trilinear register bug

* fix trilinear bug

* add trilinear aligncorners test

* add test for bicubic, still has bug

* reimpl bicubic2d kernel

* align cubic interpolate with pytorch

* add bicubic cuda kernel

* fix bug

* align api with pytorch

* add doctest for interpolate

* update interpolate docs

* fix comments

* fix check_of_return comment

* fix comment

* fix comments

* fix commnet

* fix comment

* fix comments

* fix comments

* fix comments

* fix comments

* fix comment

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: Luyang <flowingsun007@163.com>

* piprepo->pipindex

* refine

* save

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Xiaoyu Xu <xiaoyulink@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* modified SetInputArgModifyFn

* Delete the CHECK changes in the assign_op.cpp file

* Format

* Modified the OutputArgModifyFn interface

* add return

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* OutputArgModifier return maybe part_1

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* input_arg_modifier return maybe

* change lambda for JUST macro

* fix conflicts

Co-authored-by: aishangjj <702572275@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: Shenghang Tsai <jackalcooper@gmail.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* trigger ci test on graph

* add module scope

* Block.paramter or buffer return different value based on context

* add test on scope name prefix

* refine scope build

* refine scope build

* lambda get parameter to create right scope

* check para get

* fix vm:PhysicalRun

* lazy Tensor only create once

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add_timeout_for_wait_gpu_slot

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add gelu test

* add test for activation module

* add acos test

* add acosh test

* add tensor sigmoid test

* remove wrong acos

* add arccosh test

* remove acosh arccosh atan2 unittest

* remove arccosh acosh atan2 unittest to test_math_ops.py

Co-authored-by: Xiaoyu Zhang <35585791+BBuf@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add unique.cuh

* CHECK workspace_size

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* new static check based on clang-tidy

Signed-off-by: daquexian <daquexian566@gmail.com>

* set compiler as clang to generate right compile_commands.json, skip build

Signed-off-by: daquexian <daquexian566@gmail.com>

* include clang headers

Signed-off-by: daquexian <daquexian566@gmail.com>

* build third party

Signed-off-by: daquexian <daquexian566@gmail.com>

* install deps

Signed-off-by: daquexian <daquexian566@gmail.com>

* build third party by gcc, generate compile_commands.json by clang

Signed-off-by: daquexian <daquexian566@gmail.com>

* rm CMakeCache.txt

Signed-off-by: daquexian <daquexian566@gmail.com>

* run of_git_version

Signed-off-by: daquexian <daquexian566@gmail.com>

* download clang-tidy from oneflow-inc/llvm-project repo

Signed-off-by: daquexian <daquexian566@gmail.com>

* rename check name

Signed-off-by: daquexian <daquexian566@gmail.com>

* remove old clang plugin

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: o...

* printf tensor index in cpp

* revert printf debug information

* add isinstance tuple and int condition

* format code

* implement and register __iter__ method

* Change to catch exception from C++ and throw IndexError

* add indexError error message

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* auto_zero_padding -> auto_truncating in cpp

* auto_truncating for python interface

* fix element count

* fix

* fix bug auto_truncating<->dim1_varying_length

* auto_truncating -> truncate

* rm not in if

* of format

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>