when send too many message whose number is bigger then qp_init_attr.cap.max_send_wr,
I use the msg_pendding_list to store the extra message and wait for some time to send these message of the msg_pendding_list

    when we send  too many message which are greater then qp_init_attr.cap.max_send_wr  it may have some proble
    we use the msg_pendding_list_ to store the extra messages when the message is greater than qp_init_attr.cap -2021-07-15-20:00

    when we send  too many message which are greater then qp_init_attr.cap.max_send_wr  it may have some proble
    we use the msg_pendding_list_ to store the extra messages when the message is greater than qp_init_attr.cap and clear todo  and clear extra line and license and todo

    when we send  too many message which are greater then qp_init_attr.cap.max_send_wr  it may have some proble
    we use the msg_pendding_list_ to store the extra messages when the message is greater than qp_init_attr.cap and clear todo  and clear extra line and license

    when we send  too many message which are greater then qp_init_attr.cap.max_send_wr  it may have some proble
    we use the msg_pendding_list_ to store the extra messages when the message is greater than qp_init_attr.cap and clear todo  and clear extra line

    when we send  too many message which are greater then qp_init_attr.cap.max_send_wr  it may have some proble
    we use the msg_pendding_list_ to store the extra messages when the message is greater than qp_init_attr.cap and clear todo

when we send  too many message which are greater then qp_init_attr.cap.max_send_wr  it may have some problem
we use the msg_pendding_list_ to store the extra messages when the message is greater than qp_init_attr.cap.max_send_wr

* half implement of InputOpExpr

* Half implement of run InputOpExpr

* FeedInputOp/FeedVariableOp/FetchOutputOp Expr/Conf; Impl of LazyInterpret::Apply(FeedInputOpExpr)

* FeedInputOpExpr runnable

* refine note in multi-client session

* refine code

* refine test_input_op_expr.py

* Using with oneflow._oneflow_internal.lazy_mode.gard(True)

* using unittest skip if not in eager

* add JUST

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Modified the OutputArgModifyFn interface

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* OutputArgModifier return maybe part_1

* maybe error stack from CheckAndConstructOp to OutputArgModifier callback function

* add JUST for hander in ForEachOperator

Co-authored-by: aishangjj <702572275@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add gather_nd and scatter_nd

* add testcase

* stable_api

* add docs and doctest

* add scatter_nd grad

* gradfunc

* delete some code comments

* add grad test

* add docs

* update words usage

* delete some import

* update

* update

* auto format by CI

Co-authored-by: Yao Chi <later@usopp.net>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

* graph api

* add graph dummy test

* add test

* add recursive module mode

* graph.build test pass

* add detail check on graph inner node

* support config and train

* add repr for debug

* test buffer

* test buffer add

* refine test

* add comment

* refine test

* refactor Node to Block

* add named_state

* refine Graph.named_state()

* add state_tensortuple

* graph._compile()

* add mc session 0

* nn.graph: state tuple to private var; add BlockType; add simple multi client session

* NNGraphIf

* rm old graph.cpp

* nn.graph: add cpp NNGraph; export and call NNGraph

* add comment

* nn.Graph: rm prototype MultiClientSession

* nn.Graph: rm prototype MultiClientSession test

* nn.Graph: add TODO

* nn.Graph: format for review

* nn.Graph: format

* nn.Graph: format

* nn.Graph: pass flake8 check

* job_build_ctx

* support lazy context

* format

* lazy mode

* format

* format

* lazy mode add test

* debug session

* init session and job_build_context

* rm temp code

* build default scope

* add default scope

* add scope proto for debug

* chech scope

* format

* refine MultiClientSession.resource

* address review

* lazy init scope stack in single-client, instantly init scope stack after MultiClientSession created in multi-client

* fix typo

* address review

* fix clear default session

* merger and test

Co-authored-by: Xinqi Li <lixinqi0703106@163.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: leaves-zwx <kunta0932@gmail.com>
Co-authored-by: cheng cheng <472491134@qq.com>

* move generator test into ops folder to accelerate tests

* ensure ONEFLOW_TEST_CPU_ONLY

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* refine

* refine

* refine

* add inplace testcase

* check inplace valid

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>

* refine

* refine inplace name

* reformat

* auto format by CI

Co-authored-by: Yinggang Wang <wyg19970408@gmail.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* address review

* lazy init scope stack in single-client, instantly init scope stack after MultiClientSession created in multi-client

* fix typo

* address review

* fix clear default session

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* add asinh and arcsinh test

* add asinh and arcsinh test

* add sinh test

* add atan2 module test

* add softplus test, softplus module has bug

* comment softplus

* add elu module autotest

* refine

* auto format by CI

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* init nn.image.flip

* clean code

* add unit test.

* revert python.cmake

* auto format by CI

* resolve comments

* add functor

* fix image_flip

* fix testcase

* fix warning:  Unexpected indentation

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: simonJJJ <821898965@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* init of op diag

* init of op diag

* modify  op diag

* merge testcase

* delete no use tensor diag

* amend diag api docs

* resolve conficts

* resolve confict

* resolve confict

* resolve confict

* auto format by CI

* add tensor.diag

* auto format by CI

* amend API description

* add non_square test

* amend diag API description

* amend diag API description

* compact test code

* compact code

* auto format by CI

* amend API description

* amend API description

* motify functional API

* motify test_diag

* amend diag functional API

* amend diag docstring

* auto format by CI

* amend code standards

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* fix(Tensor): fix Tensor.grad setter bug

* refine error message

* fix(Tensor): fix bug

* add grad = grad * constant test

* update tensor.py to pr head

* format code

* fix(Parameter): fix Parameter construction bug

* fix bug

Co-authored-by: tea321000 <834914152@qq.com>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* add argmax test

* fix ci error

* fix docstring warning

* fix tensor greater and less bug

* fix conflict

* add test_flow_xxx_against_pytorch func

* add autotest part1

* add asin and arcsin module test

* auto format by CI

* try to fix export oneflow.python.test ci error

* delete __init__.py

* delete autotest tensor api

* fix ci export test_tensor_against_pytorch bug

* recover

* add tensor test

* auto format by CI

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Optimize tensor getitem.

* Fix merge

* Refine code style

* Fix typo and copy input if it's getitem is identity.

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

Signed-off-by: daquexian <daquexian566@gmail.com>

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* Add in_top_k function

* auto format by CI

* fix bug

* Update oneflow/python/ops/math_ops.py

Co-authored-by: Yao Chi <later@usopp.net>

Co-authored-by: oneflow-ci-bot <ci-bot@oneflow.org>
Co-authored-by: Yao Chi <later@usopp.net>
Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>

* startup: use python, not C extension

* rst changes test

* refine

* add F.rst

* add bernoulli add docstr example

* add doctest for functional method

* del docstr utils immediately in init.py

* refine test case

* use sin, cos instead of add, mul

* refine

Co-authored-by: oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>