Rewrite activation function (#5465)
* add activation
* rename swish to silu
* add selu
* add four activation op
* add softsign test
* add silu mish selu softsign
* Add softsign docs
* Add functional impl
* small fix for softsign backward
* remove flow.mish test
* add silu module test
* add selu test
* fix docs
* fix softsign docs
* fix format
* fix static cast
* merge master functional api yaml
* add torch style unittest
* Remove assert and add torch unittest
* add tensor def
* remove softsign test temporary
* add return maybe ok
* migrate nn ops to single_client dir
* migrate unittest
* remove lazy unittest
* add unittest
* fix to new directory
* Remove useless docs and single client test
* add doc
* fix docs
* add docs in oneflow and tensor namespace
* add torch autotest
* fix to new autotest
* remove outdated python code
* remove useless docs
* enlarge unittest tolerance
* Add static cast for const value
* skip softsign unittest
* skip tensor softsign
Co-authored-by:
oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com>
Showing
- docs/source/nn.rst 5 additions, 2 deletionsdocs/source/nn.rst
- docs/source/oneflow.rst 4 additions, 0 deletionsdocs/source/oneflow.rst
- docs/source/tensor.rst 3 additions, 0 deletionsdocs/source/tensor.rst
- oneflow/core/autograd/gradient_funcs/mish.cpp 54 additions, 0 deletionsoneflow/core/autograd/gradient_funcs/mish.cpp
- oneflow/core/autograd/gradient_funcs/selu.cpp 54 additions, 0 deletionsoneflow/core/autograd/gradient_funcs/selu.cpp
- oneflow/core/autograd/gradient_funcs/silu.cpp 54 additions, 0 deletionsoneflow/core/autograd/gradient_funcs/silu.cpp
- oneflow/core/autograd/gradient_funcs/softsign.cpp 54 additions, 0 deletionsoneflow/core/autograd/gradient_funcs/softsign.cpp
- oneflow/core/functional/functional_api.yaml 32 additions, 0 deletionsoneflow/core/functional/functional_api.yaml
- oneflow/core/functional/impl/activation_functor.cpp 58 additions, 0 deletionsoneflow/core/functional/impl/activation_functor.cpp
- oneflow/user/kernels/activation_kernels.cpp 5 additions, 1 deletiononeflow/user/kernels/activation_kernels.cpp
- oneflow/user/kernels/activation_kernels.cu 77 additions, 1 deletiononeflow/user/kernels/activation_kernels.cu
- oneflow/user/kernels/activation_kernels.h 110 additions, 0 deletionsoneflow/user/kernels/activation_kernels.h
- oneflow/user/ops/mish_op.cpp 92 additions, 0 deletionsoneflow/user/ops/mish_op.cpp
- oneflow/user/ops/selu_op.cpp 91 additions, 0 deletionsoneflow/user/ops/selu_op.cpp
- oneflow/user/ops/silu_op.cpp 91 additions, 0 deletionsoneflow/user/ops/silu_op.cpp
- oneflow/user/ops/softsign_op.cpp 91 additions, 0 deletionsoneflow/user/ops/softsign_op.cpp
- python/oneflow/__init__.py 6 additions, 0 deletionspython/oneflow/__init__.py
- python/oneflow/nn/__init__.py 3 additions, 0 deletionspython/oneflow/nn/__init__.py
- python/oneflow/nn/modules/activation.py 212 additions, 2 deletionspython/oneflow/nn/modules/activation.py
- python/oneflow/test/modules/test_activation.py 152 additions, 0 deletionspython/oneflow/test/modules/test_activation.py
Please register or sign in to comment