Dev flow.utils.data part2 (#5500)
* refine and add test case * support ellipsis type slice * refine * refine * support slice assign ellipsis type * refine * register fn to localtensor * basic implemetation of dataloader * format * refine as comments * fix comments * remove useless code * add test case into unnitest * refine ascomments * refine as comments * register uint8 dtypeto support dataloader * add cifer mnist dataset support * format * refine * refine * use F.interpolation * export dataloader dataset * export more api * refine * refine * add oneflow_test_cache_dir * refien * refine * refine * refine * update datasets links * refine * fix comment * unset export * auto format by CI * add requirement * always reisntall test deps * update requirement * add pillow as oneflow dependency * auto format by CI * add sessiong context debug str * remove debug str * remove experimental * fix export * update * fixbug * auto format by CI * refine * refine Co-authored-by:oneflow-ci-bot <69100618+oneflow-ci-bot@users.noreply.github.com> Co-authored-by:
tsai <jackalcooper@gmail.com> Co-authored-by:
oneflow-ci-bot <ci-bot@oneflow.org>
Showing
- .github/workflows/test.yml 7 additions, 1 deletion.github/workflows/test.yml
- dev-requirements.txt 1 addition, 0 deletionsdev-requirements.txt
- python/oneflow/test/dataloader/test_cifar_dataset.py 132 additions, 0 deletionspython/oneflow/test/dataloader/test_cifar_dataset.py
- python/oneflow/test/dataloader/test_fashion_mnist_dataset.py 173 additions, 0 deletionspython/oneflow/test/dataloader/test_fashion_mnist_dataset.py
- python/oneflow/test/dataloader/test_lenet.py 177 additions, 0 deletionspython/oneflow/test/dataloader/test_lenet.py
- python/oneflow/test/dataloader/test_numpy_dataset.py 3 additions, 7 deletionspython/oneflow/test/dataloader/test_numpy_dataset.py
- python/oneflow/test/dataloader/test_tensor_dataset.py 3 additions, 7 deletionspython/oneflow/test/dataloader/test_tensor_dataset.py
- python/oneflow/utils/data/__init__.py 1 addition, 0 deletionspython/oneflow/utils/data/__init__.py
- python/oneflow/utils/data/_utils/__init__.py 20 additions, 4 deletionspython/oneflow/utils/data/_utils/__init__.py
- python/oneflow/utils/data/_utils/collate.py 32 additions, 16 deletionspython/oneflow/utils/data/_utils/collate.py
- python/oneflow/utils/data/dataloader.py 154 additions, 40 deletionspython/oneflow/utils/data/dataloader.py
- python/oneflow/utils/data/dataset.py 62 additions, 24 deletionspython/oneflow/utils/data/dataset.py
- python/oneflow/utils/data/decorator.py 41 additions, 26 deletionspython/oneflow/utils/data/decorator.py
- python/oneflow/utils/data/sampler.py 79 additions, 30 deletionspython/oneflow/utils/data/sampler.py
- python/oneflow/utils/vision/__init__.py 17 additions, 0 deletionspython/oneflow/utils/vision/__init__.py
- python/oneflow/utils/vision/datasets/__init__.py 19 additions, 0 deletionspython/oneflow/utils/vision/datasets/__init__.py
- python/oneflow/utils/vision/datasets/cifar.py 192 additions, 0 deletionspython/oneflow/utils/vision/datasets/cifar.py
- python/oneflow/utils/vision/datasets/mnist.py 308 additions, 0 deletionspython/oneflow/utils/vision/datasets/mnist.py
- python/oneflow/utils/vision/datasets/utils.py 552 additions, 0 deletionspython/oneflow/utils/vision/datasets/utils.py
- python/oneflow/utils/vision/datasets/vision.py 113 additions, 0 deletionspython/oneflow/utils/vision/datasets/vision.py
Please register or sign in to comment