Skip to content
Snippets Groups Projects
Commit fe8d2e31 authored by Madeline's avatar Madeline
Browse files

0824

parent 67ff439a
Branches
No related tags found
No related merge requests found
Show whitespace changes
Inline Side-by-side
official/cv/shufflenetv2/README.md
+ 57
− 104
View file @ fe8d2e31
# Contents # Contents
- [Contents](#contents)
- [ShuffleNetV2 Description](#shufflenetv2-description) - [ShuffleNetV2 Description](#shufflenetv2-description)
- [Model architecture](#model-architecture) - [Model Architecture](#model-architecture)
- [Dataset](#dataset) - [Dataset](#dataset)
- [Environment Requirements](#environment-requirements) - [Environment Requirements](#environment-requirements)
- [Script description](#script-description) - [Script Description](#script-description)
- [Script and sample code](#script-and-sample-code) - [Script and Sample Code](#script-and-sample-code)
- [Training process](#training-process) - [Training Process](#training-process)
- [Usage](#usage) - [Evaluation Process](#evaluation-process)
- [Launch](#launch) - [Evaluation](#evaluation)
- [Result](#result)
- [Eval process](#eval-process)
- [Usage](#usage-1)
- [Launch](#launch-1)
- [Result](#result-1)
- [Inference Process](#inference-process) - [Inference Process](#inference-process)
- [Export MindIR](#export-mindir) - [Export MindIR](#export-mindir)
- [Infer on Ascend310](#infer-on-ascend310) - [Infer on Ascend310](#infer-on-ascend310)
- [result](#result-2) - [result](#result-2)
- [Infer with ONNX](#infer-with-onnx) - [Model Description](#model-description)
- [result](#result-3)
- [Model description](#model-description)
- [Performance](#performance) - [Performance](#performance)
- [Training Performance](#training-performance) - [Training Performance](#evaluation-performance)
- [Inference Performance](#inference-performance) - [Inference Performance](#evaluation-performance)
- [ModelZoo Homepage](#modelzoo-homepage) - [ModelZoo Homepage](#modelzoo-homepage)
# [ShuffleNetV2 Description](#contents) # [ShuffleNetV2 Description](#contents)
...@@ -48,15 +41,23 @@ Dataset used: [imagenet](http://www.image-net.org/) ...@@ -48,15 +41,23 @@ Dataset used: [imagenet](http://www.image-net.org/)
- Data format: RGB images. - Data format: RGB images.
- Note: Data will be processed in src/dataset.py - Note: Data will be processed in src/dataset.py
Dataset for transfermation:[flower_photos](https://storage.googleapis.com/download.tensorflow.org/example_images/flower_photos.tgz)
- Dataset size: 221MB, 3670 colorful images in 5 classes
- Train: 177MB, 2934images
- Test: 44MB, 736 images
- Data format: RGB images.
- Note: Data will be processed in src/dataset.py
# [Environment Requirements](#contents) # [Environment Requirements](#contents)
- Hardware(Ascend/GPU) - Hardware(Ascend/GPU/CPU)
- Prepare hardware environment with Ascend or GPU processor. - Prepare hardware environment with Ascend, GPU or CPU processor.
- Framework - Framework
- [MindSpore](https://www.mindspore.cn/install/en) - [MindSpore](https://www.mindspore.cn/install/en)
- For more information, please check the resources below: - For more information, please check the resources below:
- [MindSpore Tutorials](https://www.mindspore.cn/tutorials/en/master/index.html) - [MindSpore Tutorials](https://www.mindspore.cn/tutorials/en/master/index.html)
- [MindSpore Python API](https://www.mindspore.cn/docs/en/master/index.html) - [MindSpore Python API](https://www.mindspore.cn/docs/api/en/master/index.html)
# [Script description](#contents) # [Script description](#contents)
...@@ -71,7 +72,6 @@ Dataset used: [imagenet](http://www.image-net.org/) ...@@ -71,7 +72,6 @@ Dataset used: [imagenet](http://www.image-net.org/)
+--run_eval_for_ascend.sh # shell script for Ascend evaluation +--run_eval_for_ascend.sh # shell script for Ascend evaluation
+--run_eval_for_gpu.sh # shell script for GPU evaluation +--run_eval_for_gpu.sh # shell script for GPU evaluation
+--run_standalone_train_for_gpu.sh # shell script for standalone GPU training +--run_standalone_train_for_gpu.sh # shell script for standalone GPU training
+--run_onnx.sh # shell script for onnx inference
+-- src +-- src
+--config.py # parameter configuration +--config.py # parameter configuration
+--CrossEntropySmooth.py # loss function for GPU training +--CrossEntropySmooth.py # loss function for GPU training
...@@ -79,10 +79,11 @@ Dataset used: [imagenet](http://www.image-net.org/) ...@@ -79,10 +79,11 @@ Dataset used: [imagenet](http://www.image-net.org/)
+--loss.py # loss function for network +--loss.py # loss function for network
+--lr_generator.py # learning rate config +--lr_generator.py # learning rate config
+--shufflenetv2.py # ShuffleNetV2 model network +--shufflenetv2.py # ShuffleNetV2 model network
+-- cpu_transfer.py # transfer script
+-- dataset_split.py # splitting dataset for transfermation script
+-- quick_start.py # quick_start script
+-- train.py # training script +-- train.py # training script
+-- eval.py # evaluation script +-- eval.py # evaluation script
+-- infer_shufflenetv2_onnx.py # onnx inference script
``` ```
## [Training process](#contents) ## [Training process](#contents)
...@@ -101,7 +102,7 @@ You can start training using python or shell scripts. The usage of shell scripts ...@@ -101,7 +102,7 @@ You can start training using python or shell scripts. The usage of shell scripts
# training example # training example
python: python:
GPU: mpirun --allow-run-as-root -n 8 --output-filename log_output --merge-stderr-to-stdout python train.py --is_distributed=True --platform='GPU' --dataset_path='~/imagenet' > train.log 2>&1 & GPU: mpirun --allow-run-as-root -n 8 --output-filename log_output --merge-stderr-to-stdout python train.py --is_distributed=True --platform='GPU' --dataset_path='~/imagenet' > train.log 2>&1 &
CPU: python cpu_transfer.py --checkpoint_input_path ./input_ckpt/shufflenetv2_top1acc69.63_top5acc88.72.ckpt --checkpoint_save_path ./save_ckpt/Graph_mode --train_dataset ./data/flower_photos_split/train --use_pynative_mode False --platform CPU
shell: shell:
GPU: cd scripts & sh run_distribute_train_for_gpu.sh 8 0,1,2,3,4,5,6,7 ~/imagenet GPU: cd scripts & sh run_distribute_train_for_gpu.sh 8 0,1,2,3,4,5,6,7 ~/imagenet
``` ```
...@@ -126,7 +127,7 @@ You can start evaluation using python or shell scripts. The usage of shell scrip ...@@ -126,7 +127,7 @@ You can start evaluation using python or shell scripts. The usage of shell scrip
python: python:
Ascend: python eval.py --platform='Ascend' --dataset_path='~/imagenet' --checkpoint='checkpoint_file' > eval.log 2>&1 & Ascend: python eval.py --platform='Ascend' --dataset_path='~/imagenet' --checkpoint='checkpoint_file' > eval.log 2>&1 &
GPU: CUDA_VISIBLE_DEVICES=0 python eval.py --platform='GPU' --dataset_path='~/imagenet/val/' --checkpoint='checkpoint_file'> eval.log 2>&1 & GPU: CUDA_VISIBLE_DEVICES=0 python eval.py --platform='GPU' --dataset_path='~/imagenet/val/' --checkpoint='checkpoint_file'> eval.log 2>&1 &
CPU: python eval.py --dataset_path ./data/flower_photos_split/eval --checkpoint_dir ./save_ckpt/Graph_mode --platform CPU --checkpoint ./save_ckpt/Graph_mode/shufflenetv2_1-154_18.ckpt --enable_checkpoint_dir True --use_pynative_mode False
shell: shell:
Ascend: cd scripts & sh run_eval_for_ascend.sh '~/imagenet' 'checkpoint_file' Ascend: cd scripts & sh run_eval_for_ascend.sh '~/imagenet' 'checkpoint_file'
GPU: cd scripts & sh run_eval_for_gpu.sh '~/imagenet' 'checkpoint_file' GPU: cd scripts & sh run_eval_for_gpu.sh '~/imagenet' 'checkpoint_file'
...@@ -166,72 +167,24 @@ bash run_infer_310.sh [MINDIR_PATH] [DATASET_NAME] [DATASET_PATH] [NEED_PREPROCE ...@@ -166,72 +167,24 @@ bash run_infer_310.sh [MINDIR_PATH] [DATASET_NAME] [DATASET_PATH] [NEED_PREPROCE
- `NEED_PREPROCESS` can be y or n. - `NEED_PREPROCESS` can be y or n.
- `DEVICE_ID` is optional, default value is 0. - `DEVICE_ID` is optional, default value is 0.
#### result ### result
Inference result is saved in current path, you can find result like this in acc.log file. Inference result is saved in current path, you can find result like this in acc.log file.
Top1 acc: 0.69608 Top1 acc: 0.69608
Top5 acc: 0.88726 Top5 acc: 0.88726
### Infer with ONNX ### Infer on CPU After Transfermation
Before Inferring, you need to export the ONNX model.
- Download the [ckpt] of the model from the [mindspore official website](https://mindspore.cn/resources/hub/details?MindSpore/1.8/shufflenetv2_imagenet2012 ). ```Python
- Find the following code of `trian.py` # CPU inference
python eval.py --dataset_path [eval dataset] --checkpoint_dir [ckpt dir for eavl ] --platform [CPU] --checkpoint [ckpt path for eval] --enable_checkpoint_dir [True/False]--use_pynative_mode [True/False]
```python
context.set_context(mode=context.GRAPH_MODE, device_target=config.device_target, save_graphs=False)
```
- Change to
```python
context.set_context(mode=context.GRAPH_MODE, device_target=config.device_target, save_graphs=True)
```
The command to export the ONNX model is as follows
```shell
python export.py --device_target "GPU" --ckpt_file [ckpt_path] --file_format "ONNX" --file_name shufflenetv2.onnx
``` ```
- Infer with Python scripts ### result
```shell Inference result is saved in current path, you can find result like this in acc.log file.
python infer_shufflenetv2_onnx.py --onnx_path [onnx_path] --onnx_dataset_path [onnx_dataset_path] --platform [platform] --device_id [device_id] Top1 acc: 0.86
``` Top5 acc: 1
- Infer with shell scripts
```shell
bash ./scripts/run_onnx.sh [ONNX_PATH] [DATASET_PATH] [PLATFORM] [DEVICE_ID]
```
Infer results are output in `infer_onnx.log`
- Note 1: the above scripts need to be run in the shufflenetv2 directory.
- Note 2: the validation data set needs to be preprocessed in the form of folder. For example,
```reStructuredText
imagenet
-- val
-- n01985128
-- n02110063
-- n03041632
-- ...
```
- Note 3: `PLATFORM` only supports CPU and GPU.
- Note 4: `DEVICE_ID` is optional, default value is 0.
#### result
The reasoning results are output on the command line, and the results are as follows
```log
ACC_TOP1 = 0.69472
ACC_TOP5 = 0.88652
```
# [Model description](#contents) # [Model description](#contents)
...@@ -239,30 +192,30 @@ ACC_TOP5 = 0.88652 ...@@ -239,30 +192,30 @@ ACC_TOP5 = 0.88652
### Training Performance ### Training Performance
| Parameters | Ascend 910 | GPU | | Parameters | Ascend 910 | GPU |CPU(Transfer) |
| -------------------------- | ----------------------------- |-------------------------------| | -------------------------- | ----------------------------- |-------------------------------|-------------------------------|
| Model Version | ShuffleNetV2 | ShuffleNetV2 | | Model Version | ShuffleNetV2 | ShuffleNetV2 | ShuffleNetV2 |
| Resource | Ascend 910 | NV SMX2 V100-32G | | Resource | Ascend 910 | NV SMX2 V100-32G |Intel(R)Core(TM) i5-7200U CPU@2.50GHz(4 CPUs) |
| uploaded Date | 10/09/2021 (month/day/year) | 09/24/2020 (month/day/year) | | uploaded Date | 10/09/2021 (month/day/year) | 09/24/2020 (month/day/year) | 08/30/2022 (month/day/year) |
| MindSpore Version | 1.3.0 | 1.0.0 | | MindSpore Version | 1.3.0 | 1.0.0 | 1.8 |
| Dataset | ImageNet | ImageNet | | Dataset | ImageNet | ImageNet |Flower_photos |
| Training Parameters | src/config.py | src/config.py | | Training Parameters | src/config.py | src/config.py | src/config.py |
| Optimizer | Momentum | Momentum | | Optimizer | Momentum | Momentum | Momentum |
| Loss Function | SoftmaxCrossEntropyWithLogits | CrossEntropySmooth | | Loss Function | SoftmaxCrossEntropyWithLogits | CrossEntropySmooth | CrossEntropySmooth |
| Accuracy | 69.59%(TOP1) | 69.4%(TOP1) | | Accuracy | 69.59%(TOP1) | 69.4%(TOP1) | 86.4%(TOP1) |
| Total time | 11.6 h 8ps | 49 h 8ps | | Total time | 11.6 h 8ps | 49 h 8ps |15h18m8.6s |
### Inference Performance ### Inference Performance
| Parameters | Ascend 910 | GPU | | Parameters | Ascend 910 | GPU | CPU(Transfer) |
| -------------------------- | ----------------------------- |-------------------------------| | -------------------------- | ----------------------------- |-------------------------------|-------------------------------|
| Resource | Ascend 910 | NV SMX2 V100-32G | | Resource | Ascend 910 | NV SMX2 V100-32G |Intel(R)Core(TM) i5-7200U CPU@2.50GHz(4 CPUs) |
| uploaded Date | 10/09/2021 (month/day/year) | 09/24/2020 (month/day/year) | | uploaded Date | 10/09/2021 (month/day/year) | 09/24/2020 (month/day/year) | 08/30/2022 (month/day/year) |
| MindSpore Version | 1.3.0 | 1.0.0 | | MindSpore Version | 1.3.0 | 1.0.0 | 1.8.0 |
| Dataset | ImageNet | ImageNet | | Dataset | ImageNet | ImageNet |Flower_photos |
| batch_size | 125 | 128 | | batch_size | 125 | 128 |128 |
| outputs | probability | probability | | outputs | probability | probability | probability |
| Accuracy | acc=69.59%(TOP1) | acc=69.4%(TOP1) | | Accuracy | acc=69.59%(TOP1) | acc=69.4%(TOP1) | acc=86.4%(TOP1) |
# [ModelZoo Homepage](#contents) # [ModelZoo Homepage](#contents)
......
official/cv/shufflenetv2/cpu_transfer.py 0 → 100644
+ 133
− 0
View file @ fe8d2e31
# Copyright 2022 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
"""train ShuffleNetV2"""
import argparse
import ast
import time
from mindspore import context
from mindspore import Tensor
from mindspore.common import set_seed
from mindspore.nn.optim.momentum import Momentum
from mindspore.train.model import Model
from mindspore.train.callback import ModelCheckpoint, CheckpointConfig, TimeMonitor, LossMonitor
from mindspore.train.serialization import load_checkpoint, load_param_into_net
from mindspore.train.loss_scale_manager import FixedLossScaleManager
from mindspore import nn
from src.lr_generator import get_lr_basic
from src.shufflenetv2 import ShuffleNetV2
from src.dataset import create_dataset
from src.CrossEntropySmooth import CrossEntropySmooth
from src.config import config_cpu
set_seed(1)
def modelarts_pre_process():
pass
def filter_checkpoint_parameter_by_list(origin_dict, param_filter):
for key in list(origin_dict.keys()):
for name in param_filter:
if name in key:
print("Delete parameter from checkpoint: ", key)
del origin_dict[key]
break
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='image classification transformation')
parser.add_argument('--checkpoint_input_path', type=str, default='',
help='the checkpoint of ShuffleNetV2 (Default: None)')
parser.add_argument('--checkpoint_save_path', type=str, default='',
help='the directory that saves the ckpt')
parser.add_argument('--train_dataset', type=str, default='',
help='the training data for transformation')
parser.add_argument('--use_pynative_mode', type=ast.literal_eval, default=False,
help='whether to use pynative mode for device(Default: False)')
parser.add_argument('--platform', type=str, default='CPU', choices=('Ascend', 'GPU', 'CPU'),
help='run platform(Default:Ascend)')
args_opt = parser.parse_args()
if args_opt.use_pynative_mode:
context.set_context(mode=context.PYNATIVE_MODE, device_target=args_opt.platform,
device_id=config_cpu.device_id)
else:
context.set_context(mode=context.GRAPH_MODE, device_target=args_opt.platform,
device_id=config_cpu.device_id, save_graphs=False)
# define network
net = ShuffleNetV2(model_size=config_cpu.model_size, n_class=config_cpu.num_classes)
# define dataset
train_dataset = create_dataset(args_opt.train_dataset, True, config_cpu.rank, config_cpu.group_size,
num_parallel_workers=config_cpu.num_parallel_workers,
batch_size=config_cpu.train_batch_size,
drop_remainder=config_cpu.drop_remainder, shuffle=True,
cutout=config_cpu.cutout, cutout_length=config_cpu.cutout_length,
normalize=config_cpu.normalize,
enable_tobgr=config_cpu.enable_tobgr)
# load ckpt
if args_opt.checkpoint_input_path:
ckpt = load_checkpoint(args_opt.checkpoint_input_path)
if config_cpu.remove_classifier_parameter:
filter_list = [x.name for x in net.classifier.get_parameters()]
filter_checkpoint_parameter_by_list(ckpt, filter_list)
load_param_into_net(net, ckpt)
# loss
if not config_cpu.use_nn_default_loss:
loss = CrossEntropySmooth(sparse=True, reduction="mean",
smooth_factor=config_cpu.label_smooth_factor,
num_classes=config_cpu.num_classes)
else:
loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
# get learning rate
batches_per_epoch = train_dataset.get_dataset_size()
lr = get_lr_basic(lr_init=config_cpu.lr_init, total_epochs=config_cpu.epoch_size,
steps_per_epoch=batches_per_epoch, is_stair=True)
lr = Tensor(lr)
# define optimization
optimizer = Momentum(params=net.trainable_params(),
learning_rate=lr,
momentum=config_cpu.momentum,
weight_decay=config_cpu.weight_decay)
# model
loss_scale_manager = FixedLossScaleManager(config_cpu.loss_scale, drop_overflow_update=False)
model = Model(net,
loss_fn=loss,
optimizer=optimizer,
amp_level=config_cpu.amp_level,
metrics={'acc'},
loss_scale_manager=loss_scale_manager)
# define callbacks
cb = [TimeMonitor(), LossMonitor()]
if args_opt.checkpoint_save_path:
save_ckpt_path = args_opt.checkpoint_save_path
config_ck = CheckpointConfig(save_checkpoint_steps=config_cpu.save_checkpoint_epochs * batches_per_epoch,
keep_checkpoint_max=config_cpu.keep_checkpoint_max)
ckpt_cb = ModelCheckpoint("shufflenetv2", directory=save_ckpt_path, config=config_ck)
print("============== Starting Training ==============")
start_time = time.time()
# begin train
cb += [ckpt_cb]
model.train(config_cpu.epoch_size,
train_dataset,
callbacks=cb,
dataset_sink_mode=False)
print("time: ", (time.time() - start_time) * 1000)
print("============== Train Success ==============")
official/cv/shufflenetv2/eval.py
+ 10
− 5
View file @ fe8d2e31
# Copyright 2020-2021 Huawei Technologies Co., Ltd # Copyright 2020-2022 Huawei Technologies Co., Ltd
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -24,7 +24,7 @@ from mindspore import context ...@@ -24,7 +24,7 @@ from mindspore import context
from mindspore.train.model import Model from mindspore.train.model import Model
from mindspore.train.serialization import load_checkpoint, load_param_into_net from mindspore.train.serialization import load_checkpoint, load_param_into_net
from src.config import config_gpu, config_ascend from src.config import config_gpu, config_ascend, config_cpu
from src.dataset import create_dataset from src.dataset import create_dataset
from src.shufflenetv2 import ShuffleNetV2 from src.shufflenetv2 import ShuffleNetV2
from src.CrossEntropySmooth import CrossEntropySmooth from src.CrossEntropySmooth import CrossEntropySmooth
...@@ -40,7 +40,7 @@ if __name__ == '__main__': ...@@ -40,7 +40,7 @@ if __name__ == '__main__':
parser.add_argument('--checkpoint_dir', type=str, default='', parser.add_argument('--checkpoint_dir', type=str, default='',
help='the directory that contains one or more checkpoint files(Default: None)') help='the directory that contains one or more checkpoint files(Default: None)')
parser.add_argument('--dataset_path', type=str, default='../imagenet', help='Dataset path') parser.add_argument('--dataset_path', type=str, default='../imagenet', help='Dataset path')
parser.add_argument('--platform', type=str, default='Ascend', choices=('Ascend', 'GPU'), parser.add_argument('--platform', type=str, default='Ascend', choices=('Ascend', 'GPU', 'CPU'),
help='run platform(Default:Ascend)') help='run platform(Default:Ascend)')
parser.add_argument('--device_id', type=int, default=0, help='device id(Default:0)') parser.add_argument('--device_id', type=int, default=0, help='device id(Default:0)')
...@@ -48,7 +48,7 @@ if __name__ == '__main__': ...@@ -48,7 +48,7 @@ if __name__ == '__main__':
parser.add_argument('--data_url', type=str, default=None, help='Dataset path for modelarts') parser.add_argument('--data_url', type=str, default=None, help='Dataset path for modelarts')
parser.add_argument('--train_url', type=str, default=None, help='Output path for modelarts') parser.add_argument('--train_url', type=str, default=None, help='Output path for modelarts')
parser.add_argument('--use_pynative_mode', type=ast.literal_eval, default=False, parser.add_argument('--use_pynative_mode', type=ast.literal_eval, default=True,
help='whether to use pynative mode for device(Default: False)') help='whether to use pynative mode for device(Default: False)')
parser.add_argument('--normalize', type=ast.literal_eval, default=True, parser.add_argument('--normalize', type=ast.literal_eval, default=True,
help='whether to normalize the dataset(Default: True)') help='whether to normalize the dataset(Default: True)')
...@@ -70,6 +70,9 @@ if __name__ == '__main__': ...@@ -70,6 +70,9 @@ if __name__ == '__main__':
if args_opt.platform == 'GPU': if args_opt.platform == 'GPU':
config = config_gpu config = config_gpu
drop_remainder = True drop_remainder = True
elif args_opt.platform == 'CPU':
config = config_cpu
drop_remainder = True
else: else:
config = config_ascend config = config_ascend
drop_remainder = False drop_remainder = False
...@@ -132,7 +135,8 @@ if __name__ == '__main__': ...@@ -132,7 +135,8 @@ if __name__ == '__main__':
group_size=1, batch_size=config.val_batch_size, group_size=1, batch_size=config.val_batch_size,
drop_remainder=drop_remainder, shuffle=False, drop_remainder=drop_remainder, shuffle=False,
normalize=args_opt.normalize, normalize=args_opt.normalize,
enable_tobgr=args_opt.enable_tobgr) enable_tobgr=args_opt.enable_tobgr,
num_parallel_workers=config.num_parallel_workers)
if not args_opt.use_nn_default_loss: if not args_opt.use_nn_default_loss:
loss = CrossEntropySmooth(sparse=True, reduction="mean", loss = CrossEntropySmooth(sparse=True, reduction="mean",
smooth_factor=config.label_smooth_factor, num_classes=config.num_classes) smooth_factor=config.label_smooth_factor, num_classes=config.num_classes)
...@@ -140,6 +144,7 @@ if __name__ == '__main__': ...@@ -140,6 +144,7 @@ if __name__ == '__main__':
loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean') loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
net = ShuffleNetV2(n_class=config.num_classes) net = ShuffleNetV2(n_class=config.num_classes)
eval_metrics = {'Loss': nn.Loss(), eval_metrics = {'Loss': nn.Loss(),
'Top1-Acc': nn.Top1CategoricalAccuracy(), 'Top1-Acc': nn.Top1CategoricalAccuracy(),
'Top5-Acc': nn.Top5CategoricalAccuracy()} 'Top5-Acc': nn.Top5CategoricalAccuracy()}
......
official/cv/shufflenetv2/quick_start.py 0 → 100644
+ 88
− 0
View file @ fe8d2e31
# Copyright 2022 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
import argparse
import numpy as np
import matplotlib.pyplot as plt
from src.shufflenetv2 import ShuffleNetV2
from src.config import config_cpu
import mindspore.nn as nn
from mindspore.train.serialization import load_checkpoint, load_param_into_net
from mindspore import Tensor
from mindspore import dataset as ds
import mindspore.dataset.transforms.c_transforms as C2
import mindspore.dataset.vision.c_transforms as C
import mindspore.common.dtype as mstype
from mindspore.train import Model
if __name__ == '__main__':
parser = argparse.ArgumentParser(description='image classification transformation quick_start')
parser.add_argument('--quick_start_ckpt', type=str, default='./save_ckpt/Graph_mode/shufflenetv2-204_22.ckpt',
help='the checkpoint of ShuffleNetV2 (Default: None)')
args_opt = parser.parse_args()
network = ShuffleNetV2(model_size=config_cpu.model_size, n_class=config_cpu.num_classes)
if args_opt.quick_start_ckpt:
ckpt = load_checkpoint(args_opt.quick_start_ckpt)
load_param_into_net(network, ckpt)
print(args_opt.quick_start_ckpt, ' is loaded')
#
# # define loss
net_loss = nn.SoftmaxCrossEntropyWithLogits(sparse=True, reduction='mean')
#
# # define opt
net_opt = nn.Momentum(network.trainable_params(), learning_rate=0.01, momentum=0.9)
model = Model(network, loss_fn=net_loss, optimizer=net_opt, metrics={'accuracy'})
#load test data
test_loader = ds.ImageFolderDataset(config_cpu.eval_dataset_path, shuffle=True)
trans = [
C.RandomCropDecodeResize(224),
C.RandomHorizontalFlip(prob=0.5),
C.RandomColorAdjust(brightness=0.4, contrast=0.4, saturation=0.4),
C.Normalize(mean=[0.485 * 255, 0.456 * 255, 0.406 * 255], std=[0.229 * 255, 0.224 * 255, 0.225 * 255]),
C.HWC2CHW()
]
type_cast_op = C2.TypeCast(mstype.int32)
test_loader = test_loader.map(input_columns="image",
operations=trans,
num_parallel_workers=config_cpu.num_parallel_workers,
python_multiprocessing=True)
test_loader = test_loader.map(input_columns="label",
operations=type_cast_op,
num_parallel_workers=config_cpu.num_parallel_workers,
python_multiprocessing=True)
test_loader = test_loader.create_dict_iterator()
images = []
labels = []
plt.figure()
for i in range(1, 7):
data = next(test_loader)
image = data["image"].asnumpy()
label = data["label"].asnumpy()
plt.subplot(2, 3, i)
plt.imshow(image.T)
images.append(image)
labels.append(int(label))
plt.show()
# # user model.predict to get the classification of the input image
output = model.predict(Tensor(images))
predicted = np.argmax(output.asnumpy(), axis=1)
#
# # output the actual classification and the predicted classifiaction
print(f'Predicted: "{predicted}", Actual: "{labels}"')
official/cv/shufflenetv2/src/config.py
+ 44
− 1
View file @ fe8d2e31
# Copyright 2020-2021 Huawei Technologies Co., Ltd # Copyright 2020-2022 Huawei Technologies Co., Ltd
# #
# Licensed under the Apache License, Version 2.0 (the "License"); # Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License. # you may not use this file except in compliance with the License.
...@@ -17,6 +17,49 @@ network config setting, will be used in train.py and eval.py ...@@ -17,6 +17,49 @@ network config setting, will be used in train.py and eval.py
""" """
from easydict import EasyDict as edict from easydict import EasyDict as edict
#cpu
config_cpu = edict({
'device_target': 'CPU',
'device_id': 0,
'rank': 0,
'group_size': 1,
'num_classes': 5,
# model_size: "shuffleNetV1 model size choices 2.0x, 1.5x, 1.0x, 0.5x"
'model_size': "1.0x",
'epoch_size': 250,
'warmup_epochs': 4,
'keep_checkpoint_max': 5,
'num_parallel_workers': 4,
'drop_remainder': True,
'normalize': True,
'enable_tobgr': False,
### Cutout Config
'cutout': False,
'cutout_length': 56,
###batch
'batch_size': 128,
'train_batch_size': 128,
'val_batch_size': 128,
### Loss Config
'label_smooth_factor': 0.1,
### Learning Rate Config
'lr_init': 0.50,
'weight_decay': 0.00009,
### Optimization Config
'momentum': 0.9,
'loss_scale': 1,
'use_nn_default_loss': True,
'amp_level': 'O0',
'remove_classifier_parameter': True,
'save_checkpoint_epochs': 1,
})
config_gpu = edict({ config_gpu = edict({
'enable_modelarts': False, 'enable_modelarts': False,
......
official/cv/shufflenetv2/src/dataset_split.py 0 → 100644
+ 61
− 0
View file @ fe8d2e31
# Copyright 2022 Huawei Technologies Co., Ltd
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ============================================================================
import os
import random
from shutil import rmtree, copy2
random.seed(1)
input_path = './data/flower_photos'
train_path = './data/flower_photos_split/train'
eval_path = './data/flower_photos_split/eval'
def dataset_split():
if os.path.exists(train_path):
rmtree(train_path)
if os.path.exists(eval_path):
rmtree(eval_path)
os.makedirs(train_path)
os.makedirs(eval_path)
flowers = os.listdir(input_path)
for flower in flowers:
if os.path.isdir(os.path.join(input_path, flower)):
flower_path = os.path.join(input_path, flower)
images = os.listdir(flower_path)
images_path = list()
for image in images:
image_path = os.path.join(flower_path, image)
images_path.append(image_path)
random.shuffle(images_path)
offset = int(len(images_path) * 0.8)
train_set = images_path[:offset]
eval_set = images_path[offset:]
aim_train_path = os.path.join(train_path, flower)
aim_eval_path = os.path.join(eval_path, flower)
os.makedirs(aim_train_path)
os.makedirs(aim_eval_path)
for image in train_set:
print("copying " + image + " to " + aim_train_path)
copy2(image, aim_train_path)
for image in eval_set:
print("copying " + image + " to " + aim_eval_path)
copy2(image, aim_eval_path)
if __name__ == '__main__':
dataset_split()
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment