Skip to content
Snippets Groups Projects
Unverified Commit 9f522f37 authored by i-robot's avatar i-robot Committed by Gitee
Browse files

!3469 modify some network scripts

Merge pull request !3469 from anzhengqi/modify-networks
parents 5657d124 cf90d1e1
No related branches found
No related tags found
No related merge requests found
......@@ -76,7 +76,6 @@ Note that you can run the scripts based on the dataset mentioned in original pap
```bash
pip install Cython
pip install pycocotools
pip install mmcv=0.2.14
```
# [Quick Start](#contents)
......
......@@ -74,7 +74,6 @@ MaskRCNN是一个两级目标检测网络,作为FasterRCNN的扩展模型,
```bash
pip install Cython
pip install pycocotools
pip install mmcv=0.2.14
```
# 快速入门
......
Cython
pycocotools
mmcv==0.2.14
numpy
onnxruntime-gpu
opencv-python
mmcv
pycocotools
pyyaml
......@@ -66,7 +66,7 @@ def train_mobilenetv1():
ms.set_context(mode=ms.GRAPH_MODE, device_target=target, save_graphs=False)
# Set mempool block size in PYNATIVE_MODE for improving memory utilization, which will not take effect in GRAPH_MODE
if ms.get_context("mode") == context.PYNATIVE_MODE:
if ms.get_context("mode") == ms.PYNATIVE_MODE:
ms.set_context(mempool_block_size="31GB")
if config.parameter_server:
......
......@@ -469,8 +469,7 @@ config for lossscale and etc.
```text
Parameters for dataset and network (Pre-Training/Fine-Tuning/Evaluation):
seq_length length of input sequence: N, default is 128
vocab_size size of each embedding vector: N, must be consistent with the dataset you use. Default is 21128.
Usually, we use 21128 for CN vocabs and 30522 for EN vocabs according to the origin paper.
vocab_size size of each embedding vector: N, must be consistent with the dataset you use. Default is 21128. Usually, we use 21128 for CN vocabs and 30522 for EN vocabs according to the origin paper.
hidden_size size of bert encoder layers: N, default is 768
num_hidden_layers number of hidden layers: N, default is 12
num_attention_heads number of attention heads: N, default is 12
......@@ -901,7 +900,7 @@ Refer to the [ModelZoo FAQ](https://gitee.com/mindspore/models#FAQ) for some com
- **Q: Why the training process failed with error about operator `Gather`?**
**A**: Bert use operator `Gather` for embedding. The size of vocab is configured by `vocab_size` in yaml config file. If the vocab used to construct the dataset is larger than config, the operator will failed for the violation access.
**A**: Bert use operator `Gather` for embedding. The size of vocab is configured by `vocab_size` in yaml config file, usually 21128 for CN vocabs, 30522 for EN vocabs. If the vocab used to construct the dataset is larger than config, the operator will failed for the violation access.
- **Q: Why the modification in yaml config file doesn't take effect?**
......
......@@ -470,7 +470,7 @@ config for lossscale and etc.
```text
数据集和网络参数(预训练/微调/评估):
seq_length 输入序列的长度,默认为128
vocab_size 各内嵌向量大小,需与所采用的数据集相同。默认为21136
vocab_size 各内嵌向量大小,需与所采用的数据集相同。默认为21128。根据论文,一般对于中文数据集,取值为21128,英文数据集为30522。
hidden_size BERT的encoder层数,默认为768
num_hidden_layers 隐藏层数,默认为12
num_attention_heads 注意头的数量,默认为12
......@@ -874,7 +874,7 @@ run_pretrain.py中设置了随机种子,确保分布式训练中每个节点
- **Q: 运行过程中报错Gather算子错误是什么问题?**
**A**: Bert模型中的使用Gather算子完成embedding操作,操作会根据输入数据的值来映射字典表,字典表的大小由配置文件中的`vocab_size`来决定,当实际使用的数据集编码时使用的字典表大小超过配置的大小时,操作gather算子时就会发出越界访问的错误,从而Gather算子会报错中止程序。
**A**: Bert模型中的使用Gather算子完成embedding操作,操作会根据输入数据的值来映射字典表,字典表的大小由配置文件中的`vocab_size`来决定,一般中文为21128,英文为30522,当实际使用的数据集编码时使用的字典表大小超过配置的大小时,操作gather算子时就会发出越界访问的错误,从而Gather算子会报错中止程序。
- **Q: 修改了yaml文件中的配置,为什么没有效果?**
......
......@@ -120,7 +120,7 @@ def run_transformer_train():
ms.set_context(reserve_class_name_in_scope=False)
# Set mempool block size in PYNATIVE_MODE for improving memory utilization, which will not take effect in GRAPH_MODE
if ms.get_context("mode") == context.PYNATIVE_MODE:
if ms.get_context("mode") == ms.PYNATIVE_MODE:
ms.set_context(mempool_block_size="31GB")
if config.device_target == "GPU":
......
......@@ -125,8 +125,7 @@ def check_minmax(args, data, min_value=0.99, max_value=1.01):
def get_model(args):
'''get_model'''
net = get_backbone(args)
if args.fp16:
net.add_flags_recursive(fp16=True)
net.add_flags_recursive(fp16=True)
if args.weight.endswith('.ckpt'):
param_dict = load_checkpoint(args.weight)
param_dict_new = {}
......@@ -267,7 +266,7 @@ def run_eval(args):
format(compile_time_used))
img_transforms = transforms.Compose([vision.ToTensor(),
vision.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5), is_hwc=False)])
vision.Normalize((0.5, 0.5, 0.5), (0.5, 0.5, 0.5), is_hwc=False)])
#for test images
args.logger.info('INFO, start step1, calculate test img embedding, weight file = {}'.format(args.weight))
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment