Skip to content
Snippets Groups Projects
Commit 7afad0e1 authored by anzhengqi's avatar anzhengqi
Browse files

add network tinybert scipy version info

parent 25a1b898
Branches
No related tags found
No related merge requests found
......@@ -65,6 +65,7 @@ The backbone structure of TinyBERT is transformer, the transformer contains four
- Prepare hardware environment with Ascend or GPU processor.
- Framework
- [MindSpore](https://gitee.com/mindspore/mindspore)
- scipy>=1.7
- For more information, please check the resources below:
- [MindSpore Tutorials](https://www.mindspore.cn/tutorials/en/master/index.html)
- [MindSpore Python API](https://www.mindspore.cn/docs/api/en/master/index.html)
......@@ -100,7 +101,7 @@ The backbone structure of TinyBERT is transformer, the transformer contains four
Please follow the instructions in the link below:
https:gitee.com/mindspore/models/tree/master/utils/hccl_tools.
For dataset, if you want to set the format and parameters, a schema configuration file with JSON format needs to be created, please refer to [tfrecord](https://www.mindspore.cn/docs/programming_guide/en/master/dataset_loading.html#tfrecord) format.
For dataset, if you want to set the format and parameters, a schema configuration file with JSON format needs to be created, please refer to [tfrecord](https://www.mindspore.cn/docs/en/master/api_python/dataset/mindspore.dataset.TFRecordDataset.html) format.
```text
For general task, schema file contains ["input_ids", "input_mask", "segment_ids"].
......@@ -627,6 +628,8 @@ In gd_config.yaml and td_config/*.yaml, we set the hidden_dropout_prob and atten
In run_general_distill.py, we set the random seed to make sure distribute training has the same init weight.
If accuracy < standard, may be scipy version < 1.7.
# [ModelZoo Homepage](#contents)
Please check the official [homepage](https://gitee.com/mindspore/models).
......@@ -70,6 +70,7 @@ TinyBERT模型的主干结构是转换器,转换器包含四个编码器模块
- 使用Ascend或GPU处理器准备硬件环境。
- 框架
- [MindSpore](https://gitee.com/mindspore/mindspore)
- scipy>=1.7
- 更多关于Mindspore的信息,请查看以下资源:
- [MindSpore教程](https://www.mindspore.cn/tutorials/zh-CN/master/index.html)
- [MindSpore Python API](https://www.mindspore.cn/docs/api/zh-CN/master/index.html)
......@@ -105,7 +106,7 @@ TinyBERT模型的主干结构是转换器,转换器包含四个编码器模块
详情参见如下链接:
https:gitee.com/mindspore/models/tree/master/utils/hccl_tools.
如需设置数据集格式和参数,请创建JSON格式的视图配置文件,详见[TFRecord](https://www.mindspore.cn/docs/programming_guide/zh-CN/master/dataset_loading.html#tfrecord) 格式。
如需设置数据集格式和参数,请创建JSON格式的视图配置文件,详见[TFRecord](https://www.mindspore.cn/docs/zh-CN/master/api_python/dataset/mindspore.dataset.TFRecordDataset.html) 格式。
```text
For general task, schema file contains ["input_ids", "input_mask", "segment_ids"].
......@@ -623,6 +624,8 @@ gd_config.yaml和td_config/*.yaml文件中设置了hidden_dropout_prob和attenti
run_general_distill.py文件中设置了随机种子,确保分布式训练初始权重相同。
若结果精度不达标,可能原因为使用的scipy版本低于1.7
# ModelZoo主页
请浏览官网[主页](https://gitee.com/mindspore/models)。
easydict
scipy>=1.7.1
\ No newline at end of file
......@@ -410,8 +410,7 @@ def run_main():
if lists:
lists.sort(key=lambda fn: os.path.getmtime(td_phase1_save_ckpt_dir + '/' + fn))
name_ext = os.path.splitext(lists[-1])
if name_ext[-1] != ".ckpt":
raise ValueError("Invalid file, checkpoint file should be .ckpt file")
assert name_ext[-1] == ".ckpt", "Invalid file, checkpoint file should be .ckpt file"
newest_ckpt_file = os.path.join(td_phase1_save_ckpt_dir, lists[-1])
# run task distill
run_task_distill(newest_ckpt_file)
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment