Newer
Older
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
# Contents
- [NAML Description](#NAML-description)
- [Dataset](#dataset)
- [Environment Requirements](#environment-requirements)
- [Script Description](#script-description)
- [Script and Sample Code](#script-and-sample-code)
- [Training Process](#training-process)
- [Model Export](#model-export)
- [Model Description](#model-description)
- [Performance](#performance)
- [Evaluation Performance](#evaluation-performance)
- [Inference Performance](#evaluation-performance)
- [Description of Random Situation](#description-of-random-situation)
- [ModelZoo Homepage](#modelzoo-homepage)
# [NAML Description](#contents)
NAML is a multi-view news recommendation approach. The core of NAML is a news encoder and a user encoder. The newsencoder is composed of a title encoder, a abstract encoder, a category encoder and a subcategory encoder. In the user encoder, we learn representations of users from their browsed news. Besides, we apply additive attention to learn more informative news and user representations by selecting important words and news.
[Paper](https://arxiv.org/abs/1907.05576) Chuhan Wu, Fangzhao Wu, Mingxiao An, Jianqiang Huang, Yongfeng Huang and Xing Xie: Neural News Recommendation with Attentive Multi-View Learning, IJCAI 2019
# [Dataset](#contents)
Dataset used: [MIND](https://msnews.github.io/)
MIND contains about 160k English news articles and more than 15 million impression logs generated by 1 million users.
You can download the dataset and put the directory in structure as follows:
```path
└─MINDlarge
├─MINDlarge_train
├─MINDlarge_dev
└─MINDlarge_utils
```
# [Environment Requirements](#contents)
- Hardware（Ascend/GPU）
- Prepare hardware environment with Ascend, GPU processor.
- Framework
- [MindSpore](https://www.mindspore.cn/install/en)
- For more information, please check the resources below：
- [MindSpore Tutorials](https://www.mindspore.cn/tutorials/en/master/index.html)
- [MindSpore Python API](https://www.mindspore.cn/docs/api/en/master/index.html)
# [Script description](#contents)
## [Script and sample code](#contents)
```path
├── naml
├── README.md # descriptions about NAML
├── model_utils
│ ├──__init__.py # module init file
│ ├──config.py # Parse arguments
│ ├──device_adapter.py # Device adapter for ModelArts
│ ├──local_adapter.py # Local adapter
│ ├──moxing_adapter.py # Moxing adapter for ModelArts
├── scripts
│ ├──run_distribute_train.sh # shell script for distribute training
│ ├──run_train.sh # shell script for training
│ ├──run_eval.sh # shell script for evaluation
│ ├──run_infer_310.sh # shell script for 310 inference
├── src
│ ├──__init__.py # module init file
│ ├──callback.py # callback file
│ ├──dataset.py # creating dataset
│ ├──naml.py # NAML architecture
│ ├──utils.py # utils to load ckpt_file for fine tune or incremental learn
├── MINDdemo_config.yaml # Configurations for demo
├── MINDlarge_config.yaml # Configurations for large
├── MINDsmall_config.yaml # Configurations for small
├── ascend310_infer # application for 310 inference
├── train.py # training script
├── eval.py # evaluation script
├── export.py # export mindir script
├── preprocess.py # preprocess input data
└── postprocess.py # post process for 310 inference
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
```
## [Training process](#contents)
### Usage
You can start training using python or shell scripts. The usage of shell scripts as follows:
- running on Ascend
```shell
# train standalone
bash run_train.sh [PLATFORM] [DEVICE_ID] [DATASET] [DATASET_PATH]
# train distribute
bash run_distribute_train.sh [PLATFORM] [DEVICE_NUM] [DATASET] [DATASET_PATH] [RANK_TABLE_FILE]
# evaluation
bash run_eval.sh [PLATFORM] [DEVICE_ID] [DATASET] [DATASET_PATH] [CHECKPOINT_PATH]
```
- `PLATFORM` should be Ascend.
- `DEVICE_ID` is the device id you want to run the network.
- `DATASET` MIND dataset, support large, small and demo.
- `DATASET_PATH` is the dataset path, the structure as [Dataset](#dataset).
- `CHECKPOINT_PATH` is a pre-trained checkpoint path.
- `RANK_TABLE_FILE` is HCCL configuration file when running on Ascend.
For distributed training, a hccl configuration file with JSON format needs to be created in advance.
Please follow the instructions in the link below:
<https://gitee.com/mindspore/models/tree/master/utils/hccl_tools>.
- ModelArts (If you want to run in modelarts, please check the official documentation of [modelarts](https://support.huaweicloud.com/modelarts/), and you can start training as follows)
- Train large dataset 1p/8p on ModelArts
```python
# (1) Add "config_path='/path_to_code/MINDlarge_config.yaml'" on the website UI interface.
# (2) Perform a or b.
# a. Set "enable_modelarts=True" on MINDlarge_config.yaml file.
# Set "platform=Ascend" on MINDlarge_config.yaml file.
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
# Set "dataset='large'" on MINDlarge_config.yaml file.
# Set "dataset_path='/cache/data/MINDlarge'" on MINDlarge_config.yaml file.
# Set "save_checkpoint_path='./checkpoint'" on MINDlarge_config.yaml file.
# Set "weight_decay=False" on MINDlarge_config.yaml file.
# Set "sink_mode=True" on MINDlarge_config.yaml file.
# Set other parameters on MINDlarge_config.yaml file you need.
# b. Add "enable_modelarts=True" on the website UI interface.
# Add "platform=Ascend" on the website UI interface.
# Add "dataset=large" on the website UI interface.
# Add "dataset_path=/cache/data/MINDlarge" on the website UI interface.
# Add "save_checkpoint_path=./checkpoint" on the website UI interface.
# Add "weight_decay=False" on the website UI interface.
# Add "sink_mode=True" on the website UI interface.
# Add other parameters on the website UI interface.
# (3) Upload dataset to S3 bucket.
# (4) Set the code directory to "/path/naml" on the website UI interface.
# (5) Set the startup file to "train.py" on the website UI interface.
# (6) Set the "Dataset path" and "Output file path" and "Job log path" to your path on the website UI interface.
# (7) Create your job.
```
- Eval large dataset 1p on ModelArts
```python
# (1) Add "config_path='/path_to_code/MINDlarge_config.yaml'" on the website UI interface.
# (2) Perform a or b.
# a. Set "enable_modelarts=True" on MINDlarge_config.yaml file.
# Set "platform=Ascend" on MINDlarge_config.yaml file.
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
# Set "dataset='large'" on MINDlarge_config.yaml file.
# Set "dataset_path='/cache/data/MINDlarge'" on MINDlarge_config.yaml file.
# Set "checkpoint_url='s3://dir_to_trained_ckpt/'" on MINDlarge_config.yaml file.
# Set "checkpoint_path='/cache/checkpoint_path/model.ckpt'" on MINDlarge_config.yaml file.
# Set other parameters on MINDlarge_config.yaml file you need.
# b. Add "enable_modelarts=True" on the website UI interface.
# Add "platform=Ascend" on the website UI interface.
# Add "dataset=large" on the website UI interface.
# Add "dataset_path=/cache/data/MINDlarge" on the website UI interface.
# Add "checkpoint_url=s3://dir_to_trained_ckpt/" on the website UI interface.
# Add "checkpoint_path=/cache/checkpoint_path/model.ckpt" on the website UI interface.
# Add other parameters on the website UI interface.
# (3) Upload or copy your trained model to S3 bucket.
# (4) Upload dataset to S3 bucket.
# (5) Set the code directory to "/path/naml" on the website UI interface.
# (6) Set the startup file to "eval.py" on the website UI interface.
# (7) Set the "Dataset path" and "Output file path" and "Job log path" to your path on the website UI interface.
# (8) Create your job.
```
- Export 1p on ModelArts
```python
# (1) Add "config_path='/path_to_code/MINDlarge_config.yaml'" on the website UI interface.
# (2) Perform a or b.
# a. Set "enable_modelarts=True" on MINDlarge_config.yaml file.
# Set "platform=Ascend" on MINDlarge_config.yaml file.
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
# Set "file_format='AIR'" on MINDlarge_config.yaml file.
# Set "batch_size=1" on MINDlarge_config.yaml file.
# Set "checkpoint_url='s3://dir_to_trained_ckpt/'" on MINDlarge_config.yaml file.
# Set "checkpoint_path='/cache/checkpoint_path/model.ckpt'" on MINDlarge_config.yaml file.
# Set other parameters on MINDlarge_config.yaml file you need.
# b. Add "enable_modelarts=True" on the website UI interface.
# Add "platform=Ascend" on the website UI interface.
# Add "file_format=AIR" on the website UI interface.
# Add "batch_size=1" on the website UI interface.
# Add "checkpoint_url=s3://dir_to_trained_ckpt/" on the website UI interface.
# Add "checkpoint_path=/cache/checkpoint_path/model.ckpt" on the website UI interface.
# Add other parameters on the website UI interface.
# (3) Upload or copy your trained model to S3 bucket.
# (4) Set the code directory to "/path/naml" on the website UI interface.
# (5) Set the startup file to "export.py" on the website UI interface.
# (6) Set the "Dataset path" and "Output file path" and "Job log path" to your path on the website UI interface.
# (7) Create your job.
```
## [Model Export](#contents)
```shell
python export.py --platform [PLATFORM] --checkpoint_path [CHECKPOINT_PATH] --file_format [EXPORT_FORMAT] --batch_size [BATCH_SIZE]
```
- `EXPORT_FORMAT` should be in ["AIR", "MINDIR"]
## [Infer on Ascend310](#contents)
```shell
# Ascend310 inference
bash run_infer_310.sh [NEWS_MODEL] [USER_MODEL] [DEVICE_ID]
```
- `NEWS_MODEL` specifies path of news "MINDIR" OR "AIR" model.
- `USER_MODEL` specifies path of user "MINDIR" OR "AIR" model.
- `DEVICE_ID` is optional, default value is 0.
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
# [Model Description](#contents)
## [Performance](#contents)
### Evaluation Performance
| Parameters | Ascend |
| -------------------------- | ------------------------------------------------------------ |
| Model Version | NAML |
| Resource | Ascend 910; CPU 2.60GHz, 56cores; Memory 314G; OS Euler2.8 |
| uploaded Date | 02/23/2021 (month/day/year) |
| MindSpore Version | 1.2.0 |
| Dataset | MINDlarge |
| Training Parameters | epoch=1, steps=52869, batch_size=64, lr=0.001 |
| Optimizer | Adam |
| Loss Function | Softmax Cross Entropy |
| outputs | probability |
| Speed | 1pc: 62 ms/step |
| Total time | 1pc: 54 mins |
### Inference Performance
| Parameters | Ascend |
| ------------------- | --------------------------- |
| Model Version | NAML |
| Resource | Ascend 910; OS Euler2.8 |
| Uploaded Date | 02/23/2021 (month/day/year) |
| MindSpore Version | 1.2.0 |
| Dataset | MINDlarge |
| batch_size | 64 |
| outputs | probability |
| Accuracy | AUC: 0.66 |
### Inference on Ascend310 Performance
| Parameters | Ascend |
| ------------------- | --------------------------- |
| Model Version | NAML |
| Resource | Ascend 310 |
| Uploaded Date | 12/10/2021 (month/day/year) |
| MindSpore Version | 1.6.0 |
# [Description of Random Situation](#contents)
<!-- In dataset.py, we set the seed inside “create_dataset" function. We also use random seed in train.py. -->
In train.py, we set the seed which is used by numpy.random, mindspore.common.Initializer, mindspore.ops.composite.random_ops and mindspore.nn.probability.distribution.
# [ModelZoo Homepage](#contents)
Please check the official [homepage](https://gitee.com/mindspore/models).