|-i|设置包含列表，只有路径中名称包含其中一个列表的脚本文件将被执行，如果有多个，如果没有指定，用'，'分隔，指的是包含的所有情况set the including list, and only script files in the path whose name contains one of the lists will be executed, if more than one, separated by `,`, if not specified, refers to all cases included|
|-e|设置排除列表，如果路径下的脚本文件的名称包含一个排除列表，则不会被执行，如果有多个，用'，'分隔，如果没有指定，表示不排除任何情况set the excluding list, and script files in the path whose name contains one of the lists will not be executed, if more than one, separated by `,`, if not specified, refers to none of the cases excluded|
-p set the path of test cases needed to be executed by mo-tester, the default value is configured by the `path` in `run.yaml`
-m set the method that mo-tester will run with, the default value is configured by the `method` in `run.yaml`
-t set the type of the format that mo-tester executes the SQL command in, the default value is configured by the `type` in `run.yaml`
-r set The success rate that test cases should reach, the default value is configured by the `rate` in `run.yaml`
-i set the including list, and only script files in the path whose name contains one of the lists will be executed, if more than one, separated by `,`, if not specified, refers to all cases included
-e set the excluding list, and script files in the path whose name contains one of the lists will not be executed, if more than one, separated by `,`, if not specified, refers to none of the cases excluded
-g means SQL commands which is marked with [bvt:issue] flag will not be executed,this flag starts with [-- @bvt:issue#{issueNO.}],and ends with [-- @bvt:issue],eg:
@@ -39,7 +39,7 @@ MO-Tester includes testing cases in the following table.
* Make sure you have installed jdk8.
* Launch MatrixOne or other database instance. Please refer to more information about [how to install and launch MatrixOne](https://docs.matrixorigin.io/0.5.0/MatrixOne/Get-Started/install-standalone-matrixone/).
* Launch MatrixOne or other database instance. Please refer to more information about [how to install and launch MatrixOne](../Get-Started/install-standalone-matrixone/).
* Clone `mo-tester` repository.
...
...
@@ -49,7 +49,7 @@ MO-Tester includes testing cases in the following table.
## 2. Configure `mo-tester`
* In `mo.yml` file, configure the server address, default database name, username&password, etc. MO-tester is based on java, so these parameters are required for the JDBC driver. Below is a default example for a local standalone version MatrixOne.
* In `mo.yml` file, configure the server address, default database name, username, and password, etc. MO-tester is based on java, so these parameters are required for the JDBC(JDBC，Java Database Connectivity) driver. Below is a default example for a local standalone version MatrixOne.
```
#jdbc
...
...
@@ -77,42 +77,41 @@ MO-Tester includes testing cases in the following table.
## 3. Run mo-tester
* With the simple below command, all the SQL test cases will automatically run and generate reports and error messages to `report/report.txt` and `report/error.txt`.
* With the simple below command, all the SQL test cases will automatically run and generate reports and error messages to *report/report.txt* and *report/error.txt*.
```
> ./run.sh
```
If you'd like to adjust the test range, you can just change the `path` parameter of `run.yml`. And you can also specify some parameters when executing the command `run.sh`, parameters are as followings:
If you'd like to adjust the test range, you can just change the `path` parameter of `run.yml`. And you can also specify some parameters when executing the command `run.sh`, parameters are as followings:
|Parameters|Description|
|---|---|
|-p|set the path of test cases needed to be executed by mo-tester, the default value is configured by the `path` in `run.yaml`|
|-m|set the method that mo-tester will run with, the default value is configured by the `method` in `run.yaml`|
|-t| set the type of the format that mo-tester executes the SQL command in, the default value is configured by the `type` in `run.yaml`|
|-r|set The success rate that test cases should reach, the default value is configured by the `rate` in `run.yaml`|
|-i|set the including list, and only script files in the path whose name contains one of the lists will be executed, if more than one, separated by `,`, if not specified, refers to all cases included|
|-e|set the excluding list, and script files in the path whose name contains one of the lists will not be executed, if more than one, separated by `,`, if not specified, refers to none of the cases excluded|
|-g|means SQL commands which is marked with [bvt:issue] flag will not be executed,this flag starts with [-- @bvt:issue#{issueNO.}],and ends with [-- @bvt:issue],eg:<br>-- @bvt:issue#3236<br/><br>select date_add("1997-12-31 23:59:59",INTERVAL "-10000:1" HOUR_MINUTE);<br/><br>select date_add("1997-12-31 23:59:59",INTERVAL "-100 1" YEAR_MONTH);<br/><br>-- @bvt:issue<br/><br>Those two sql commands are associated with issue#3236, and they will not be executed in bvt test, until the flag is removed when issue#3236 is fixed.<br/>|
|-n|means the metadata of the resultset will be ignored when comparing the result|
**Examples**:
```
-p set the path of test cases needed to be executed by mo-tester, the default value is configured by the `path` in `run.yaml`
-m set the method that mo-tester will run with, the default value is configured by the `method` in `run.yaml`
-t set the type of the format that mo-tester executes the SQL command in, the default value is configured by the `type` in `run.yaml`
-r set The success rate that test cases should reach, the default value is configured by the `rate` in `run.yaml`
-i set the including list, and only script files in the path whose name contains one of the lists will be executed, if more than one, separated by `,`, if not specified, refers to all cases included
-e set the excluding list, and script files in the path whose name contains one of the lists will not be executed, if more than one, separated by `,`, if not specified, refers to none of the cases excluded
-g means SQL commands which is marked with [bvt:issue] flag will not be executed,this flag starts with [-- @bvt:issue#{issueNO.}],and ends with [-- @bvt:issue],eg:
Those two sql commands are associated with issue#3236, and they will not be executed in bvt test, until the flag is removed when issue#3236 is fixed.
-n means the metadata of the resultset will be ignored when comparing the result
Examples:
bash run.sh -p case -m run -t script -r 100 -i select,subquery -e substring -g
./run.sh -p case -m run -t script -r 100 -i select,subquery -e substring -g
```
If you want to automatically generate SQL results for the new SQL cases, you can just change the `method` parameter of `run.yml`to `genrs`. Running the `run.sh` scripts will directly record test results in the `result/` path with their original filenames.
If you want to automatically generate SQL results for the new SQL cases, you can just change the `method` parameter of `run.yml`file to `genrs`, or you can just change the command `-m run` to `-m genrs`. Running the `run.sh` scripts will directly record test results in the `result/` path with their original filenames. For more information on example, see <ahref="#new_test_scenario">Example 4</a>.
Note: every time running `run.sh` will overwrite the `error`, `report`, and `success` reports.
!!! note
Every time running `run.sh` will overwrite the report of the *error.txt* file, *report.txt* file, and *success.txt* file.
## 4. Check the report
* Once the test is finished, `mo-tester` generates `error`, `report` and `success` reports in `txt` format.
* Once the test is finished, *mo-tester* generates *error.txt* file, *report.txt* file and *success.txt* file reports .
* An example of `report.txt` looks like this:
* An example of *report.txt* file looks like this:
2. To run the test cases in the *cases/transaction/* path of the *mo-tester* repository, see the following commands:
```
./run.sh -p cases/transaction/
```
3. Check the result reports in the *error.txt* file, *report.txt* file, and *success.txt* file in the *report/* path. The example of the expected *report.txt* looks like this:
2. To run the test cases *cases/transaction/atomicity.sql*, see the following commands:
```
./run.sh -p cases/transaction/atomicity.sql
```
3. Check the result reports in the *error.txt* file, *report.txt* file, and *success.txt* file in the *report/* path. The example of the expected *report.txt* looks like this:
- Method 2: Open the *run.yml* file, change the *method* parameter from the default `run` to `genrs`, and run the following command to generate the test result.
```
./run.sh -p cases/local_test/new_test.sql
```
3. Check the result file in the *result/* path.
4. To run the test cases *cases/local_test/new_test.sql*, see the following commands:
```
./run.sh -p cases/local_test/new_test.sql -m run -g
```
4. Check the result reports in the *error.txt* file, *report.txt* file, and *success.txt* file in the *report/* path. The example of the expected *report.txt* looks like this:
