At present, all our services have their own data directories, and when running tests, the data directories are scattered all over the place, so we need to unify the data in one data directory

* Add index for catalog

* remove append

* Add accountid

* Remove empty table error code

* Change debug info

* add task init related state in rsm

* query task state

* set task table user

* generate task table user with uuid

* taskservice: refector task service init

* hakeeper create init tasks

* fix typo

* fix typo

* fix refresh storage not ready

* fix task runner bug

* add state transition diagram

* fix cn heartbeat and add task service test

* complete task service

* fix mysql task storage

* close db connections

* fix syntax error in hakeeper

* fix task_test.go ut

* register init executors

* allow special user skip auth

* remove println

* add init in tests

* partially fix ut error

* fix goroutine leak

* task_test.go tests can pass

* tmp

* tmp

* fix

* add external mysql

* fix mo frontend

* fix sqladdress adjust

* always enable hakeeper

* fix sca errors

* fix ut

* fix some data race

* fix dead lock

* fix data race

* fix task register

* fix data race

* skip network partition

* fix tests

* fix ut

* make sure bvt must start after mo system init

Co-authored-by: Ziran Wei <weiziran125@gmail.com>
Co-authored-by: fengttt <fengttt@gmail.com>

* Add launch-flush-tae-CN-tae-DN

* Update ckp

Co-authored-by: fengttt <fengttt@gmail.com>

adapt log data write in filepath {account}/log/yyyy/mm/dd/metric/*.csv
- result as Table `SingleStatementTable`, `SingleRawLogTable`, and View `logView`, `errorView`, `spanView`
- Table `SingleStatementTable` handle `statement` data, storing in path `account/log/yyyy/mm/dd/statement_info/*.csv`
- Table `SingleRawLogTable` handle `log`, `error`, `span` data, storing in path `account/log/yyyy/mm/dd/rawlog/*.csv`
- View `logView`, only show `log` data in Table`SingleRawLogTable`
- view schema like `create view logView as select ... from rawlog where raw_item = 'log_info'`
- View `errorView` like logView
- View `spanView` like logView

others
- add config `pathBuilder` chose which rule exportor would use.
- value `DBTable`, build path like `{db_name}/{tbl_name}_*.csv`
- value `AccountDate`,  build path like `account/{log,merged}/yyyy/mm/dd/datatype/*.csv`

Approved by: @zhangxu19830126, @aressu1985, @fengttt, @aptend, @daviszhen

* add new helper

* default dn.toml without flushing

* fix standalone

* skip primary key in ut

* base sql_txn_errors collect all transaction error case

* save metric in same folder

* metric sql_transaction_errors with 'other' type, means exec (auto) rollback cnt

* new metric: sql_transaction_total

Co-authored-by: fengttt <fengttt@gmail.com>

BVT takes a long time after using log-service, which has 3 replicas. So change to single replica to reduce the test time.

Approved by: @jiangxinmeng1, @zhangxu19830126

Approved by: @zhangxu19830126, @fengttt, @daviszhen, @sukki37, @XuPeng-SH, @nnsgmsone

Approved by: @zhangxu19830126

etc: add cn2 to launch-mem-engine
memoryengine: fix DDL target shards

Approved by: @zhangxu19830126

Implement a new mpool, which will replace old mempool, mmu, mheap, hmu, allocator. mpool is a simple, but real memory allocator.

Updated vec and tae etc, to use mpool.

Co-authored-by: Feng Tian <fengttt@gmail.com>
Co-authored-by: XuPeng-SH <xupeng3112@163.com>

Run multiple CN servers help us catch problems like https://github.com/matrixorigin/matrixone/issues/5441 earlier.

Approved by: @zhangxu19830126

1. add and fix CN-CLIENT to send pipeline message to remote node.
2. add and fix pipeline-message-handle logic at remote node.
3. fix some remote run bugs.

Approved by: @reusee, @zhangxu19830126, @nnsgmsone

txnstorage: more bench codes
txnstorage: fix nullable attribute
etc: rename cn-standalone-non-dist-memory-test.toml to cn-non-dist-memory.toml
txnstorage: fix potential deadlocks
txnstorage: fix NewPhysicalIter

Approved by: @zhangxu19830126

set console format log, as running bvt.
it can show real newline or tab character in the message field, instead of '\n' or '\t'.
if developers want to log some formated-json, or panic stacktrace, they can benift from it.

log example
- new (console format)
```
2022/09/23 15:09:02.943420 +0800 INFO db/db.go:115 [PrintStats][Caller=github.com/matrixorigin/matrixone/pkg/vm/engine/tae/db.(*catalogStatsMonitor).PostExecute]:{
"CatalogStats": {
"MaxDBID": 1002,
"MaxTID": 1027,
"MaxSID": 1004,
"MaxBID": 1004
},
"TxnStats": {
"MaxTS": [
...
2022/09/23 15:10:59.183640 +0800 ERROR frontend/routine.go:123 routine execute request failed. error:internal error: panic runtime error: invalid memory address or nil pointer dereference: goroutine 145 [running]:
runtime/debug.Stack()
/Users/reus/gotip/src/runtime/debug/stack.go:24 +0x64
github.com/matrixorigin/matrixone/pkg/common/moerr.ConvertPanicError({0x102b96a40?, 0x103dd9760})
/Users/reus/mo/mem/pkg/common/moerr/error.go:410 +0x5c
github.com/matrixorigin/matrixone/pkg/frontend.(*MysqlCmdExecutor).ExecRequest.func1()
/Users/reus/mo/mem/pkg/frontend/mysql_cmd_executor.go:2434 +0xe0
panic({0x102b96a40, 0x103dd9760})
/Users/reus/gotip/src/runtime/panic.go:884 +0x204
github.com/matrixorigin/matrixone/pkg/frontend.(*TenantInfo).GetTenant(...)
/Users/reus/mo/mem/pkg/frontend/authenticate.go:51
...
```
- original (json format)
```
{"level":"INFO","time":"2022/09/23 15:04:39.036862 +0800","caller":"db/replay.go:196","msg":"CATALOG[CNT=1][CommitId=0,MaxTS=0-0,LSN=0]\n\t\tDB[1][name=0-mo_catalog][C@1-0,D@0-0]\n\t\t\tTBL[1][name=mo_database][C@1-0,D@0-0]\n\t\t\t\t[A]SEG[101][C@1-0,D@0-0]\n\t\t\t\t\t[A]BLK[201][C@1-0,D@0-0]\n\t\t\tTBL[2][name=mo_tables][C@1-0,D@0-0]\n\t\t\t\t[A]SEG[102][C@1-0,D@0-0]\n\t\t\t\t\t[A]BLK[202][C@1-0,D@0-0]\n\t\t\tTBL[3][name=mo_columns][C@1-0,D@0-0]\n\t\t\t\t[A]SEG[103][C@1-0,D@0-0]\n\t\t\t\t\t[A]BLK[203][C@1-0,D@0-0]"}
...
```

Approved by: @reusee, @zhangxu19830126

initialize the clock in main

Approved by: @nnsgmsone, @reusee, @daviszhen, @cnutshell

Remove standalone service mode, and use launch to instead. usage:  ./mo-service -launch ./etc/launch-mem-engine/launch.toml

Approved by: @nnsgmsone

* add distributed mem-engine example

Signed-off-by: Aylei <rayingecho@gmail.com>

* remove redundant file

Signed-off-by: Aylei <rayingecho@gmail.com>

engine: add transaction interface
disttae: get the schema information of a table
disttae: add multi-account logic

Approved by: @reusee, @zhangxu19830126, @XuPeng-SH, @daviszhen

cnservice: add non-distributed memory CN engine type

Approved by: @nnsgmsone, @zhangxu19830126

- support FileService do Write CSV data
- init schema support CVS params, in `pkg/util/metric/metric.go`, `pkg/util/trace/schema.go`
- adapt config[fileservice] to get stringwriter obj, in `pkg/util/export/fs_writer.go`
- add [cn.fileservice] in cn-service.toml
- add [[fileservice]] in log-service.toml
- mo-service support metric/trace/log/... collection,  in `pkg/cnservice/server.go`, `pkg/dnservice/store.go`, `pkg/logservice/service.go`

Approved by: @yingfeng, @daviszhen, @aptend, @fengttt, @reusee, @cnutshell, @zhangxu19830126, @lni

Manage CNStore metadata at the CN node. Provide data for CN and HAKeeper's heartbeat process

Approved by: @nnsgmsone, @daviszhen, @reusee, @lni

sql: refactor external table with Process.FileService

init file service on CN and pass to frontend

Co-authored-by: fengttt <fengttt@gmail.com>

    mo-service, cnservice, etc: codes for starting standalone mem engine and distributed TAE (#4700)

mo-service, cnservice, etc: codes for starting standalone mem engine and distributed TAE

Approved by: @zhangxu19830126, @lni, @nnsgmsone

* add standalone cn config

* fix typo

* debug log for default

* support mo-service-standalone ci

Co-authored-by: ouyuanning <52741323@qq.com>

cnservice: initialize hakeeper client, txn sender, txn client lazily

Approved by: @zhangxu19830126, @nnsgmsone

cnservice: init hakeeper client

cnservice: init txn sender

cnservice: init txn client

frontend: add wrappedTAETx, wrappedTAEEngine; rename InitDB to InitTAE

frontend: use TxnOperator instead of moengine.Txn

txn: add get txn metadata to txnOperator

replace engine.Snapshot with TxnOperator

Co-authored-by: zhangxu <zhangxu19830126@gmail.com>

1. Connect the cn-service with the frontend and the computation engine.
2. Change the configuraion mechanism to the general method. Remove the auto-generated configuration.

Approved by: @zhangxu19830126, @yingfeng, @fengttt, @aptend, @reusee, @nnsgmsone, @lni

Approved by: @lni

use []string in toml files, no more semicoln separated strings

remove the HAKeeperReplicaID field in bootstrap config