Resolves: #5262

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>

Improve session manager.
Issue #5174

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

Resolves: #5257 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

- set dd request's time tick on master
- send time tick into dd channel

See also: #5291

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

* Add SaveBinlogPaths grpc in dataservice

* fix merged compile error

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

- add cases for collection with data
- cancel import *

See also: #5302 #5304 

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>

DDL won't auto-flush now

See also: #5289 , #5220 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

See also: #5253 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>

Add SessionManager.
Resolves: #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Resolves: #5293 

Restore segment flush stream & statistic stream to last success pos

Signed-off-by: Congqi Xia <congqi.xia@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

- add cases for creating collections
- change unique str from var to function
- improve coding standards

See also: #5273 #5285 

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>

Before this PR, DataNode considered auto-flush a valid flush
complete. It's wrong. So I open this PR to correct this behaviour
in DataNode.

Now binlog paths from auto-flush will be buffered in replica,
waiting until the manul flush to save into etcd all together.

See also: #5220, #5268
A follow up job of #5271 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

This document includes changes about dataservice and new features 
to support the recovery after it crashed.

Signed-off-by: sunby <bingyi.sun@zilliz.com>

Add exclusive parameter.
Issue #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

query base on timeline, add timestamp on these queries:
- ShowCollections
- HasCollection
- DescribeCollection

Resolves: #5219

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Auto-flush of a segment is not considered flush-completed.
So we need to buffer binlog paths generated by auto-flush.

See also: #5268 #5220 

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

let meta support snapshot
so collection could query meta with timestamp

Resolves: #5219

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>

Add session util.
Issue #5174 

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

The design procedure can be seen: #5220 

Updata PhysicalChannelNames and VirtualChannelNames based
on ShardsNum when create collection, and update collVChan2Chan
map.

Resolves: #5240 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

- update get_connection to create_connection
- add case level for collection
- optimize collection cases


Signed-off-by: ThreadDao <yufen.zong@zilliz.com>

* Change volumes with pytest docker-compose

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

* Fix run pytest bug

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Add a part of the test case for the Collection interface

See also: #5224 #5231 

Signed-off-by: ThreadDao <yufen.zong@zilliz.com>

Remove CreateIndexTask struct and indexTaskQueue

Resolves: #5228 

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

Signed-off-by: Xiangyu Wang <xiangyu.wang@zilliz.com>

Signed-off-by: jeffoverflow <zhifeng.zhang@zilliz.com>

just remove standalone and milvus from $GOPATH/bin,
not every thing, on command of make clean

Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Resend unsuccessful ddMsg when master start

Signed-off-by: yudong.cai <yudong.cai@zilliz.com>

* Deny jenkins slave pod to remain active for a while

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Signed-off-by: quicksilver <zhifeng.zhang@zilliz.com>

Add session util doc.

Signed-off-by: godchen <qingxiang.chen@zilliz.com>

- FlushMsg proto message is no longer in use

- We don't depend on "github.com/prometheus/common/log"

Signed-off-by: yangxuan <xuan.yang@zilliz.com>

* add datanode design doc
Signed-off-by: yangxuan <xuan.yang@zilliz.com>

design doc about master service recover on power failure
Signed-off-by: yefu.chen <yefu.chen@zilliz.com>

GetEntityByID needs the flushed segment to be sorted by RowID field, 
then do binary search to get the target id and entities.

See also: #5177 

Signed-off-by: fishpenguin <kun.yu@zilliz.com>