!96 [西安电子科技大学][高校贡献][Mindspore][Fairmot]-高性能预训练模型提交+mxbase+sdk+modelarts

Merge pull request !96 from 陈志鹏/code_fairmot

.jenkins/check/config/filter_cpplint.txt

@@ -49,4 +49,7 @@
 "models/official/cv/posenet/infer/mxbase/src/Posenet.cpp" "runtime/references"
 "models/official/cv/posenet/infer/mxbase/src/main.cpp" "runtime/references"
 
-"models/research/cv/ibnnet/infer/mxbase/src/IbnnetOpencv.h" "runtime/references"
\ No newline at end of file
+"models/research/cv/ibnnet/infer/mxbase/src/IbnnetOpencv.h" "runtime/references"
+
+"models/research/cv/fairmot/infer/mxbase/src/Fairmot.h" "runtime/references"
+"models/research/cv/fairmot/infer/mxbase/src/PostProcess/FairmotMindsporePost.h" "runtime/references"
\ No newline at end of file

research/cv/fairmot/Dockerfile

+ARG FROM_IMAGE_NAME
+FROM ${FROM_IMAGE_NAME}
+
+COPY requirements.txt .
+RUN pip3.7 install -r requirements.txt
\ No newline at end of file

research/cv/fairmot/fairmot_train.py

@@ -21,7 +21,6 @@ from mindspore import dtype as mstype
 from mindspore import Model
 import mindspore.nn as nn
 import mindspore.dataset as ds
-from mindspore.common import set_seed
 from mindspore.context import ParallelMode
 from mindspore.train.callback import TimeMonitor, ModelCheckpoint, CheckpointConfig
 from mindspore.communication.management import init
@@ -34,8 +33,7 @@ from src.utils.lr_schedule import dynamic_lr
 from src.utils.jde import JointDataset
 from src.utils.callback import LossCallback
 
-set_seed(1234)
-ds.config.set_seed(1234)
+
 def train(opt):
     """train fairmot."""
     local_data_path = '/cache/data'
@@ -71,7 +69,8 @@ def train(opt):
         context.set_context(device_id=device_id, mode=context.GRAPH_MODE, device_target="Ascend", save_graphs=False)
         context.set_auto_parallel_context(parallel_mode=ParallelMode.DATA_PARALLEL,
                                           gradients_mean=True,
-                                          device_num=device_num
+                                          device_num=device_num,
+                                          parameter_broadcast=True
                                           )
         init()
     else:

research/cv/fairmot/infer/Dockerfile

+ARG FROM_IMAGE_NAME
+FROM ${FROM_IMAGE_NAME}
+
+COPY requirements.txt .
+RUN pip3.7 install -r requirements.txt
\ No newline at end of file

research/cv/fairmot/infer/README_CN.md

+# 推理
+
+## 模型转换
+
+   ```bash
+
+   cd infer/convert
+
+   ```
+
+1. 准备模型文件。
+
+   AIR模型为在昇腾910服务器上导出的模型，导出AIR模型的详细步骤请参见“模型训练”。
+
+2. 执行以下命令，进行模型转换。
+
+   转换详细信息可查看转换脚本和对应的AIPP配置文件，转换命令如下。
+
+   **bash convert/convert_om.sh** *air_path* *aipp_cfg_path* *om_path*
+
+   | 参数          | 说明                                              |
+   | ------------- | ------------------------------------------------- |
+   | air_path      | 转换脚本AIR文件路径。                             |
+   | aipp_cfg_path | AIPP配置文件路径。                                |
+   | om_path       | 生成的OM文件名，转换脚本会在此基础上添加.om后缀。 |
+
+   转换示例如下所示。
+
+   ```bash
+
+   # 转换模型
+   bash convert_om.sh fairmot.air aipp_rgb.cfg fairmot
+
+   ```
+
+## 推理数据集下载
+
+1. 下载推理数据集[MOT20](https://motchallenge.net/data/MOT20/)。
+
+2. 将数据集存放在`infer/data/MOT20`目录下。
+
+3. 目录格式为：
+
+```text
+   └─fairmot
+      ├─infer
+        ├─data
+        │ ├─MOT20
+        │ │ └─train
+        │ │    ├─MOT20-01
+        │ │    ├─MOT20-02
+        │ │    ├─MOT20-03
+        │ │    └─MOT20-05
+        │ └─data.json
+        ├─infer
+        │ ├─convert                   //  模型转换脚本
+        │ ├─mxbase                   // mxbase 推理脚本
+        │ └─sdk                   // sdk推理脚本
+```
+
+## mxBase推理
+
+   ```bash
+
+   cd infer/mxbase
+
+   ```
+
+1. 编译工程。
+
+   目前mxBase推理仅实现了基于DVPP方式推理。
+
+   ```bash
+
+   bash build.sh
+
+   ```
+
+2. （可选）修改配置文件。
+
+   可根据实际情况修改，配置文件位于`mxbase/src/main.cpp`中，可修改参数如下。
+
+   ```c++
+
+   namespace {
+   const uint32_t DEVICE_ID = 0;
+   } // namespace
+   ...
+
+   ```
+
+3. 运行推理服务。
+
+   运行推理服务：
+   **./build/fairmot_mindspore**  *om_path* *img_path*
+   | 参数       | 说明                           |
+   | ---------- | ------------------------------ |
+   | om_path | om存放路径。如：`../convert/fairmot.om`。 |
+   | img_path | 推理图片路径。如：`../../data/MOT20/`。 |
+
+   ```bash
+
+   ./build/fairmot_mindspore ../convert/fairmot.om ../../data/MOT20/
+
+   ```
+
+4. 观察结果。
+   推理结果以txt格式保存，路径为`../../data/MOT20/result_Files`.
+
+5. 可视化结果并得到精度。
+
+   运行精度测试以及可视化：
+   **python3.7 mx_base_eval.py**  *result_path*
+   | 参数       | 说明                           |
+   | ---------- | ------------------------------ |
+   | result_path | 推理结果路径。如：“../../data/MOT20”。 |
+
+   ```bash
+
+   cd ../../
+   python3.7 mx_base_eval.py --data_dir data/MOT20
+
+   ```
+
+6. 查看精度结果以及可视化结果。
+
+   图片保存在`data/MOT20/result`里。精度测试结果在运行完`mxbase_eval.py`以后会在终端显示并以xlsx文件格式保存在`data/MOT20`。
+
+## MindX SDK推理
+
+   ```bash
+
+   cd infer/sdk
+
+   ```
+
+1. （可选）修改配置文件。
+
+   1. 可根据实际情况修改pipeline文件。
+
+      ```python
+
+      ├── config
+      │   ├── config.py
+      │   └──  fairmot.pipeline # PIPELINE文件
+
+      ```
+
+2. 模型推理。
+
+   1. 执行推理。
+
+      切换到sdk目录下，执行推理脚本。
+      **python main.py**  *img_path* *pipeline_path* *infer_result_path* *infer_mode*
+      | 参数        | 说明                                  |
+      | ----------- | ------------------------------------- |
+      | img_path  | 推理图片路径。如：“../data/MOT20”。        |
+      | pipeline_path | 存放pipeline路径。如："./config/fairmot.pipeline"。 |
+      | infer_result_path | 存放推理结果的路径。如："../data/infer_result"。 |
+      | infer_mode | 推理模式，默认为infer，可用eval直接得到精度对比 |
+
+      ```bash
+
+      python3.7 main.py --img_path ../data/MOT20 --pipeline_path ./config/fairmot.pipeline --infer_result_path ../data/infer_result
+
+      ```
+
+   2. 查看推理结果。
+      推理结果以bin文件形式保存在`../data/infer_result`目录下。
+3. 执行精度测试以及可视化。
+
+   切换到fairmot目录下，执行推理脚本：
+      **python3.7 sdk_eval.py**  *img_path*
+
+   ```bash
+   cd ../
+   python3.7 sdk_eval.py --data_dir ./data/MOT20
+   ```
+
+4. 查看精度结果以及可视化结果。
+
+   图片保存在`data/MOT20/results`里。精度测试结果在运行完`sdk_eval.py`以后会在终端显示并以xlsx文件格式保存在`data/MOT20`。

research/cv/fairmot/infer/convert/aipp_rgb.cfg

+aipp_op {
+    aipp_mode : static
+    input_format : RGB888_U8
+    related_input_rank : 0
+    csc_switch : false
+    rbuv_swap_switch : true
+    var_reci_chn_0 : 0.003921568627451
+    var_reci_chn_1 : 0.003921568627451
+    var_reci_chn_2 : 0.003921568627451
+}
\ No newline at end of file

research/cv/fairmot/infer/convert/convert_om.sh

+#!/bin/bash
+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+
+if [ $# -ne 3 ]; then
+  echo "Wrong parameter format."
+  echo "Usage:"
+  echo "         bash $0 [INPUT_AIR_PATH] [AIPP_PATH] [OUTPUT_OM_PATH_NAME]"
+  echo "Example: "
+  echo "         bash convert_om.sh  xxx.air ./aipp.cfg xx"
+
+  exit 1
+fi
+
+input_air_path=$1
+aipp_cfg_file=$2
+output_om_path=$3
+
+export install_path=/usr/local/Ascend/
+
+export ASCEND_ATC_PATH=${install_path}/atc
+export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH
+export PATH=/usr/local/python3.7.5/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH
+export PYTHONPATH=${install_path}/atc/python/site-packages:${install_path}/latest/atc/python/site-packages/auto_tune.egg/auto_tune:${install_path}/atc/python/site-packages/schedule_search.egg
+export ASCEND_OPP_PATH=${install_path}/opp
+
+export ASCEND_SLOG_PRINT_TO_STDOUT=1
+
+echo "Input AIR file path: ${input_air_path}"
+echo "Output OM file path: ${output_om_path}"
+
+atc --input_format=NCHW \
+  --framework=1 \
+  --model="${input_air_path}" \
+  --input_shape="x:1, 3, 608, 1088" \
+  --output="${output_om_path}" \
+  --insert_op_conf="${aipp_cfg_file}" \
+  --soc_version=Ascend310 \
+  --op_select_implmode=high_precision \
+  --output_type=FP32

research/cv/fairmot/infer/docker_start_infer.sh

+#!/bin/bash
+
+#coding = utf-8
+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the BSD 3-Clause License  (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://opensource.org/licenses/BSD-3-Clause
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+docker_image=$1
+data_dir=$2
+
+function show_help() {
+    echo "Usage: docker_start.sh docker_image data_dir"
+}
+
+function param_check() {
+    if [ -z "${docker_image}" ]; then
+        echo "please input docker_image"
+        show_help
+        exit 1
+    fi
+
+    if [ -z "${data_dir}" ]; then
+        echo "please input data_dir"
+        show_help
+        exit 1
+    fi
+}
+
+param_check
+
+docker run -it \
+  --device=/dev/davinci3 \
+  --device=/dev/davinci_manager \
+  --device=/dev/devmm_svm \
+  --device=/dev/hisi_hdc \
+  -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
+  -v ${data_dir}:${data_dir} \
+  ${docker_image} \
+  /bin/bash

research/cv/fairmot/infer/mxbase/CMakeLists.txt

+cmake_minimum_required(VERSION 3.5.2)
+project(fairmot)
+add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0)
+
+#set(PLUGIN_NAME "fairmot_mindspore_post")
+set(TARGET_LIBRARY fairmot_mindspore_post)
+set(TARGET_MAIN fairmot_mindspore)
+
+set(ACL_LIB_PATH $ENV{ASCEND_HOME}/nnrt/latest/acllib)
+
+include_directories(${CMAKE_CURRENT_BINARY_DIR})
+
+include_directories($ENV{MX_SDK_HOME}/include)
+include_directories($ENV{MX_SDK_HOME}/opensource/include)
+include_directories($ENV{MX_SDK_HOME}/opensource/include/opencv4)
+include_directories($ENV{MX_SDK_HOME}/opensource/include/gstreamer-1.0)
+include_directories($ENV{MX_SDK_HOME}/opensource/include/glib-2.0)
+include_directories($ENV{MX_SDK_HOME}/opensource/lib/glib-2.0/include)
+message($ENV{MX_SDK_HOME})
+link_directories($ENV{MX_SDK_HOME}/lib)
+link_directories($ENV{MX_SDK_HOME}/opensource/lib/)
+
+
+add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations)
+add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}")
+add_compile_options("-Dgoogle=mindxsdk_private")
+
+add_definitions(-DENABLE_DVPP_INTERFACE)
+
+include_directories(${ACL_LIB_PATH}/include)
+link_directories(${ACL_LIB_PATH}/lib64/)
+
+add_library(${TARGET_LIBRARY} SHARED src/PostProcess/FairmotMindsporePost.cpp)
+
+target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0)
+target_link_libraries(${TARGET_LIBRARY} plugintoolkit mxpidatatype mxbase)
+target_link_libraries(${TARGET_LIBRARY} -Wl,-z,relro,-z,now,-z,noexecstack -s)
+
+message("TARGET_LIBRARY:${TARGET_LIBRARY}.")
+
+
+add_executable(${TARGET_MAIN} src/main.cpp src/Fairmot.cpp)
+target_link_libraries(${TARGET_MAIN} ${TARGET_LIBRARY} glog  cpprest mxbase libascendcl.so opencv_world)

research/cv/fairmot/infer/mxbase/build.sh

+#!/bin/bash
+
+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+
+# env
+
+mkdir -p build
+cd build || exit
+
+function make_plugin() {
+  if ! cmake ..; then
+    echo "cmake failed."
+    return 1
+  fi
+
+  if ! (make); then
+    echo "make failed."
+    return 1
+  fi
+
+  return 0
+}
+
+if make_plugin; then
+  echo "INFO: Build successfully."
+else
+  echo "ERROR: Build failed."
+fi
+
+cd - || exit

research/cv/fairmot/infer/mxbase/src/Fairmot.cpp

+/*
+ * Copyright 2021 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "Fairmot.h"
+
+#include <dirent.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#define BOOST_BIND_GLOBAL_PLACEHOLDERS
+#include <algorithm>
+#include <map>
+#include <memory>
+#include <string>
+#include <utility>
+#include <vector>
+#include <boost/property_tree/json_parser.hpp>
+#include <opencv4/opencv2/core.hpp>
+#include <opencv4/opencv2/opencv.hpp>
+
+#include "MxBase/DeviceManager/DeviceManager.h"
+#include "MxBase/Log/Log.h"
+#include "acl/acl.h"
+namespace {
+const uint32_t YUV_BYTE_NU = 3;
+const uint32_t YUV_BYTE_DE = 2;
+const uint32_t FRAME_RATE = 25;
+const uint32_t MODEL_HEIGHT = 768;
+const uint32_t MODEL_WIDTH = 1280;
+const float CONF_THRES = 0.4;
+}  // namespace
+
+void PrintTensorShape(const std::vector<MxBase::TensorDesc> &tensorDescVec,
+                      const std::string &tensorName) {
+  LogInfo << "The shape of " << tensorName << " is as follows:";
+  for (size_t i = 0; i < tensorDescVec.size(); ++i) {
+    LogInfo << "  Tensor " << i << ":";
+    for (size_t j = 0; j < tensorDescVec[i].tensorDims.size(); ++j) {
+      LogInfo << "   dim: " << j << ": " << tensorDescVec[i].tensorDims[j];
+    }
+  }
+}
+
+APP_ERROR Fairmot::Init(const InitParam &initParam) {
+  deviceId_ = initParam.deviceId;
+  APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices();
+  if (ret != APP_ERR_OK) {
+    LogError << "Init devices failed, ret=" << ret << ".";
+    return ret;
+  }
+  ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId);
+  if (ret != APP_ERR_OK) {
+    LogError << "Set context failed, ret=" << ret << ".";
+    return ret;
+  }
+  dvppWrapper_ = std::make_shared<MxBase::DvppWrapper>();
+  ret = dvppWrapper_->Init();
+  if (ret != APP_ERR_OK) {
+    LogError << "DvppWrapper init failed, ret=" << ret << ".";
+    return ret;
+  }
+  model_ = std::make_shared<MxBase::ModelInferenceProcessor>();
+  ret = model_->Init(initParam.modelPath, modelDesc_);
+  if (ret != APP_ERR_OK) {
+    LogError << "ModelInferenceProcessor init failed, ret=" << ret << ".";
+    return ret;
+  }
+
+  PrintTensorShape(modelDesc_.inputTensors, "Model Input Tensors");
+  PrintTensorShape(modelDesc_.outputTensors, "Model Output Tensors");
+
+  MxBase::ConfigData configData;
+  const std::string checkTensor = initParam.checkTensor ? "true" : "false";
+
+  configData.SetJsonValue("CLASS_NUM", std::to_string(initParam.classNum));
+  configData.SetJsonValue("SCORE_THRESH",
+                          std::to_string(initParam.scoreThresh));
+  configData.SetJsonValue("IOU_THRESH", std::to_string(initParam.iouThresh));
+  configData.SetJsonValue("CHECK_MODEL", checkTensor);
+
+  auto jsonStr = configData.GetCfgJson().serialize();
+  std::map<std::string, std::shared_ptr<void>> config;
+  config["postProcessConfigContent"] = std::make_shared<std::string>(jsonStr);
+  post_ = std::make_shared<MxBase::FairmotMindsporePost>();
+  ret = post_->Init(config);
+  if (ret != APP_ERR_OK) {
+    LogError << "Fairmot init failed, ret=" << ret << ".";
+    return ret;
+  }
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::DeInit() {
+  dvppWrapper_->DeInit();
+  model_->DeInit();
+  post_->DeInit();
+  MxBase::DeviceManager::GetInstance()->DestroyDevices();
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::Inference(const std::vector<MxBase::TensorBase> &inputs,
+                             std::vector<MxBase::TensorBase> &outputs) {
+  auto dtypes = model_->GetOutputDataType();
+  for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) {
+    std::vector<uint32_t> shape = {};
+    for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) {
+      shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]);
+    }
+    MxBase::TensorBase tensor(shape, dtypes[i],
+                              MxBase::MemoryData::MemoryType::MEMORY_DEVICE,
+                              deviceId_);
+    APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor);
+    if (ret != APP_ERR_OK) {
+      LogError << "TensorBaseMalloc failed, ret=" << ret << ".";
+      return ret;
+    }
+    outputs.push_back(tensor);
+  }
+  MxBase::DynamicInfo dynamicInfo = {};
+  dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH;
+  auto startTime = std::chrono::high_resolution_clock::now();
+  APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo);
+  auto endTime = std::chrono::high_resolution_clock::now();
+  double costMs = std::chrono::duration<double, std::milli>(endTime - startTime)
+                      .count();  // save time
+  inferCostTimeMilliSec += costMs;
+  if (ret != APP_ERR_OK) {
+    LogError << "ModelInference failed, ret=" << ret << ".";
+    return ret;
+  }
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::PostProcess(const std::vector<MxBase::TensorBase> &inputs,
+                               MxBase::JDETracker &tracker,
+                               MxBase::Files &file) {
+  APP_ERROR ret = post_->Process(inputs, tracker, file);
+  if (ret != APP_ERR_OK) {
+    LogError << "Process failed, ret=" << ret << ".";
+    return ret;
+  }
+  return APP_ERR_OK;
+}
+
+void SaveInferResult(const std::vector<MxBase::ObjectInfo> &objInfos,
+                     const std::string &resultPath) {
+  if (objInfos.empty()) {
+    LogWarn << "The predict result is empty.";
+    return;
+  }
+
+  namespace pt = boost::property_tree;
+  pt::ptree root, data;
+  int index = 0;
+  for (auto &obj : objInfos) {
+    ++index;
+    LogInfo << "BBox[" << index << "]:[x0=" << obj.x0 << ", y0=" << obj.y0
+            << ", x1=" << obj.x1 << ", y1=" << obj.y1
+            << "], confidence=" << obj.confidence << ", classId=" << obj.classId
+            << ", className=" << obj.className << std::endl;
+    pt::ptree item;
+    item.put("classId", obj.classId);
+    item.put("className", obj.className);
+    item.put("confidence", obj.confidence);
+    item.put("x0", obj.x0);
+    item.put("y0", obj.y0);
+    item.put("x1", obj.x1);
+    item.put("y1", obj.y1);
+
+    data.push_back(std::make_pair("", item));
+  }
+  root.add_child("data", data);
+  pt::json_parser::write_json(resultPath, root, std::locale(), false);
+}
+std::string RealPath(std::string path) {
+  char realPathMem[PATH_MAX] = {0};
+  char *realPathRet = nullptr;
+  realPathRet = realpath(path.data(), realPathMem);
+  if (realPathRet == nullptr) {
+    std::cout << "File: " << path << " is not exist.";
+    return "";
+  }
+
+  std::string realPath(realPathMem);
+  std::cout << path << " realpath is: " << realPath << std::endl;
+  return realPath;
+}
+
+DIR *OpenDir(std::string dirName) {
+  if (dirName.empty()) {
+    std::cout << " dirName is null ! " << std::endl;
+    return nullptr;
+  }
+  std::string realPath = RealPath(dirName);
+  struct stat s;
+  lstat(realPath.c_str(), &s);
+  if (!S_ISDIR(s.st_mode)) {
+    std::cout << "dirName is not a valid directory !" << std::endl;
+    return nullptr;
+  }
+  DIR *dir = opendir(realPath.c_str());
+  if (dir == nullptr) {
+    std::cout << "Can not open dir " << dirName << std::endl;
+    return nullptr;
+  }
+  std::cout << "Successfully opened the dir " << dirName << std::endl;
+  return dir;
+}
+
+std::vector<std::string> GetAllFiles(std::string dirName) {
+  struct dirent *filename;
+  DIR *dir = OpenDir(dirName);
+  if (dir == nullptr) {
+    return {};
+  }
+  std::vector<std::string> res;
+  while ((filename = readdir(dir)) != nullptr) {
+    std::string dName = std::string(filename->d_name);
+    if (dName == "." || dName == ".." || filename->d_type != DT_REG) {
+      continue;
+    }
+    res.emplace_back(std::string(dirName) + "/" + filename->d_name);
+  }
+  std::sort(res.begin(), res.end());
+  return res;
+}
+
+APP_ERROR Fairmot::ReadImageCV(const std::string &imgPath, cv::Mat &imageMat,
+                               ImageShape &imgShape) {
+  imageMat = cv::imread(imgPath, cv::IMREAD_COLOR);
+  imgShape.width = imageMat.cols;
+  imgShape.height = imageMat.rows;
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::ResizeImage(const cv::Mat &srcImageMat, cv::Mat &dstImageMat,
+                               ImageShape &imgShape) {
+  int height = 608;
+  int width = 1088;
+  float ratio = std::min(static_cast<float>(height) / srcImageMat.rows,
+                         static_cast<float>(width) / srcImageMat.cols);
+  std::vector<int> new_shape = {
+      static_cast<int>(round(srcImageMat.rows * ratio)),
+      static_cast<int>(round(srcImageMat.cols * ratio))};
+  int tmp = 2;
+  float dw = static_cast<float>((width - new_shape[1])) / tmp;
+  float dh = static_cast<float>((height - new_shape[0])) / tmp;
+  int top = round(dh - 0.1);
+  int bottom = round(dh + 0.1);
+  int left = round(dw - 0.1);
+  int right = round(dw + 0.1);
+  cv::Mat tmp_img;
+  cv::resize(srcImageMat, tmp_img, cv::Size(new_shape[1], new_shape[0]), 0, 0,
+             cv::INTER_AREA);
+  cv::Scalar value = cv::Scalar(127.5, 127.5, 127.5);
+  cv::copyMakeBorder(tmp_img, dstImageMat, top, bottom, left, right,
+                     cv::BORDER_CONSTANT, value);
+  imgShape.width = dstImageMat.cols;
+  imgShape.height = dstImageMat.rows;
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::CVMatToTensorBase(const cv::Mat &imageMat,
+                                     MxBase::TensorBase &tensorBase) {
+  const uint32_t dataSize = imageMat.cols * imageMat.rows * imageMat.channels();
+  MxBase::MemoryData memoryDataDst(dataSize, MxBase::MemoryData::MEMORY_DEVICE,
+                                   deviceId_);
+  MxBase::MemoryData memoryDataSrc(imageMat.data, dataSize,
+                                   MxBase::MemoryData::MEMORY_HOST_MALLOC);
+
+  APP_ERROR ret =
+      MxBase::MemoryHelper::MxbsMallocAndCopy(memoryDataDst, memoryDataSrc);
+  if (ret != APP_ERR_OK) {
+    LogError << GetError(ret) << "Memory malloc failed.";
+    return ret;
+  }
+  std::vector<uint32_t> shape = {static_cast<uint32_t>(imageMat.rows),
+                                 static_cast<uint32_t>(imageMat.cols),
+                                 static_cast<uint32_t>(imageMat.channels())};
+  tensorBase = MxBase::TensorBase(memoryDataDst, false, shape,
+                                  MxBase::TENSOR_DTYPE_UINT8);
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::GetMetaMap(const ImageShape imgShape,
+                              const ImageShape resizeimgShape,
+                              MxBase::JDETracker &tracker) {
+  std::vector<float> c = {static_cast<float>(imgShape.width) / 2,
+                          static_cast<float>(imgShape.height) / 2};
+  float s = std::max(static_cast<float>(resizeimgShape.width) /
+                         static_cast<float>(resizeimgShape.height) *
+                         static_cast<float>(imgShape.height),
+                     static_cast<float>(imgShape.width)) *
+            1.0;
+  tracker.c = c;
+  tracker.s = s;
+  int tmp = 4;
+  tracker.out_height = resizeimgShape.height / tmp;
+  tracker.out_width = resizeimgShape.width / tmp;
+  return APP_ERR_OK;
+}
+
+void Fairmot::WriteResult(const std::string &result_filename,
+                          std::vector<MxBase::Results *> results) {
+  FILE *fp;
+  fp = std::fopen(result_filename.c_str(), "w");
+  for (int i = 0; i < results.size(); i++) {
+    std::vector<cv::Mat> online_tlwhs = (*results[i]).online_tlwhs;
+    std::vector<int> online_ids = (*results[i]).online_ids;
+    int frame_id = (*results[i]).frame_id;
+    for (int j = 0; j < online_tlwhs.size(); j++) {
+      if (online_ids[j] < 0) {
+        continue;
+      }
+      double x1, y1, w, h;
+      x1 = online_tlwhs[j].at<double>(0, 0);
+      y1 = online_tlwhs[j].at<double>(0, 1);
+      w = online_tlwhs[j].at<double>(0, 2);
+      h = online_tlwhs[j].at<double>(0, 3);
+      double x2, y2;
+      x2 = x1 + w;
+      y2 = y1 + h;
+      fprintf(fp, "%d,%d,%.13lf,%.13lf,%.13lf,%.13lf,%d,%d,%d,%d\n", frame_id,
+              (online_ids[j]), x1, y1, w, h, 1, -1, -1, -1);
+    }
+  }
+  fclose(fp);
+}
+
+APP_ERROR Fairmot::Process(const std::string &imgPath) {
+  ImageShape imageShape{};
+  ImageShape resizedImageShape{};
+  std::vector<std::string> seqs = {"/MOT20-01", "/MOT20-02", "/MOT20-03",
+                                   "/MOT20-05"};
+  std::string homePath = imgPath + "/result_Files";
+  if (access(homePath.c_str(), 0) != 0) {
+    std::string cmd = "mkdir " + homePath;
+    system(cmd.data());
+  }
+  for (const auto &seq : seqs) {
+    std::string result_filename = homePath + seq + ".txt";
+    std::string image_path = imgPath + "/train" + seq + "/img1";
+    std::vector<std::string> images = GetAllFiles(image_path);
+    int frame_rate = 25;
+    MxBase::JDETracker tracker(frame_rate);
+    MxBase::Files file;
+    for (const auto &image_file : images) {
+      LogInfo << image_file;
+      int tmp = 20;
+      if (file.frame_id % tmp == 0) {
+        LogInfo << "Processing frame " << file.frame_id;
+      }
+      cv::Mat imageMat;
+      APP_ERROR ret = ReadImageCV(image_file, imageMat, imageShape);
+      if (ret != APP_ERR_OK) {
+        LogError << "ReadImage failed, ret=" << ret << ".";
+        return ret;
+      }
+      ret = ResizeImage(imageMat, imageMat, resizedImageShape);
+      if (ret != APP_ERR_OK) {
+        LogError << "ResizeImage failed, ret=" << ret << ".";
+        return ret;
+      }
+      ret = GetMetaMap(imageShape, resizedImageShape, tracker);
+      if (ret != APP_ERR_OK) {
+        LogError << "GetMetaMap failed, ret=" << ret << ".";
+        return ret;
+      }
+      MxBase::TensorBase tensorBase;
+      ret = CVMatToTensorBase(imageMat, tensorBase);
+      if (ret != APP_ERR_OK) {
+        LogError << "CVMatToTensorBase failed, ret=" << ret << ".";
+        return ret;
+      }
+      std::vector<MxBase::TensorBase> inputs = {};
+      std::vector<MxBase::TensorBase> outputs = {};
+      inputs.push_back(tensorBase);
+      auto startTime = std::chrono::high_resolution_clock::now();
+      ret = Inference(inputs, outputs);
+      auto endTime = std::chrono::high_resolution_clock::now();
+      double costMs =
+          std::chrono::duration<double, std::milli>(endTime - startTime)
+              .count();  // save time
+      inferCostTimeMilliSec += costMs;
+      if (ret != APP_ERR_OK) {
+        LogError << "Inference failed, ret=" << ret << ".";
+        return ret;
+      }
+      tracker.seq = seq;
+      tracker.image_file = image_file;
+      ret = PostProcess(outputs, tracker, file);
+      if (ret != APP_ERR_OK) {
+        LogError << "PostProcess failed, ret=" << ret << ".";
+        return ret;
+      }
+    }
+    WriteResult(result_filename, file.results);
+  }
+  return APP_ERR_OK;
+}

research/cv/fairmot/infer/mxbase/src/Fairmot.h

+/*
+ * Copyright 2021 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include <memory>
+#include <string>
+#include <vector>
+#include <opencv2/opencv.hpp>
+
+#include "MxBase/DeviceManager/DeviceManager.h"
+#include "MxBase/DvppWrapper/DvppWrapper.h"
+#include "MxBase/ModelInfer/ModelInferenceProcessor.h"
+#include "MxBase/PostProcessBases/ObjectPostProcessBase.h"
+#include "MxBase/Tensor/TensorContext/TensorContext.h"
+#include "PostProcess/FairmotMindsporePost.h"
+
+struct InitParam {
+  uint32_t deviceId;
+  std::string labelPath;
+  uint32_t classNum;
+  float iouThresh;
+  float scoreThresh;
+
+  bool checkTensor;
+  std::string modelPath;
+};
+
+struct ImageShape {
+  uint32_t width;
+  uint32_t height;
+};
+
+class Fairmot {
+ public:
+  APP_ERROR Init(const InitParam &initParam);
+  APP_ERROR DeInit();
+  APP_ERROR Inference(const std::vector<MxBase::TensorBase> &inputs,
+                      std::vector<MxBase::TensorBase> &outputs);
+  APP_ERROR PostProcess(const std::vector<MxBase::TensorBase> &inputs,
+                        MxBase::JDETracker &tracker, MxBase::Files &file);
+  APP_ERROR Process(const std::string &imgPath);
+  APP_ERROR ReadImageCV(const std::string &imgPath, cv::Mat &imageMat,
+                        ImageShape &imgShape);
+  APP_ERROR ResizeImage(const cv::Mat &srcImageMat, cv::Mat &dstImageMat,
+                        ImageShape &imgShape);
+  APP_ERROR CVMatToTensorBase(const cv::Mat &imageMat,
+                              MxBase::TensorBase &tensorBase);
+  APP_ERROR GetMetaMap(const ImageShape imgShape,
+                       const ImageShape resizeimgShape,
+                       MxBase::JDETracker &tracker);
+  void WriteResult(const std::string &result_filename,
+                   std::vector<MxBase::Results *> results);
+
+ private:
+  std::shared_ptr<MxBase::DvppWrapper> dvppWrapper_;
+  std::shared_ptr<MxBase::ModelInferenceProcessor> model_;
+  std::shared_ptr<MxBase::FairmotMindsporePost> post_;
+  MxBase::ModelDesc modelDesc_;
+  uint32_t deviceId_ = 0;
+  double inferCostTimeMilliSec = 0.0;
+};

research/cv/fairmot/infer/mxbase/src/PostProcess/FairmotMindsporePost.cpp

+/*
+ * Copyright 2021 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#define BOOST_BIND_GLOBAL_PLACEHOLDERS
+#include "FairmotMindsporePost.h"
+
+#include <dirent.h>
+#include <float.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <iostream>
+#include <memory>
+#include <string>
+#include <boost/property_tree/json_parser.hpp>
+#include <opencv2/core/hal/hal.hpp>
+#include <opencv2/opencv.hpp>
+
+#include "MxBase/CV/ObjectDetection/Nms/Nms.h"
+#include "acl/acl.h"
+
+namespace {
+// Output Tensor
+const int OUTPUT_TENSOR_SIZE = 2;
+const int OUTPUT_ID_FEATURE_INDEX = 0;
+const int OUTPUT_DETS_INDEX = 1;
+
+const int OUTPUT_ID_FEATURE_SIZE = 2;
+const int OUTPUT_DETS_SIZE = 3;
+const float CONF_THRES = 0.3;
+const int TRACK_BUFFER = 30;
+const int K = 500;
+const int TrackState_New = 0;
+const int TrackState_Tracked = 1;
+const int TrackState_Lost = 2;
+const int TrackState_Removed = 3;
+const int MIN_BOX_AREA = 100;
+}  // namespace
+
+namespace MxBase {
+BaseTrack basetrack;
+KalmanFilter kalmanfilter;
+void FairmotMindsporePost::get_result(int *collist, int cols, cost *d, cost min,
+                                      int rows, int &endofpath, row *colsol,
+                                      cost **assigncost, cost *v, int *pred) {
+  col low = 0, up = 0;  // columns in 0..low-1 are ready, now none.
+  int k, j, i, j1;
+  cost h, v2;
+  boolean unassignedfound = FALSE;
+  do {
+    if (up == low) {
+      min = d[collist[up++]];
+      for (k = up; k < cols; k++) {
+        j = collist[k];
+        h = d[j];
+        if (h <= min) {
+          if (h < min) {
+            up = low;  // restart list at index low.
+            min = h;
+          }
+          collist[k] = collist[up];
+          collist[up++] = j;
+        }
+      }
+      for (k = low; k < up; k++)
+        if (colsol[collist[k]] < 0) {
+          endofpath = collist[k];
+          unassignedfound = TRUE;
+          break;
+        }
+    }
+    if (!unassignedfound) {
+      j1 = collist[low];
+      low++;
+      i = colsol[j1];
+      if (i > rows) {
+        i = 0;
+      }
+      h = assigncost[i][j1] - v[j1] - min;
+      for (k = up; k < cols; k++) {
+        j = collist[k];
+        v2 = assigncost[i][j] - v[j] - h;
+        if (v2 < d[j]) {
+          pred[j] = i;
+          if (v2 == min) {
+            if (colsol[j] < 0) {
+              endofpath = j;
+              unassignedfound = TRUE;
+              break;
+            } else {
+              collist[k] = collist[up];
+              collist[up++] = j;
+            }
+          }
+          d[j] = v2;
+        }
+      }
+    }
+  } while (!unassignedfound);
+}
+void FairmotMindsporePost::func(int &numfree, int *free, cost **assigncost,
+                                row *colsol, cost *v, col *rowsol, int cols) {
+  int j2, loopcnt = 0;  // do-loop to be done twice.
+  do {
+    loopcnt++;
+    int k = 0;
+    row prvnumfree;
+    prvnumfree = numfree;
+    numfree = 0;
+    while (k < prvnumfree) {
+      int i = free[k++];
+      int umin = assigncost[i][0] - v[0];
+      int j1 = 0;
+      int usubmin = BIG;
+      for (int j = 1; j < cols; j++) {
+        int h = assigncost[i][j] - v[j];
+        if (h < usubmin) {
+          if (h >= umin) {
+            usubmin = h;
+            j2 = j;
+          } else {
+            usubmin = umin;
+            umin = h;
+            j2 = j1;
+            j1 = j;
+          }
+        }
+      }
+      int i0 = colsol[j1];
+      if (umin < usubmin) {
+        v[j1] = v[j1] - (usubmin - umin);
+      } else {
+        if (i0 > -1) {
+          j1 = j2;
+          i0 = colsol[j2];
+        }
+      }
+      rowsol[i] = j1;
+      colsol[j1] = i;
+      if (i0 > -1) {
+        if (umin < usubmin) {
+          free[--k] = i0;
+        } else {
+          free[numfree++] = i0;
+        }
+      }
+    }
+  } while (loopcnt < 2);  // repeat once.
+}
+
+void FairmotMindsporePost::lap(cost **assigncost, col *rowsol, row *colsol,
+                               cost *u, cost *v, int rows, int cols) {
+  int i, numfree = 0, f, *pred = new row[rows], *free = new row[rows], j, j1,
+         endofpath, *collist = new col[cols], *matches = new col[cols];
+  cost min, *d = new cost[rows];
+  for (i = 0; i < rows; i++) matches[i] = 0;
+  for (j = cols; j--;) {  // reverse order gives better results.
+    row imin = 0;
+    min = assigncost[0][j];
+    for (i = 1; i < rows; i++)
+      if (assigncost[i][j] < min) {
+        min = assigncost[i][j];
+        imin = i;
+      }
+    v[j] = min;
+    if (++matches[imin] == 1) {
+      rowsol[imin] = j;
+      colsol[j] = imin;
+    } else if (v[j] < v[rowsol[imin]]) {
+      int j_1 = rowsol[imin];
+      rowsol[imin] = j;
+      colsol[j] = imin;
+      colsol[j_1] = -1;
+    } else {
+      colsol[j] = -1;
+    }
+  }
+  for (i = 0; i < rows; i++)
+    if (matches[i] == 0) {
+      free[numfree++] = i;
+    } else if (matches[i] == 1) {
+      j1 = rowsol[i];
+      min = BIG;
+      for (j = 0; j < cols; j++)
+        if (j != j1)
+          if (assigncost[i][j] - v[j] < min) min = assigncost[i][j] - v[j];
+      v[j1] = v[j1] - min;
+    }
+  func(numfree, free, assigncost, colsol, v, rowsol, cols);
+  for (f = 0; f < numfree; f++) {
+    row freerow;
+    freerow = free[f];  // start row of augmenting path.
+    for (j = cols; j--;) {
+      d[j] = assigncost[freerow][j] - v[j];
+      pred[j] = freerow;
+      collist[j] = j;  // init column list.
+    }
+    get_result(collist, cols, d, min, rows, endofpath, colsol, assigncost, v,
+               pred);
+    do {
+      i = pred[endofpath];
+      colsol[endofpath] = i;
+      j1 = endofpath;
+      endofpath = rowsol[i];
+      rowsol[i] = j1;
+    } while (i != freerow);
+  }
+  delete[] matches;
+}
+
+int BaseTrack::next_id() {
+  this->count += 1;
+  return this->count;
+}
+
+Results::Results(uint32_t frame_id, const std::vector<cv::Mat> &online_tlwhs,
+                 const std::vector<int> &online_ids) {
+  this->frame_id = frame_id;
+  this->online_tlwhs = online_tlwhs;
+  this->online_ids = online_ids;
+}
+
+JDETracker::JDETracker(uint32_t frame_rate) {
+  this->det_thresh = CONF_THRES;
+  this->buffer_size = static_cast<int>(frame_rate / 30.0 * TRACK_BUFFER);
+  this->max_time_lost = this->buffer_size;
+  this->max_per_image = K;
+  KalmanFilter kalman_filter;
+  this->kalman_filter = kalman_filter;
+}
+STack::STack() {}
+STack::STack(cv::Mat tlwh, float score, cv::Mat temp_feat,
+             uint32_t buffer_size) {
+  tlwh.convertTo(this->tlwh, CV_64FC1);
+  this->is_activated = false;
+  this->score = score;
+  this->tracklet_len = 0;
+  this->update_features(temp_feat);
+  this->alpha = 0.9;
+}
+cv::Mat STack::gettlwh() {
+  if (this->mean.rows == 0) {
+    return this->tlwh.clone();
+  } else {
+    cv::Mat ret =
+        this->mean(cv::Range(0, this->mean.rows), cv::Range(0, 4)).clone();
+    ret.at<double>(0, 2) *= ret.at<double>(0, 3);
+    for (size_t i = 0; i < 2; i++) {
+      ret.at<double>(0, i) -= ret.at<double>(0, i + 2) / 2;
+    }
+    return ret;
+  }
+}
+cv::Mat STack::tlbr() {
+  cv::Mat ret = this->gettlwh();
+  for (size_t i = 0; i < 2; i++) {
+    ret.at<double>(0, i + 2) += ret.at<double>(0, i);
+  }
+  return ret;
+}
+cv::Mat STack::tlwh_to_xyah(cv::Mat tlwh) {
+  cv::Mat ret = tlwh;
+  for (size_t i = 0; i < 2; i++) {
+    ret.at<double>(0, i) += ret.at<double>(0, i + 2) / 2;
+  }
+  ret.at<double>(0, 2) /= ret.at<double>(0, 3);
+  return ret;
+}
+void STack::activate(const KalmanFilter &kalman_filter, uint32_t frame_id) {
+  this->kalman_filter = kalman_filter;
+  this->track_id = basetrack.next_id();
+  this->kalman_filter.initiate(this->tlwh_to_xyah(this->tlwh), this->mean,
+                               this->covariance);
+  this->tracklet_len = 0;
+  this->state = TrackState_Tracked;
+  this->is_activated = false;
+  if (frame_id == 1) {
+    this->is_activated = true;
+  }
+  this->frame_id = frame_id;
+  this->start_frame = frame_id;
+}
+void STack::update_features(cv::Mat temp_feat) {
+  cv::Mat feat;
+  cv::normalize(temp_feat, feat);
+  this->curr_feat = feat;
+  if (this->smooth_feat.empty()) {
+    this->smooth_feat = feat;
+  } else {
+    this->smooth_feat =
+        this->alpha * this->smooth_feat + (1 - this->alpha) * feat;
+  }
+  this->features.push_back(feat);
+  cv::normalize(this->smooth_feat, this->smooth_feat);
+}
+
+void STack::update(STack new_track, int frame_id, bool update_feature) {
+  this->frame_id = frame_id;
+  this->tracklet_len += 1;
+  cv::Mat new_tlwh = new_track.gettlwh();
+  this->kalman_filter.update(this->mean, this->covariance,
+                             this->tlwh_to_xyah(new_tlwh));
+  this->state = TrackState_Tracked;
+  this->is_activated = true;
+  this->score = new_track.score;
+  if (update_feature == true) {
+    this->update_features(new_track.curr_feat);
+  }
+}
+
+void STack::re_activate(STack new_track, int frame_id, bool new_id) {
+  this->kalman_filter.update(this->mean, this->covariance,
+                             this->tlwh_to_xyah(new_track.gettlwh()));
+  this->update_features(new_track.curr_feat);
+  this->tracklet_len = 0;
+  this->state = TrackState_Tracked;
+  this->is_activated = true;
+  this->frame_id = frame_id;
+  if (new_id) {
+    this->track_id = basetrack.next_id();
+  }
+}
+
+void KalmanFilter::initiate(cv::Mat measurement, cv::Mat &mean,
+                            cv::Mat &covariance) {
+  cv::Mat mean_pos = measurement;
+  cv::Mat mean_vel = cv::Mat::zeros(mean_pos.rows, mean_pos.cols, CV_64FC1);
+  hconcat(mean_pos, mean_vel, mean);
+  double tmp[1][8] = {
+      2 * this->std_weight_position * measurement.at<double>(0, 3),
+      2 * this->std_weight_position * measurement.at<double>(0, 3),
+      1e-2,
+      2 * this->std_weight_position * measurement.at<double>(0, 3),
+      10 * this->std_weight_velocity * measurement.at<double>(0, 3),
+      10 * this->std_weight_velocity * measurement.at<double>(0, 3),
+      1e-5,
+      10 * this->std_weight_velocity * measurement.at<double>(0, 3)};
+  cv::Mat std = cv::Mat(1, 8, CV_64FC1, tmp);
+  std = std.mul(std);
+  covariance = cv::Mat::eye(std.cols, std.cols, CV_64FC1);
+  for (size_t i = 0; i < std.cols; i++)
+    covariance.at<double>(i, i) = std.at<double>(0, i);
+}
+void KalmanFilter::multi_predict(cv::Mat &mean,
+                                 std::vector<cv::Mat> &covariance) {
+  cv::Mat std_pos(4, mean.rows, CV_64FC1);
+  cv::Mat std_vel(4, mean.rows, CV_64FC1);
+  for (size_t i = 0; i < 4; i++)
+    for (size_t j = 0; j < mean.rows; j++)
+      if (i == 2) {
+        std_pos.at<double>(i, j) = 1e-2;
+        std_vel.at<double>(i, j) = 1e-5;
+      } else {
+        std_pos.at<double>(i, j) =
+            this->std_weight_position * mean.at<double>(j, 3);
+        std_vel.at<double>(i, j) =
+            this->std_weight_velocity * mean.at<double>(j, 3);
+      }
+  cv::Mat sqr;
+  vconcat(std_pos, std_vel, sqr);
+  sqr = sqr.mul(sqr).t();
+  std::vector<cv::Mat> motion_cov;
+  for (size_t i = 0; i < mean.rows; i++) {
+    cv::Mat diag = cv::Mat::eye(sqr.cols, sqr.cols, CV_64FC1);
+    for (size_t j = 0; j < sqr.cols; j++) {
+      diag.at<double>(j, j) = sqr.at<double>(i, j);
+    }
+    motion_cov.push_back(diag);
+  }
+  mean = mean * this->motion_mat.t();
+  std::vector<cv::Mat> left;
+  for (size_t i = 0; i < covariance.size(); i++) {
+    left.push_back(this->motion_mat * covariance[i]);
+  }
+  for (size_t i = 0; i < covariance.size(); i++) {
+    covariance[i] = left[i] * this->motion_mat.t() + motion_cov[i];
+  }
+}
+KalmanFilter::KalmanFilter() {
+  this->ndim = 4;
+  this->dt = 1.0;
+  this->motion_mat = cv::Mat::eye(2 * (this->ndim), 2 * (this->ndim), CV_64FC1);
+  for (size_t i = 0; i < this->ndim; ++i) {
+    this->motion_mat.at<double>(i, this->ndim + i) = this->dt;
+  }
+  this->update_mat = cv::Mat::eye(this->ndim, 2 * (this->ndim), CV_64FC1);
+  this->std_weight_position = 1.0 / 20;
+  this->std_weight_velocity = 1.0 / 160;
+}
+cv::Mat KalmanFilter::GatingDistance(cv::Mat mean, cv::Mat covariance,
+                                     cv::Mat measurements, bool only_position,
+                                     const std::string &metric) {
+  this->project(mean, covariance);
+  cv::Mat d(measurements.rows, measurements.cols, CV_64FC1);
+  for (size_t i = 0; i < measurements.rows; i++) {
+    d.row(i) = measurements.row(i) - mean;
+  }
+  cv::Mat cholesky_factor = cv::Mat::zeros(covariance.size(), CV_64F);
+  for (int i = 0; i < covariance.rows; ++i) {
+    int j;
+    double sum;
+    for (j = 0; j < i; ++j) {
+      sum = 0;
+      for (int k = 0; k < j; ++k) {
+        sum +=
+            cholesky_factor.at<double>(i, k) * cholesky_factor.at<double>(j, k);
+      }
+      cholesky_factor.at<double>(i, j) = (covariance.at<double>(i, j) - sum) /
+                                         cholesky_factor.at<double>(j, j);
+    }
+    sum = 0;
+    assert(i == j);
+    for (int k = 0; k < j; ++k) {
+      sum +=
+          cholesky_factor.at<double>(j, k) * cholesky_factor.at<double>(j, k);
+    }
+    cholesky_factor.at<double>(j, j) = sqrt(covariance.at<double>(j, j) - sum);
+  }
+  cv::Mat z;
+  cv::solve(cholesky_factor, d.t(), z);
+  z = z.mul(z);
+  cv::Mat squared_maha(1, z.cols, CV_64FC1);
+  for (size_t i = 0; i < z.cols; i++) {
+    double sum = 0;
+    for (size_t t = 0; t < z.rows; t++) {
+      sum += z.at<double>(t, i);
+    }
+    squared_maha.at<double>(0, i) = sum;
+  }
+  return squared_maha;
+}
+
+void KalmanFilter::project(cv::Mat &mean, cv::Mat &covariance) {
+  cv::Mat std(1, 4, CV_64FC1);
+  for (size_t i = 0; i < 4; i++)
+    if (i == 2)
+      std.at<double>(0, i) = 1e-1;
+    else
+      std.at<double>(0, i) = this->std_weight_position * mean.at<double>(0, 3);
+  std = std.mul(std);
+  cv::Mat innovation_cov = cv::Mat::eye(std.cols, std.cols, CV_64FC1);
+  for (size_t j = 0; j < std.cols; j++) {
+    innovation_cov.at<double>(j, j) = std.at<double>(0, j);
+  }
+  cv::Mat tmp(mean.rows, this->update_mat.rows, CV_64FC1);
+  for (size_t i = 0; i < this->update_mat.rows; i++) {
+    tmp.at<double>(0, i) = this->update_mat.row(i).dot(mean);
+  }
+  mean = tmp;
+  covariance =
+      this->update_mat * covariance * this->update_mat.t() + innovation_cov;
+}
+
+void KalmanFilter::update(cv::Mat &mean, cv::Mat &covariance,
+                          cv::Mat measurement) {
+  cv::Mat projected_mean = mean.clone();
+  cv::Mat projected_cov = covariance.clone();
+  this->project(projected_mean, projected_cov);
+  cv::Mat chol_factor;
+  this->cholesky_decomposition(projected_cov, chol_factor);
+  cv::Mat b = covariance * this->update_mat.t();
+  b = b.t();
+  cv::Mat kalman_gain(chol_factor.rows, b.cols, CV_64FC1);
+  for (size_t i = 0; i < b.cols; i++) {
+    f64_vec_t x = {0, 0, 0, 0};
+    f64_vec_t *f_x = &x;
+    cv::Mat mat_b(b.rows, 1, CV_64FC1);
+    mat_b = b.col(i);
+    this->chol_subtitute(chol_factor, mat_b, f_x, chol_factor.rows);
+    for (size_t j = 0; j < chol_factor.rows; j++) {
+      if ((*f_x)[j] < 0.001)
+        kalman_gain.at<double>(j, i) = 0;
+      else
+        kalman_gain.at<double>(j, i) = (*f_x)[j];
+    }
+  }
+  kalman_gain = kalman_gain.t();
+  cv::Mat innovation = measurement - projected_mean;
+  mean += innovation * kalman_gain.t();
+  covariance -= kalman_gain * projected_cov * kalman_gain.t();
+}
+
+void KalmanFilter::chol_subtitute(cv::Mat chol_factor, cv::Mat mat_b,
+                                  f64_vec_t *f_x, int n) {
+  f64_mat_t L;
+  f64_vec_t b;
+  for (int i = 0; i < n; ++i)
+    for (int j = 0; j < n; ++j) L[i][j] = chol_factor.at<double>(i, j);
+  for (int i = 0; i < n; ++i) b[i] = mat_b.at<double>(i, 0);
+  f64_mat_t *f_L = &L;
+  f64_vec_t *f_b = &b;
+  int i, j;
+  double f_sum;
+
+  double *pX;
+  double *pXj;
+  const double *pL;
+  const double *pB;
+  /** @llr - This function shall solve the unknown vector f_x given a matrix f_L
+   * and a vector f_b with the relation f_L*fL'*f_x = f_b.*/
+
+  pX = &(*f_x)[0];
+  pB = &(*f_b)[0];
+  /* Copy f_b into f_x */
+  for (i = 0u; i < n; i++) {
+    (*pX) = (*pB);
+    pX++;
+    pB++;
+  }
+  /* Solve Ly = b  for y */
+  pXj = &(*f_x)[0];
+  for (i = 0u; i < n; i++) {
+    double *pXi;
+    double fLii;
+    f_sum = (*f_x)[i];
+    fLii = (*f_L)[i][i];
+    pXi = &(*f_x)[0];
+    pL = &(*f_L)[i][0];
+
+    for (j = 0u; j < i; j++) {
+      f_sum -= (*pL) * (*pXi);
+      pL++;
+      pXi++;
+    }
+    (*pXj) = f_sum / fLii;
+    pXj++;
+  }
+  /* Solve L'x = y for x */
+  for (i = 1u; i <= n; i++) {
+    f_sum = (*f_x)[n - i];
+    pXj = &(*f_x)[n - i + 1u];
+    pL = &(*f_L)[n - i + 1u][n - i];
+    for (j = n - i + 1u; j < n; j++) {
+      f_sum -= (*pL) * (*pXj);
+      pXj++;
+      pL += n; /* PRQA S 0488 */
+    }
+    (*f_x)[n - i] = f_sum / (*f_L)[n - i][n - i];
+  }
+}
+
+void KalmanFilter::cholesky_decomposition(const cv::Mat &A, cv::Mat &L) {
+  L = cv::Mat::zeros(A.size(), CV_64F);
+  int rows = A.rows;
+
+  for (int i = 0; i < rows; ++i) {
+    int j;
+    float sum;
+
+    for (j = 0; j < i; ++j) {
+      sum = 0;
+      for (int k = 0; k < j; ++k) {
+        sum += L.at<double>(i, k) * L.at<double>(j, k);
+      }
+      L.at<double>(i, j) = (A.at<double>(i, j) - sum) / L.at<double>(j, j);
+    }
+    sum = 0;
+    assert(i == j);
+    for (int k = 0; k < j; ++k) {
+      sum += L.at<double>(j, k) * L.at<double>(j, k);
+    }
+    L.at<double>(j, j) = sqrt(A.at<double>(j, j) - sum);
+  }
+}
+
+FairmotMindsporePost &FairmotMindsporePost::operator=(
+    const FairmotMindsporePost &other) {
+  if (this == &other) {
+    return *this;
+  }
+  PostProcessBase::operator=(other);
+  return *this;
+}
+
+APP_ERROR FairmotMindsporePost::Init(
+    const std::map<std::string, std::shared_ptr<void>> &postConfig) {
+  LogInfo << "Begin to initialize FairmotMindsporePost.";
+  APP_ERROR ret = PostProcessBase::Init(postConfig);
+  if (ret != APP_ERR_OK) {
+    LogError << GetError(ret) << "Fail to superinit  in PostProcessBase.";
+    return ret;
+  }
+
+  LogInfo << "End to initialize FairmotMindsporePost.";
+  return APP_ERR_OK;
+}
+
+APP_ERROR FairmotMindsporePost::DeInit() {
+  LogInfo << "Begin to deinitialize FairmotMindsporePost.";
+  LogInfo << "End to deinitialize FairmotMindsporePost.";
+  return APP_ERR_OK;
+}
+
+bool FairmotMindsporePost::IsValidTensors(
+    const std::vector<TensorBase> &tensors) const {
+  if (tensors.size() < OUTPUT_TENSOR_SIZE) {
+    LogError << "The number of tensor (" << tensors.size()
+             << ") is less than required (" << OUTPUT_TENSOR_SIZE << ")";
+    return false;
+  }
+
+  auto idFeatureShape = tensors[OUTPUT_ID_FEATURE_INDEX].GetShape();
+  if (idFeatureShape.size() != OUTPUT_ID_FEATURE_SIZE) {
+    LogError << "The number of tensor[" << OUTPUT_ID_FEATURE_INDEX
+             << "] dimensions (" << idFeatureShape.size()
+             << ") is not equal to (" << OUTPUT_ID_FEATURE_SIZE << ")";
+    return false;
+  }
+
+  auto detsShape = tensors[OUTPUT_DETS_INDEX].GetShape();
+  if (detsShape.size() != OUTPUT_DETS_SIZE) {
+    LogError << "The number of tensor[" << OUTPUT_DETS_INDEX << "] dimensions ("
+             << detsShape.size() << ") is not equal to ("
+             << OUTPUT_ID_FEATURE_SIZE << ")";
+    return false;
+  }
+
+  return true;
+}
+
+void FairmotMindsporePost::TransformPreds(const cv::Mat &coords,
+                                          MxBase::JDETracker tracker,
+                                          cv::Mat &target_coords) {
+  target_coords = cv::Mat::zeros(coords.rows, coords.cols, CV_32FC1);
+  float scale = tracker.s;
+  uint32_t h = tracker.out_height;
+  uint32_t w = tracker.out_width;
+  float scale_value[1][2] = {scale, scale};
+  cv::Mat scale_tmp = cv::Mat(1, 2, CV_32FC1, scale_value);
+  float src_w = scale_tmp.at<float>(0, 0);
+  uint32_t dst_w = w;
+  uint32_t dst_h = h;
+  float sn = 0.0;
+  float cs = 1.0;
+  uint32_t src_point_0 = 0;
+  float src_point_1 = src_w * (-0.5);
+  float src_dir_value[1][2] = {src_point_0 * cs - src_point_1 * sn,
+                               src_point_0 * sn + src_point_1 * cs};
+  cv::Mat src_dir = cv::Mat(1, 2, CV_32FC1, src_dir_value);
+  float dst_dir_value[1][2] = {0, static_cast<float>(dst_w * (-0.5))};
+  cv::Mat dst_dir = cv::Mat(1, 2, CV_32FC1, dst_dir_value);
+  cv::Mat src = cv::Mat::zeros(3, 2, CV_32FC1);
+  cv::Mat dst = cv::Mat::zeros(3, 2, CV_32FC1);
+  float center_value[1][2] = {tracker.c[0], tracker.c[1]};
+  cv::Mat center = cv::Mat(1, 2, CV_32FC1, center_value);
+  cv::Mat shift = cv::Mat::zeros(1, 2, CV_32FC1);
+  cv::Mat src_0 = scale_tmp.mul(shift) + center;
+  cv::Mat src_1 = scale_tmp.mul(shift) + center + src_dir;
+  cv::Mat direct = src_0 - src_1;
+  float direct_tmp_value[1][2] = {-direct.at<float>(0, 1),
+                                  direct.at<float>(0, 0)};
+  cv::Mat direct_tmp = cv::Mat(1, 2, CV_32FC1, direct_tmp_value);
+  cv::Mat src_2 = src_1 + direct_tmp;
+  float dst_0_value[1][2] = {static_cast<float>(dst_w * (0.5)),
+                             static_cast<float>(dst_h * (0.5))};
+  cv::Mat dst_0 = cv::Mat(1, 2, CV_32FC1, dst_0_value);
+  float dst_1_value[1][2] = {
+      static_cast<float>(dst_w * (0.5)) + dst_dir.at<float>(0, 0),
+      static_cast<float>(dst_h * (0.5)) + dst_dir.at<float>(0, 1)};
+  cv::Mat dst_1 = cv::Mat(1, 2, CV_32FC1, dst_1_value);
+  direct = dst_0 - dst_1;
+  float direct_value[1][2] = {-direct.at<float>(0, 1), direct.at<float>(0, 0)};
+  direct_tmp = cv::Mat(1, 2, CV_32FC1, direct_value);
+  cv::Mat dst_2 = dst_1 + direct_tmp;
+  for (size_t y = 0; y < src.cols; ++y) {
+    src.at<float>(0, y) = src_0.at<float>(0, y);
+    src.at<float>(1, y) = src_1.at<float>(0, y);
+    src.at<float>(2, y) = src_2.at<float>(0, y);
+    dst.at<float>(0, y) = dst_0.at<float>(0, y);
+    dst.at<float>(1, y) = dst_1.at<float>(0, y);
+    dst.at<float>(2, y) = dst_2.at<float>(0, y);
+  }
+  cv::Mat trans(2, 3, CV_32FC1);
+  trans = cv::getAffineTransform(dst, src);
+  trans.convertTo(trans, CV_32F);
+  for (size_t x = 0; x < coords.rows; ++x) {
+    float pt_value[3][1] = {coords.at<float>(x, 0), coords.at<float>(x, 1),
+                            1.0};
+    cv::Mat pt = cv::Mat(3, 1, CV_32FC1, pt_value);
+    for (size_t y = 0; y < 2; ++y) {
+      cv::Mat new_pt = trans * pt;
+      target_coords.at<float>(x, y) = new_pt.at<float>(y);
+    }
+  }
+}
+
+void FairmotMindsporePost::PostProcess(cv::Mat &det,
+                                       const MxBase::JDETracker &tracker) {
+  cv::Mat coords_0 = det(cv::Range(0, det.rows), cv::Range(0, 2));
+  cv::Mat coords_1 = det(cv::Range(0, det.rows), cv::Range(2, 4));
+  cv::Mat target_coords_0;
+  cv::Mat target_coords_1;
+  TransformPreds(coords_0, tracker, target_coords_0);
+  TransformPreds(coords_1, tracker, target_coords_1);
+  for (size_t x = 0; x < det.rows; ++x) {
+    for (size_t y = 0; y < 4; ++y) {
+      if (y < 2) {
+        det.at<float>(x, y) = target_coords_0.at<float>(x, y);
+      } else {
+        det.at<float>(x, y) = target_coords_1.at<float>(x, y - 2);
+      }
+    }
+  }
+  det = det(cv::Range(0, det.rows), cv::Range(0, 5));
+}
+void FairmotMindsporePost::TensorBaseToCVMat(cv::Mat &imageMat,
+                                             const MxBase::TensorBase &tensor) {
+  TensorBase Data = tensor;
+  uint32_t outputModelWidth;
+  uint32_t outputModelHeight;
+  auto shape = Data.GetShape();
+  if (shape.size() == 2) {
+    outputModelWidth = shape[0];
+    outputModelHeight = shape[1];
+  } else {
+    outputModelWidth = shape[1];
+    outputModelHeight = shape[2];
+  }
+  auto *data = reinterpret_cast<float *>(GetBuffer(Data, 0));
+  cv::Mat dataMat(outputModelWidth, outputModelHeight, CV_32FC1);
+  for (size_t x = 0; x < outputModelWidth; ++x) {
+    for (size_t y = 0; y < outputModelHeight; ++y) {
+      dataMat.at<float>(x, y) = data[x * outputModelHeight + y];
+    }
+  }
+  imageMat = dataMat.clone();
+}
+
+std::vector<cv::Mat> FairmotMindsporePost::get_lap(cost **assigncost,
+                                                   col *rowsol, row *colsol,
+                                                   cost *u, cost *v, int row,
+                                                   int col, int dim) {
+  std::vector<cv::Mat> results;
+  lap(assigncost, rowsol, colsol, u, v, dim, dim);
+  cv::Mat x(1, row, CV_32FC1), y(1, col, CV_32FC1);
+  for (int i = 0; i < row; i++)
+    x.at<float>(0, i) = rowsol[i] > (col - 1) ? (-1) : rowsol[i];
+  for (int j = 0; j < col; j++)
+    y.at<float>(0, j) = colsol[j] > (row - 1) ? (-1) : colsol[j];
+  cv::Mat matches(0, 2, CV_32FC1);
+  for (size_t i = 0; i < x.cols; i++) {
+    if (x.at<float>(0, i) >= 0) {
+      cv::Mat tmp(1, 2, CV_32FC1);
+      tmp.at<float>(0, 0) = i;
+      tmp.at<float>(0, 1) = x.at<float>(0, i);
+      matches.push_back(tmp);
+    }
+  }
+  std::vector<int> a, b;
+  for (size_t i = 0; i < x.cols; i++)
+    if (x.at<float>(0, i) < 0) a.push_back(i);
+  for (size_t i = 0; i < y.cols; i++)
+    if (y.at<float>(0, i) < 0) b.push_back(i);
+  cv::Mat unmatched_a(1, a.size(), CV_32FC1),
+      unmatched_b(1, b.size(), CV_32FC1);
+  for (size_t i = 0; i < a.size(); i++) unmatched_a.at<float>(0, i) = a[i];
+  for (size_t i = 0; i < b.size(); i++) unmatched_b.at<float>(0, i) = b[i];
+  results.push_back(matches);
+  results.push_back(unmatched_a);
+  results.push_back(unmatched_b);
+  return results;
+}
+
+std::vector<cv::Mat> FairmotMindsporePost::LinearAssignment(cv::Mat cost_matrix,
+                                                            float thresh) {
+  if (cost_matrix.rows == 0) {
+    std::vector<cv::Mat> results;
+    cv::Mat matches(0, 2, CV_32FC1), u_track,
+        u_detection(1, cost_matrix.cols, CV_32FC1);
+    for (size_t i = 0; i < cost_matrix.cols; ++i)
+      u_detection.at<float>(0, i) = i;
+    results.push_back(matches);
+    results.push_back(u_track);
+    results.push_back(u_detection);
+    return results;
+  } else {
+    int row = cost_matrix.rows, col = cost_matrix.cols;
+    int N = row > col ? row : col;
+    cv::Mat cost_c_extended = cv::Mat::ones(2 * N, 2 * N, CV_64FC1);
+    cost_c_extended *= thresh;
+    if (row != col) {
+      double min = 0, max = 0;
+      double *minp = &min, *maxp = &max;
+      cv::minMaxIdx(cost_matrix, minp, maxp);
+      for (size_t i = 0; i < N; i++)
+        for (size_t j = 0; j < N; j++)
+          cost_c_extended.at<double>(i, j) = (*maxp) + thresh + 1;
+    }
+    for (size_t i = 0; i < row; i++)
+      for (size_t j = 0; j < col; j++)
+        cost_c_extended.at<double>(i, j) = cost_matrix.at<double>(i, j);
+    cost_matrix = cost_c_extended;
+    int dim = 2 * N, *rowsol = new int[dim], *colsol = new int[dim];
+    double **costMatrix = new double *[dim], *u = new double[dim],
+           *v = new double[dim];
+    for (int i = 0; i < dim; i++) costMatrix[i] = new double[dim];
+    for (int i = 0; i < dim; ++i)
+      for (int j = 0; j < dim; ++j)
+        costMatrix[i][j] = cost_matrix.at<double>(i, j);
+    return get_lap(costMatrix, rowsol, colsol, u, v, row, col, dim);
+  }
+}
+
+void FairmotMindsporePost::FuseMotion(MxBase::KalmanFilter &kf,
+                                      cv::Mat &cost_matrix,
+                                      std::vector<STack *> tracks,
+                                      std::vector<STack *> detections,
+                                      bool only_position, float lambda_) {
+  if (cost_matrix.rows != 0) {
+    int gating_dim;
+    if (only_position = false)
+      gating_dim = 2;
+    else
+      gating_dim = 4;
+    float gating_threshold = kalmanfilter.chi2inv95[gating_dim];
+    cv::Mat measurements(detections.size(), 4, CV_64FC1);
+    for (size_t i = 0; i < detections.size(); i++)
+      measurements.row(i) =
+          (*detections[i]).tlwh_to_xyah((*detections[i]).gettlwh()) + 0;
+    for (size_t i = 0; i < tracks.size(); i++) {
+      cv::Mat gating_distance =
+          kf.GatingDistance((*tracks[i]).mean, (*tracks[i]).covariance,
+                            measurements, only_position);
+      for (size_t t = 0; t < gating_distance.cols; t++)
+        if (gating_distance.at<double>(0, t) > gating_threshold)
+          cost_matrix.at<double>(i, t) = DBL_MAX;
+      cost_matrix.row(i) =
+          lambda_ * cost_matrix.row(i) + (1 - lambda_) * gating_distance;
+    }
+  }
+}
+std::vector<STack *> FairmotMindsporePost::JointStracks(
+    std::vector<STack *> tlista, std::vector<STack *> tlistb) {
+  std::vector<STack *> res;
+  std::map<int, int> exists;
+  for (size_t t = 0; t < tlista.size(); t++) {
+    exists[(*tlista[t]).track_id] = 1;
+    res.push_back(tlista[t]);
+  }
+  for (size_t t = 0; t < tlistb.size(); t++) {
+    int tid = (*tlistb[t]).track_id;
+    if (exists[tid] == 0) {
+      exists[tid] = 1;
+      res.push_back(tlistb[t]);
+    }
+  }
+  return res;
+}
+void FairmotMindsporePost::RemoveDuplicateStracks(
+    std::vector<STack *> &stracksa, std::vector<STack *> &stracksb) {
+  cv::Mat pdist = IouDistance(stracksa, stracksb);
+  std::vector<size_t> p, q, dupa, dupb;
+  std::vector<STack *> resa;
+  std::vector<STack *> resb;
+  for (size_t i = 0; i < pdist.rows; i++)
+    for (size_t j = 0; j < pdist.cols; j++)
+      if (pdist.at<double>(i, j) < 0.15) {
+        p.push_back(i);
+        q.push_back(j);
+      }
+  for (size_t i = 0; i < p.size(); i++) {
+    int timep = (*stracksa[p[i]]).frame_id - (*stracksa[p[i]]).start_frame;
+    int timeq = (*stracksb[q[i]]).frame_id - (*stracksb[q[i]]).start_frame;
+    if (timep > timeq) {
+      dupb.push_back(q[i]);
+    } else {
+      dupa.push_back(p[i]);
+    }
+  }
+  for (size_t i = 0; i < stracksa.size(); i++) {
+    if (std::find(dupa.begin(), dupa.end(), i) == dupa.end()) {
+      resa.push_back(stracksa[i]);
+    }
+  }
+  for (size_t i = 0; i < stracksb.size(); i++) {
+    if (std::find(dupb.begin(), dupb.end(), i) == dupb.end()) {
+      resb.push_back(stracksb[i]);
+    }
+  }
+  stracksa = resa;
+  stracksb = resb;
+}
+std::vector<STack *> FairmotMindsporePost::SubStracks(
+    std::vector<STack *> tlista, std::vector<STack *> tlistb) {
+  std::vector<STack *> res;
+  std::map<size_t, STack *> stracks;
+  std::map<size_t, STack *>::iterator it;
+  std::vector<size_t> key;
+  std::vector<size_t> del_key;
+  for (size_t t = 0; t < tlista.size(); t++) {
+    key.push_back((*tlista[t]).track_id);
+    stracks[(*tlista[t]).track_id] = tlista[t];
+  }
+  for (size_t t = 0; t < tlistb.size(); t++) {
+    int tid = (*tlistb[t]).track_id;
+    it = stracks.find(tid);
+    if (it != stracks.end()) {
+      del_key.push_back(tid);
+      stracks.erase(it);
+    }
+  }
+  for (size_t i = 0; i < key.size(); i++) {
+    bool flag = false;
+    for (size_t j = 0; j < del_key.size(); j++) {
+      if (del_key[j] == key[i]) {
+        flag = true;
+      }
+      if (flag == true) {
+        break;
+      }
+    }
+    if (flag == false) {
+      res.push_back(stracks[key[i]]);
+    }
+  }
+  return res;
+}
+cv::Mat FairmotMindsporePost::BboxOverlaps(std::vector<cv::Mat> boxes,
+                                           std::vector<cv::Mat> query_boxes) {
+  int N = boxes.size();
+  int K = query_boxes.size();
+  cv::Mat overlaps = cv::Mat::zeros(N, K, CV_64FC1);
+  for (size_t k = 0; k < K; k++) {
+    double box_area =
+        (query_boxes[k].at<double>(0, 2) - query_boxes[k].at<double>(0, 0) +
+         1) *
+        (query_boxes[k].at<double>(0, 3) - query_boxes[k].at<double>(0, 1) + 1);
+    for (size_t n = 0; n < N; n++) {
+      double iw =
+          std::min(boxes[n].at<double>(0, 2), query_boxes[k].at<double>(0, 2)) -
+          std::max(boxes[n].at<double>(0, 0), query_boxes[k].at<double>(0, 0)) +
+          1;
+      if (iw > 0) {
+        double ih = std::min(boxes[n].at<double>(0, 3),
+                             query_boxes[k].at<double>(0, 3)) -
+                    std::max(boxes[n].at<double>(0, 1),
+                             query_boxes[k].at<double>(0, 1)) +
+                    1;
+        if (ih > 0) {
+          double ua = static_cast<double>(
+              (boxes[n].at<double>(0, 2) - boxes[n].at<double>(0, 0) + 1) *
+                  (boxes[n].at<double>(0, 3) - boxes[n].at<double>(0, 1) + 1) +
+              box_area - iw * ih);
+          overlaps.at<double>(n, k) = iw * ih / ua;
+        }
+      }
+    }
+  }
+  return overlaps;
+}
+cv::Mat FairmotMindsporePost::IouDistance(std::vector<STack *> atracks,
+                                          std::vector<STack *> btracks) {
+  std::vector<cv::Mat> atlbrs;
+  std::vector<cv::Mat> btlbrs;
+  cv::Mat cost_matrix;
+  for (size_t i = 0; i < atracks.size(); i++) {
+    atlbrs.push_back((*atracks[i]).tlbr());
+  }
+  for (size_t i = 0; i < btracks.size(); i++) {
+    btlbrs.push_back((*btracks[i]).tlbr());
+  }
+  cv::Mat ious = cv::Mat::zeros(atlbrs.size(), btlbrs.size(), CV_64FC1);
+  if (!ious.empty()) {
+    ious = BboxOverlaps(atlbrs, btlbrs);
+    cost_matrix = 1 - ious;
+  } else {
+    cost_matrix = cv::Mat::zeros(atlbrs.size(), btlbrs.size(), CV_64FC1);
+  }
+  return cost_matrix;
+}
+void FairmotMindsporePost::MultiPredict(std::vector<STack *> &stracks) {
+  if (stracks.size() > 0) {
+    cv::Mat multi_mean(stracks.size(), (*stracks[0]).mean.cols, CV_64FC1);
+    std::vector<cv::Mat> multi_covariance;
+    for (size_t i = 0; i < stracks.size(); i++) {
+      multi_mean.row(i) = (*stracks[i]).mean.clone() + 0;
+      multi_covariance.push_back((*stracks[i]).covariance);
+    }
+    for (size_t i = 0; i < stracks.size(); i++) {
+      if ((*stracks[i]).state != TrackState_Tracked) {
+        multi_mean.at<double>(i, 7) = 0;
+      }
+    }
+    kalmanfilter.multi_predict(multi_mean, multi_covariance);
+    for (size_t i = 0; i < multi_covariance.size(); i++) {
+      (*stracks[i]).mean = multi_mean.row(i);
+      (*stracks[i]).covariance = multi_covariance[i];
+    }
+  }
+}
+
+std::vector<STack *> FairmotMindsporePost::Get_output_stracks(
+    MxBase::JDETracker &tracker, const std::vector<STack *> &activated_starcks,
+    const std::vector<STack *> &refind_stracks,
+    std::vector<STack *> lost_stracks, std::vector<STack *> removed_stracks) {
+  std::vector<STack *> det_tmp;
+  for (size_t i = 0; i < tracker.tracked_stracks.size(); i++) {
+    if ((*tracker.tracked_stracks[i]).state == TrackState_Tracked) {
+      det_tmp.push_back(tracker.tracked_stracks[i]);
+    }
+  }
+  std::vector<STack *>().swap(tracker.tracked_stracks);
+  tracker.tracked_stracks = det_tmp;
+  std::vector<STack *>().swap(det_tmp);
+  tracker.tracked_stracks =
+      JointStracks(tracker.tracked_stracks, activated_starcks);
+  tracker.tracked_stracks =
+      JointStracks(tracker.tracked_stracks, refind_stracks);
+  tracker.lost_stracks =
+      SubStracks(tracker.lost_stracks, tracker.tracked_stracks);
+  for (size_t i = 0; i < lost_stracks.size(); i++) {
+    tracker.lost_stracks.push_back(lost_stracks[i]);
+  }
+  tracker.lost_stracks =
+      SubStracks(tracker.lost_stracks, tracker.removed_stracks);
+  for (size_t i = 0; i < removed_stracks.size(); i++) {
+    tracker.removed_stracks.push_back(removed_stracks[i]);
+  }
+  std::vector<STack *> output_stracks;
+  RemoveDuplicateStracks(tracker.tracked_stracks, tracker.lost_stracks);
+  // get scores of lost tracks
+  for (size_t i = 0; i < tracker.tracked_stracks.size(); i++) {
+    if ((*tracker.tracked_stracks[i]).is_activated) {
+      output_stracks.push_back(tracker.tracked_stracks[i]);
+    }
+  }
+  return output_stracks;
+}
+
+void FairmotMindsporePost::Get_detections(cv::Mat det,
+                                          std::vector<STack *> &detections,
+                                          cv::Mat id_feature) {
+  if (det.rows > 0) {
+    cv::Mat det_tmp = det(cv::Range(0, det.rows), cv::Range(0, 5)).clone();
+    for (size_t x = 0; x < det.rows; ++x) {
+      cv::Mat tlbrs = det_tmp.row(x);
+      cv::Mat f = id_feature.row(x);
+      cv::Mat ret = tlbrs(cv::Range(0, tlbrs.rows), cv::Range(0, 4));
+      for (size_t y = 0; y < 2; ++y) {
+        ret.at<float>(0, y + 2) -= ret.at<float>(0, y);
+      }
+      STack *stack = new STack(ret, tlbrs.at<float>(0, 4), f, 30);
+      detections.push_back(stack);
+    }
+  }
+}
+
+void FairmotMindsporePost::Get_dists(cv::Mat &dists,
+                                     std::vector<STack *> &detections,
+                                     std::vector<STack *> &strack_pool) {
+  if (dists.rows != 0) {
+    cv::Mat det_features(detections.size(), (*detections[0]).curr_feat.cols,
+                         CV_32FC1);
+    cv::Mat track_features(strack_pool.size(),
+                           (*strack_pool[0]).smooth_feat.cols, CV_32FC1);
+    for (size_t i = 0; i < detections.size(); i++)
+      det_features.row(i) = (*detections[i]).curr_feat + 0;
+    det_features.convertTo(det_features, CV_64F);
+    for (size_t i = 0; i < strack_pool.size(); i++)
+      track_features.row(i) = (*strack_pool[i]).smooth_feat + 0;
+    track_features.convertTo(track_features, CV_64F);
+    // cv::Mat cdist(track_features.rows, det_features.rows, CV_64FC1);
+    for (size_t i = 0; i < dists.rows; i++)
+      for (size_t j = 0; j < dists.cols; j++) {
+        cv::normalize(det_features.row(j), det_features.row(j));
+        cv::normalize(track_features.row(i), track_features.row(i));
+        dists.at<double>(i, j) =
+            1 - track_features.row(i).dot(det_features.row(j));
+      }
+  }
+}
+
+void FairmotMindsporePost::Update_Starcks(
+    const std::vector<STack *> &strack_pool, std::vector<STack *> &detections,
+    const cv::Mat &matches, std::vector<STack *> &activated_starcks,
+    std::vector<STack *> &refind_stracks, const MxBase::JDETracker &tracker) {
+  for (size_t i = 0; i < matches.rows; i++) {
+    STack *track = strack_pool[matches.at<float>(i, 0)];
+    STack *dets = detections[matches.at<float>(i, 1)];
+    if ((*track).state == TrackState_Tracked) {
+      (*track).update((*detections[matches.at<float>(i, 1)]), tracker.frame_id);
+      activated_starcks.push_back(track);
+    } else {
+      (*track).re_activate(*dets, tracker.frame_id, false);
+      refind_stracks.push_back(track);
+    }
+  }
+}
+
+std::vector<STack *> FairmotMindsporePost::ObjectDetectionOutput(
+    const std::vector<TensorBase> &tensors, MxBase::JDETracker &tracker) {
+  tracker.frame_id += 1;
+  cv::Mat id_feature, det, matches, u_track, u_detection, u_unconfirmed, dists;
+  std::vector<STack *> activated_starcks, refind_stracks, lost_stracks,
+      removed_stracks, detections, unconfirmed, tracked_stracks, det_tmp,
+      r_tracked_stracks, output_stracks, strack_pool;
+  TensorBaseToCVMat(det, tensors[1]);
+  TensorBaseToCVMat(id_feature, tensors[0]);
+  PostProcess(det, tracker);
+  cv::Mat scores = det(cv::Range(0, det.rows), cv::Range(4, 5));
+  cv::Mat new_det(0, det.cols, CV_32FC1), new_id(0, id_feature.cols, CV_32FC1);
+  for (size_t x = 0; x < det.rows; ++x) {
+    if (det.at<float>(x, 4) > CONF_THRES) {
+      new_det.push_back(det.row(x));
+      new_id.push_back(id_feature.row(x));
+    }
+  }
+  det = new_det;
+  id_feature = new_id;
+  Get_detections(det, detections, id_feature);
+  for (size_t i = 0; i < tracker.tracked_stracks.size(); i++)
+    if (!(*tracker.tracked_stracks[i]).is_activated)
+      unconfirmed.push_back(tracker.tracked_stracks[i]);
+    else
+      tracked_stracks.push_back(tracker.tracked_stracks[i]);
+  strack_pool = JointStracks(tracked_stracks, tracker.lost_stracks);
+  MultiPredict(strack_pool);
+  dists = cv::Mat::zeros(strack_pool.size(), detections.size(), CV_64FC1);
+  Get_dists(dists, detections, strack_pool);
+  FuseMotion(tracker.kalman_filter, dists, strack_pool, detections);
+  std::vector<cv::Mat> results;
+  results = LinearAssignment(dists, 0.4);
+  matches = results[0];
+  u_track = results[1];
+  u_detection = results[2];
+  Update_Starcks(strack_pool, detections, matches, activated_starcks,
+                 refind_stracks, tracker);
+  for (size_t i = 0; i < u_detection.cols; ++i)
+    det_tmp.push_back(
+        detections[static_cast<int>(u_detection.at<float>(0, i))]);
+  detections = det_tmp;
+  std::vector<STack *>().swap(det_tmp);
+  for (size_t i = 0; i < u_track.cols; ++i)
+    if ((*strack_pool[u_track.at<float>(0, i)]).state == TrackState_Tracked)
+      r_tracked_stracks.push_back(strack_pool[u_track.at<float>(0, i)]);
+  dists = IouDistance(r_tracked_stracks, detections);
+  results = LinearAssignment(dists, 0.5);
+  matches = results[0];
+  u_track = results[1];
+  u_detection = results[2];
+  Update_Starcks(r_tracked_stracks, detections, matches, activated_starcks,
+                 refind_stracks, tracker);
+  for (size_t i = 0; i < u_track.cols; i++) {
+    STack *track = r_tracked_stracks[u_track.at<float>(0, i)];
+    if ((*track).state != TrackState_Lost) {
+      (*track).state = TrackState_Lost;
+      lost_stracks.push_back(track);
+    }
+  }
+  for (size_t i = 0; i < u_detection.cols; ++i)
+    det_tmp.push_back(
+        detections[static_cast<int>(u_detection.at<float>(0, i))]);
+  detections = det_tmp;
+  std::vector<STack *>().swap(det_tmp);
+  dists = IouDistance(unconfirmed, detections);
+  results = LinearAssignment(dists, 0.7);
+  matches = results[0];
+  u_unconfirmed = results[1];
+  u_detection = results[2];
+  for (size_t i = 0; i < matches.rows; i++) {
+    (*unconfirmed[matches.at<float>(i, 0)])
+        .update((*detections[matches.at<float>(i, 1)]), tracker.frame_id);
+    activated_starcks.push_back(unconfirmed[matches.at<float>(i, 0)]);
+  }
+  for (size_t i = 0; i < u_unconfirmed.cols; i++) {
+    STack *track = unconfirmed[u_unconfirmed.at<float>(0, i)];
+    (*track).state = TrackState_Removed;
+    removed_stracks.push_back(track);
+  }
+  for (int j = 0; j < u_detection.cols; j++) {
+    auto inew = u_detection.at<float>(0, j);
+    STack *track = detections[inew];
+    if ((*track).score < tracker.det_thresh) continue;
+    (*track).activate(tracker.kalman_filter, tracker.frame_id);
+    activated_starcks.push_back(track);
+  }
+  for (size_t i = 0; i < tracker.lost_stracks.size(); i++) {
+    if (tracker.frame_id - (*tracker.lost_stracks[i]).frame_id >
+        tracker.max_time_lost) {
+      (*tracker.lost_stracks[i]).state = TrackState_Removed;
+      removed_stracks.push_back(tracker.lost_stracks[i]);
+    }
+  }
+  output_stracks =
+      Get_output_stracks(tracker, activated_starcks, refind_stracks,
+                         lost_stracks, removed_stracks);
+  return output_stracks;
+}
+
+APP_ERROR FairmotMindsporePost::Process(const std::vector<TensorBase> &tensors,
+                                        MxBase::JDETracker &tracker,
+                                        MxBase::Files &file) {
+  LogDebug << "Begin to process FairmotMindsporePost.";
+  auto inputs = tensors;
+  APP_ERROR ret = CheckAndMoveTensors(inputs);
+  if (ret != APP_ERR_OK) {
+    LogError << "CheckAndMoveTensors failed, ret=" << ret;
+    return ret;
+  }
+  std::vector<STack *> online_targets;
+  online_targets = ObjectDetectionOutput(inputs, tracker);
+  std::vector<cv::Mat> online_tlwhs;
+  std::vector<int> online_ids;
+  for (size_t i = 0; i < online_targets.size(); i++) {
+    cv::Mat tlwh = (*online_targets[i]).gettlwh();
+    int tid = (*online_targets[i]).track_id;
+    double tmp = tlwh.at<double>(0, 2) / tlwh.at<double>(0, 3);
+    bool vertical = false;
+    if (tmp > 1.6) {
+      vertical = true;
+    }
+    if ((tlwh.at<double>(0, 2) * tlwh.at<double>(0, 3) > MIN_BOX_AREA) &&
+        vertical == false) {
+      online_tlwhs.push_back(tlwh);
+      online_ids.push_back(tid);
+    }
+  }
+  Results *result = new Results(file.frame_id + 1, online_tlwhs, online_ids);
+  file.results.push_back(result);
+  file.frame_id += 1;
+  LogInfo << "End to process FairmotMindsporePost.";
+  return APP_ERR_OK;
+}
+
+extern "C" {
+std::shared_ptr<MxBase::FairmotMindsporePost> GetObjectInstance() {
+  LogInfo << "Begin to get FairmotMindsporePost instance.";
+  auto instance = std::make_shared<FairmotMindsporePost>();
+  LogInfo << "End to get FairmotMindsporePost Instance";
+  return instance;
+}
+}
+
+}  // namespace MxBase

research/cv/fairmot/infer/mxbase/src/PostProcess/FairmotMindsporePost.h

+/*
+ * Copyright 2021 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <algorithm>
+#include <map>
+#include <memory>
+#include <string>
+#include <vector>
+#include <opencv4/opencv2/opencv.hpp>
+typedef int row;
+typedef double f64_mat_t[4][4]; /**< a matrix */
+typedef double f64_vec_t[4];
+#define ROW_TYPE INT
+typedef int col;
+#define COL_TYPE INT
+typedef double cost;
+#define COST_TYPE DOUBLE
+#define BIG 100000
+#if !defined TRUE
+#define TRUE 1
+#endif
+#if !defined FALSE
+#define FALSE 0
+#endif
+
+/*************** DATA TYPES *******************/
+
+typedef int boolean;
+#include "MxBase/CV/Core/DataType.h"
+#include "MxBase/ErrorCode/ErrorCode.h"
+#include "MxBase/PostProcessBases/PostProcessBase.h"
+
+namespace MxBase {
+class Results {
+ public:
+  Results(uint32_t frame_id, const std::vector<cv::Mat> &online_tlwhs,
+          const std::vector<int> &online_ids);
+  uint32_t frame_id;
+  std::vector<cv::Mat> online_tlwhs;
+  std::vector<int> online_ids;
+};
+class TrackState {
+ public:
+  uint32_t New = 0;
+  uint32_t Tracked = 1;
+  uint32_t Lost = 2;
+  uint32_t Removed = 3;
+};
+class BaseTrack {
+ public:
+  uint32_t trackId = 0;
+  bool activated = false;
+  uint32_t base_state;
+  int next_id();
+
+ private:
+  uint32_t count = 0;
+};
+class KalmanFilter {
+ public:
+  std::map<int, float> chi2inv95 = {{1, 3.8415}, {2, 5.9915}, {3, 7.8147},
+                                    {4, 9.4877}, {5, 11.070}, {6, 12.592},
+                                    {7, 14.067}, {8, 15.507}, {9, 16.919}};
+  uint32_t ndim;
+  float dt;
+  KalmanFilter();
+  void chol_subtitute(cv::Mat chol_factor, cv::Mat b, f64_vec_t *f_x, int n);
+  void cholesky_decomposition(const cv::Mat &A, cv::Mat &L);
+  void initiate(cv::Mat measurement, cv::Mat &mean, cv::Mat &covariance);
+  void multi_predict(cv::Mat &mean, std::vector<cv::Mat> &covariance);
+  cv::Mat GatingDistance(cv::Mat mean, cv::Mat covariance, cv::Mat measurements,
+                         bool only_position = false,
+                         const std::string &metric = "maha");
+  void project(cv::Mat &mean, cv::Mat &covariance);
+  void update(cv::Mat &mean, cv::Mat &covariance, cv::Mat measurement);
+
+ private:
+  cv::Mat motion_mat;
+  cv::Mat update_mat;
+  float std_weight_position;
+  float std_weight_velocity;
+};
+
+class STack : public BaseTrack {
+ public:
+  cv::Mat mean, covariance;
+  cv::Mat smooth_feat;
+  cv::Mat curr_feat;
+  bool is_activated;
+  KalmanFilter kalman_filter;
+  float score;
+  float alpha;
+  uint32_t tracklet_len;
+  int track_id;
+  uint32_t state;
+  uint32_t start_frame;
+  uint32_t frame_id;
+  std::vector<cv::Mat> features;
+  STack();
+  STack(cv::Mat tlwh, float score, cv::Mat temp_feat, uint32_t buffer_size);
+  void activate(const KalmanFilter &kalman_filter, uint32_t frame_id);
+  void re_activate(STack new_track, int frame_id, bool new_id = false);
+  cv::Mat tlwh_to_xyah(cv::Mat tlwh);
+  cv::Mat tlbr();
+  cv::Mat gettlwh();
+  void update(STack new_track, int frame_id, bool update_feature = true);
+
+ private:
+  cv::Mat tlwh;
+  void update_features(cv::Mat temp_feat);
+};
+class JDETracker {
+ public:
+  explicit JDETracker(uint32_t frame_rate);
+  std::vector<STack *> tracked_stracks;
+  std::vector<STack *> lost_stracks;
+  std::vector<STack *> removed_stracks;
+  uint32_t frame_id = 0;
+  uint32_t out_height = 0;
+  uint32_t out_width = 0;
+  std::vector<float> c;
+  float det_thresh;
+  float s = 0;
+  int buffer_size;
+  int max_time_lost;
+  int max_per_image;
+  std::string seq;
+  std::string image_file;
+  cv::Mat mean;
+  cv::Mat std;
+  KalmanFilter kalman_filter;
+};
+class Files {
+ public:
+  uint32_t frame_id = 0;
+  std::vector<Results *> results;
+};
+class FairmotMindsporePost : public PostProcessBase {
+ public:
+  FairmotMindsporePost() = default;
+
+  ~FairmotMindsporePost() = default;
+
+  FairmotMindsporePost(const FairmotMindsporePost &other) = default;
+
+  FairmotMindsporePost &operator=(const FairmotMindsporePost &other);
+
+  APP_ERROR Init(
+      const std::map<std::string, std::shared_ptr<void>> &postConfig) override;
+
+  APP_ERROR DeInit() override;
+
+  APP_ERROR Process(const std::vector<TensorBase> &tensors,
+                    MxBase::JDETracker &tracker, MxBase::Files &file);
+
+  bool IsValidTensors(const std::vector<TensorBase> &tensors) const override;
+
+ private:
+  std::vector<STack *> ObjectDetectionOutput(
+      const std::vector<TensorBase> &tensors, MxBase::JDETracker &tracker);
+  void TransformPreds(const cv::Mat &coords, MxBase::JDETracker tracker,
+                      cv::Mat &target_coords);
+  void PostProcess(cv::Mat &det, const MxBase::JDETracker &tracker);
+  void TensorBaseToCVMat(cv::Mat &imageMat, const MxBase::TensorBase &tensor);
+  void FuseMotion(MxBase::KalmanFilter &kalman_filter, cv::Mat &cost_matrix,
+                  std::vector<STack *> tracks, std::vector<STack *> detections,
+                  bool only_position = false, float lambda_ = 0.98);
+  std::vector<cv::Mat> LinearAssignment(cv::Mat cost_matrix, float thresh);
+  void lap(cost **assigncost, col *rowsol, row *colsol, cost *u, cost *v,
+           int row, int col);
+  std::vector<cv::Mat> get_lap(cost **assigncost, col *rowsol, row *colsol,
+                               cost *u, cost *v, int row, int col, int dim);
+  std::vector<STack *> JointStracks(std::vector<STack *> tlista,
+                                    std::vector<STack *> tlistb);
+  std::vector<STack *> SubStracks(std::vector<STack *> tlista,
+                                  std::vector<STack *> tlistb);
+  void RemoveDuplicateStracks(std::vector<STack *> &stracksa,
+                              std::vector<STack *> &stracksb);
+  cv::Mat IouDistance(std::vector<STack *> atracks,
+                      std::vector<STack *> btracks);
+  cv::Mat BboxOverlaps(std::vector<cv::Mat> boxes,
+                       std::vector<cv::Mat> query_boxes);
+  std::vector<STack *> Get_output_stracks(
+      MxBase::JDETracker &tracker,
+      const std::vector<STack *> &activated_starcks,
+      const std::vector<STack *> &refind_stracks,
+      std::vector<STack *> lost_stracks, std::vector<STack *> removed_stracks);
+  void Get_detections(cv::Mat det, std::vector<STack *> &detections,
+                      cv::Mat id_feature);
+  void Get_dists(cv::Mat &dists, std::vector<STack *> &detections,
+                 std::vector<STack *> &strack_pool);
+  void Update_Starcks(const std::vector<STack *> &strack_pool,
+                      std::vector<STack *> &detections, const cv::Mat &matches,
+                      std::vector<STack *> &activated_starcks,
+                      std::vector<STack *> &refind_stracks,
+                      const MxBase::JDETracker &tracker);
+  void get_result(int *collist, int cols, cost *d, cost min, int rows,
+                  int &endofpath, row *colsol, cost **assigncost, cost *v,
+                  int *pred);
+  void func(int &numfree, int *free, cost **assigncost, row *colsol, cost *v,
+            col *rowsol, int cols);
+  void MultiPredict(std::vector<STack *> &stracks);
+  const uint32_t DEFAULT_CLASS_NUM_MS = 80;
+  const float DEFAULT_SCORE_THRESH_MS = 0.7;
+  const float DEFAULT_IOU_THRESH_MS = 0.5;
+  const uint32_t DEFAULT_RPN_MAX_NUM_MS = 1000;
+  const uint32_t DEFAULT_MAX_PER_IMG_MS = 128;
+
+  uint32_t classNum_ = DEFAULT_CLASS_NUM_MS;
+  float scoreThresh_ = DEFAULT_SCORE_THRESH_MS;
+  float iouThresh_ = DEFAULT_IOU_THRESH_MS;
+  uint32_t rpnMaxNum_ = DEFAULT_RPN_MAX_NUM_MS;
+  uint32_t maxPerImg_ = DEFAULT_MAX_PER_IMG_MS;
+};
+
+extern "C" {
+std::shared_ptr<MxBase::FairmotMindsporePost> GetObjectInstance();
+}
+}  // namespace MxBase

research/cv/fairmot/infer/mxbase/src/main.cpp

+/*
+ * Copyright 2021 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "Fairmot.h"
+#include "MxBase/Log/Log.h"
+
+namespace {
+const uint32_t DEVICE_ID = 0;
+}  // namespace
+
+int main(int argc, char *argv[]) {
+  int num = 2;
+  if (argc <= num) {
+    LogWarn << "Please input image path, such as './Fairmot_mindspore "
+               "[om_file_path] [img_path]'.";
+    return APP_ERR_OK;
+  }
+
+  InitParam initParam = {};
+  initParam.deviceId = DEVICE_ID;
+
+  initParam.checkTensor = true;
+
+  initParam.modelPath = argv[1];
+  auto inferFairmot = std::make_shared<Fairmot>();
+  APP_ERROR ret = inferFairmot->Init(initParam);
+  if (ret != APP_ERR_OK) {
+    LogError << "Fairmot init failed, ret=" << ret << ".";
+    return ret;
+  }
+
+  std::string imgPath = argv[2];
+  ret = inferFairmot->Process(imgPath);
+  if (ret != APP_ERR_OK) {
+    LogError << "Fairmot process failed, ret=" << ret << ".";
+    inferFairmot->DeInit();
+    return ret;
+  }
+  inferFairmot->DeInit();
+  return APP_ERR_OK;
+}

research/cv/fairmot/infer/requirements.txt

+motmetrics
+lap
+openpyxl
+cython_bbox

research/cv/fairmot/infer/sdk/api/infer.py

+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+"""sdk infer"""
+import json
+import logging
+import MxpiDataType_pb2 as MxpiDataType
+from StreamManagerApi import StreamManagerApi, MxDataInput, InProtobufVector, MxProtobufIn, StringVector
+
+from config import config as cfg
+
+
+class SdkApi:
+    """sdk api"""
+    INFER_TIMEOUT = cfg.INFER_TIMEOUT
+    STREAM_NAME = cfg.STREAM_NAME
+
+    def __init__(self, pipeline_cfg):
+        self.pipeline_cfg = pipeline_cfg
+        self._stream_api = None
+        self._data_input = None
+        self._device_id = None
+
+    def init(self):
+        """sdk init """
+        with open(self.pipeline_cfg, 'r') as fp:
+            self._device_id = int(
+                json.loads(fp.read())[self.STREAM_NAME]["stream_config"]
+                ["deviceId"])
+            print("The device id: {}.".format(self._device_id))
+
+        # create api
+        self._stream_api = StreamManagerApi()
+
+        # init stream mgr
+        ret = self._stream_api.InitManager()
+        if ret != 0:
+            print(f"Failed to init stream manager, ret={ret}.")
+            return False
+
+        # create streams
+        with open(self.pipeline_cfg, 'rb') as fp:
+            pipe_line = fp.read()
+
+        ret = self._stream_api.CreateMultipleStreams(pipe_line)
+        if ret != 0:
+            print(f"Failed to create stream, ret={ret}.")
+            return False
+
+        self._data_input = MxDataInput()
+
+        return True
+
+    def __del__(self):
+        """del sdk"""
+        if not self._stream_api:
+            return
+
+        self._stream_api.DestroyAllStreams()
+
+    def send_data_input(self, stream_name, plugin_id, input_data):
+        """input data use SendData"""
+        data_input = MxDataInput()
+        data_input.data = input_data
+        unique_id = self._stream_api.SendData(stream_name, plugin_id,
+                                              data_input)
+        if unique_id < 0:
+            logging.error("Fail to send data to stream.")
+            return False
+        return True
+
+    def _send_protobuf(self, stream_name, plugin_id, element_name, buf_type,
+                       pkg_list):
+        """input data use SendProtobuf"""
+        protobuf = MxProtobufIn()
+        protobuf.key = element_name.encode("utf-8")
+        protobuf.type = buf_type
+        protobuf.protobuf = pkg_list.SerializeToString()
+        protobuf_vec = InProtobufVector()
+        protobuf_vec.push_back(protobuf)
+        err_code = self._stream_api.SendProtobuf(stream_name, plugin_id,
+                                                 protobuf_vec)
+        if err_code != 0:
+            logging.error(
+                "Failed to send data to stream, stream_name(%s), plugin_id(%s), element_name(%s), "
+                "buf_type(%s), err_code(%s).", stream_name, plugin_id,
+                element_name, buf_type, err_code)
+            return False
+        return True
+
+    def send_img_input(self, stream_name, plugin_id, element_name, input_data,
+                       img_size):
+        """use cv input to sdk"""
+        vision_list = MxpiDataType.MxpiVisionList()
+        vision_vec = vision_list.visionVec.add()
+        vision_vec.visionInfo.format = 1
+        vision_vec.visionInfo.width = img_size[1]
+        vision_vec.visionInfo.height = img_size[0]
+        vision_vec.visionInfo.widthAligned = img_size[1]
+        vision_vec.visionInfo.heightAligned = img_size[0]
+        vision_vec.visionData.memType = 0
+        vision_vec.visionData.dataStr = input_data
+        vision_vec.visionData.dataSize = len(input_data)
+        buf_type = b"MxTools.MxpiVisionList"
+        return self._send_protobuf(stream_name, plugin_id, element_name,
+                                   buf_type, vision_list)
+
+    def get_result(self, stream_name, out_plugin_id=0):
+        """get_result"""
+        key_vec = StringVector()
+        key_vec.push_back(b'mxpi_modelinfer0')
+        infer_result = self._stream_api.GetProtobuf(
+            stream_name, out_plugin_id, key_vec)
+        result = MxpiDataType.MxpiTensorPackageList()
+        result.ParseFromString(infer_result[0].messageBuf)
+        return result.tensorPackageVec[0].tensorVec[0].dataStr, result.tensorPackageVec[0].tensorVec[1].dataStr

research/cv/fairmot/infer/sdk/config/config.py

+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+"""config"""
+
+STREAM_NAME = "im_fairmot"
+MODEL_WIDTH = 1088
+MODEL_HEIGHT = 608
+
+INFER_TIMEOUT = 100000
+
+TENSOR_DTYPE_FLOAT32 = 0
+TENSOR_DTYPE_FLOAT16 = 1
+TENSOR_DTYPE_INT8 = 2

research/cv/fairmot/infer/sdk/config/fairmot.pipeline

+{
+    "im_fairmot": {
+        "stream_config": {
+            "deviceId": "0"
+        },
+        "appsrc0": {
+            "props": {
+                "blocksize": "409600"
+            },
+            "factory": "appsrc",
+            "next": "mxpi_modelinfer0"
+        },
+        "mxpi_modelinfer0": {
+            "props": {
+                "dataSource": "appsrc0",
+                "modelPath": "../convert/fairmot.om",
+                "tensorFormat": "1"
+            },
+            "factory": "mxpi_modelinfer",
+            "next": "mxpi_dataserialize0"
+        },
+        "mxpi_dataserialize0": {
+            "props": {
+                "outputDataKeys": "mxpi_modelinfer0"
+            },
+            "factory": "mxpi_dataserialize",
+            "next": "appsink0"
+        },
+        "appsink0": {
+            "factory": "appsink"
+        }
+    }
+}
\ No newline at end of file