!96 [西安电子科技大学][高校贡献][Mindspore][Fairmot]-高性能预训练模型提交+mxbase+sdk+modelarts

Merge pull request !96 from 陈志鹏/code_fairmot

.jenkins/check/config/filter_cpplint.txt

@@ -49,4 +49,7 @@
 "models/official/cv/posenet/infer/mxbase/src/Posenet.cpp" "runtime/references"
 "models/official/cv/posenet/infer/mxbase/src/main.cpp" "runtime/references"
 
-"models/research/cv/ibnnet/infer/mxbase/src/IbnnetOpencv.h" "runtime/references"
\ No newline at end of file
+"models/research/cv/ibnnet/infer/mxbase/src/IbnnetOpencv.h" "runtime/references"
+
+"models/research/cv/fairmot/infer/mxbase/src/Fairmot.h" "runtime/references"
+"models/research/cv/fairmot/infer/mxbase/src/PostProcess/FairmotMindsporePost.h" "runtime/references"
\ No newline at end of file

research/cv/fairmot/Dockerfile

+ARG FROM_IMAGE_NAME
+FROM ${FROM_IMAGE_NAME}
+
+COPY requirements.txt .
+RUN pip3.7 install -r requirements.txt
\ No newline at end of file

research/cv/fairmot/fairmot_train.py

@@ -21,7 +21,6 @@ from mindspore import dtype as mstype
 from mindspore import Model
 import mindspore.nn as nn
 import mindspore.dataset as ds
-from mindspore.common import set_seed
 from mindspore.context import ParallelMode
 from mindspore.train.callback import TimeMonitor, ModelCheckpoint, CheckpointConfig
 from mindspore.communication.management import init
@@ -34,8 +33,7 @@ from src.utils.lr_schedule import dynamic_lr
 from src.utils.jde import JointDataset
 from src.utils.callback import LossCallback
 
-set_seed(1234)
-ds.config.set_seed(1234)
+
 def train(opt):
     """train fairmot."""
     local_data_path = '/cache/data'
@@ -71,7 +69,8 @@ def train(opt):
         context.set_context(device_id=device_id, mode=context.GRAPH_MODE, device_target="Ascend", save_graphs=False)
         context.set_auto_parallel_context(parallel_mode=ParallelMode.DATA_PARALLEL,
                                           gradients_mean=True,
-                                          device_num=device_num
+                                          device_num=device_num,
+                                          parameter_broadcast=True
                                           )
         init()
     else:

research/cv/fairmot/infer/Dockerfile

+ARG FROM_IMAGE_NAME
+FROM ${FROM_IMAGE_NAME}
+
+COPY requirements.txt .
+RUN pip3.7 install -r requirements.txt
\ No newline at end of file

research/cv/fairmot/infer/README_CN.md

+# 推理
+
+## 模型转换
+
+   ```bash
+
+   cd infer/convert
+
+   ```
+
+1. 准备模型文件。
+
+   AIR模型为在昇腾910服务器上导出的模型，导出AIR模型的详细步骤请参见“模型训练”。
+
+2. 执行以下命令，进行模型转换。
+
+   转换详细信息可查看转换脚本和对应的AIPP配置文件，转换命令如下。
+
+   **bash convert/convert_om.sh** *air_path* *aipp_cfg_path* *om_path*
+
+   | 参数          | 说明                                              |
+   | ------------- | ------------------------------------------------- |
+   | air_path      | 转换脚本AIR文件路径。                             |
+   | aipp_cfg_path | AIPP配置文件路径。                                |
+   | om_path       | 生成的OM文件名，转换脚本会在此基础上添加.om后缀。 |
+
+   转换示例如下所示。
+
+   ```bash
+
+   # 转换模型
+   bash convert_om.sh fairmot.air aipp_rgb.cfg fairmot
+
+   ```
+
+## 推理数据集下载
+
+1. 下载推理数据集[MOT20](https://motchallenge.net/data/MOT20/)。
+
+2. 将数据集存放在`infer/data/MOT20`目录下。
+
+3. 目录格式为：
+
+```text
+   └─fairmot
+      ├─infer
+        ├─data
+        │ ├─MOT20
+        │ │ └─train
+        │ │    ├─MOT20-01
+        │ │    ├─MOT20-02
+        │ │    ├─MOT20-03
+        │ │    └─MOT20-05
+        │ └─data.json
+        ├─infer
+        │ ├─convert                   //  模型转换脚本
+        │ ├─mxbase                   // mxbase 推理脚本
+        │ └─sdk                   // sdk推理脚本
+```
+
+## mxBase推理
+
+   ```bash
+
+   cd infer/mxbase
+
+   ```
+
+1. 编译工程。
+
+   目前mxBase推理仅实现了基于DVPP方式推理。
+
+   ```bash
+
+   bash build.sh
+
+   ```
+
+2. （可选）修改配置文件。
+
+   可根据实际情况修改，配置文件位于`mxbase/src/main.cpp`中，可修改参数如下。
+
+   ```c++
+
+   namespace {
+   const uint32_t DEVICE_ID = 0;
+   } // namespace
+   ...
+
+   ```
+
+3. 运行推理服务。
+
+   运行推理服务：
+   **./build/fairmot_mindspore**  *om_path* *img_path*
+   | 参数       | 说明                           |
+   | ---------- | ------------------------------ |
+   | om_path | om存放路径。如：`../convert/fairmot.om`。 |
+   | img_path | 推理图片路径。如：`../../data/MOT20/`。 |
+
+   ```bash
+
+   ./build/fairmot_mindspore ../convert/fairmot.om ../../data/MOT20/
+
+   ```
+
+4. 观察结果。
+   推理结果以txt格式保存，路径为`../../data/MOT20/result_Files`.
+
+5. 可视化结果并得到精度。
+
+   运行精度测试以及可视化：
+   **python3.7 mx_base_eval.py**  *result_path*
+   | 参数       | 说明                           |
+   | ---------- | ------------------------------ |
+   | result_path | 推理结果路径。如：“../../data/MOT20”。 |
+
+   ```bash
+
+   cd ../../
+   python3.7 mx_base_eval.py --data_dir data/MOT20
+
+   ```
+
+6. 查看精度结果以及可视化结果。
+
+   图片保存在`data/MOT20/result`里。精度测试结果在运行完`mxbase_eval.py`以后会在终端显示并以xlsx文件格式保存在`data/MOT20`。
+
+## MindX SDK推理
+
+   ```bash
+
+   cd infer/sdk
+
+   ```
+
+1. （可选）修改配置文件。
+
+   1. 可根据实际情况修改pipeline文件。
+
+      ```python
+
+      ├── config
+      │   ├── config.py
+      │   └──  fairmot.pipeline # PIPELINE文件
+
+      ```
+
+2. 模型推理。
+
+   1. 执行推理。
+
+      切换到sdk目录下，执行推理脚本。
+      **python main.py**  *img_path* *pipeline_path* *infer_result_path* *infer_mode*
+      | 参数        | 说明                                  |
+      | ----------- | ------------------------------------- |
+      | img_path  | 推理图片路径。如：“../data/MOT20”。        |
+      | pipeline_path | 存放pipeline路径。如："./config/fairmot.pipeline"。 |
+      | infer_result_path | 存放推理结果的路径。如："../data/infer_result"。 |
+      | infer_mode | 推理模式，默认为infer，可用eval直接得到精度对比 |
+
+      ```bash
+
+      python3.7 main.py --img_path ../data/MOT20 --pipeline_path ./config/fairmot.pipeline --infer_result_path ../data/infer_result
+
+      ```
+
+   2. 查看推理结果。
+      推理结果以bin文件形式保存在`../data/infer_result`目录下。
+3. 执行精度测试以及可视化。
+
+   切换到fairmot目录下，执行推理脚本：
+      **python3.7 sdk_eval.py**  *img_path*
+
+   ```bash
+   cd ../
+   python3.7 sdk_eval.py --data_dir ./data/MOT20
+   ```
+
+4. 查看精度结果以及可视化结果。
+
+   图片保存在`data/MOT20/results`里。精度测试结果在运行完`sdk_eval.py`以后会在终端显示并以xlsx文件格式保存在`data/MOT20`。

research/cv/fairmot/infer/convert/aipp_rgb.cfg

+aipp_op {
+    aipp_mode : static
+    input_format : RGB888_U8
+    related_input_rank : 0
+    csc_switch : false
+    rbuv_swap_switch : true
+    var_reci_chn_0 : 0.003921568627451
+    var_reci_chn_1 : 0.003921568627451
+    var_reci_chn_2 : 0.003921568627451
+}
\ No newline at end of file

research/cv/fairmot/infer/convert/convert_om.sh

+#!/bin/bash
+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+
+if [ $# -ne 3 ]; then
+  echo "Wrong parameter format."
+  echo "Usage:"
+  echo "         bash $0 [INPUT_AIR_PATH] [AIPP_PATH] [OUTPUT_OM_PATH_NAME]"
+  echo "Example: "
+  echo "         bash convert_om.sh  xxx.air ./aipp.cfg xx"
+
+  exit 1
+fi
+
+input_air_path=$1
+aipp_cfg_file=$2
+output_om_path=$3
+
+export install_path=/usr/local/Ascend/
+
+export ASCEND_ATC_PATH=${install_path}/atc
+export LD_LIBRARY_PATH=${install_path}/atc/lib64:$LD_LIBRARY_PATH
+export PATH=/usr/local/python3.7.5/bin:${install_path}/atc/ccec_compiler/bin:${install_path}/atc/bin:$PATH
+export PYTHONPATH=${install_path}/atc/python/site-packages:${install_path}/latest/atc/python/site-packages/auto_tune.egg/auto_tune:${install_path}/atc/python/site-packages/schedule_search.egg
+export ASCEND_OPP_PATH=${install_path}/opp
+
+export ASCEND_SLOG_PRINT_TO_STDOUT=1
+
+echo "Input AIR file path: ${input_air_path}"
+echo "Output OM file path: ${output_om_path}"
+
+atc --input_format=NCHW \
+  --framework=1 \
+  --model="${input_air_path}" \
+  --input_shape="x:1, 3, 608, 1088" \
+  --output="${output_om_path}" \
+  --insert_op_conf="${aipp_cfg_file}" \
+  --soc_version=Ascend310 \
+  --op_select_implmode=high_precision \
+  --output_type=FP32

research/cv/fairmot/infer/docker_start_infer.sh

+#!/bin/bash
+
+#coding = utf-8
+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the BSD 3-Clause License  (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# https://opensource.org/licenses/BSD-3-Clause
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+docker_image=$1
+data_dir=$2
+
+function show_help() {
+    echo "Usage: docker_start.sh docker_image data_dir"
+}
+
+function param_check() {
+    if [ -z "${docker_image}" ]; then
+        echo "please input docker_image"
+        show_help
+        exit 1
+    fi
+
+    if [ -z "${data_dir}" ]; then
+        echo "please input data_dir"
+        show_help
+        exit 1
+    fi
+}
+
+param_check
+
+docker run -it \
+  --device=/dev/davinci3 \
+  --device=/dev/davinci_manager \
+  --device=/dev/devmm_svm \
+  --device=/dev/hisi_hdc \
+  -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
+  -v ${data_dir}:${data_dir} \
+  ${docker_image} \
+  /bin/bash

research/cv/fairmot/infer/mxbase/CMakeLists.txt

+cmake_minimum_required(VERSION 3.5.2)
+project(fairmot)
+add_definitions(-D_GLIBCXX_USE_CXX11_ABI=0)
+
+#set(PLUGIN_NAME "fairmot_mindspore_post")
+set(TARGET_LIBRARY fairmot_mindspore_post)
+set(TARGET_MAIN fairmot_mindspore)
+
+set(ACL_LIB_PATH $ENV{ASCEND_HOME}/nnrt/latest/acllib)
+
+include_directories(${CMAKE_CURRENT_BINARY_DIR})
+
+include_directories($ENV{MX_SDK_HOME}/include)
+include_directories($ENV{MX_SDK_HOME}/opensource/include)
+include_directories($ENV{MX_SDK_HOME}/opensource/include/opencv4)
+include_directories($ENV{MX_SDK_HOME}/opensource/include/gstreamer-1.0)
+include_directories($ENV{MX_SDK_HOME}/opensource/include/glib-2.0)
+include_directories($ENV{MX_SDK_HOME}/opensource/lib/glib-2.0/include)
+message($ENV{MX_SDK_HOME})
+link_directories($ENV{MX_SDK_HOME}/lib)
+link_directories($ENV{MX_SDK_HOME}/opensource/lib/)
+
+
+add_compile_options(-std=c++11 -fPIC -fstack-protector-all -pie -Wno-deprecated-declarations)
+add_compile_options("-DPLUGIN_NAME=${PLUGIN_NAME}")
+add_compile_options("-Dgoogle=mindxsdk_private")
+
+add_definitions(-DENABLE_DVPP_INTERFACE)
+
+include_directories(${ACL_LIB_PATH}/include)
+link_directories(${ACL_LIB_PATH}/lib64/)
+
+add_library(${TARGET_LIBRARY} SHARED src/PostProcess/FairmotMindsporePost.cpp)
+
+target_link_libraries(${TARGET_LIBRARY} glib-2.0 gstreamer-1.0 gobject-2.0 gstbase-1.0 gmodule-2.0)
+target_link_libraries(${TARGET_LIBRARY} plugintoolkit mxpidatatype mxbase)
+target_link_libraries(${TARGET_LIBRARY} -Wl,-z,relro,-z,now,-z,noexecstack -s)
+
+message("TARGET_LIBRARY:${TARGET_LIBRARY}.")
+
+
+add_executable(${TARGET_MAIN} src/main.cpp src/Fairmot.cpp)
+target_link_libraries(${TARGET_MAIN} ${TARGET_LIBRARY} glog  cpprest mxbase libascendcl.so opencv_world)

research/cv/fairmot/infer/mxbase/build.sh

+#!/bin/bash
+
+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+
+# env
+
+mkdir -p build
+cd build || exit
+
+function make_plugin() {
+  if ! cmake ..; then
+    echo "cmake failed."
+    return 1
+  fi
+
+  if ! (make); then
+    echo "make failed."
+    return 1
+  fi
+
+  return 0
+}
+
+if make_plugin; then
+  echo "INFO: Build successfully."
+else
+  echo "ERROR: Build failed."
+fi
+
+cd - || exit

research/cv/fairmot/infer/mxbase/src/Fairmot.cpp

+/*
+ * Copyright 2021 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include "Fairmot.h"
+
+#include <dirent.h>
+#include <stdio.h>
+#include <sys/stat.h>
+#define BOOST_BIND_GLOBAL_PLACEHOLDERS
+#include <algorithm>
+#include <map>
+#include <memory>
+#include <string>
+#include <utility>
+#include <vector>
+#include <boost/property_tree/json_parser.hpp>
+#include <opencv4/opencv2/core.hpp>
+#include <opencv4/opencv2/opencv.hpp>
+
+#include "MxBase/DeviceManager/DeviceManager.h"
+#include "MxBase/Log/Log.h"
+#include "acl/acl.h"
+namespace {
+const uint32_t YUV_BYTE_NU = 3;
+const uint32_t YUV_BYTE_DE = 2;
+const uint32_t FRAME_RATE = 25;
+const uint32_t MODEL_HEIGHT = 768;
+const uint32_t MODEL_WIDTH = 1280;
+const float CONF_THRES = 0.4;
+}  // namespace
+
+void PrintTensorShape(const std::vector<MxBase::TensorDesc> &tensorDescVec,
+                      const std::string &tensorName) {
+  LogInfo << "The shape of " << tensorName << " is as follows:";
+  for (size_t i = 0; i < tensorDescVec.size(); ++i) {
+    LogInfo << "  Tensor " << i << ":";
+    for (size_t j = 0; j < tensorDescVec[i].tensorDims.size(); ++j) {
+      LogInfo << "   dim: " << j << ": " << tensorDescVec[i].tensorDims[j];
+    }
+  }
+}
+
+APP_ERROR Fairmot::Init(const InitParam &initParam) {
+  deviceId_ = initParam.deviceId;
+  APP_ERROR ret = MxBase::DeviceManager::GetInstance()->InitDevices();
+  if (ret != APP_ERR_OK) {
+    LogError << "Init devices failed, ret=" << ret << ".";
+    return ret;
+  }
+  ret = MxBase::TensorContext::GetInstance()->SetContext(initParam.deviceId);
+  if (ret != APP_ERR_OK) {
+    LogError << "Set context failed, ret=" << ret << ".";
+    return ret;
+  }
+  dvppWrapper_ = std::make_shared<MxBase::DvppWrapper>();
+  ret = dvppWrapper_->Init();
+  if (ret != APP_ERR_OK) {
+    LogError << "DvppWrapper init failed, ret=" << ret << ".";
+    return ret;
+  }
+  model_ = std::make_shared<MxBase::ModelInferenceProcessor>();
+  ret = model_->Init(initParam.modelPath, modelDesc_);
+  if (ret != APP_ERR_OK) {
+    LogError << "ModelInferenceProcessor init failed, ret=" << ret << ".";
+    return ret;
+  }
+
+  PrintTensorShape(modelDesc_.inputTensors, "Model Input Tensors");
+  PrintTensorShape(modelDesc_.outputTensors, "Model Output Tensors");
+
+  MxBase::ConfigData configData;
+  const std::string checkTensor = initParam.checkTensor ? "true" : "false";
+
+  configData.SetJsonValue("CLASS_NUM", std::to_string(initParam.classNum));
+  configData.SetJsonValue("SCORE_THRESH",
+                          std::to_string(initParam.scoreThresh));
+  configData.SetJsonValue("IOU_THRESH", std::to_string(initParam.iouThresh));
+  configData.SetJsonValue("CHECK_MODEL", checkTensor);
+
+  auto jsonStr = configData.GetCfgJson().serialize();
+  std::map<std::string, std::shared_ptr<void>> config;
+  config["postProcessConfigContent"] = std::make_shared<std::string>(jsonStr);
+  post_ = std::make_shared<MxBase::FairmotMindsporePost>();
+  ret = post_->Init(config);
+  if (ret != APP_ERR_OK) {
+    LogError << "Fairmot init failed, ret=" << ret << ".";
+    return ret;
+  }
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::DeInit() {
+  dvppWrapper_->DeInit();
+  model_->DeInit();
+  post_->DeInit();
+  MxBase::DeviceManager::GetInstance()->DestroyDevices();
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::Inference(const std::vector<MxBase::TensorBase> &inputs,
+                             std::vector<MxBase::TensorBase> &outputs) {
+  auto dtypes = model_->GetOutputDataType();
+  for (size_t i = 0; i < modelDesc_.outputTensors.size(); ++i) {
+    std::vector<uint32_t> shape = {};
+    for (size_t j = 0; j < modelDesc_.outputTensors[i].tensorDims.size(); ++j) {
+      shape.push_back((uint32_t)modelDesc_.outputTensors[i].tensorDims[j]);
+    }
+    MxBase::TensorBase tensor(shape, dtypes[i],
+                              MxBase::MemoryData::MemoryType::MEMORY_DEVICE,
+                              deviceId_);
+    APP_ERROR ret = MxBase::TensorBase::TensorBaseMalloc(tensor);
+    if (ret != APP_ERR_OK) {
+      LogError << "TensorBaseMalloc failed, ret=" << ret << ".";
+      return ret;
+    }
+    outputs.push_back(tensor);
+  }
+  MxBase::DynamicInfo dynamicInfo = {};
+  dynamicInfo.dynamicType = MxBase::DynamicType::STATIC_BATCH;
+  auto startTime = std::chrono::high_resolution_clock::now();
+  APP_ERROR ret = model_->ModelInference(inputs, outputs, dynamicInfo);
+  auto endTime = std::chrono::high_resolution_clock::now();
+  double costMs = std::chrono::duration<double, std::milli>(endTime - startTime)
+                      .count();  // save time
+  inferCostTimeMilliSec += costMs;
+  if (ret != APP_ERR_OK) {
+    LogError << "ModelInference failed, ret=" << ret << ".";
+    return ret;
+  }
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::PostProcess(const std::vector<MxBase::TensorBase> &inputs,
+                               MxBase::JDETracker &tracker,
+                               MxBase::Files &file) {
+  APP_ERROR ret = post_->Process(inputs, tracker, file);
+  if (ret != APP_ERR_OK) {
+    LogError << "Process failed, ret=" << ret << ".";
+    return ret;
+  }
+  return APP_ERR_OK;
+}
+
+void SaveInferResult(const std::vector<MxBase::ObjectInfo> &objInfos,
+                     const std::string &resultPath) {
+  if (objInfos.empty()) {
+    LogWarn << "The predict result is empty.";
+    return;
+  }
+
+  namespace pt = boost::property_tree;
+  pt::ptree root, data;
+  int index = 0;
+  for (auto &obj : objInfos) {
+    ++index;
+    LogInfo << "BBox[" << index << "]:[x0=" << obj.x0 << ", y0=" << obj.y0
+            << ", x1=" << obj.x1 << ", y1=" << obj.y1
+            << "], confidence=" << obj.confidence << ", classId=" << obj.classId
+            << ", className=" << obj.className << std::endl;
+    pt::ptree item;
+    item.put("classId", obj.classId);
+    item.put("className", obj.className);
+    item.put("confidence", obj.confidence);
+    item.put("x0", obj.x0);
+    item.put("y0", obj.y0);
+    item.put("x1", obj.x1);
+    item.put("y1", obj.y1);
+
+    data.push_back(std::make_pair("", item));
+  }
+  root.add_child("data", data);
+  pt::json_parser::write_json(resultPath, root, std::locale(), false);
+}
+std::string RealPath(std::string path) {
+  char realPathMem[PATH_MAX] = {0};
+  char *realPathRet = nullptr;
+  realPathRet = realpath(path.data(), realPathMem);
+  if (realPathRet == nullptr) {
+    std::cout << "File: " << path << " is not exist.";
+    return "";
+  }
+
+  std::string realPath(realPathMem);
+  std::cout << path << " realpath is: " << realPath << std::endl;
+  return realPath;
+}
+
+DIR *OpenDir(std::string dirName) {
+  if (dirName.empty()) {
+    std::cout << " dirName is null ! " << std::endl;
+    return nullptr;
+  }
+  std::string realPath = RealPath(dirName);
+  struct stat s;
+  lstat(realPath.c_str(), &s);
+  if (!S_ISDIR(s.st_mode)) {
+    std::cout << "dirName is not a valid directory !" << std::endl;
+    return nullptr;
+  }
+  DIR *dir = opendir(realPath.c_str());
+  if (dir == nullptr) {
+    std::cout << "Can not open dir " << dirName << std::endl;
+    return nullptr;
+  }
+  std::cout << "Successfully opened the dir " << dirName << std::endl;
+  return dir;
+}
+
+std::vector<std::string> GetAllFiles(std::string dirName) {
+  struct dirent *filename;
+  DIR *dir = OpenDir(dirName);
+  if (dir == nullptr) {
+    return {};
+  }
+  std::vector<std::string> res;
+  while ((filename = readdir(dir)) != nullptr) {
+    std::string dName = std::string(filename->d_name);
+    if (dName == "." || dName == ".." || filename->d_type != DT_REG) {
+      continue;
+    }
+    res.emplace_back(std::string(dirName) + "/" + filename->d_name);
+  }
+  std::sort(res.begin(), res.end());
+  return res;
+}
+
+APP_ERROR Fairmot::ReadImageCV(const std::string &imgPath, cv::Mat &imageMat,
+                               ImageShape &imgShape) {
+  imageMat = cv::imread(imgPath, cv::IMREAD_COLOR);
+  imgShape.width = imageMat.cols;
+  imgShape.height = imageMat.rows;
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::ResizeImage(const cv::Mat &srcImageMat, cv::Mat &dstImageMat,
+                               ImageShape &imgShape) {
+  int height = 608;
+  int width = 1088;
+  float ratio = std::min(static_cast<float>(height) / srcImageMat.rows,
+                         static_cast<float>(width) / srcImageMat.cols);
+  std::vector<int> new_shape = {
+      static_cast<int>(round(srcImageMat.rows * ratio)),
+      static_cast<int>(round(srcImageMat.cols * ratio))};
+  int tmp = 2;
+  float dw = static_cast<float>((width - new_shape[1])) / tmp;
+  float dh = static_cast<float>((height - new_shape[0])) / tmp;
+  int top = round(dh - 0.1);
+  int bottom = round(dh + 0.1);
+  int left = round(dw - 0.1);
+  int right = round(dw + 0.1);
+  cv::Mat tmp_img;
+  cv::resize(srcImageMat, tmp_img, cv::Size(new_shape[1], new_shape[0]), 0, 0,
+             cv::INTER_AREA);
+  cv::Scalar value = cv::Scalar(127.5, 127.5, 127.5);
+  cv::copyMakeBorder(tmp_img, dstImageMat, top, bottom, left, right,
+                     cv::BORDER_CONSTANT, value);
+  imgShape.width = dstImageMat.cols;
+  imgShape.height = dstImageMat.rows;
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::CVMatToTensorBase(const cv::Mat &imageMat,
+                                     MxBase::TensorBase &tensorBase) {
+  const uint32_t dataSize = imageMat.cols * imageMat.rows * imageMat.channels();
+  MxBase::MemoryData memoryDataDst(dataSize, MxBase::MemoryData::MEMORY_DEVICE,
+                                   deviceId_);
+  MxBase::MemoryData memoryDataSrc(imageMat.data, dataSize,
+                                   MxBase::MemoryData::MEMORY_HOST_MALLOC);
+
+  APP_ERROR ret =
+      MxBase::MemoryHelper::MxbsMallocAndCopy(memoryDataDst, memoryDataSrc);
+  if (ret != APP_ERR_OK) {
+    LogError << GetError(ret) << "Memory malloc failed.";
+    return ret;
+  }
+  std::vector<uint32_t> shape = {static_cast<uint32_t>(imageMat.rows),
+                                 static_cast<uint32_t>(imageMat.cols),
+                                 static_cast<uint32_t>(imageMat.channels())};
+  tensorBase = MxBase::TensorBase(memoryDataDst, false, shape,
+                                  MxBase::TENSOR_DTYPE_UINT8);
+  return APP_ERR_OK;
+}
+
+APP_ERROR Fairmot::GetMetaMap(const ImageShape imgShape,
+                              const ImageShape resizeimgShape,
+                              MxBase::JDETracker &tracker) {
+  std::vector<float> c = {static_cast<float>(imgShape.width) / 2,
+                          static_cast<float>(imgShape.height) / 2};
+  float s = std::max(static_cast<float>(resizeimgShape.width) /
+                         static_cast<float>(resizeimgShape.height) *
+                         static_cast<float>(imgShape.height),
+                     static_cast<float>(imgShape.width)) *
+            1.0;
+  tracker.c = c;
+  tracker.s = s;
+  int tmp = 4;
+  tracker.out_height = resizeimgShape.height / tmp;
+  tracker.out_width = resizeimgShape.width / tmp;
+  return APP_ERR_OK;
+}
+
+void Fairmot::WriteResult(const std::string &result_filename,
+                          std::vector<MxBase::Results *> results) {
+  FILE *fp;
+  fp = std::fopen(result_filename.c_str(), "w");
+  for (int i = 0; i < results.size(); i++) {
+    std::vector<cv::Mat> online_tlwhs = (*results[i]).online_tlwhs;
+    std::vector<int> online_ids = (*results[i]).online_ids;
+    int frame_id = (*results[i]).frame_id;
+    for (int j = 0; j < online_tlwhs.size(); j++) {
+      if (online_ids[j] < 0) {
+        continue;
+      }
+      double x1, y1, w, h;
+      x1 = online_tlwhs[j].at<double>(0, 0);
+      y1 = online_tlwhs[j].at<double>(0, 1);
+      w = online_tlwhs[j].at<double>(0, 2);
+      h = online_tlwhs[j].at<double>(0, 3);
+      double x2, y2;
+      x2 = x1 + w;
+      y2 = y1 + h;
+      fprintf(fp, "%d,%d,%.13lf,%.13lf,%.13lf,%.13lf,%d,%d,%d,%d\n", frame_id,
+              (online_ids[j]), x1, y1, w, h, 1, -1, -1, -1);
+    }
+  }
+  fclose(fp);
+}
+
+APP_ERROR Fairmot::Process(const std::string &imgPath) {
+  ImageShape imageShape{};
+  ImageShape resizedImageShape{};
+  std::vector<std::string> seqs = {"/MOT20-01", "/MOT20-02", "/MOT20-03",
+                                   "/MOT20-05"};
+  std::string homePath = imgPath + "/result_Files";
+  if (access(homePath.c_str(), 0) != 0) {
+    std::string cmd = "mkdir " + homePath;
+    system(cmd.data());
+  }
+  for (const auto &seq : seqs) {
+    std::string result_filename = homePath + seq + ".txt";
+    std::string image_path = imgPath + "/train" + seq + "/img1";
+    std::vector<std::string> images = GetAllFiles(image_path);
+    int frame_rate = 25;
+    MxBase::JDETracker tracker(frame_rate);
+    MxBase::Files file;
+    for (const auto &image_file : images) {
+      LogInfo << image_file;
+      int tmp = 20;
+      if (file.frame_id % tmp == 0) {
+        LogInfo << "Processing frame " << file.frame_id;
+      }
+      cv::Mat imageMat;
+      APP_ERROR ret = ReadImageCV(image_file, imageMat, imageShape);
+      if (ret != APP_ERR_OK) {
+        LogError << "ReadImage failed, ret=" << ret << ".";
+        return ret;
+      }
+      ret = ResizeImage(imageMat, imageMat, resizedImageShape);
+      if (ret != APP_ERR_OK) {
+        LogError << "ResizeImage failed, ret=" << ret << ".";
+        return ret;
+      }
+      ret = GetMetaMap(imageShape, resizedImageShape, tracker);
+      if (ret != APP_ERR_OK) {
+        LogError << "GetMetaMap failed, ret=" << ret << ".";
+        return ret;
+      }
+      MxBase::TensorBase tensorBase;
+      ret = CVMatToTensorBase(imageMat, tensorBase);
+      if (ret != APP_ERR_OK) {
+        LogError << "CVMatToTensorBase failed, ret=" << ret << ".";
+        return ret;
+      }
+      std::vector<MxBase::TensorBase> inputs = {};
+      std::vector<MxBase::TensorBase> outputs = {};
+      inputs.push_back(tensorBase);
+      auto startTime = std::chrono::high_resolution_clock::now();
+      ret = Inference(inputs, outputs);
+      auto endTime = std::chrono::high_resolution_clock::now();
+      double costMs =
+          std::chrono::duration<double, std::milli>(endTime - startTime)
+              .count();  // save time
+      inferCostTimeMilliSec += costMs;
+      if (ret != APP_ERR_OK) {
+        LogError << "Inference failed, ret=" << ret << ".";
+        return ret;
+      }
+      tracker.seq = seq;
+      tracker.image_file = image_file;
+      ret = PostProcess(outputs, tracker, file);
+      if (ret != APP_ERR_OK) {
+        LogError << "PostProcess failed, ret=" << ret << ".";
+        return ret;
+      }
+    }
+    WriteResult(result_filename, file.results);
+  }
+  return APP_ERR_OK;
+}

research/cv/fairmot/infer/mxbase/src/Fairmot.h

+/*
+ * Copyright 2021 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#include <memory>
+#include <string>
+#include <vector>
+#include <opencv2/opencv.hpp>
+
+#include "MxBase/DeviceManager/DeviceManager.h"
+#include "MxBase/DvppWrapper/DvppWrapper.h"
+#include "MxBase/ModelInfer/ModelInferenceProcessor.h"
+#include "MxBase/PostProcessBases/ObjectPostProcessBase.h"
+#include "MxBase/Tensor/TensorContext/TensorContext.h"
+#include "PostProcess/FairmotMindsporePost.h"
+
+struct InitParam {
+  uint32_t deviceId;
+  std::string labelPath;
+  uint32_t classNum;
+  float iouThresh;
+  float scoreThresh;
+
+  bool checkTensor;
+  std::string modelPath;
+};
+
+struct ImageShape {
+  uint32_t width;
+  uint32_t height;
+};
+
+class Fairmot {
+ public:
+  APP_ERROR Init(const InitParam &initParam);
+  APP_ERROR DeInit();
+  APP_ERROR Inference(const std::vector<MxBase::TensorBase> &inputs,
+                      std::vector<MxBase::TensorBase> &outputs);
+  APP_ERROR PostProcess(const std::vector<MxBase::TensorBase> &inputs,
+                        MxBase::JDETracker &tracker, MxBase::Files &file);
+  APP_ERROR Process(const std::string &imgPath);
+  APP_ERROR ReadImageCV(const std::string &imgPath, cv::Mat &imageMat,
+                        ImageShape &imgShape);
+  APP_ERROR ResizeImage(const cv::Mat &srcImageMat, cv::Mat &dstImageMat,
+                        ImageShape &imgShape);
+  APP_ERROR CVMatToTensorBase(const cv::Mat &imageMat,
+                              MxBase::TensorBase &tensorBase);
+  APP_ERROR GetMetaMap(const ImageShape imgShape,
+                       const ImageShape resizeimgShape,
+                       MxBase::JDETracker &tracker);
+  void WriteResult(const std::string &result_filename,
+                   std::vector<MxBase::Results *> results);
+
+ private:
+  std::shared_ptr<MxBase::DvppWrapper> dvppWrapper_;
+  std::shared_ptr<MxBase::ModelInferenceProcessor> model_;
+  std::shared_ptr<MxBase::FairmotMindsporePost> post_;
+  MxBase::ModelDesc modelDesc_;
+  uint32_t deviceId_ = 0;
+  double inferCostTimeMilliSec = 0.0;
+};

research/cv/fairmot/infer/mxbase/src/PostProcess/FairmotMindsporePost.h

+/*
+ * Copyright 2021 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include <algorithm>
+#include <map>
+#include <memory>
+#include <string>
+#include <vector>
+#include <opencv4/opencv2/opencv.hpp>
+typedef int row;
+typedef double f64_mat_t[4][4]; /**< a matrix */
+typedef double f64_vec_t[4];
+#define ROW_TYPE INT
+typedef int col;
+#define COL_TYPE INT
+typedef double cost;
+#define COST_TYPE DOUBLE
+#define BIG 100000
+#if !defined TRUE
+#define TRUE 1
+#endif
+#if !defined FALSE
+#define FALSE 0
+#endif
+
+/*************** DATA TYPES *******************/
+
+typedef int boolean;
+#include "MxBase/CV/Core/DataType.h"
+#include "MxBase/ErrorCode/ErrorCode.h"
+#include "MxBase/PostProcessBases/PostProcessBase.h"
+
+namespace MxBase {
+class Results {
+ public:
+  Results(uint32_t frame_id, const std::vector<cv::Mat> &online_tlwhs,
+          const std::vector<int> &online_ids);
+  uint32_t frame_id;
+  std::vector<cv::Mat> online_tlwhs;
+  std::vector<int> online_ids;
+};
+class TrackState {
+ public:
+  uint32_t New = 0;
+  uint32_t Tracked = 1;
+  uint32_t Lost = 2;
+  uint32_t Removed = 3;
+};
+class BaseTrack {
+ public:
+  uint32_t trackId = 0;
+  bool activated = false;
+  uint32_t base_state;
+  int next_id();
+
+ private:
+  uint32_t count = 0;
+};
+class KalmanFilter {
+ public:
+  std::map<int, float> chi2inv95 = {{1, 3.8415}, {2, 5.9915}, {3, 7.8147},
+                                    {4, 9.4877}, {5, 11.070}, {6, 12.592},
+                                    {7, 14.067}, {8, 15.507}, {9, 16.919}};
+  uint32_t ndim;
+  float dt;
+  KalmanFilter();
+  void chol_subtitute(cv::Mat chol_factor, cv::Mat b, f64_vec_t *f_x, int n);
+  void cholesky_decomposition(const cv::Mat &A, cv::Mat &L);
+  void initiate(cv::Mat measurement, cv::Mat &mean, cv::Mat &covariance);
+  void multi_predict(cv::Mat &mean, std::vector<cv::Mat> &covariance);
+  cv::Mat GatingDistance(cv::Mat mean, cv::Mat covariance, cv::Mat measurements,
+                         bool only_position = false,
+                         const std::string &metric = "maha");
+  void project(cv::Mat &mean, cv::Mat &covariance);
+  void update(cv::Mat &mean, cv::Mat &covariance, cv::Mat measurement);
+
+ private:
+  cv::Mat motion_mat;
+  cv::Mat update_mat;
+  float std_weight_position;
+  float std_weight_velocity;
+};
+
+class STack : public BaseTrack {
+ public:
+  cv::Mat mean, covariance;
+  cv::Mat smooth_feat;
+  cv::Mat curr_feat;
+  bool is_activated;
+  KalmanFilter kalman_filter;
+  float score;
+  float alpha;
+  uint32_t tracklet_len;
+  int track_id;
+  uint32_t state;
+  uint32_t start_frame;
+  uint32_t frame_id;
+  std::vector<cv::Mat> features;
+  STack();
+  STack(cv::Mat tlwh, float score, cv::Mat temp_feat, uint32_t buffer_size);
+  void activate(const KalmanFilter &kalman_filter, uint32_t frame_id);
+  void re_activate(STack new_track, int frame_id, bool new_id = false);
+  cv::Mat tlwh_to_xyah(cv::Mat tlwh);
+  cv::Mat tlbr();
+  cv::Mat gettlwh();
+  void update(STack new_track, int frame_id, bool update_feature = true);
+
+ private:
+  cv::Mat tlwh;
+  void update_features(cv::Mat temp_feat);
+};
+class JDETracker {
+ public:
+  explicit JDETracker(uint32_t frame_rate);
+  std::vector<STack *> tracked_stracks;
+  std::vector<STack *> lost_stracks;
+  std::vector<STack *> removed_stracks;
+  uint32_t frame_id = 0;
+  uint32_t out_height = 0;
+  uint32_t out_width = 0;
+  std::vector<float> c;
+  float det_thresh;
+  float s = 0;
+  int buffer_size;
+  int max_time_lost;
+  int max_per_image;
+  std::string seq;
+  std::string image_file;
+  cv::Mat mean;
+  cv::Mat std;
+  KalmanFilter kalman_filter;
+};
+class Files {
+ public:
+  uint32_t frame_id = 0;
+  std::vector<Results *> results;
+};
+class FairmotMindsporePost : public PostProcessBase {
+ public:
+  FairmotMindsporePost() = default;
+
+  ~FairmotMindsporePost() = default;
+
+  FairmotMindsporePost(const FairmotMindsporePost &other) = default;
+
+  FairmotMindsporePost &operator=(const FairmotMindsporePost &other);
+
+  APP_ERROR Init(
+      const std::map<std::string, std::shared_ptr<void>> &postConfig) override;
+
+  APP_ERROR DeInit() override;
+
+  APP_ERROR Process(const std::vector<TensorBase> &tensors,
+                    MxBase::JDETracker &tracker, MxBase::Files &file);
+
+  bool IsValidTensors(const std::vector<TensorBase> &tensors) const override;
+
+ private:
+  std::vector<STack *> ObjectDetectionOutput(
+      const std::vector<TensorBase> &tensors, MxBase::JDETracker &tracker);
+  void TransformPreds(const cv::Mat &coords, MxBase::JDETracker tracker,
+                      cv::Mat &target_coords);
+  void PostProcess(cv::Mat &det, const MxBase::JDETracker &tracker);
+  void TensorBaseToCVMat(cv::Mat &imageMat, const MxBase::TensorBase &tensor);
+  void FuseMotion(MxBase::KalmanFilter &kalman_filter, cv::Mat &cost_matrix,
+                  std::vector<STack *> tracks, std::vector<STack *> detections,
+                  bool only_position = false, float lambda_ = 0.98);
+  std::vector<cv::Mat> LinearAssignment(cv::Mat cost_matrix, float thresh);
+  void lap(cost **assigncost, col *rowsol, row *colsol, cost *u, cost *v,
+           int row, int col);
+  std::vector<cv::Mat> get_lap(cost **assigncost, col *rowsol, row *colsol,
+                               cost *u, cost *v, int row, int col, int dim);
+  std::vector<STack *> JointStracks(std::vector<STack *> tlista,
+                                    std::vector<STack *> tlistb);
+  std::vector<STack *> SubStracks(std::vector<STack *> tlista,
+                                  std::vector<STack *> tlistb);
+  void RemoveDuplicateStracks(std::vector<STack *> &stracksa,
+                              std::vector<STack *> &stracksb);
+  cv::Mat IouDistance(std::vector<STack *> atracks,
+                      std::vector<STack *> btracks);
+  cv::Mat BboxOverlaps(std::vector<cv::Mat> boxes,
+                       std::vector<cv::Mat> query_boxes);
+  std::vector<STack *> Get_output_stracks(
+      MxBase::JDETracker &tracker,
+      const std::vector<STack *> &activated_starcks,
+      const std::vector<STack *> &refind_stracks,
+      std::vector<STack *> lost_stracks, std::vector<STack *> removed_stracks);
+  void Get_detections(cv::Mat det, std::vector<STack *> &detections,
+                      cv::Mat id_feature);
+  void Get_dists(cv::Mat &dists, std::vector<STack *> &detections,
+                 std::vector<STack *> &strack_pool);
+  void Update_Starcks(const std::vector<STack *> &strack_pool,
+                      std::vector<STack *> &detections, const cv::Mat &matches,
+                      std::vector<STack *> &activated_starcks,
+                      std::vector<STack *> &refind_stracks,
+                      const MxBase::JDETracker &tracker);
+  void get_result(int *collist, int cols, cost *d, cost min, int rows,
+                  int &endofpath, row *colsol, cost **assigncost, cost *v,
+                  int *pred);
+  void func(int &numfree, int *free, cost **assigncost, row *colsol, cost *v,
+            col *rowsol, int cols);
+  void MultiPredict(std::vector<STack *> &stracks);
+  const uint32_t DEFAULT_CLASS_NUM_MS = 80;
+  const float DEFAULT_SCORE_THRESH_MS = 0.7;
+  const float DEFAULT_IOU_THRESH_MS = 0.5;
+  const uint32_t DEFAULT_RPN_MAX_NUM_MS = 1000;
+  const uint32_t DEFAULT_MAX_PER_IMG_MS = 128;
+
+  uint32_t classNum_ = DEFAULT_CLASS_NUM_MS;
+  float scoreThresh_ = DEFAULT_SCORE_THRESH_MS;
+  float iouThresh_ = DEFAULT_IOU_THRESH_MS;
+  uint32_t rpnMaxNum_ = DEFAULT_RPN_MAX_NUM_MS;
+  uint32_t maxPerImg_ = DEFAULT_MAX_PER_IMG_MS;
+};
+
+extern "C" {
+std::shared_ptr<MxBase::FairmotMindsporePost> GetObjectInstance();
+}
+}  // namespace MxBase

research/cv/fairmot/infer/mxbase/src/main.cpp

+/*
+ * Copyright 2021 Huawei Technologies Co., Ltd. All rights reserved.
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "Fairmot.h"
+#include "MxBase/Log/Log.h"
+
+namespace {
+const uint32_t DEVICE_ID = 0;
+}  // namespace
+
+int main(int argc, char *argv[]) {
+  int num = 2;
+  if (argc <= num) {
+    LogWarn << "Please input image path, such as './Fairmot_mindspore "
+               "[om_file_path] [img_path]'.";
+    return APP_ERR_OK;
+  }
+
+  InitParam initParam = {};
+  initParam.deviceId = DEVICE_ID;
+
+  initParam.checkTensor = true;
+
+  initParam.modelPath = argv[1];
+  auto inferFairmot = std::make_shared<Fairmot>();
+  APP_ERROR ret = inferFairmot->Init(initParam);
+  if (ret != APP_ERR_OK) {
+    LogError << "Fairmot init failed, ret=" << ret << ".";
+    return ret;
+  }
+
+  std::string imgPath = argv[2];
+  ret = inferFairmot->Process(imgPath);
+  if (ret != APP_ERR_OK) {
+    LogError << "Fairmot process failed, ret=" << ret << ".";
+    inferFairmot->DeInit();
+    return ret;
+  }
+  inferFairmot->DeInit();
+  return APP_ERR_OK;
+}

research/cv/fairmot/infer/requirements.txt

+motmetrics
+lap
+openpyxl
+cython_bbox

research/cv/fairmot/infer/sdk/api/infer.py

+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+"""sdk infer"""
+import json
+import logging
+import MxpiDataType_pb2 as MxpiDataType
+from StreamManagerApi import StreamManagerApi, MxDataInput, InProtobufVector, MxProtobufIn, StringVector
+
+from config import config as cfg
+
+
+class SdkApi:
+    """sdk api"""
+    INFER_TIMEOUT = cfg.INFER_TIMEOUT
+    STREAM_NAME = cfg.STREAM_NAME
+
+    def __init__(self, pipeline_cfg):
+        self.pipeline_cfg = pipeline_cfg
+        self._stream_api = None
+        self._data_input = None
+        self._device_id = None
+
+    def init(self):
+        """sdk init """
+        with open(self.pipeline_cfg, 'r') as fp:
+            self._device_id = int(
+                json.loads(fp.read())[self.STREAM_NAME]["stream_config"]
+                ["deviceId"])
+            print("The device id: {}.".format(self._device_id))
+
+        # create api
+        self._stream_api = StreamManagerApi()
+
+        # init stream mgr
+        ret = self._stream_api.InitManager()
+        if ret != 0:
+            print(f"Failed to init stream manager, ret={ret}.")
+            return False
+
+        # create streams
+        with open(self.pipeline_cfg, 'rb') as fp:
+            pipe_line = fp.read()
+
+        ret = self._stream_api.CreateMultipleStreams(pipe_line)
+        if ret != 0:
+            print(f"Failed to create stream, ret={ret}.")
+            return False
+
+        self._data_input = MxDataInput()
+
+        return True
+
+    def __del__(self):
+        """del sdk"""
+        if not self._stream_api:
+            return
+
+        self._stream_api.DestroyAllStreams()
+
+    def send_data_input(self, stream_name, plugin_id, input_data):
+        """input data use SendData"""
+        data_input = MxDataInput()
+        data_input.data = input_data
+        unique_id = self._stream_api.SendData(stream_name, plugin_id,
+                                              data_input)
+        if unique_id < 0:
+            logging.error("Fail to send data to stream.")
+            return False
+        return True
+
+    def _send_protobuf(self, stream_name, plugin_id, element_name, buf_type,
+                       pkg_list):
+        """input data use SendProtobuf"""
+        protobuf = MxProtobufIn()
+        protobuf.key = element_name.encode("utf-8")
+        protobuf.type = buf_type
+        protobuf.protobuf = pkg_list.SerializeToString()
+        protobuf_vec = InProtobufVector()
+        protobuf_vec.push_back(protobuf)
+        err_code = self._stream_api.SendProtobuf(stream_name, plugin_id,
+                                                 protobuf_vec)
+        if err_code != 0:
+            logging.error(
+                "Failed to send data to stream, stream_name(%s), plugin_id(%s), element_name(%s), "
+                "buf_type(%s), err_code(%s).", stream_name, plugin_id,
+                element_name, buf_type, err_code)
+            return False
+        return True
+
+    def send_img_input(self, stream_name, plugin_id, element_name, input_data,
+                       img_size):
+        """use cv input to sdk"""
+        vision_list = MxpiDataType.MxpiVisionList()
+        vision_vec = vision_list.visionVec.add()
+        vision_vec.visionInfo.format = 1
+        vision_vec.visionInfo.width = img_size[1]
+        vision_vec.visionInfo.height = img_size[0]
+        vision_vec.visionInfo.widthAligned = img_size[1]
+        vision_vec.visionInfo.heightAligned = img_size[0]
+        vision_vec.visionData.memType = 0
+        vision_vec.visionData.dataStr = input_data
+        vision_vec.visionData.dataSize = len(input_data)
+        buf_type = b"MxTools.MxpiVisionList"
+        return self._send_protobuf(stream_name, plugin_id, element_name,
+                                   buf_type, vision_list)
+
+    def get_result(self, stream_name, out_plugin_id=0):
+        """get_result"""
+        key_vec = StringVector()
+        key_vec.push_back(b'mxpi_modelinfer0')
+        infer_result = self._stream_api.GetProtobuf(
+            stream_name, out_plugin_id, key_vec)
+        result = MxpiDataType.MxpiTensorPackageList()
+        result.ParseFromString(infer_result[0].messageBuf)
+        return result.tensorPackageVec[0].tensorVec[0].dataStr, result.tensorPackageVec[0].tensorVec[1].dataStr

research/cv/fairmot/infer/sdk/config/config.py

+# Copyright 2021 Huawei Technologies Co., Ltd
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+# ============================================================================
+"""config"""
+
+STREAM_NAME = "im_fairmot"
+MODEL_WIDTH = 1088
+MODEL_HEIGHT = 608
+
+INFER_TIMEOUT = 100000
+
+TENSOR_DTYPE_FLOAT32 = 0
+TENSOR_DTYPE_FLOAT16 = 1
+TENSOR_DTYPE_INT8 = 2

research/cv/fairmot/infer/sdk/config/fairmot.pipeline

+{
+    "im_fairmot": {
+        "stream_config": {
+            "deviceId": "0"
+        },
+        "appsrc0": {
+            "props": {
+                "blocksize": "409600"
+            },
+            "factory": "appsrc",
+            "next": "mxpi_modelinfer0"
+        },
+        "mxpi_modelinfer0": {
+            "props": {
+                "dataSource": "appsrc0",
+                "modelPath": "../convert/fairmot.om",
+                "tensorFormat": "1"
+            },
+            "factory": "mxpi_modelinfer",
+            "next": "mxpi_dataserialize0"
+        },
+        "mxpi_dataserialize0": {
+            "props": {
+                "outputDataKeys": "mxpi_modelinfer0"
+            },
+            "factory": "mxpi_dataserialize",
+            "next": "appsink0"
+        },
+        "appsink0": {
+            "factory": "appsink"
+        }
+    }
+}
\ No newline at end of file