目录
1.执行流程
2. SegmentDynObject::SegmentDynObject
3. SegmentDynObject::GetSegmentation
1.执行流程
我们输入到命令行五个参数：
stereo_kitti path_to_vocabulary path_to_settings path_to_sequence (path_to_masks)分别是DynaSLAM双目例程中的可执行文件stereo_kittt、ORB字典文件、配置文件、序列文件、以及可选的存储mask的文件。
输入完成后，我们便执行起了双目例程，首先进入的是LoadImages函数：
ORB-SLAM2 ---- LoadImages函数解析
https://blog.csdn.net/qq_41694024/article/details/126285573 LoadImages函数的主要作用是将双目图像分割：将左目图像存储进vstrImageLeft容器中、右目图像存储进vstrImageRight容器中、对应的时间戳存储进vTimestamps中。
随后我们就开始加载MaskRCNN网络了：
// Initialize Mask R-CNN cout << "Loading Mask R-CNN. This could take a while..." << endl; DynaSLAM::SegmentDynObject *MaskNet; if (argc==5){ MaskNet = new DynaSLAM::SegmentDynObject(); } cout << "Mask net loaded!" << endl;
2. SegmentDynObject::SegmentDynObject
这个类方法首先做的就是通过ImportSettings加载Mask R-CNN网络配置：
void SegmentDynObject::ImportSettings(){ std::string strSettingsFile = "./Examples/RGB-D/MaskSettings.yaml"; cv::FileStorage fs(strSettingsFile.c_str(), cv::FileStorage::READ); fs["py_path"] >> this->py_path; fs["module_name"] >> this->module_name; fs["class_name"] >> this->class_name; fs["get_dyn_seg"] >> this->get_dyn_seg;%YAML:1.0 py_path: "./src/python/" module_name: "MaskRCNN" class_name: "Mask" get_dyn_seg: "GetDynSeg"主要是加载了几个参数：
@py_path ： src/python目录
@module_name ： MaskRCNN文件
@class_name：Mask类
@get_dyn_seg：GetDynSeg MaskRCNN中的方法获取mask信息
def GetDynSeg(image,image2=None): h = image.shape[0] w = image.shape[1] if len(image.shape) == 2: im = np.zeros((h,w,3)) im[:,:,0]=image im[:,:,1]=image im[:,:,2]=image image = im #if image2 is not None: # args+=[image2] # Run detection results = model.detect([image], verbose=0) # Visualize results r = results[0] i = 0 mask = np.zeros((h,w)) for roi in r['rois']: if class_names[r['class_ids'][i]] == 'person': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'bicycle': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'car': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'motorcycle': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'airplane': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'bus': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'train': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'truck': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'boat': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'bird': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'cat': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'dog': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'horse': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'sheep': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'cow': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'elephant': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'bear': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'zebra': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. if class_names[r['class_ids'][i]] == 'giraffe': image_m = r['masks'][:,:,i] mask[image_m == 1] = 1. i+=1 #print('GetSeg mask shape:',mask.shape) return masksetenv起到改变或增加环境变量的作用，用法如下：
int setenv(const char *name,const char * value,int overwrite);
@name为环境变量名称字符串。
@value则为变量内容。@overwrite用来决定是否要改变已存在的环境变量。
如果overwrite不为0，则改变环境变量原有内容，原有内容会被改为参数value所指的变量内容。如果overwrite为0，且该环境变量已有内容，则参数value会被忽略。
@reuturn 执行成功则返回0，有错误发生时返回-1。#include<stdlib.h> main() { char * p; if((p=getenv(“USER”))) printf(“USER =%s\n”,p); setenv(“USER”,”test”,1); printf(“USER=%s\n”,getenv(“USEr”)); unsetenv(“USER”); printf(“USER=%s\n”,getenv(“USER”)); }执行结果：USER = root USER = test USER = (null)
setenv("PYTHONPATH", this->py_path.c_str(), 1);这里就是将./src/python文件夹设置为this->py_path。初始化Python虚拟环境。
namespace DynaSLAM { class SegmentDynObject{ private: NDArrayConverter *cvt; /*!< Converter to NumPy Array from cv::Mat */ PyObject *py_module; /*!< Module of python where the Mask algorithm is implemented */ PyObject *py_class; /*!< Class to be instanced */ PyObject *net; /*!< Instance of the class */ std::string py_path; /*!< Path to be included to the environment variable PYTHONPATH */ std::string module_name; /*!< Detailed description after the member */ std::string class_name; /*!< Detailed description after the member */ std::string get_dyn_seg; /*!< Detailed description after the member */ void ImportSettings(); public: SegmentDynObject(); ~SegmentDynObject(); cv::Mat GetSegmentation(cv::Mat &image, std::string dir="no_save", std::string rgb_name="no_file"); };初始化Numpy向cv::Mat的转化器cvt、导入py文库文件的过程：
知识点请见博客：
如何在C++中使用一个Python类-[PyImport_ImportModule、PyModule_GetDict、PyDict_GetItemString、PyObject_CallFuncti]
SegmentDynObject::SegmentDynObject(){ std::cout << "Importing Mask R-CNN Settings..." << std::endl; //fs["py_path"] >> this->py_path; //fs["module_name"] >> this->module_name; //fs["class_name"] >> this->class_name; //fs["get_dyn_seg"] >> this->get_dyn_seg; ImportSettings(); std::string x; setenv("PYTHONPATH", this->py_path.c_str(), 1); x = getenv("PYTHONPATH"); Py_Initialize(); this->cvt = new NDArrayConverter(); this->py_module = PyImport_ImportModule(this->module_name.c_str()); assert(this->py_module != NULL); this->py_class = PyObject_GetAttrString(this->py_module, this->class_name.c_str()); assert(this->py_class != NULL); this->net = PyInstance_New(this->py_class, NULL, NULL); assert(this->net != NULL); std::cout << "Creating net instance..." << std::endl; cv::Mat image = cv::Mat::zeros(480,640,CV_8UC3); //Be careful with size!! std::cout << "Loading net parameters..." << std::endl; GetSegmentation(image); }py_module 的类型是PyObject *类型，通过PyImport_ImportModule方法读入src/python目录下的MaskRCNN.py文件，如果读入文件为空报错。
py_class 的类型是PyObject *类型，通过PyObject_GetAttrString方法导入已导入模块（MaskRCNN.py）中的方法或类（MaskRCNN.py中的Mask类）。
net 的类型是PyObject *类型，通过PyInstance_New方法使用类构造对象。
这样我们顺利地构造了MaskRCNN.py中的Mask类。
下面我们创建网络实例：Creating net instance....，首先创建一个480*640的空白图像。
调用GetSegmentation函数对该张图片进行处理。
3. SegmentDynObject::GetSegmentation
cv::Mat SegmentDynObject::GetSegmentation(cv::Mat &image,std::string dir, std::string name){ cv::Mat seg = cv::imread(dir+"/"+name,CV_LOAD_IMAGE_UNCHANGED); if(seg.empty()){ PyObject* py_image = cvt->toNDArray(image.clone()); assert(py_image != NULL); PyObject* py_mask_image = PyObject_CallMethod(this->net, const_cast<char*>(this->get_dyn_seg.c_str()),"(O)",py_image); seg = cvt->toMat(py_mask_image).clone(); seg.cv::Mat::convertTo(seg,CV_8U);//0 background y 1 foreground if(dir.compare("no_save")!=0){ DIR* _dir = opendir(dir.c_str()); if (_dir) {closedir(_dir);} else if (ENOENT == errno) { const int check = mkdir(dir.c_str(), S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH); if (check == -1) { std::string str = dir; str.replace(str.end() - 6, str.end(), ""); mkdir(str.c_str(), S_IRWXU | S_IRWXG | S_IROTH | S_IXOTH); } } cv::imwrite(dir+"/"+name,seg); } } return seg; }本方法将传入的image进行语义分割，得到mask图。
首先将image图片转换成tensor格式存储在py_image在，判断是否读取成功。
通过PyObject_CallMethod方法调用（this -> net 是MaskRCNN.py中Mask类实例化的一个对象，this->get_dyn_seg 是该类中的一个方法求得图像的mask信息，再将其转化成图片格式保存起来。返回图像。
现在Mask R-CNN网络初始化完毕，屏幕输出Mask net loaded!。
