基于改进目标检测的动态场景SLAM研究

doi:10.16182/j.issn1004731x.joss.22-1332

摘要/Abstract

摘要：

针对单目SLAM在动态场景下存在的对极约束误匹配问题，提出一种基于目标检测的动态特征点选择 方法，通过在特征提取时剔除SLAM系统前端图像帧中动态特征点，提高SLAM的定位精度。提出了一个改进的目标检测网络，利用重叠面积、距离相似度和余弦相似度构建描述边界框的回归损失函数，实现目标的准确定位，获得当前图像帧中物体特征点范围。判断物体类别，对于标记为动态的物体根据目标检测结果剔除前端图像帧中的动态特征点。根据静态特征点，采用对极约束进行两帧图像间的特征匹配估计位姿，对单目相机运动进行跟踪、建图与闭环检测。通过对目标检测网络的主干进行结构重参数化改进，提升推理过程的速度，保证整体系统运行的实时性。在公开数据集KITTI的11个序列上的实验结果表明：改进后的系统比ORB-SLAM3系统定位精度提升了23.4%，帧率可以达到30 帧/s以上，在保证实时运行的条件下能有效提高动态场景下单目SLAM系统定位精度。

关键词: 视觉SLAM, 对极约束, 特征匹配, 目标检测, IoU损失函数, 结构重参数化

Abstract:

Aiming at the epipolar constraint matching problem of monocular SLAM in dynamic scenes adynamic feature point selection method based on object detection is proposed, in which the dynamic feature points in the front-end image frame of SLAM system is eliminated during feature extraction to improve the localization accuracy of SLAM. An improved target detection network is proposed to construct a loss function to describe the bounding box by using the overlap area, distance similarity and cosine similarity, which can achieve the accurate localization of target objects and obtain the range of object feature points in the current image frame. The object category is judged in SLAM, and the dynamic feature points in the front-end image frame are rejected according to the target detection result for the objects marked as dynamic. Based on the static feature point results, the epipolar geometry is used for the feature matching between two frames to estimate pose the to carry out the tracking, map building and closed-loop detection of monocular camera motion. The speed of the inference process is improved by the structural reparameterization of the backbone of target detection network to ensure the real-time operation of the overall system. Experimental results on KITTI dataset show that the improved system improves the localization accuracy by 23.4% over ORB-SLAM3 system, and the frame rate can reach more than 30fps. The algorithm can effectively improve the localization accuracy of monocular SLAM system in dynamic scenes under the condition of ensuring the real-time operation.

Key words: visual SLAM, epipolar geometry, feature matching, object detection, IoU loss function, structural reparameterization

中图分类号:

TP391.9

史蓝兮,颜文旭,倪宏宇等 . 基于改进目标检测的动态场景SLAM研究[J]. 系统仿真学报, 2024, 36(4): 1028-1042.

Shi Lanxi,Yan Wenxu,Ni Hongyu,et al . Research on Dynamic Scene SLAM Based on Improved Object Detection[J]. Journal of System Simulation, 2024, 36(4): 1028-1042.

图/表 29

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

表1

图12

图13

图14

图15

图16

图17

表2

图18

图19

图20

图21

表3

图22

图23

表4

表5

表6

参考文献 24

1	Davison A J, Reid I D, Molton N D, et al. MonoSLAM: Real-time Single Camera SLAM[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2007, 29(6): 1052-1067.
2	张良桥, 陈国良, 许晓东, 等. 一种用于图像特征提取的改进ORB-SLAM算法[J]. 测绘通报, 2019(3): 16-20.
	Zhang Liangqiao, Chen Guoliang, Xu Xiaodong, et al. An Improved ORB-SLAM Algorithm for Feature Extraction[J]. Bulletin of Surveying and Mapping, 2019(3): 16-20.
3	Zhou Lipu, Wang Shengze, Kaess M. DPLVO: Direct Point-line Monocular Visual Odometry[J]. IEEE Robotics and Automation Letters, 2021, 6(4): 7113-7120.
4	Ban Xicheng, Wang Hongjian, Chen Tao, et al. Monocular Visual Odometry Based on Depth and Optical Flow Using Deep Learning[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-19.
5	Li Jinquan, Pei Ling, Zou Danping, et al. Attention-SLAM: A Visual Monocular SLAM Learning from Human Gaze[J]. IEEE Sensors Journal, 2021, 21(5): 6408-6420.
6	Lee Sehyung, Lim Jongwoo, Il Hong Suh. Progressive Feature Matching: Incremental Graph Construction and Optimization[J]. IEEE Transactions on Image Processing, 2020, 29: 6992-7005.
7	胡立华, 左威健, 张继福. 采用逆近邻与影响空间的图像特征误匹配剔除方法[J]. 计算机辅助设计与图形学学报, 2022, 34(3): 449-458.
	Hu Lihua, Zuo Weijian, Zhang Jifu. A Mismatch Elimination Method Based on Reverse Nearest Neighborhood and Influence Space[J]. Journal of Computer-Aided Design & Computer Graphics, 2022, 34(3): 449-458.
8	任彬, 宋海丽, 赵增旭, 等. 基于RANSAC的视觉里程计优化方法研究[J]. 仪器仪表学报, 2022, 43(6): 205-212.
	Ren Bin, Song Haili, Zhao Zengxu, et al. Study on Optimization Method of Visual Odometry Based on RANSAC[J]. Chinese Journal of Scientific Instrument, 2022, 43(6): 205-212.
9	盛超, 潘树国, 赵涛, 等. 基于图像语义分割的动态场景下的单目SLAM算法[J]. 测绘通报, 2020(1): 40-44.
	Sheng Chao, Pan Shuguo, Zhao Tao, et al. Monocular SLAM System in Dynamic Scenes Based on Image Semantic Segmentation[J]. Bulletin of Surveying and Mapping, 2020(1): 40-44.
10	Chang Jianfang, Dong Na, Li Donghui. A Real-time Dynamic Object Segmentation Framework for SLAM System in Dynamic Scenes[J]. IEEE Transactions on Instrumentation and Measurement, 2021, 70: 1-9.
11	刘瑞军, 王向上, 张晨, 等. 基于深度学习的视觉SLAM综述[J]. 系统仿真学报, 2020, 32(7): 1244-1256.
	Liu Ruijun, Wang Xiangshang, Zhang Chen, et al. A Survey on Visual SLAM Based on Deep Learning[J]. Journal of System Simulation, 2020, 32(7): 1244-1256.
12	张翠文, 张长伦, 何强, 等. 目标检测中框回归损失函数的研究[J]. 计算机工程与应用, 2021, 57(20): 97-103.
	Zhang Cuiwen, Zhang Changlun, He Qiang, et al. Research on Loss Function of Box Regression in Object Detection[J]. Computer Engineering and Applications, 2021, 57(20): 97-103.
13	Rezatofighi H, Tsoi N, Gwak J Y, et al. Generalized Intersection over Union: A Metric and a Loss for Bounding Box Regression[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 658-666.
14	Zheng Zhaohui, Wang Ping, Ren Dongwei, et al. Enhancing Geometric Factors in Model Learning and Inference for Object Detection and Instance Segmentation[J]. IEEE Transactions on Cybernetics, 2022, 52(8): 8574-8586.
15	Xue Hongtao, Wu Meng, Zhang Ziming, et al. Intelligent Diagnosis of Mechanical Faults of In-wheel Motor Based on Improved Artificial Hydrocarbon Networks[J]. ISA Transactions, 2022, 120: 360-371.
16	Zoph B, Vasudevan V, Shlens J, et al. Learning Transferable Architectures for Scalable Image Recognition[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 8697-8710.
17	Ding Xiaohan, Zhang Xiangyu, Ma Ningning, et al. RepVGG: Making VGG-style ConvNets Great Again[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2021: 13728-13737.
18	Campos Carlos, Elvira Richard, J Gómez Rodríguez Juan, et al. ORB-SLAM3: An Accurate Open-source Library for Visual, Visual-inertial, and Multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6): 1874-1890.
19	艾青林, 刘刚江, 徐巧宁. 动态环境下基于改进几何与运动约束的机器人RGB-D SLAM算法[J]. 机器人, 2021, 43(2): 167-176.
	Ai Qinglin, Liu Gangjiang, Xu Qiaoning. An RGB-D SLAM Algorithm for Robot Based on the Improved Geometric and Motion Constraints in Dynamic Environment[J]. Robot, 2021, 43(2): 167-176.
20	Shao Chunyan, Zhang Chi, Fang Zaojun, et al. A Deep Learning-based Semantic Filter for RANSAC-based Fundamental Matrix Calculation and the ORB-SLAM System[J]. IEEE Access, 2020, 8: 3212-3223.
21	João Carlos Virgolino Soares, Gattass Marcelo, Marco Antonio Meggiolaro. Visual SLAM in Human Populated Environments: Exploring the Trade-off between Accuracy and Speed of YOLO and Mask R-CNN[C]//2019 19th International Conference on Advanced Robotics (ICAR). Piscataway, NJ, USA: IEEE, 2019: 135-140.
22	Bescos Berta, Fácil José M, Civera Javier, et al. DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083.
23	Geiger A, Lenz P, Stiller C, et al. Vision Meets Robotics: The KITTI Dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237.
24	Wang Ke, Cao Chuan, Ma Sai, et al. An Optimization-Based Multi-sensor Fusion Approach Towards Global Drift-free Motion Estimation[J]. IEEE Sensors Journal, 2021, 21(10): 12228-12235.

损失函数	准确率	召回率	mAP@.5	mAP@.5:.95
L_CIoU	0.952	0.893	0.957	0.837
L_New-IoU	0.956	0.980	0.974	0.856

序列	平均值			标准差			均方根误差
序列	ORB-SLAM3	文献[24]	本文	ORB-SLAM3	文献[24]	本文	ORB-SLAM3	文献[24]	本文
00	0.513	13.238	0.396	0.286	5.222	0.245	0.587	15.165	0.466
01	―	6.505	2.021	―	2.631	1.407	―	7.017	2.462
02	0.735	9.155	0.727	0.535	6.015	0.618	0.907	10.594	0.954
03	0.042	0.865	0.025	0.042	0.267	0.019	0.059	0.906	0.032
04	0.043	0.247	0.006	0.022	0.126	0.003	0.048	0.277	0.007
05	0.164	5.997	0.185	0.056	3.093	0.075	0.173	6.748	0.200
06	0.513	3.157	0.434	0.286	1.786	0.259	0.587	3.627	0.505
07	0.229	1.353	0.175	0.114	1.515	0.101	0.256	2.484	0.212
08	4.770	4.913	3.758	3.743	3.432	2.723	6.063	5.993	4.641
09	0.608	4.866	0.287	0.570	3.953	0.126	0.834	6.269	0.314
10	0.293	3.158	0.379	0.295	1.778	0.331	0.474	3.753	0.503

序列	均方根误差/%			旋转误差/((˚)/m)
序列	ORB-SLAM3	文献[24]	本文	ORB-SLAM3	文献[24]	本文
00	1.334	1.791	1.468	0.767	0.216	0.474
01	―	3.632	0.958	―	0.064	2.352
02	0.692	1.207	0.166	0.272	0.158	0.302
03	0.591	1.174	0.403	0.107	0.058	0.169
04	0.669	0.617	0.552	0.147	0.046	0.14
05	1.022	0.988	1.347	0.328	0.077	0.497
06	2.601	0.797	1.607	0.229	0.057	0.193
07	1.122	0.812	1.126	0.191	0.075	0.182
08	10.855	1.479	6.392	0.184	0.072	0.19
09	4.113	1.612	1.041	0.282	0.067	0.246
10	2.022	1.318	1.504	0.183	0.083	0.186

序列	平均值			标准差			均方根误差
序列	ORB-SLAM3	O+Y	本文方法	ORB-SLAM3	O+Y	本文方法	ORB-SLAM3	O+Y	本文方法
04	0.043	0.034	0.006	0.022	0.02	0.003	0.048	0.039	0.007
07	0.229	0.377	0.175	0.114	0.237	0.101	0.256	0.446	0.212

序列	ORB-SLAM3		本文算法
序列	中位数	平均数	中位数	平均数
平均值	24.5	26.6	29.8	30.9
03	24.7	26.6	28.9	30.2
06	24.4	26.5	29.9	31.4
07	24.5	26.7	30.5	31.1