面向移动机器人大视角运动的图神经网络视觉SLAM算法

doi:10.16182/j.issn1004731x.joss.23-0053

摘要/Abstract

摘要：

针对移动机器人在大视角运动下光照变化剧烈或遭遇纹理稀疏场景易出现特征点提取困难，极端角度下特征难以匹配导致对极几何计算误差较大问题，提出一种融合改进图神经网络的视觉SLAM算法。基于先验位置估计的特征提取网络，通过先验位置估计实现图像特征点快速均匀检测与描述，构建真实准确的特征点信息。基于图注意力机制的特征匹配网络，通过消息传递图神经网络聚合特征点信息，使用自我与联合注意力机制对前后图像帧分权重特征匹配。将特征提取与匹配图神经网络与ORB-SLAM2系统后端非线性优化、闭环修正、局部建图模块融合，提出一个完整的单目视觉GNN-SLAM系统。实验结果表明：该算法在KITTI数据集上与ORB-SLAM2算法相比，绝对轨迹误差降低37.59%，相对位姿误差降低19.67%。

关键词: 同步定位与地图构建, 大视角运动, 图神经网络, 图注意力机制, 移动机器人

Abstract:

Aimed at the difficulty of feature point extraction in mobile robots with drastic changes in illumination or sparse texture scenes under large-angle view motion, difficulty in matching features at extreme angles leads to large errors in Epipolar Geometry calculations,a fusion of an improved graph neural network based visual SLAM algorithm (GNN-SLAM)is proposed.The priori location estimation feature extraction network is proposed to achieve fast and uniform detection and description of image feature points by a priori location estimation and to construct real and accurate feature point information.The graph attention mechanism feature matching network is proposed to aggregate feature point information through message passing graph neural network, and then use self and joint attention mechanism for before and after image frame weighted feature matching.The feature extraction and matching map neural network is fused with the back-end nonlinear optimization, closed-loop correction, and local mapping modules of the ORB-SLAM2 system to propose a complete monocular vision GNN-SLAM system. The experimental results show that:compared with the ORB-SLAM2 algorithm on the KITTI dataset, the absolute trajectory error of this algorithm is reduced by 37.59%, and the relative pose error is reduced by 19.67%.

Key words: SLAM, large-angle view motion, graph neural network, graph attention mechanism, mobile robot

中图分类号:

TP242

刘金辉,陈孟元,韩朋朋等 . 面向移动机器人大视角运动的图神经网络视觉SLAM算法[J]. 系统仿真学报, 2024, 36(5): 1043-1060.

Liu Jinhui,Chen Mengyuan,Han Pengpeng,et al . A Graph Neural Network Visual SLAM Algorithm for Large-angle View Motion[J]. Journal of System Simulation, 2024, 36(5): 1043-1060.

图/表 16

图1

图2

图3

图4

图5

图6

表1

图7

图8

表2

图9

表3

图10

图11

图12

表4

参考文献 33

1	Smith R C, Cheeseman P. On the Representation and Estimation of Spatial Uncertainty[J]. The International Journal of Robotics Research, 1986, 5(4): 56-68.
2	Dissanayake M W M G, Newman P, Clark S, et al. A Solution to the Simultaneous Localization and Map Building (SLAM) Problem[J]. IEEE Transactions on Robotics and Automation, 2001, 17(3): 229-241.
3	王晨捷, 罗斌, 李成源, 等. 无人机视觉SLAM协同建图与导航[J]. 测绘学报, 2020, 49(6): 767-776.
	Wang Chenjie, Luo Bin, Li Chengyuan, et al. The Collaborative Mapping and Navigation Based on Visual SLAM in UAV Platform[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(6): 767-776.
4	Mur-Artal Raúl, Tardós Juan D. ORB-SLAM2: An Open-source SLAM System for Monocular, Stereo, and RGB-D Cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262.
5	Rublee E, Rabaud V, Konolige K, et al. ORB: An Efficient Alternative to SIFT or SURF[C]//2011 International Conference on Computer Vision. Piscataway, NJ, USA: IEEE, 2011: 2564-2571.
6	聂伟, 文怀志, 谢良波, 等. 一种基于单目视觉的无人机室内定位方法[J]. 电子与信息学报, 2022, 44(3): 906-914.
	Nie Wei, Wen Huaizhi, Xie Liangbo, et al. Indoor Localization of UAV Using Monocular Vision[J]. Journal of Electronics & Information Technology, 2022, 44(3): 906-914.
7	丁文东, 徐德, 刘希龙, 等. 移动机器人视觉里程计综述[J]. 自动化学报, 2018, 44(3): 385-400.
	Ding Wendong, Xu De, Liu Xilong, et al. Review on Visual Odometry for Mobile Robots[J]. Acta Automatica Sinica, 2018, 44(3): 385-400.
8	陈孟元, 张玉坤, 田德红, 等. 基于兴趣倾向机制的仿生SLAM算法[J]. 电子与信息学报, 2022, 44(5): 1743-1753.
	Chen Mengyuan, Zhang Yukun, Tian Dehong, et al. Bionic SLAM Algorithm Based on Interest Tendency Mechanism[J]. Journal of Electronics & Information Technology, 2022, 44(5): 1743-1753.
9	邸凯昌, 万文辉, 赵红颖, 等. 视觉SLAM技术的进展与应用[J]. 测绘学报, 2018, 47(6): 770-779.
	Di Kaichang, Wan Wenhui, Zhao Hongying, et al. Progress and Applications of Visual SLAM[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6): 770-779.
10	牛珉玉, 黄宜庆. 基于动态耦合与空间数据关联的RGB-D SLAM算法[J]. 机器人, 2022, 44(3): 333-342.
	Niu Minyu, Huang Yiqing. An RGB-D SLAM Algorithm Based on Dynamic Coupling and Spatial Data Association[J]. Robot, 2022, 44(3): 333-342.
11	林景栋, 吴欣怡, 柴毅, 等. 卷积神经网络结构优化综述[J]. 自动化学报, 2020, 46(1): 24-37.
	Lin Jingdong, Wu Xinyi, Chai Yi, et al. Structure Optimization of Convolutional Neural Networks: A Survey[J]. Acta Automatica Sinica, 2020, 46(1): 24-37.
12	Tateno Keisuke, Tombari Federico, Laina Iro, et al. CNN-SLAM: Real-time Dense Monocular SLAM with Learned Depth Prediction[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 6565-6574.
13	Hou Yi, Zhang Hong, Zhou Shilin. Convolutional Neural Network-based Image Representation for Visual Loop Closure Detection[C]//2015 IEEE International Conference on Information and Automation. Piscataway, NJ, USA: IEEE, 2015: 2238-2245.
14	李维鹏, 张国良, 姚二亮, 等. 基于场景显著区域的改进闭环检测算法[J]. 机器人, 2017, 39(1): 23-35.
	Li Weipeng, Zhang Guoliang, Yao Erliang, et al. An Improved Loop Closure Detection Algorithm Based on Scene Salient Regions[J]. Robot, 2017, 39(1): 23-35.
15	Ganti Pranav, Waslander Steven L. Network Uncertainty Informed Semantic Feature Selection for Visual SLAM[C]//2019 16th Conference on Computer and Robot Vision (CRV). Piscataway, NJ, USA: IEEE, 2019: 121-128.
16	Tang Jiexiong, Ericson Ludvig, Folkesson John, et al. GCNv2: Efficient Correspondence Prediction for Real-time SLAM[J]. IEEE Robotics and Automation Letters, 2019, 4(4): 3505-3512.
17	Li Dongjiang, Shi Xuesong, Long Qiwei, et al. DXSLAM: A Robust and Efficient Visual SLAM System with Deep Features[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway, NJ, USA: IEEE, 2020: 4958-4965.
18	Lee J B, Rossi R A, Kim S, et al. Attention Models in Graphs: A Survey[J]. ACM Transactions on Knowledge Discovery from Data, 2019, 13(6): 62.
19	Zhuang Chenyi, Ma Qiang. Dual Graph Convolutional Networks for Graph-based Semi-supervised Classification[C]//Proceedings of the 2018 World Wide Web Conference. Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee, 2018: 499-508.
20	Zhang Xiwu, Su Yan, Zhu Xinhua. Loop Closure Detection for Visual SLAM Systems Using Convolutional Neural Network[C]//2017 23rd International Conference on Automation and Computing (ICAC). Piscataway, NJ, USA: IEEE, 2017: 1-6.
21	Scarselli Franco, Gori Marco, Ah Chung Tsoi, et al. The Graph Neural Network Model[J]. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80.
22	徐冰冰, 岑科廷, 黄俊杰, 等. 图卷积神经网络综述[J]. 计算机学报, 2020, 43(5): 755-780.
	Xu Bingbing, Cen Keting, Huang Junjie, et al. A Survey on Graph Convolutional Neural Network[J]. Chinese Journal of Computers, 2020, 43(5): 755-780.
23	吴博, 梁循, 张树森, 等. 图神经网络前沿进展与应用[J]. 计算机学报, 2022, 45(1): 35-68.
	Wu Bo, Liang Xun, Zhang Shusen, et al. Advances and Applications in Graph Neural Network[J]. Chinese Journal of Computers, 2022, 45(1): 35-68.
24	Jia Kui, Chan T H, Zeng Zinan, et al. ROML: A Robust Feature Correspondence Approach for Matching Objects in a Set of Images[J]. International Journal of Computer Vision, 2016, 117(2): 173-197.
25	Sarlin Paul-Edouard, DeTone D, Malisiewicz T, et al. SuperGlue: Learning Feature Matching with Graph Neural Networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2020: 4937-4946.
26	Elmoogy Ahmed, Dong Xiaodai, Lu Tao, et al. Pose-GNN: Camera Pose Estimation System Using Graph Neural Networks[EB/OL]. (2021-03-17) [2022-12-15]. .
27	Wang Limin, Guo Sheng, Huang Weilin, et al. Places205-VGGNet Models for Scene Recognition[EB/OL]. (2015-08-07) [2022-11-07]. .
28	Meng Zihang, Ravi S N, Singh V. Physarum Powered Differentiable Linear Programming Layers and Applications[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(10): 8939-8949.
29	Lin T Y, Maire M, Belongie S, et al. Microsoft COCO: Common Objects in Context[C]//Computer Vision-ECCV 2014. Cham: Springer International Publishing, 2014: 740-755.
30	Sturm Jürgen, Engelhard Nikolas, Endres Felix, et al. A Benchmark for the Evaluation of RGB-D SLAM Systems[C]//2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, NJ, USA: IEEE, 2012: 573-580.
31	Geiger A, Lenz P, Stiller C, et al. Vision Meets Robotics: The KITTI Dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237.
32	Balntas V, Lenc K, Vedaldi A, et al. HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 3852-3861.
33	DeTone D, Malisiewicz T, Rabinovich A. SuperPoint: Self-supervised Interest Point Detection and Description[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway, NJ, USA: IEEE, 2018: 337-33712.

数据集	ORB	Superpoint	GCNv2	本文
i_books	0.572	0.591	0.627	0.693
i_castle	0.583	0.614	0.641	0.716
i_fruits	0.537	0.569	0.599	0.593
i_ski	0.564	0.576	0.598	0.645
i_whitebuilding	0.524	0.513	0.545	0.617

数据集	ORB+FLANN			Superpoint+FLANN			GCNv2+FLANN			本文
数据集	@30°	@60°	@90°	@30°	@60°	@90°	@30°	@60°	@90°	@30°	@60°	@90°
v_abstract	0.643	0.465	0.298	0.620	0.473	0.239	0.728	0.533	0.295	0.802	0.739	0.507
v_bees	0.651	0.482	0.262	0.702	0.536	0.388	0.648	0.499	0.386	0.857	0.686	0.569
v_beyus	0.682	0.503	0.312	0.608	0.449	0.256	0.612	0.428	0.324	0.829	0.737	0.571
v_home	0.605	0.396	0.274	0.633	0.405	0.293	0.730	0.475	0.252	0.796	0.706	0.535
v_woman	0.597	0.391	0.238	0.652	0.433	0.264	0.671	0.416	0.241	0.840	0.744	0.522

数据集	ORB-SLAM2		DX-SLAM		GCNv2-SLAM		本文
数据集	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE
fr1/360	―	―	―	―	0.065	0.082	0.145	0.199
fr1/floor	0.390	0.469	―	―	0.138	0.167	0.019	0.038
fr1/desk	0.081	0.101	0.300	0.364	0.188	0.230	0.039	0.050
fr1/desk2	0.571	0.999	0.343	0.416	0.156	0.197	0.047	0.075
fr1/room	0.295	0.354	0.181	0.212	0.246	0.305	0.070	0.111
fr2/desk	0.931	1.149	0.581	0.712	0.160	0.199	0.109	0.133
fr3/long_office_household	1.222	1.553	0.791	1.007	0.363	0.455	0.169	0.176

算法	特征提取/ms	特征匹配/ms	总时间/ms	平均帧率/(帧/s)
ORB-SLAM2	12.88	15.06	27.94	30
DX-SLAM	32.42	39.24	71.66	15
GCNv2-SLAM	19.45	31.57	51.02	20
本文	16.32	30.79	47.11	20

[1]	吕金虎, 蒋弘毅, 刘德元, 谭少林. 基于图神经网络的复杂系统建模与仿真[J]. 系统仿真学报, 2025, 37(7): 1624-1638.
[2]	王子怡, 张凯, 钱殿伟, 刘玉贞. 一种基于DRL的分布式装备体系优选方法[J]. 系统仿真学报, 2025, 37(6): 1565-1573.
[3]	周晓晖, 李研强, 王勇, 赵德财, 杨逍瑶. 基于双启发式信息蚁群算法的机器人路径规划[J]. 系统仿真学报, 2025, 37(5): 1280-1289.
[4]	喻蝶, 鲍柏仲, 司言, 段暕, 詹小斌, 史铁林. 基于搜索步优化A*算法的移动机器人路径规划[J]. 系统仿真学报, 2025, 37(4): 1041-1050.
[5]	林桂娟, 李子涵, 王宇. 基于全局关键点提取的改进A*算法全局路径规划研究[J]. 系统仿真学报, 2025, 37(3): 667-678.
[6]	李炯逸, 李强, 张新闻, Htet Zin Myo, 蔡永斌. 移动机器人用改进的双向A*二次路径规划算法[J]. 系统仿真学报, 2025, 37(2): 498-507.
[7]	李孝斌, 胡冰, 尹超, 李波, 马军. 基于时空图卷积的汽车配件供应链需求预测与仿真分析[J]. 系统仿真学报, 2025, 37(12): 3060-3074.
[8]	张文康, 孙霄峰, 钟一平, 尹勇. 基于图神经网络的船舶液舱晃荡数值仿真[J]. 系统仿真学报, 2025, 37(12): 3087-3099.
[9]	张弛, 魏巍. 基于改进人工势场法的移动机器人路径规划[J]. 系统仿真学报, 2025, 37(11): 2918-2926.
[10]	杨兰英, 李超, 邹海锋, 万江涛, 张仁强, 刘惠, 卢宏. 基于改进蚁群算法与A*算法相融合的机器人路径规划优化[J]. 系统仿真学报, 2025, 37(11): 2956-2965.
[11]	罗毅, 邓嘉. 基于改进RRT-Connect与DWA融合的移动机器人路径规划[J]. 系统仿真学报, 2025, 37(10): 2545-2556.
[12]	许建民, 宋雷, 邓冬冬, 陈尧箬, 杨炜. 基于多尺度A*与优化DWA算法融合的移动机器人路径规划[J]. 系统仿真学报, 2025, 37(1): 257-270.
[13]	姚万业, 庞泽伟, 孙沛杰, 王祝. 基于窗口化匹配估计的ORB-SLAM算法研究[J]. 系统仿真学报, 2024, 36(9): 2032-2042.
[14]	孙海杰, 伞红军, 肖乐, 姚得鑫, 陈久朋, 杨晓园. 一种改进的移动机器人路径规划算法[J]. 系统仿真学报, 2024, 36(9): 2193-2207.
[15]	康亮, 杜奕, 尹丽华. 一种面向异质多移动机器人的改进猫群算法[J]. 系统仿真学报, 2024, 36(8): 1958-1968.