面向移动机器人大视角运动的图神经网络视觉SLAM算法

doi:10.16182/j.issn1004731x.joss.23-0053

摘要/Abstract

摘要：

针对移动机器人在大视角运动下光照变化剧烈或遭遇纹理稀疏场景易出现特征点提取困难，极端角度下特征难以匹配导致对极几何计算误差较大问题，提出一种融合改进图神经网络的视觉SLAM算法。基于先验位置估计的特征提取网络，通过先验位置估计实现图像特征点快速均匀检测与描述，构建真实准确的特征点信息。基于图注意力机制的特征匹配网络，通过消息传递图神经网络聚合特征点信息，使用自我与联合注意力机制对前后图像帧分权重特征匹配。将特征提取与匹配图神经网络与ORB-SLAM2系统后端非线性优化、闭环修正、局部建图模块融合，提出一个完整的单目视觉GNN-SLAM系统。实验结果表明：该算法在KITTI数据集上与ORB-SLAM2算法相比，绝对轨迹误差降低37.59%，相对位姿误差降低19.67%。

关键词: 同步定位与地图构建, 大视角运动, 图神经网络, 图注意力机制, 移动机器人

Abstract:

Aimed at the difficulty of feature point extraction in mobile robots with drastic changes in illumination or sparse texture scenes under large-angle view motion, difficulty in matching features at extreme angles leads to large errors in Epipolar Geometry calculations,a fusion of an improved graph neural network based visual SLAM algorithm (GNN-SLAM)is proposed.The priori location estimation feature extraction network is proposed to achieve fast and uniform detection and description of image feature points by a priori location estimation and to construct real and accurate feature point information.The graph attention mechanism feature matching network is proposed to aggregate feature point information through message passing graph neural network, and then use self and joint attention mechanism for before and after image frame weighted feature matching.The feature extraction and matching map neural network is fused with the back-end nonlinear optimization, closed-loop correction, and local mapping modules of the ORB-SLAM2 system to propose a complete monocular vision GNN-SLAM system. The experimental results show that:compared with the ORB-SLAM2 algorithm on the KITTI dataset, the absolute trajectory error of this algorithm is reduced by 37.59%, and the relative pose error is reduced by 19.67%.

Key words: SLAM, large-angle view motion, graph neural network, graph attention mechanism, mobile robot

中图分类号:

TP242

刘金辉,陈孟元,韩朋朋等 . 面向移动机器人大视角运动的图神经网络视觉SLAM算法[J]. 系统仿真学报, 2024, 36(5): 1043-1060.

Liu Jinhui,Chen Mengyuan,Han Pengpeng,et al . A Graph Neural Network Visual SLAM Algorithm for Large-angle View Motion[J]. Journal of System Simulation, 2024, 36(5): 1043-1060.

图/表 16

图1

图2

图3

图4

图5

图6

表1

图7

图8

表2

图9

表3

图10

图11

图12

表4

参考文献 33

1	Smith R C, Cheeseman P. On the Representation and Estimation of Spatial Uncertainty[J]. The International Journal of Robotics Research, 1986, 5(4): 56-68.
2	Dissanayake M W M G, Newman P, Clark S, et al. A Solution to the Simultaneous Localization and Map Building (SLAM) Problem[J]. IEEE Transactions on Robotics and Automation, 2001, 17(3): 229-241.
3	王晨捷, 罗斌, 李成源, 等. 无人机视觉SLAM协同建图与导航[J]. 测绘学报, 2020, 49(6): 767-776.
	Wang Chenjie, Luo Bin, Li Chengyuan, et al. The Collaborative Mapping and Navigation Based on Visual SLAM in UAV Platform[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(6): 767-776.
4	Mur-Artal Raúl, Tardós Juan D. ORB-SLAM2: An Open-source SLAM System for Monocular, Stereo, and RGB-D Cameras[J]. IEEE Transactions on Robotics, 2017, 33(5): 1255-1262.
5	Rublee E, Rabaud V, Konolige K, et al. ORB: An Efficient Alternative to SIFT or SURF[C]//2011 International Conference on Computer Vision. Piscataway, NJ, USA: IEEE, 2011: 2564-2571.
6	聂伟, 文怀志, 谢良波, 等. 一种基于单目视觉的无人机室内定位方法[J]. 电子与信息学报, 2022, 44(3): 906-914.
	Nie Wei, Wen Huaizhi, Xie Liangbo, et al. Indoor Localization of UAV Using Monocular Vision[J]. Journal of Electronics & Information Technology, 2022, 44(3): 906-914.
7	丁文东, 徐德, 刘希龙, 等. 移动机器人视觉里程计综述[J]. 自动化学报, 2018, 44(3): 385-400.
	Ding Wendong, Xu De, Liu Xilong, et al. Review on Visual Odometry for Mobile Robots[J]. Acta Automatica Sinica, 2018, 44(3): 385-400.
8	陈孟元, 张玉坤, 田德红, 等. 基于兴趣倾向机制的仿生SLAM算法[J]. 电子与信息学报, 2022, 44(5): 1743-1753.
	Chen Mengyuan, Zhang Yukun, Tian Dehong, et al. Bionic SLAM Algorithm Based on Interest Tendency Mechanism[J]. Journal of Electronics & Information Technology, 2022, 44(5): 1743-1753.
9	邸凯昌, 万文辉, 赵红颖, 等. 视觉SLAM技术的进展与应用[J]. 测绘学报, 2018, 47(6): 770-779.
	Di Kaichang, Wan Wenhui, Zhao Hongying, et al. Progress and Applications of Visual SLAM[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6): 770-779.
10	牛珉玉, 黄宜庆. 基于动态耦合与空间数据关联的RGB-D SLAM算法[J]. 机器人, 2022, 44(3): 333-342.
	Niu Minyu, Huang Yiqing. An RGB-D SLAM Algorithm Based on Dynamic Coupling and Spatial Data Association[J]. Robot, 2022, 44(3): 333-342.
11	林景栋, 吴欣怡, 柴毅, 等. 卷积神经网络结构优化综述[J]. 自动化学报, 2020, 46(1): 24-37.
	Lin Jingdong, Wu Xinyi, Chai Yi, et al. Structure Optimization of Convolutional Neural Networks: A Survey[J]. Acta Automatica Sinica, 2020, 46(1): 24-37.
12	Tateno Keisuke, Tombari Federico, Laina Iro, et al. CNN-SLAM: Real-time Dense Monocular SLAM with Learned Depth Prediction[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 6565-6574.
13	Hou Yi, Zhang Hong, Zhou Shilin. Convolutional Neural Network-based Image Representation for Visual Loop Closure Detection[C]//2015 IEEE International Conference on Information and Automation. Piscataway, NJ, USA: IEEE, 2015: 2238-2245.
14	李维鹏, 张国良, 姚二亮, 等. 基于场景显著区域的改进闭环检测算法[J]. 机器人, 2017, 39(1): 23-35.
	Li Weipeng, Zhang Guoliang, Yao Erliang, et al. An Improved Loop Closure Detection Algorithm Based on Scene Salient Regions[J]. Robot, 2017, 39(1): 23-35.
15	Ganti Pranav, Waslander Steven L. Network Uncertainty Informed Semantic Feature Selection for Visual SLAM[C]//2019 16th Conference on Computer and Robot Vision (CRV). Piscataway, NJ, USA: IEEE, 2019: 121-128.
16	Tang Jiexiong, Ericson Ludvig, Folkesson John, et al. GCNv2: Efficient Correspondence Prediction for Real-time SLAM[J]. IEEE Robotics and Automation Letters, 2019, 4(4): 3505-3512.
17	Li Dongjiang, Shi Xuesong, Long Qiwei, et al. DXSLAM: A Robust and Efficient Visual SLAM System with Deep Features[C]//2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway, NJ, USA: IEEE, 2020: 4958-4965.
18	Lee J B, Rossi R A, Kim S, et al. Attention Models in Graphs: A Survey[J]. ACM Transactions on Knowledge Discovery from Data, 2019, 13(6): 62.
19	Zhuang Chenyi, Ma Qiang. Dual Graph Convolutional Networks for Graph-based Semi-supervised Classification[C]//Proceedings of the 2018 World Wide Web Conference. Republic and Canton of Geneva, CHE: International World Wide Web Conferences Steering Committee, 2018: 499-508.
20	Zhang Xiwu, Su Yan, Zhu Xinhua. Loop Closure Detection for Visual SLAM Systems Using Convolutional Neural Network[C]//2017 23rd International Conference on Automation and Computing (ICAC). Piscataway, NJ, USA: IEEE, 2017: 1-6.
21	Scarselli Franco, Gori Marco, Ah Chung Tsoi, et al. The Graph Neural Network Model[J]. IEEE Transactions on Neural Networks, 2009, 20(1): 61-80.
22	徐冰冰, 岑科廷, 黄俊杰, 等. 图卷积神经网络综述[J]. 计算机学报, 2020, 43(5): 755-780.
	Xu Bingbing, Cen Keting, Huang Junjie, et al. A Survey on Graph Convolutional Neural Network[J]. Chinese Journal of Computers, 2020, 43(5): 755-780.
23	吴博, 梁循, 张树森, 等. 图神经网络前沿进展与应用[J]. 计算机学报, 2022, 45(1): 35-68.
	Wu Bo, Liang Xun, Zhang Shusen, et al. Advances and Applications in Graph Neural Network[J]. Chinese Journal of Computers, 2022, 45(1): 35-68.
24	Jia Kui, Chan T H, Zeng Zinan, et al. ROML: A Robust Feature Correspondence Approach for Matching Objects in a Set of Images[J]. International Journal of Computer Vision, 2016, 117(2): 173-197.
25	Sarlin Paul-Edouard, DeTone D, Malisiewicz T, et al. SuperGlue: Learning Feature Matching with Graph Neural Networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2020: 4937-4946.
26	Elmoogy Ahmed, Dong Xiaodai, Lu Tao, et al. Pose-GNN: Camera Pose Estimation System Using Graph Neural Networks[EB/OL]. (2021-03-17) [2022-12-15]. .
27	Wang Limin, Guo Sheng, Huang Weilin, et al. Places205-VGGNet Models for Scene Recognition[EB/OL]. (2015-08-07) [2022-11-07]. .
28	Meng Zihang, Ravi S N, Singh V. Physarum Powered Differentiable Linear Programming Layers and Applications[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(10): 8939-8949.
29	Lin T Y, Maire M, Belongie S, et al. Microsoft COCO: Common Objects in Context[C]//Computer Vision-ECCV 2014. Cham: Springer International Publishing, 2014: 740-755.
30	Sturm Jürgen, Engelhard Nikolas, Endres Felix, et al. A Benchmark for the Evaluation of RGB-D SLAM Systems[C]//2012 IEEE/RSJ International Conference on Intelligent Robots and Systems. Piscataway, NJ, USA: IEEE, 2012: 573-580.
31	Geiger A, Lenz P, Stiller C, et al. Vision Meets Robotics: The KITTI Dataset[J]. The International Journal of Robotics Research, 2013, 32(11): 1231-1237.
32	Balntas V, Lenc K, Vedaldi A, et al. HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 3852-3861.
33	DeTone D, Malisiewicz T, Rabinovich A. SuperPoint: Self-supervised Interest Point Detection and Description[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway, NJ, USA: IEEE, 2018: 337-33712.

数据集	ORB	Superpoint	GCNv2	本文
i_books	0.572	0.591	0.627	0.693
i_castle	0.583	0.614	0.641	0.716
i_fruits	0.537	0.569	0.599	0.593
i_ski	0.564	0.576	0.598	0.645
i_whitebuilding	0.524	0.513	0.545	0.617

数据集	ORB+FLANN			Superpoint+FLANN			GCNv2+FLANN			本文
数据集	@30°	@60°	@90°	@30°	@60°	@90°	@30°	@60°	@90°	@30°	@60°	@90°
v_abstract	0.643	0.465	0.298	0.620	0.473	0.239	0.728	0.533	0.295	0.802	0.739	0.507
v_bees	0.651	0.482	0.262	0.702	0.536	0.388	0.648	0.499	0.386	0.857	0.686	0.569
v_beyus	0.682	0.503	0.312	0.608	0.449	0.256	0.612	0.428	0.324	0.829	0.737	0.571
v_home	0.605	0.396	0.274	0.633	0.405	0.293	0.730	0.475	0.252	0.796	0.706	0.535
v_woman	0.597	0.391	0.238	0.652	0.433	0.264	0.671	0.416	0.241	0.840	0.744	0.522

数据集	ORB-SLAM2		DX-SLAM		GCNv2-SLAM		本文
数据集	ATE	RPE	ATE	RPE	ATE	RPE	ATE	RPE
fr1/360	―	―	―	―	0.065	0.082	0.145	0.199
fr1/floor	0.390	0.469	―	―	0.138	0.167	0.019	0.038
fr1/desk	0.081	0.101	0.300	0.364	0.188	0.230	0.039	0.050
fr1/desk2	0.571	0.999	0.343	0.416	0.156	0.197	0.047	0.075
fr1/room	0.295	0.354	0.181	0.212	0.246	0.305	0.070	0.111
fr2/desk	0.931	1.149	0.581	0.712	0.160	0.199	0.109	0.133
fr3/long_office_household	1.222	1.553	0.791	1.007	0.363	0.455	0.169	0.176

算法	特征提取/ms	特征匹配/ms	总时间/ms	平均帧率/(帧/s)
ORB-SLAM2	12.88	15.06	27.94	30
DX-SLAM	32.42	39.24	71.66	15
GCNv2-SLAM	19.45	31.57	51.02	20
本文	16.32	30.79	47.11	20

[1]	雷旭, 陈静夷, 陈潇阳. 改进哈里斯鹰算法的仓储机器人路径规划研究[J]. 系统仿真学报, 2024, 36(5): 1081-1092.
[2]	张瑞, 周丽, 刘正洋. 融合RRT*与DWA算法的移动机器人动态路径规划[J]. 系统仿真学报, 2024, 36(4): 957-968.
[3]	刘福琳, 李庆鑫. 多移动机器人混合避障算法的编队策略[J]. 系统仿真学报, 2024, 36(3): 726-734.
[4]	唐昀超, 祁少军, 朱立学, 卓献荣, 张芸齐, 孟繁. 移动机器人避障运动研究[J]. 系统仿真学报, 2024, 36(1): 1-26.
[5]	杨来义, 毕敬, 苑海涛. 基于SAC算法的移动机器人智能路径规划[J]. 系统仿真学报, 2023, 35(8): 1726-1736.
[6]	陈奕梅, 石小凡, 李宝全. 基于改进虚拟弹簧模型的多机器人编队控制[J]. 系统仿真学报, 2023, 35(6): 1235-1244.
[7]	于雅楠, 史敦煌, 华春杰. 特征点法SLAM视觉里程计自适应优化算法[J]. 系统仿真学报, 2022, 34(1): 104-112.
[8]	王霄汉, 张霖, 任磊, 谢堃钰, 王昆玉, 叶飞, 陈真. 基于强化学习的车间调度问题研究简述[J]. 系统仿真学报, 2021, 33(12): 2782-2791.
[9]	尤波, 王明睿, 李智, 丁亮. 基于模型预测控制的轮式移动机器人轨迹规划[J]. 系统仿真学报, 2020, 32(4): 591-600.
[10]	赵萍, 雷新宇, 陈波芝, 杨矫云. 基于TI-A*的多机器人动态规划协调方法研究[J]. 系统仿真学报, 2019, 31(5): 925-935.
[11]	张新良, 李阳, 赵运基. 一种基于时空切片的SLAM关键帧提取方法[J]. 系统仿真学报, 2018, 30(5): 1724-1729.
[12]	许亚芳, 孙作雷, 曾连荪, 张波. 基于多次测量更新的移动机器人SLAM仿真[J]. 系统仿真学报, 2015, 27(6): 1288-1293.
[13]	苏程, 王砚麟, 刘继涛, 赵志刚. 紧耦合多机器人联合吊运系统虚拟平台仿真分析[J]. 系统仿真学报, 2015, 27(12): 2981-2987.
[14]	刘子龙, 胡少凯, 刘洁, 王亚刚. 非完整移动机器人在线辨识级联路径跟随控制[J]. 系统仿真学报, 2015, 27(11): 2748-2755.