Improvement of SLAM Localization Accuracy in AR by Enhancing YOLOv8

doi:10.16182/j.issn1004731x.joss.24-0564

Abstract

Abstract:

In the presence of dynamic interference in the environment, traditional simultaneous localization and mapping (SLAM) methods often experience reduced precision and stability in the registration of virtual objects during three-dimensional registration in augmented reality (AR). To address these issues, an improved method for dynamic scenes based on semantic segmentation and optical flow tracking was proposed. The convolutional block attention module (CBAM) attention mechanism was incorporated into YOLOv8 to enhance its focus on dynamic objects in the environment, thereby improving detection performance and accuracy. The semantic segmentation functionality of the improved YOLOv8 was integrated into the front-end of ORB-SLAM3 to segment dynamic objects in the scene and remove dynamic feature points that affect map construction. The optical flow method was further used to track moving objects, thereby improving the positioning accuracy of the camera. Validation was conducted on the TUM dataset and in real-world scenarios. The results indicate that, compared to traditional ORB-SLAM3, the proposed method improves positioning accuracy in dynamic scenes, significantly enhancing the stability of 3D registration in AR.

Key words: augmented reality(AR), three-dimensional registration, YOLOv8, ORB-SLAM3, semantic segmentation, dynamic detection

CLC Number:

TP391.9

Liu Jia, Zhang Zengwei, Chen Dapeng, Huang Nanxuan, Wang Bin, Song Hong. Improvement of SLAM Localization Accuracy in AR by Enhancing YOLOv8[J]. Journal of System Simulation, 2025, 37(11): 2701-2713.

Figures/Tables 18

Fig. 1

Fig. 2

Fig. 3

Table 1

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Table 2

Table 3

Table 4

Fig. 13

Fig. 14

References 33

[1]	Chen Yunqiang, Wang Qing, Chen Hong, et al. An Overview of Augmented Reality Technology[J]. Journal of Physics: Conference Series, 2019, 1237(2): 022082.
[2]	Li Jinyu, Yang Bangbang, Chen Danpeng, et al. Survey and Evaluation of Monocular Visual-inertial SLAM Algorithms for Augmented Reality[J]. Virtual Reality & Intelligent Hardware, 2019, 1(4): 386-410.
[3]	侯守明, 韩吉, 张煜东, 等. 基于视觉的增强现实三维注册技术综述[J]. 系统仿真学报, 2019, 31(11): 2206-2215.
	Hou Shouming, Han Ji, Zhang Yudong, et al. Survey of Vision-based Augmented Reality 3D Registration Technology[J]. Journal of System Simulation, 2019, 31(11): 2206-2215.
[4]	Lowe D G. Distinctive Image Features from Scale-invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[5]	Bay Herbert, Tuytelaars Tinne, Luc Van Gool. SURF: Speeded Up Robust Features[C]//Computer Vision – ECCV 2006. Berlin: Springer Berlin Heidelberg, 2006: 404-417.
[6]	Rublee E, Rabaud V, Konolige K, et al. ORB: An Efficient Alternative to SIFT or SURF[C]//2011 International Conference on Computer Vision. Piscataway: IEEE, 2011: 2564-2571.
[7]	Leonard J J, Durrant-Whyte H F. Simultaneous Map Building and Localization for an Autonomous Mobile Robot[C]//Proceedings IROS '91: IEEE/RSJ International Workshop on Intelligent Robots and Systems '91. Piscataway: IEEE, 1991: 1442-1447 vol.3.
[8]	Klein G, Murray D. Parallel Tracking and Mapping for Small AR Workspaces[C]//2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. Piscataway: IEEE, 2007: 225-234.
[9]	Mur-Artal Raúl, Montiel J M M, Tardós Juan D. ORB-SLAM: A Versatile and Accurate Monocular SLAM System[J]. IEEE Transactions on Robotics, 2015, 31(5): 1147-1163.
[10]	Campos Carlos, Elvira Richard, J Gómez Rodríguez Juan, et al. ORB-SLAM3: An Accurate Open-source Library for Visual, Visual-inertial, and Multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6): 1874-1890.
[11]	Li Jinyu, Pan Xiaokun, Huang Gan, et al. RD-VIO: Robust Visual-inertial Odometry for Mobile Augmented Reality in Dynamic Environments[J]. IEEE Transactions on Visualization and Computer Graphics, 2024, 30(10): 6941-6955.
[12]	Tan Wei, Liu Haomin, Dong Zilong, et al. Robust Monocular SLAM in Dynamic Environments[C]//2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Piscataway: IEEE, 2013: 209-218.
[13]	史蓝兮, 颜文旭, 倪宏宇, 等. 基于改进目标检测的动态场景SLAM研究[J]. 系统仿真学报, 2024, 36(4): 1028-1042.
	Shi Lanxi, Yan Wenxu, Ni Hongyu, et al. Research on Dynamic Scene SLAM Based on Improved Object Detection[J]. Journal of System Simulation, 2024, 36(4): 1028-1042.
[14]	Saputra M R U, Markham A, Trigoni N. Visual SLAM and Structure from Motion in Dynamic Environments: A Survey[J]. ACM Computing Surveys, 2018, 51(2): 37.
[15]	Alcantarilla Pablo F, Yebes José J, Almazán Javier, et al. On Combining Visual SLAM and Dense Scene Flow to Increase the Robustness of Localization and Mapping in Dynamic Environments[C]//2012 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2012: 1290-1297.
[16]	Mei Jianfeng, Zuo Tao, Song Dong. Highly Dynamic Visual SLAM Dense Map Construction Based on Indoor Environments[J]. IEEE Access, 2024, 12: 38717-38731.
[17]	Bescos Berta, Fácil José M, Civera Javier, et al. DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083.
[18]	Wu Wenxin, Guo Liang, Gao Hongli, et al. YOLO-SLAM: A Semantic SLAM System Towards Dynamic Environment with Geometric Constraint[J]. Neural Computing and Applications, 2022, 34(8): 6011-6026.
[19]	Zhang Yuhao. NGD-SLAM: Towards Real-time SLAM for Dynamic Environments without GPU[EB/OL]. (2024-05-12) [访问日期]. .
[20]	Han Shuangquan, Xi Zhihong. Dynamic Scene Semantics SLAM Based on Semantic Segmentation[J]. IEEE Access, 2020, 8: 43563-43570.
[21]	Gabriel Fischer Abati, João Carlos Virgolino Soares, Vivian Suzano Medeiros, et al. Panoptic-SLAM: Visual SLAM in Dynamic Environments Using Panoptic Segmentation[EB/OL]. (2024-05-03) [访问日期]. .
[22]	Yu Chao, Liu Zuxin, Liu Xinjun, et al. DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway: IEEE, 2018: 1168-1174.
[23]	Zhu Xizhou, Cheng Dazhi, Zhang Zheng, et al. An Empirical Study of Spatial Attention Mechanisms in Deep Networks[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 6687-6696.
[24]	Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation Networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
[25]	Woo Sanghyun, Park Jongchan, Lee J Y, et al. CBAM: Convolutional Block Attention Module[C]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[26]	Lin T Y, Maire M, Belongie S, et al. Microsoft COCO: Common Objects in Context[C]//Computer Vision – ECCV 2014. Cham: Springer International Publishing, 2014: 740-755.
[27]	Tang Baihui, Liu Zhengyi, Cao Sanxing. AR Application Research Based on ORB-SLAM[C]//VR/AR and 3D Displays. Singapore: Springer Singapore, 2021: 78-88.
[28]	Singla Nishu. Motion Detection Based on Frame Difference Method[J]. International Journal of Information & Computation Technology, 2014, 4(15): 1559-1565.
[29]	Wang Zhen, Yang Xiaojun. Moving Target Detection and Tracking Based on Pyramid Lucas-Kanade Optical Flow[C]//2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC). Piscataway: IEEE, 2018: 66-69.
[30]	He Jiaming, Li Mingrui, Wang Yangyang, et al. OVD-SLAM: An Online Visual SLAM for Dynamic Environments[J]. IEEE Sensors Journal, 2023, 23(12): 13210-13219.
[31]	Liu Yubao, Miura Jun. RDS-SLAM: Real-time Dynamic SLAM Using Semantic Segmentation Methods[J]. IEEE Access, 2021, 9: 23772-23785.
[32]	Cheng Shuhong, Sun Changhe, Zhang Shijun, et al. SG-SLAM: A Real-time RGB-D Visual SLAM Toward Dynamic Scenes with Semantic and Geometric Information[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-12.
[33]	Su Peng, Luo Suyun, Huang Xiaoci. Real-time Dynamic SLAM Algorithm Based on Deep Learning[J]. IEEE Access, 2022, 10: 87754-87766.

模型	人	汽车	狗	杯子	飞盘
YOLOv8	0.810	0.718	0.911	0.735	0.640
YOLOv8+GAM	0.821	0.725	0.970	0.752	0.780
YOLOv8+CBAM	0.845	0.740	0.995	0.780	0.790

序列	ORB-SLAM3^[10]		RDS-SLAM^[30]		DS-SLAM^[22]		本文方法
序列	RMSE	SD	RMSE	SD	RMSE	SD	RMSE	SD
f3_walk_xyz	0.273 6	0.123 7	0.056 8	0.022 4	0.023 9	0.016 0	0.013 5	0.006 8
f3_walk_half	0.273 9	0.110 6	0.080 2	0.043 2	0.031 2	0.013 8	0.027 6	0.015 2
f3_walk_rpy	0.662 1	0.290 6	0.160 4	0.087 1	0.444 0	0.235 0	0.031 7	0.018 0
fr3_sit_static	0.008 5	0.004 2	0.008 4	0.004 1	0.005 9	0.003 1	0.006 0	0.003 1

序列	ORB-SLAM3^[10]		RDS-SLAM^[30]		DS-SLAM^[22]		本文方法
序列	RMSE	SD	RMSE	SD	RMSE	SD	RMSE	SD
f3_walk_xyz	0.151 2	0.123 5	0.041 6	0.022 4	0.034 0	0.023 1	0.011 5	0.006 4
f3_walk_half	0.153 1	0.096 8	0.047 9	0.035 1	0.028 4	0.014 8	0.013 4	0.007 7
f3_walk_rpy	0.385 2	0.315 6	0.132 0	0.116 2	0.161 1	0.117 5	0.019 2	0.012 2
fr3_sit_static	0.009 5	0.005 2	0.011 3	0.007 1	0.007 5	0.003 1	0.005 0	0.002 7

序列	SG-SLAM^[32]	RDS-SLAM^[30]	RTD-SLAM^[33]	本文方法
fr3_walking_xyz	94.90	91.68	95.57	95.07
fr3_walking_rpy	79.50	90.68	92.46	95.21
fr3_walking_half	95.78	84.46	90.13	89.91