基于改进YOLOv8提升SLAM在AR中的定位精度

doi:10.16182/j.issn1004731x.joss.24-0564

系统仿真学报 ›› 2025, Vol. 37 ›› Issue (11): 2701-2713.doi: 10.16182/j.issn1004731x.joss.24-0564

• 论文 •

基于改进YOLOv8提升SLAM在AR中的定位精度

刘佳¹^,²^,³, 张增伟¹^,², 陈大鹏¹^,²^,³, 黄楠瑄¹^,², 王斌¹^,², 宋泓¹^,²

^1.南京信息工程大学天长研究院，安徽滁州 239356
^2.南京信息工程大学自动化学院，江苏南京 210044
^3.江苏省大气环境与装备技术协同创新中心，江苏南京 210044

收稿日期:2024-05-24 修回日期:2024-07-20 出版日期:2025-11-18 发布日期:2025-11-27
通讯作者: 陈大鹏
第一作者简介:刘佳(1981-)，女，教授，博士，研究方向为虚拟/增强现实、计算机视觉、图像处理。
基金资助:
国家自然科学基金(62003169);江苏产业前瞻与关键技术重点项目(BE2020006-2);江苏省自然科学基金(BK20200823)

Improvement of SLAM Localization Accuracy in AR by Enhancing YOLOv8

Liu Jia¹^,²^,³, Zhang Zengwei¹^,², Chen Dapeng¹^,²^,³, Huang Nanxuan¹^,², Wang Bin¹^,², Song Hong¹^,²

^1.Tianchang Research Institute, Nanjing University of Information Science and Technology, Chuzhou 239356, China
^2.School of Automation, Nanjing University of Information Science and Technology, Nanjing 210044, China
^3.Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing 210044, China

Received:2024-05-24 Revised:2024-07-20 Online:2025-11-18 Published:2025-11-27
Contact: Chen Dapeng

摘要/Abstract

摘要：

针对传统SLAM方法在进行增强现实的三维注册时，由于环境中存在动态干扰，会出现虚拟物体注册精度和稳定性降低等问题，提出一种动态场景下基于语义分割和光流追踪的改进方法。在YOLOv8中加入CBAM注意力机制以增强其对环境中动态物体的关注度，提高检测性能和准确性；将改进YOLOv8的语义分割功能嵌入ORB-SLAM3的前端，分割出场景中的动态物体并去除影响地图构建的动态特征点，结合光流法进一步跟踪移动物体，提高相机的定位精度；在TUM数据集以及真实场景中进行验证。结果表明：相比于传统ORB-SLAM3，提出的方法在动态场景中的定位精度显著提高，提升了AR中三维注册的稳定性。

关键词: 增强现实, 三维注册, YOLOv8, ORB-SLAM3, 语义分割, 动态检测

Abstract:

In the presence of dynamic interference in the environment, traditional simultaneous localization and mapping (SLAM) methods often experience reduced precision and stability in the registration of virtual objects during three-dimensional registration in augmented reality (AR). To address these issues, an improved method for dynamic scenes based on semantic segmentation and optical flow tracking was proposed. The convolutional block attention module (CBAM) attention mechanism was incorporated into YOLOv8 to enhance its focus on dynamic objects in the environment, thereby improving detection performance and accuracy. The semantic segmentation functionality of the improved YOLOv8 was integrated into the front-end of ORB-SLAM3 to segment dynamic objects in the scene and remove dynamic feature points that affect map construction. The optical flow method was further used to track moving objects, thereby improving the positioning accuracy of the camera. Validation was conducted on the TUM dataset and in real-world scenarios. The results indicate that, compared to traditional ORB-SLAM3, the proposed method improves positioning accuracy in dynamic scenes, significantly enhancing the stability of 3D registration in AR.

Key words: augmented reality(AR), three-dimensional registration, YOLOv8, ORB-SLAM3, semantic segmentation, dynamic detection

中图分类号:

TP391.9

刘佳,张增伟,陈大鹏等 . 基于改进YOLOv8提升SLAM在AR中的定位精度[J]. 系统仿真学报, 2025, 37(11): 2701-2713.

Liu Jia,Zhang Zengwei,Chen Dapeng,et al . Improvement of SLAM Localization Accuracy in AR by Enhancing YOLOv8[J]. Journal of System Simulation, 2025, 37(11): 2701-2713.

图/表 18

图1

图2

图3

表1

图4

图5

图6

图7

图8

图9

图10

图11

图12

表2

表3

表4

图13

图14

参考文献 33

[1]	Chen Yunqiang, Wang Qing, Chen Hong, et al. An Overview of Augmented Reality Technology[J]. Journal of Physics: Conference Series, 2019, 1237(2): 022082.
[2]	Li Jinyu, Yang Bangbang, Chen Danpeng, et al. Survey and Evaluation of Monocular Visual-inertial SLAM Algorithms for Augmented Reality[J]. Virtual Reality & Intelligent Hardware, 2019, 1(4): 386-410.
[3]	侯守明, 韩吉, 张煜东, 等. 基于视觉的增强现实三维注册技术综述[J]. 系统仿真学报, 2019, 31(11): 2206-2215.
	Hou Shouming, Han Ji, Zhang Yudong, et al. Survey of Vision-based Augmented Reality 3D Registration Technology[J]. Journal of System Simulation, 2019, 31(11): 2206-2215.
[4]	Lowe D G. Distinctive Image Features from Scale-invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[5]	Bay Herbert, Tuytelaars Tinne, Luc Van Gool. SURF: Speeded Up Robust Features[C]//Computer Vision – ECCV 2006. Berlin: Springer Berlin Heidelberg, 2006: 404-417.
[6]	Rublee E, Rabaud V, Konolige K, et al. ORB: An Efficient Alternative to SIFT or SURF[C]//2011 International Conference on Computer Vision. Piscataway: IEEE, 2011: 2564-2571.
[7]	Leonard J J, Durrant-Whyte H F. Simultaneous Map Building and Localization for an Autonomous Mobile Robot[C]//Proceedings IROS '91: IEEE/RSJ International Workshop on Intelligent Robots and Systems '91. Piscataway: IEEE, 1991: 1442-1447 vol.3.
[8]	Klein G, Murray D. Parallel Tracking and Mapping for Small AR Workspaces[C]//2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality. Piscataway: IEEE, 2007: 225-234.
[9]	Mur-Artal Raúl, Montiel J M M, Tardós Juan D. ORB-SLAM: A Versatile and Accurate Monocular SLAM System[J]. IEEE Transactions on Robotics, 2015, 31(5): 1147-1163.
[10]	Campos Carlos, Elvira Richard, J Gómez Rodríguez Juan, et al. ORB-SLAM3: An Accurate Open-source Library for Visual, Visual-inertial, and Multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6): 1874-1890.
[11]	Li Jinyu, Pan Xiaokun, Huang Gan, et al. RD-VIO: Robust Visual-inertial Odometry for Mobile Augmented Reality in Dynamic Environments[J]. IEEE Transactions on Visualization and Computer Graphics, 2024, 30(10): 6941-6955.
[12]	Tan Wei, Liu Haomin, Dong Zilong, et al. Robust Monocular SLAM in Dynamic Environments[C]//2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR). Piscataway: IEEE, 2013: 209-218.
[13]	史蓝兮, 颜文旭, 倪宏宇, 等. 基于改进目标检测的动态场景SLAM研究[J]. 系统仿真学报, 2024, 36(4): 1028-1042.
	Shi Lanxi, Yan Wenxu, Ni Hongyu, et al. Research on Dynamic Scene SLAM Based on Improved Object Detection[J]. Journal of System Simulation, 2024, 36(4): 1028-1042.
[14]	Saputra M R U, Markham A, Trigoni N. Visual SLAM and Structure from Motion in Dynamic Environments: A Survey[J]. ACM Computing Surveys, 2018, 51(2): 37.
[15]	Alcantarilla Pablo F, Yebes José J, Almazán Javier, et al. On Combining Visual SLAM and Dense Scene Flow to Increase the Robustness of Localization and Mapping in Dynamic Environments[C]//2012 IEEE International Conference on Robotics and Automation. Piscataway: IEEE, 2012: 1290-1297.
[16]	Mei Jianfeng, Zuo Tao, Song Dong. Highly Dynamic Visual SLAM Dense Map Construction Based on Indoor Environments[J]. IEEE Access, 2024, 12: 38717-38731.
[17]	Bescos Berta, Fácil José M, Civera Javier, et al. DynaSLAM: Tracking, Mapping, and Inpainting in Dynamic Scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083.
[18]	Wu Wenxin, Guo Liang, Gao Hongli, et al. YOLO-SLAM: A Semantic SLAM System Towards Dynamic Environment with Geometric Constraint[J]. Neural Computing and Applications, 2022, 34(8): 6011-6026.
[19]	Zhang Yuhao. NGD-SLAM: Towards Real-time SLAM for Dynamic Environments without GPU[EB/OL]. (2024-05-12) [访问日期]. .
[20]	Han Shuangquan, Xi Zhihong. Dynamic Scene Semantics SLAM Based on Semantic Segmentation[J]. IEEE Access, 2020, 8: 43563-43570.
[21]	Gabriel Fischer Abati, João Carlos Virgolino Soares, Vivian Suzano Medeiros, et al. Panoptic-SLAM: Visual SLAM in Dynamic Environments Using Panoptic Segmentation[EB/OL]. (2024-05-03) [访问日期]. .
[22]	Yu Chao, Liu Zuxin, Liu Xinjun, et al. DS-SLAM: A Semantic Visual SLAM towards Dynamic Environments[C]//2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway: IEEE, 2018: 1168-1174.
[23]	Zhu Xizhou, Cheng Dazhi, Zhang Zheng, et al. An Empirical Study of Spatial Attention Mechanisms in Deep Networks[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 6687-6696.
[24]	Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation Networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
[25]	Woo Sanghyun, Park Jongchan, Lee J Y, et al. CBAM: Convolutional Block Attention Module[C]//Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 3-19.
[26]	Lin T Y, Maire M, Belongie S, et al. Microsoft COCO: Common Objects in Context[C]//Computer Vision – ECCV 2014. Cham: Springer International Publishing, 2014: 740-755.
[27]	Tang Baihui, Liu Zhengyi, Cao Sanxing. AR Application Research Based on ORB-SLAM[C]//VR/AR and 3D Displays. Singapore: Springer Singapore, 2021: 78-88.
[28]	Singla Nishu. Motion Detection Based on Frame Difference Method[J]. International Journal of Information & Computation Technology, 2014, 4(15): 1559-1565.
[29]	Wang Zhen, Yang Xiaojun. Moving Target Detection and Tracking Based on Pyramid Lucas-Kanade Optical Flow[C]//2018 IEEE 3rd International Conference on Image, Vision and Computing (ICIVC). Piscataway: IEEE, 2018: 66-69.
[30]	He Jiaming, Li Mingrui, Wang Yangyang, et al. OVD-SLAM: An Online Visual SLAM for Dynamic Environments[J]. IEEE Sensors Journal, 2023, 23(12): 13210-13219.
[31]	Liu Yubao, Miura Jun. RDS-SLAM: Real-time Dynamic SLAM Using Semantic Segmentation Methods[J]. IEEE Access, 2021, 9: 23772-23785.
[32]	Cheng Shuhong, Sun Changhe, Zhang Shijun, et al. SG-SLAM: A Real-time RGB-D Visual SLAM Toward Dynamic Scenes with Semantic and Geometric Information[J]. IEEE Transactions on Instrumentation and Measurement, 2023, 72: 1-12.
[33]	Su Peng, Luo Suyun, Huang Xiaoci. Real-time Dynamic SLAM Algorithm Based on Deep Learning[J]. IEEE Access, 2022, 10: 87754-87766.

模型	人	汽车	狗	杯子	飞盘
YOLOv8	0.810	0.718	0.911	0.735	0.640
YOLOv8+GAM	0.821	0.725	0.970	0.752	0.780
YOLOv8+CBAM	0.845	0.740	0.995	0.780	0.790

序列	ORB-SLAM3^[10]		RDS-SLAM^[30]		DS-SLAM^[22]		本文方法
序列	RMSE	SD	RMSE	SD	RMSE	SD	RMSE	SD
f3_walk_xyz	0.273 6	0.123 7	0.056 8	0.022 4	0.023 9	0.016 0	0.013 5	0.006 8
f3_walk_half	0.273 9	0.110 6	0.080 2	0.043 2	0.031 2	0.013 8	0.027 6	0.015 2
f3_walk_rpy	0.662 1	0.290 6	0.160 4	0.087 1	0.444 0	0.235 0	0.031 7	0.018 0
fr3_sit_static	0.008 5	0.004 2	0.008 4	0.004 1	0.005 9	0.003 1	0.006 0	0.003 1

序列	ORB-SLAM3^[10]		RDS-SLAM^[30]		DS-SLAM^[22]		本文方法
序列	RMSE	SD	RMSE	SD	RMSE	SD	RMSE	SD
f3_walk_xyz	0.151 2	0.123 5	0.041 6	0.022 4	0.034 0	0.023 1	0.011 5	0.006 4
f3_walk_half	0.153 1	0.096 8	0.047 9	0.035 1	0.028 4	0.014 8	0.013 4	0.007 7
f3_walk_rpy	0.385 2	0.315 6	0.132 0	0.116 2	0.161 1	0.117 5	0.019 2	0.012 2
fr3_sit_static	0.009 5	0.005 2	0.011 3	0.007 1	0.007 5	0.003 1	0.005 0	0.002 7

序列	SG-SLAM^[32]	RDS-SLAM^[30]	RTD-SLAM^[33]	本文方法
fr3_walking_xyz	94.90	91.68	95.57	95.07
fr3_walking_rpy	79.50	90.68	92.46	95.21
fr3_walking_half	95.78	84.46	90.13	89.91

基于改进YOLOv8提升SLAM在AR中的定位精度

Improvement of SLAM Localization Accuracy in AR by Enhancing YOLOv8

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 18

参考文献 33

相关文章 15

编辑推荐

Metrics

本文评价

[1]	鲁斌, 杨烜, 杨振宇, 高啸天. 自适应采样与重影多尺度特征融合的轻量化焊缝缺陷检测[J]. 系统仿真学报, 2025, 37(8): 1978-1990.
[2]	李杰, 刘扬, 李良, 苏本淦, 魏佳隆, 周广达, 石艳敏, 赵振. 基于跨阶段双分支特征聚合的遥感小目标检测[J]. 系统仿真学报, 2025, 37(4): 1025-1040.
[3]	王贺, 许佳宁, 闫广宇. 基于深度强化学习的AGV行人避让策略研究[J]. 系统仿真学报, 2025, 37(3): 595-606.
[4]	李若晴, 赵瑶池, 胡祝华, 戚文璐, 刘广丰. TOHF：一种针对资源受限室内VSLAM的特征提取器[J]. 系统仿真学报, 2025, 37(3): 691-703.
[5]	符强, 腾先云, 纪元法, 任风华. 基于改进特征描述的SLAM动态算法研究[J]. 系统仿真学报, 2024, 36(11): 2712-2721.
[6]	蔡兴旺, 裴耀强, 杨继华, 庞瑞峰. 煤矿掘进机系统数字孪生技术研究[J]. 系统仿真学报, 2024, 36(10): 2455-2468.
[7]	方维, 许澍虹, 韩磊, 李张文驰. AR增强装配中的跟踪注册方法研究与应用进展[J]. 系统仿真学报, 2023, 35(7): 1438-1454.
[8]	魏赟, 罗琦, 赵迎志. 基于自适应融合和注意力细化的语义分割模型[J]. 系统仿真学报, 2023, 35(6): 1226-1234.
[9]	赵为平, 陈雨, 项松, 刘远强, 王超越. 基于改进的DeepLabv3+图像语义分割算法研究[J]. 系统仿真学报, 2023, 35(11): 2333-2344.
[10]	孙一铃, 陈谊, 单桂华, 李晓兴. 基于AR技术的多人互动地球仪系统[J]. 系统仿真学报, 2022, 34(6): 1367-1374.
[11]	沈祺宗, 高春艳. 融合边缘检测模块的自然地貌语义分割模型研究[J]. 系统仿真学报, 2022, 34(2): 293-302.
[12]	孙红, 凌岳览, 张玉香. 融合边界监督策略的改进特征金字塔算法研究[J]. 系统仿真学报, 2022, 34(10): 2119-2129.
[13]	陈立家, 王凯, 李世刚, 田延飞. 基于航空影像的航海模拟器视景快速建模方法[J]. 系统仿真学报, 2021, 33(7): 1565-1573.
[14]	侯守明, 葛倩, 刘彦彦. 基于MAR的非物质文化遗产数字化保护系统研究[J]. 系统仿真学报, 2021, 33(6): 1334-1341.
[15]	魏士松, 周正东, 章栩苓, 毛玲, 贾峻山, 刘传乐. 基于桌面虚拟现实技术的航天器虚拟维修训练系统[J]. 系统仿真学报, 2021, 33(6): 1358-1363.