Monocular Semantic SLAM Method Based on Object Relation Description

doi:10.16182/j.issn1004731x.joss.20-0734

Abstract

Abstract:

Semantic information perception of the external environment and accurate positioning are the keys to autonomous navigation and operation of mobile robots. This paper proposes a method of semantic simultaneous localization and mapping (SLAM) based on a monocular camera. The system completes three-dimensional (3D) object detection while estimating the trajectory. We model the 3D objects with cuboids. Then, the semantic meanings, color distribution, size and neighborhood topology of the objects are extracted as descriptors for the accurate matching of objects between different frames. The camera pose, map points and object landmarks are optimized jointly in the backend of the system. The weight coefficient of each error term in the cost function is autonomously adjusted to improve the estimation accuracy and robustness of each state variable of the system. The experimental results show that the proposed method has high accuracy in map construction.

Key words: image segmentation, 3D object detection, topological map, graph matching, semantic SLAM

CLC Number:

TP391

Shiqi Lin, Jikai Wang, Haoyuan Pei, Hao Zhao, Zonghai Chen. Monocular Semantic SLAM Method Based on Object Relation Description[J]. Journal of System Simulation, 2022, 34(2): 278-284.

Figures/Tables 6

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Table 1

References 14

1	Dong J, Fei X, Soatto S. Visual-inertial-semantic Scene Representation for 3D Object Detection[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017: 960-970.
2	Gálvez-López D, Salas M, Tardós J D, et al. Real-time Monocular Object Slam[J]. Robotics and Autonomous Systems(S0921-8890), 2016, 75: 435-449.
3	Leibe B, Cornelis N, Cornelis K, et al. Dynamic 3d Scene Analysis from a Moving Vehicle[C]//2007 IEEE Conference on Computer Vision and Pattern Recognition. IEEE, 2007: 1-8.
4	Gay P, Rubino C, Bansal V, et al. Probabilistic Structure from Motion with Objects (Psfmo)[C]//Proceedings of the IEEE International Conference on Computer Vision. 2017: 3075-3084.
5	Lianos K N, Schonberger J L, Pollefeys M, et al. Vso: Visual Semantic Odometry[C]//Proceedings of the European Conference on Computer Vision (ECCV). 2018: 234-250.
6	Salas-Moreno R F, Newcombe R A, Strasdat H, et al. Slam++: Simultaneous Localisation and Mapping at the Level of Objects[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2013: 1352-1359.
7	Cross G, Zisserman A. Quadric reconstruction from Dual-space Geometry[C]//Sixth International Conference on Computer Vision IEEE Cat. No. 98CH36271.IEEE, 1998: 25-31.
8	Sünderhauf N, Pham T T, Latif Y, et al. Meaningful Maps with Object-Oriented Semantic Mapping[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017: 5079-5085.
9	Sünderhauf N, Pham T T, Latif Y, et al. Meaningful Maps with Object-Oriented Semantic Mapping[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2017: 5079-5085.
10	Yang S, Scherer S. Cubeslam: Monocular 3-d Object Slam[J]. IEEE Transactions on Robotics(S1941-0468), 2019, 35(4): 925-938.
11	Redmon J, Farhadi A. Yolov3: An Incremental Improvement[C]// 2018 IEEE International Conference on Computer Vision and Pattern Recognition, IEEE, 2018: 1-6.
12	Gawel A, Del Don C, Siegwart R, et al. X-view: Graph-based Semantic Multi-View Localization[J]. IEEE Robotics and Automation Letters(S2377-3766), 2018, 3(3): 1687-1694.
13	Liu Y, Petillot Y, Lane D, et al. Global Localization with Object-Level Semantics and Topology[C]// 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019: 4909-4915.
14	Kümmerle R, Grisetti G, Strasdat H, et al. g 2 o: A General Framework for Graph Optimization[C]//2011 IEEE International Conference on Robotics and Automation. IEEE, 2011: 3607-3613.

算法	01	02	03	04	05	06
CubeSLAM	0.57	0.33	0.62	0.61	0.50	0.56
Ours	0.63	0.62	0.69	0.72	0.58	0.73

[1]	Bingan Yuan, Mingen Zhong, Jingxin Ni. Research on Visual Inspection Algorithm of Crimping Appearance Defects for Wiring Harness Terminals [J]. Journal of System Simulation, 2022, 34(5): 1152-1159.
[2]	Liu Xiaojun, He Changyan, Liu Chang, Jia Jinyuan. Fast Alignment of BIM Products Based on Structure Matching [J]. Journal of System Simulation, 2021, 33(7): 1626-1637.
[3]	Wang Kun, Ji Yao, Liu Peilun, Wang Li. Infrared Image Segmentation of Aircraft Skin Based on Otsu and Improved I-Ching Divination Evolutionary Algorithm [J]. Journal of System Simulation, 2021, 33(2): 421-433.
[4]	Gu Juping, Cheng Tianyu, Hua Liang, Wang Jianping, Zhao Jian, Cao Yong. Overview of Image Segmentation and Registration for Spine Biological Modeling [J]. Journal of System Simulation, 2019, 31(2): 167-173.
[5]	Gao Shangbing, Xie Zheng, Pan Zhigeng, Qin Fangzhe, Li Rui. Novel Automatic Pavement Crack Detection Algorithm [J]. Journal of System Simulation, 2017, 29(9): 2009-2015.
[6]	Pan Xiuqiang, Shan Jinxiao, Yang Caifeng. Image Segmentation and Offset Correction Based on Minimal Relative Entropy Theoryand Level Set Method [J]. Journal of System Simulation, 2017, 29(11): 2731-2741.
[7]	Yang Jiabo, Yang Gang, Yang Meng. Augmented Reality System Based on Depth Image Segmentation and Object Tracking [J]. Journal of System Simulation, 2017, 29(11): 2788-2795.
[8]	Yang Yong, Zheng Liangren, Guo Ling, Ye Yangdong. Color-texture Image Segmentation Approach Designed by Integrating Geodesic Active Contour Model and Multilayer Graph Cut Model [J]. Journal of System Simulation, 2016, 28(9): 2159-2168.
[9]	Yang Yong, Guo Ling, Dai Wenzheng, Ye Yangdong. Regional Credible Fusion Based Color-texture Image Segmentation Approach [J]. Journal of System Simulation, 2016, 28(10): 2304-2312.
[10]	Jin Yaoli, Wang Shiliang, Xu Gang, Hu Weihua, Wang Yigang. Automatic Hair Contour Extraction Method with Complex Background [J]. Journal of System Simulation, 2015, 27(10): 2305-2309.
[11]	Miao Yongchun, Cheng Yan. Dark Color Image Segmentation Method Integrated MRF & Retinex [J]. Journal of System Simulation, 2015, 27(10): 2387-2394.