3D Object Tracking Registration Based on Improved RBOT Method

doi:10.16182/j.issn1004731x.joss.25-0214

Abstract

Abstract:

To address the limitations of region-based object tracking (RBOT) in handling isotropic objects and scenarios with similar foreground-background colors, an improved method integrating edge features is proposed. The approach employs edge detection to extract object contours and designs a region segmentation strategy incorporated into an energy function framework to optimize internal line and edge consistency, thereby enhancing adaptability in dynamic environments and improving pose estimation accuracy. Validation through augmented reality assembly experiments on an aero-engine demonstrates that the proposed method effectively reduces rotational and translational errors, achieving initialization deviations of less than 1.5° and 0.5%, respectively. For static video sequences, inter-frame pose jitter remains below 0.3° and displacement jitter below 0.1%, meeting the precision requirements of virtual-real registration for assembly guidance.

Key words: augmented reality, position estimation, edge detection, image processing, aero-engines

CLC Number:

TP391.9

Zhou Jiarui, Cui Haihua, Li Pengcheng, Gu Shihao, Hao Huipu, Zhao Xifu, Zhao Anan, Jiang Tao. 3D Object Tracking Registration Based on Improved RBOT Method[J]. Journal of System Simulation, 2026, 38(1): 14-28.

Figures/Tables 17

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Fig. 5

Fig. 6

Table 2

Fig. 7

Table 3

Fig. 8

Table 4

Fig. 9

Fig. 10

Fig. 11

Table 5

Table 6

References 26

[1]	侯廷红, 周平, 何黎明, 等. 航空发动机维修技术进展与展望[J]. 航空工程进展, 2023, 14(5): 29-34, 60.
	Hou Tinghong, Zhou Ping, He Liming, et al. Research Progress and Prospect of Aero-engine Maintenance Technology[J]. Advances in Aeronautical Science and Engineering, 2023, 14(5): 29-34, 60.
[2]	魏禛, 郭宇, 汤鹏洲, 等. 增强现实在复杂产品装配领域的关键技术研究与应用综述[J]. 计算机集成制造系统, 2022, 28(3): 649-662.
	Wei Zhen, Guo Yu, Tang Pengzhou, et al. A Review of Key Technology Research and Application of Augmented Reality in Complex Product Assembly[J]. Computer Integrated Manufacturing Systems, 2022, 28(3): 649-662.
[3]	王群淼. 增强现实技术及其在航空发动机维修中的应用研究[J]. 内燃机与配件, 2020(22): 159-160.
	Wang Qunmiao. Research on Augmented Reality Technology and its Application in Aeroengine Maintenance[J]. Internal Combustion Engine & Parts, 2020(22): 159-160.
[4]	刘远, 孟少华, 陈畅宇, 等. 增强现实技术最新研究进展及航空航天领域应用[J]. 计算机与数字工程, 2023, 51(1): 125-132.
	Liu Yuan, Meng Shaohua, Chen Changyu, et al. Research on the Latest Progress Augmented Reality Technology and its Application in Aerospace[J]. Computer & Digital Engineering, 2023, 51(1): 125-132.
[5]	Serván J, Mas F, Menéndez J L, et al. Using Augmented Reality in Airbus A400M Shop Floor Assembly Work Instructions[J]. AIP Conference Proceedings, 2012, 1431(1): 633-640.
[6]	梅英杰, 王大伟, 孙传智, 等. 基于增强现实的航空发动机机匣数字孪生测调系统[J]. 航空学报, 2024, 45(21): 80-92.
	Mei Yingjie, Wang Dawei, Sun Chuanzhi, et al. A Digital Twin Testing and Adjusting System for Aero-engine Casings Based on Augmented Reality[J]. Acta Aeronautica et Astronautica Sinica, 2024, 45(21): 80-92.
[7]	敖良忠, 吴梓祺. 基于混合现实的发动机维修培训系统[J]. 中国民航飞行学院学报, 2023, 34(2): 53-56.
	Ao Liangzhong, Wu Ziqi. Mixed Reality-based Aero-engine Maintenance Training System[J]. Journal of Civil Aviation Flight University of China, 2023, 34(2): 53-56.
[8]	方维, 许澍虹, 韩磊, 等. AR增强装配中的跟踪注册方法研究与应用进展[J]. 系统仿真学报, 2023, 35(7): 1438-1454.
	Fang Wei, Xu Shuhong, Han Lei, et al. Research and Application Progress of Tracking Registration Methods in AR Assembly[J]. Journal of System Simulation, 2023, 35(7): 1438-1454.
[9]	李峰. 基于视觉的增强现实三维注册技术研究[D]. 唐山: 华北理工大学, 2022.
	Li Feng. Research on Augmented Reality 3D Registration Technology Based on Vison[D]. Tangshan: North China University of Science and Technology, 2022.
[10]	Schmaltz C, Rosenhahn B, BROX T, et al. Region-based Pose Tracking[C]//Proceedings of the 3rd Iberian conference on Pattern Recognition and Image Analysis, Part II. Heidelberg : Springer-Verlag, 2007: 56-63.
[11]	Rosenhahn B, Brox T, Weickert J. Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Tracking[J]. International Journal of Computer Vision, 2007, 73(3): 243-262.
[12]	Kass M, Witkin A, Terzopoulos D. Snakes: Active Contour Models[J]. International Journal of Computer Vision, 1988, 1(4): 321-331.
[13]	Prisacariu V A, Reid I D. PWP3D: Real-time Segmentation and Tracking of 3D Objects[J]. International Journal of Computer Vision, 2012, 98(3): 335-354.
[14]	Tjaden H, Schwanecke U, Schömer E. Real-time Monocular Segmentation and Pose Tracking of Multiple Objects[C]//Computer Vision – ECCV 2016. Cham: Springer International Publishing, 2016: 423-438.
[15]	Tjaden H, Schwanecke U, Schömer E. Real-time Monocular Pose Estimation of 3D Objects Using Temporally Consistent Local Color Histograms[C]//2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2017: 124-132.
[16]	Tjaden H, Schwanecke U, Schömer E, et al. A Region-Based Gauss-newton Approach to Real-time Monocular Multiple Object Tracking[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(8): 1797-1812.
[17]	Zhong Leisheng, Zhao Xiaolin, Zhang Yu, et al. Occlusion-aware Region-based 3D Pose Tracking of Objects with Temporally Consistent Polar-based Local Partitioning[J]. IEEE Transactions on Image Processing, 2020, 29: 5065-5078.
[18]	黄鸿, 钟凡, 秦学英. 基于时间一致性局部颜色特征的无纹理3D物体实时跟踪[J]. 计算机辅助设计与图形学学报, 2020, 32(1): 99-111.
	Huang Hong, Zhong Fan, Qin Xueying. Real-time Textureless 3D Object Tracking Based on Temporally Consistent Local Color Feature[J]. Journal of Computer-aided Design & Computer Graphics, 2020, 32(1): 99-111.
[19]	Stoiber M, Pfanne M, Strobl K H, et al. SRT3D: A Sparse Region-based 3D Object Tracking Approach for the Real World[J]. International Journal of Computer Vision, 2022, 130(4): 1008-1030.
[20]	Stoiber M, Sundermeyer M, Triebel R. Iterative Corresponding Geometry: Fusing Region and Depth for Highly Efficient 3D Tracking of Textureless Objects[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 6845-6855.
[21]	Stoiber M, Elsayed M, Reichert A E, et al. Fusing Visual Appearance and Geometry for Multi-modality 6DoF Object Tracking[C]//2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Piscataway: IEEE, 2023: 1170-1177.
[22]	徐一成, 里鹏, 李帅, 等. 基于区域的弱纹理零件三维跟踪方法[J]. 计算机集成制造系统, 2024, 30(12): 4246-4258.
	Xu Yicheng, Li Peng, Li Shuai, et al. Region-based 3D Tracking Method for Textureless Parts[J]. Computer Integrated Manufacturing Systems, 2024, 30(12): 4246-4258.
[23]	Lowe D G. Robust Model-based Motion Tracking Through the Integration of Search and Estimation[J]. International Journal of Computer Vision, 1992, 8(2): 113-122.
[24]	Harris C, Stennett C. RAPID - A Video Rate Object Tracker[C]//Proceedings of the British Machine Vision Conference. Durham: BMVA Press, 1990: 15.1-15.6.
[25]	Dong Yanchao, Ji Lingling, Wang Senbo, et al. Accurate 6DOF Pose Tracking for Texture-less Objects[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(5): 1834-1848.
[26]	Tian Xuhui, Lin Xinran, Zhong Fan, et al. Large-displacement 3D Object Tracking with Hybrid Non-local Optimization[C]//Computer Vision – ECCV 2022. Cham: Springer Nature Switzerland, 2022: 627-643.

方法	平均运行时间/ms	旋转误差/(°)		平移误差/mm
方法	平均运行时间/ms	RMSE	MAE	RMSE	MAE
RBOT	82.81	2.01	1.10	6.04	2.86
改进RBOT	86.02	1.33	0.80	4.99	2.14

方法	平均运行时间/ms	旋转误差/(°)		平移误差/mm
方法	平均运行时间/ms	RMSE	MAE	RMSE	MAE
RBOT	82.54	2.71	1.65	6.08	4.46
改进RBOT	85.47	2.56	1.53	5.81	4.25

方法	平均运行时间/ms	旋转误差/(°)		平移误差/mm
方法	平均运行时间/ms	RMSE	MAE	RMSE	MAE
RBOT	80.14	3.35	2.37	5.72	4.57
改进RBOT	86.09	3.30	2.33	5.62	4.49

方法	位移抖动/%	旋转误差/(°)		平移误差/mm
方法	位移抖动/%	RMSE	MAE	RMSE	MAE
RBOT	0.03	0.41	0.42	0.14	0.15
改进RBOT	0.02	0.34	0.34	0.13	0.14

方法	位移抖动/%	旋转误差/(°)		平移误差/mm
方法	位移抖动/%	RMSE	MAE	RMSE	MAE
RBOT	0.34	0.10	0.13	1.65	1.65
改进RBOT	0.34	0.11	0.13	1.66	1.65