Survey of Vision-Based Augmented Reality 3D Registration Technology

doi:10.16182/j.issn1004731x.joss.19-FZ0286

Abstract

Abstract: Three-dimensional registration technology is one of the core technologies for augmented reality system implementation. In recent years, with the maturity of augmented reality hardware technology and the expansion of application fields, vision-based three-dimensional registration technology has received more attention and development. This paper introduces the basic principles and classification methods of 3D registration technology, focuses on the analysis of the principles and research status of three kinds of visual three-dimensional registration technology based on artificial identification, natural feature points and model.Furthermore, the defects and treatment methods of three visual registration technologies are discussed. The development prospect of visual 3D registration technology based on SLAM technology and deep learning method is prospected.

Key words: augmented reality, three-dimensional registration, artificial identification, natural feature points, model

CLC Number:

TP391

Hou Shouming, Han Ji, Zhang Yudong, Zhu Ziqiu. Survey of Vision-Based Augmented Reality 3D Registration Technology[J]. Journal of System Simulation, 2019, 31(11): 2206-2215.

References

[1] 朱杰杰, 潘志庚, 孙超. 增强现实技术应用综述[C]. 第四届和谐人机环境联合学术会议论文集, 武汉: 中国图象图形学会多媒体专业委员会, 2008: 1-8.
Zhu Jiejie, Pan Zhigeng, Sun Chao.A review of the application of augmented reality technology[C]. Proceedings of the 4th Joint Conference on Harmonious Human-Machine Environment, Wuhan: Multimedia Professional Committee of China Image and Graphics Society, 2008: 1-8.
[2] Azuma R T.A survey of augmented reality[J]. Presence: Teleoperators & Virtual Environments (S1054-7460), 1997, 6(4): 355-385.
[3] 王宇希, 张凤军, 刘越. 增强现实技术研究现状及发展趋势[J]. 科技导报, 2018, 36(10): 75-83.
Wang Yuxi, Zhang Fengjun, Liu Yue.Research Status and Development Trend of Augmented Reality Technology[J]. Science & Technology Review, 2018, 36(10): 75-83.
[4] 施琦, 王涌天, 陈靖. 一种基于视觉的增强现实三维注册算法[J]. 中国图象图形学报, 2002, 7(7): 56-60. Shi Qi, Wang Yongtian, Chen Jing. A Visual Augmented Reality 3D Registration Algorithm[J]. Journal of Image and Graphics, 2002, 7(7): 56-60.
[5] Zhou F, Duh H B L, Billinghurst M. Trends in augmented reality tracking, interaction and display: A review of ten years of ISMAR[C]. Proceedings of the 7^th IEEE/ACM international symposium on mixed and augmented reality. Cambridge, UK: IEEE Computer Society, 2008: 193-202.
[6] 高净业. 增强现实人机交互系统的研究[D]. 杭州: 杭州电子科技大学, 2011.
Gao Jingye.Research on augmented reality human-computer interaction system [D]. Hangzhou: Hangzhou University of Electronic Technology, 2011.
[7] Bajura M, Neumann U.Dynamic registration correction in video-based augmented reality systems[J]. IEEE Computer Graphics and Applications (S1558-1756), 1995, 15(5): 52-60.
[8] Nee A Y C, Ong S K, Chryssolouris G, et al. Augmented reality applications in design and manufacturing[J]. CIRP annals (S0007-8506), 2012, 61(2): 657-679.
[9] Fiala M.Designing highly reliable fiducial markers[J]. IEEE Transactions on Pattern analysis and machine intelligence (S1939-3539), 2009, 32(7): 1317-1324.
[10] Kato H, Billinghurst M.Marker tracking and hmd calibration for a video-based augmented reality conferencing system[C]. Proceedings 2^nd IEEE and ACM International Workshop on Augmented Reality (IWAR'99). San Francisco, CA, USA: IEEE, 1999: 85-94.
[11] Khan D, Ullah S, Yan D M, et al.Robust tracking through the design of high quality fiducial markers: an optimization tool for ARToolKit[J]. IEEE access (S2169-3536), 2018, 6: 22421-22433.
[12] Fiala M.ARTag, a fiducial marker system using digital techniques[C]. 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). San Diego, CA, USA: IEEE (S1063-6919), 2005, 2: 590-596.
[13] 王维廉, 王北星, 黄岚. 两种基于标识的增强现实注册方法的比较[J]. 东北师大学报(自然科学版), 2007(3): 32-36.
Wang Weilian, Wang Beixing, Huang Lan.Comparison of Two Registration-based Augmented Reality Registration Methods[J]. Journal of Northeast Normal University (Natural Science Edition), 2007(3): 32-36.
[14] Park J, You S, Neumann U.Natural feature tracking for extendible robust augmented realities[C]. Proc. Int. Workshop on Augmented Reality, 1998, 2(3): 2.
[15] Lowe D G.Distinctive image features from scale-invariant keypoints[J]. International journal of computer vision (S1573-1405), 2004, 60(2): 91-110.
[16] Bay H, Tuytelaars T, Van Gool L.Surf: Speeded up robust features[C]. European conference on computer vision. Berlin, Heidelberg: Springer, 2006: 404-417.
[17] Rosten E, Drummond T.Machine learning for high-speed corner detection[C]. European conference on computer vision. Berlin, Heidelberg: Springer, 2006: 430-443.
[18] Rublee E, Rabaud V, Konolige K, et al.ORB: An efficient alternative to SIFT or SURF[C]. ICCV.(S1550-5499). Barcelona, Spain: IEEE, 2011, 11(1): 2.
[19] Leutenegger S, Chli M, Siegwart R.BRISK: Binary robust invariant scalable keypoints[C]. 2011 IEEE international conference on computer vision (ICCV). IEEE. Barcelona, Spain: IEEE, 2011: 2548-2555.
[20] 刘佳, 王强, 张小瑞, 等. 移动增强现实跟踪注册技术概述[J]. 南京信息工程大学学报(自然科学版), 2018, 10(2): 203-212.
Liu Jia, Wang Qiang, Zhang Xiaorui, et al.Overview of Mobile Augmented Reality Tracking Registration Technology[J]. Journal of Nanjing University of Information Science & Technology(Natural Science), 2018, 10(2): 203-212.
[21] Carmigniani J, Furht B, Anisetti M, et al.Augmented reality technologies, systems and applications[J]. Multimedia tools and applications (S1573-7721), 2011, 51(1): 341-377.
[22] Wuest H, Vial F, Stricker D.Adaptive line tracking with multiple hypotheses for augmented reality[C]. Proceedings of the 4^th IEEE/ACM International Symposium on Mixed and Augmented Reality. Vienna, Austria, Austria: IEEE Computer Society, 2005: 62-69.
[23] Vacchetti L, Lepetit V, Fua P.Combining edge and texture information for real-time accurate 3d camera tracking[C]. Third IEEE and ACM International Symposium on Mixed and Augmented Reality. Arlington, VA, USA, USA: IEEE, 2004: 48-56.
[24] Reitmayr G, Drummond T.Going out: robust model-based tracking for outdoor augmented reality[C]. ISMAR. Santa Barbard, CA, USA: IEEE, 2006, 6: 109-118.
[25] Cho Y, Lee J, Neumann U.A multi-ring color fiducial system and an intensity-invariant detection method for scalable fiducial-tracking augmented reality[C]. IWAR. 1998: 147-165.
[26] Mohamed B, Mohamed B.Proposition of a 3D pattern for e-learning augmented reality applications based on ARToolkit library[C]. International Conference on Education and e-Learning Innovations. Sousse, Tunisia: IEEE, 2012: 1-4.
[27] Vogt S, Khamene A, Sauer F, et al.Single camera tracking of marker clusters: Multiparameter cluster optimization and experimental verification[C]. Proceedings. International Symposium on Mixed and Augmented Reality. Darmstadt, Germany: IEEE, 2002: 127-136.
[28] 刘嘉敏, 安乐祥, 常燕, 等. 增强现实中基于三角标识的三维注册方法[J]. 沈阳工业大学学报, 2013, 35(1): 79-84.
Liu Jiamin, An Lexiang, Chang Yan, et al.Three-dimensional registration method based on triangular identification in augmented reality[J]. Journal of Shenyang University of Technology, 2013, 35(1): 79-84.
[29] Liu J, Chen S, Sun H, et al.Real time tracking method by using color markers[C]. 2013 International Conference on Virtual Reality and Visualization. Xi'an, China: IEEE, 2013: 106-111.
[30] 向程谕, 王冬丽, 李建勋, 等. 基于改进SIFT特征的深度图像匹配[J]. 计算机应用, 2016, 36(增2): 135-138.
Xiang Chengyu, Wang Dongli, Li Jianxun, et al.Depth image matching based on improved SIFT features[J]. Journal of Computer Applications, 2016, 36(S2): 135-138.
[31] 王一璋, 王亚刚. 基于改进SURF算法的机器人识别匹配方法[J]. 软件导刊, 2018, 17(10): 14-17.
Wang Yizhang, Wang Yagang.A Robot Identification and Matching Method Based on Improved SURF Algorithm[J]. Software Guide, 2018, 17(10): 14-17.
[32] 何林阳, 刘晶红, 李刚, 等. 改进BRISK特征的快速图像配准算法[J].红外与激光工程, 2014, 43(8): 2722-2727.
He Linyang, Liu Jinghong, Li Gang, et al.A Fast Image Registration Algorithm with Improved BRISK Features[J]. Infrared and Laser Engineering, 2014, 43(8): 2722-2727.
[33] 白雪冰, 车进, 牟晓凯, 等. 结合快速鲁棒性特征改进ORB的特征点匹配算法[J]. 计算机应用, 2016, 36(7): 1923-1926.
Bai Xuebing, Che Jin, Mu Xiaokai, et al.Improved ORB feature point matching algorithm based on fast robustness characteristics[J]. Journal of Computer Applications, 2016, 36(7): 1923-1926.
[34] Bunnun P, Mayol-Cuevas W W. OutlinAR: an assisted interactive model building system with reduced computational effort[C]. 2008 7^th IEEE/ACM International Symposium on Mixed and Augmented Reality. Cambridge, UK: IEEE, 2008: 61-64.
[35] Klein G, Murray D W.Full-3D Edge Tracking with a Particle Filter[C]. 2006: 1119-1128.
[36] Neubert J, Pretlove J, Drummond T.Rapidly constructed appearance models for tracking in augmented reality applications[J]. UK: BMVC, Machine Vision and Applications (S1432-1769), 2012, 23(5): 843-856.
[37] Durrant-Whyte H, Bailey T.Simultaneous localization and mapping: part I[J]. IEEE robotics & automation magazine (S1558-223X), 2006, 13(2): 99-110.
[38] Bailey T, Durrant-Whyte H.Simultaneous localization and mapping (SLAM): Part II[J]. IEEE Robotics & Automation Magazine (S1558-223X), 2006, 13(3): 108-117.
[39] Klein G, Murray D.Parallel tracking and mapping for small AR workspaces[C]. Proceedings of the 2007 6^th IEEE and ACM International Symposium on Mixed and Augmented Reality. Nara, Japan: IEEE Computer Society, 2007: 1-10.
[40] Civera J, Davison A J, Montiel J M M. Inverse depth parametrization for monocular SLAM[J]. IEEE transactions on robotics (S1941-0468), 2008, 24(5): 932-945.
[41] 高翔, 安辉, 陈为, 等. 移动增强现实可视化综述[J].计算机辅助设计与图形学学报, 2018, 30(1):1-8.
Gao Xiang, An Hui, Chen Wei, et al.A Review of Mobile Augmented Reality Visualization[J]. Journal of Computer-Aided Design & Computer Graphics, 2018, 30(1): 1-8.
[42] Garon M, Lalonde J F.Deep 6-DOF tracking[J]. IEEE transactions on visualization and computer graphics (S1941-0506), 2017, 23(11): 2410-2418.