Modeling Indoor environment with Kinect

doi:10.16182/j.issn1004731x.joss.19-FZ0439

Abstract

Abstract: To solve the problems of point cloud noise, depth data mismatching and local details prone to hollowing in the process of Kinect camera reconstruction of indoor three-dimensional environment. By improving the assignment method of depth image pixel and the threshold relationship of time frame matching, this paper denoises the point cloud with optimizing mean filtering and time frame weighting methods. Combined with recent iteration algorithm (Iterative Closest Point, ICP) to complete the adjacent point cloud splicing so that it can achieve completely indoor model to build 3D environment. To test the validity of the algorithm, the experiment of reconstructing indoor environment with Kinect camera is designed. The experimental results show that by using the improved mean filter and time frame weighted method, the point cloud model quality loss compared to that before noise reduction falls by 3.33% on average. Point cloud signal-to-noise ratio is improved compared to that before noise reduction processing before by 2.18dB, which is given to illustrate the feasibility of these two optimization algorithms.

Key words: 3D reconstruction, Kinect, point cloud noise reduction, registration

CLC Number:

TP391.9

Xu Lianrui, Zhang Jinming. Modeling Indoor environment with Kinect[J]. Journal of System Simulation, 2019, 31(12): 2643-2651.

References

[1] Feng J, Teng Q, He X, et al.Reconstruction of Three-dimensional Heterogeneous Media from a Single Two-dimensional Section via Co-occurrence Correlation Function[J]. Computational Materials Science (S0927-0256), 2018, 144: 181-192.
[2] 王亚琪, 陈成军, 李希彬, 等. 基于Kinect的轴类零件三维重建研究[J]. 计算技术与自动化, 2015(4): 126-131.
Wang Yaqi, Chen Chengjun, Li Xibin, et al.Research on Kinect-Based 3D Reconstruction of Shaft-type Parts[J]. Computing Technology and Automation, 2015(4): 126-131.
[3] Macknojia R.Design and Calibration of a Network of RGB-D Sensors for Robotic Applications over Large Workspaces[D]. Université d'Ottawa/University of Ottawa, 2013.
[4] 陈晓明, 蒋乐天, 应忍冬. 基于Kinect 深度信息的实时三维重建和滤波算法研究[J]. 计算机应用研究, 2013, 30(4): 1216-1218.
Chen Xiaoming, Jiang Letian, Ying Rendong.Research of 3D reconstruction and filtering algorithm based on depth information of Kinect[J]. Journal of Computer Applications, 2013, 30(4): 1216-1218.
[5] Choi S, Zhou Q Y, Koltun V.Robust reconstruction of indoor scenes[C]// Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 5556-5565.
[6] 李翔, 李泽丰, 李治江. 基于Kinect的人体三维重建方法[J]. 计算机系统应用, 2018, 27(6): 180-185.
Li Xiang, Li Zefeng, Li Zhijiang.3D Reconstruction of Human Body Based on Kinect[J]. Journal of Computer Systems, 2018, 27(6): 180-185.
[7] NieBNer M, ZollhFer M, Izadi S, et al. Real-time 3D reconstruction at scale using voxel hashing[J]. ACM Transactions on Graphics (S0736-721X), 2013, 32(6): 1-11.
[8] Newcombe R A, Izadi S, Hilliges O, et al.Kinectfusion: Real-time dense surface mapping and tracking[C]// ISMAR. 2011, 11: 127-136.
[9] 刘鑫, 许华荣, 胡占义. 基于GPU和Kinect的快速物体重建[J]. 自动化学报, 2012, 38(8): 1288-1297.
Liu Xin, Xu Huarong, Hu Zhanyi.GPU Based Fast 3D-Object Modeling with Kinect[J]. Acta Automatica Sinica, 2012, 38(8): 1288-1297.
[10] 张娟, 张习民, 万旺根, 等. 利用显著性的点云模型客观质量评价[J]. 应用科学学报, 2014, 32(5): 441-446.
Zhang Juan, Zhang Ximin, Wan Wanggen, et al.Objective quality evaluation using significant point cloud model[J]. Journal of Applied Sciences, 2014, 32(5): 441-446.
[11] 胡正乙, 谭庆昌, 孙秋成. 基于RGB-D的室内场景实时三维重建算法[J]. 东北大学学报: (自然科学版), 2017, 38(12): 1764-1768.
Hu Zhengyi, Tan Qingchang, Sun Qiucheng.Real-time 3D reconstruction algorithm for indoor scene based on RGB-D[J]. Journal of Northeastern University:(Natural Science), 2017, 38(12): 1764-1768.
[12] 王亚龙, 张奇志, 周亚丽. 基于RGB-D相机的室内环境3D地图创建[J]. 计算机应用研究, 2015, 32(8): 2533-2537.
Wang Yalong, Zhang Qizhi, Zhou Yali.3D map creation of indoor environment based on RGB-D camera[J]. Journal of Computer Applications, 2015, 32(8): 2533-2537.
[13] Hijazi I H, Ehlers M, Zlatanova S.NIBU: a new approach to representing and analysing interior utility networks within 3D geo-information systems[J]. International Journal of Digital Earth (S1755-1315), 2012, 5(1): 21.
[14] Henry P, Krainin M, Herbst E, et al.RGB-D mapping: Using Kinect-style depth cameras for dense 3D modeling of indoor environments[J]. International Journal of Robotics Research (S0278-3649), 2013, 31(5): 647-663.
[15] Santos J M, Portugal D, Rocha R P.An evaluation of 2D SLAM techniques available in robot operating system[C]// 2013 IEEE International Symposium on Safety, Security, and Rescue Robotics (SSRR). IEEE, 2013: 1-6.
[16] Chao Y, Tillo T, Yao Z, et al.Depth Map Driven Hole Filling Algorithm Exploiting Temporal Correlation Information[J]. IEEE Transactions on Broadcasting (S0018-9316), 2014, 60(2): 394-404.
[17] Liu W, Chen X, Yang J, et al.Robust Color Guided Depth Map Restoration[J]. IEEE Trans Image Process (S1057-7149), 2016, 26(1): 315-327.
[18] Chen F, Hu T, Zuo L, et al.Depth map inpainting via sparse distortion model[J]. Digital Signal Processing (S1051-2004), 2016, 58(C): 93-101.
[19] Staranowicz A, Brown G R, Morbidi F, et al.Easy-to-use and accurate calibration of rgb-d cameras from spheres[C]// Pacific-Rim Symposium on Image and Video Technology. Springer, Berlin, Heidelberg, 2013: 265-278.
[20] Besl P J, Mckay N D.A method for registration of 3-D shapes[J]. IEEE Transactions on Pattern Analysis & Machine Intelligence (S0162-8828), 1992, 14(2): 239-256.
[21] Senin N, Colosimo B M, Pacella M.Point set augmentation through fitting for enhanced ICP registration of point clouds in multisensor coordinate metrology[J]. Robotics & Computer Integrated Manufacturing (S0736-5845), 2013, 29(1): 39-52.
[22] 储珺, 聂春梅, 王璐, 等. 基于SIFT特征的多视点云数据配准和拼接算法[J]. 半导体光电, 2011, 32(3): 442-447.
Chu Jun, Nie Chunmei, Wang Lu, et al.Multi-viewpoint cloud data registration and splicing algorithm based on SIFT feature[J]. Semiconductor Optoelectronics, 2011, 32(3): 442-447.
[23] 周春艳, 李勇, 邹峥嵘. 三维点云ICP算法改进研究[J]. 计算机技术与发展, 2011, 21(8): 75-77.
Zhou Chunyan, Li Yong, Zou Wei.Research on Improvement of 3D Point Cloud ICP Algorithm[J]. Computer Technology and Development, 2011, 21(8): 75-77.
[24] Li Y, Zheng Q, Sharf A, et al.2D-3D fusion for layer decomposition of urban facades[C]// International Conference on Computer Vision. 2011.