基于卷积神经网络的深度图姿态估计算法研究

doi:10.16182/j.issn1004731x.joss.201711003

摘要/Abstract

摘要： 随着深度相机的应用,三维场景的重建越来越简单、快速。从单视角的深度场景图像中检索出物体还是比较困难,特别是物体的姿态估计。提出了一种基于卷积神经网络的深度图像姿态估计算法。该算法采用了回归估计来实现姿态的估计。通过3D模型合成大量不同姿态的深度图像样本,从而解决回归估计需要稠密采样的训练数据问题。对于不同类别的物体,分别用线性回归估计来拟合姿态函数。在基于LeNet-5模型上修改了卷积神经网络的结构,使得该网络适用于回归估计。实验结果表明：我们的方法取得了平均误差约4.3°的估计结果,优于其他文献的方法。

关键词: 姿态估计, 卷积神经网络, 深度图像, 场景重建

Abstract: 3D scenes can be reconstructed more easily and rapidly with depth camera. However, it is difficult to retrieve items in 3D scenes from a single view depth image, especially for the pose estimation. In this paper, we present a method of pose estimation using convolutional neural network with synthesis depth data, which predicts the items' pose in 3D scenes by regression. This is achieved by (i) synthesizing large amount of depth images with different pose for linear regression using 3D model, (ii) designing a class-dependent linear regression framework, which estimates the object's pose from different classes separately, (iii) reforming LeNet-5 model by representing the loss layer as a linear regression form. The proposed algorithm is demonstrated on different data sets and achieves higher accuracy (average error 4.3°) than other algorithms.

Key words: pose estimation, convolutional neural networks, depth images, scene reconstruction

中图分类号:

TP391.4

王松, 刘复昌, 黄骥, 许威威, 董洪伟. 基于卷积神经网络的深度图姿态估计算法研究[J]. 系统仿真学报, 2017, 29(11): 2618-2623.

Wang Song, Liu Fuchang, Huang Ji, Xu Weiwei, Dong Hongwei. Pose Estimation Using Convolutional Neural Network with Synthesis Depth Data[J]. Journal of System Simulation, 2017, 29(11): 2618-2623.

参考文献

[1] Poggio T, Edelman S.A network that learns to recognize 3D objects[J]. Nature (S1476-4687), 1990, 343(6255): 263-266.
[2] Khotanzad A R, Liou J H.Neural network system for 3D object recognition and pose estimation from a single arbitrary 2D view[J]. Proceedings of SPIE - The International Society for Optical Engineering (S0277-786X ), 1992, 1709: 107-118.
[3] Savarese S, Li F F.3D generic object categorization, localization and pose estimation[C]// Proceedings / IEEE International Conference on Computer Vision. IEEE International Conference on Computer Vision. 2007: 1-8.
[4] Fanelli G, Weise T, Gall J, et al.Real Time Head Pose Estimation from Consumer Depth Cameras[M]// Pattern Recognition. Springer Berlin Heidelberg, 2011: 101-110.
[5] Shao T, Xu W, Zhou K, et al.An interactive approach to semantic modeling of indoor scenes with an RGBD camera[J]. Acm Transactions on Graphics (S0730-0301), 2012, 31(6): 439-445.
[6] Lu C P, Hager G D, Mjolsness E.Fast and globally convergent pose estimation from video images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (S0162-8828 ), 2000, 22(6): 610-622.
[7] 冷大炜, 马洪兵, 孙卫东. 基于2D-3D泛轮廓点对应的三维刚体目标的迭代姿态估计[J]. 中国科学院大学学报, 2012,29(6): 821-828.
Leng D W, Ma H B, Sun W D.Rigid object based on general 2D-3D contour point Iterative pose estimation of a 3D correspondence[J]. Journal of Graduate University of Chinese Academy of Sciences, 2012, 29(6): 821-828.
[8] Haralick R M, Joo H, Lee C N, et al.Pose Estimation from Corresponding Point Data[J]. IEEE Transactions on Systems Man & Cybernetics (S0018-9472), 1989, 19(6): 1-84.
[9] 邱丽梅. 基于人脸特征点和线性回归的3D人脸姿态估计方法[J]. 三明学院学报, 2008, 25(4): 390-394.
Qiu L M.3D Face Pose Estimation Based on Face Feature Points and Linear Regression[J]. Journal of Sanming University, 2008, 25(4): 390-394.
[10] 张浩鹏, 姜志国. 基于姿态加权核回归的航天器姿态估计[J]. 北京航空航天大学学报, 2014, 40(4): 494-499.
Zhang H P, Jiang Z G.Spacecraft attitude estimation based on attitude-weighted kernel regression[J]. Beijing Hangkong Hangtian Daxue Xuebao/journal of Beijing University of Aeronautics & Astronautics, 2014, 40(4): 494-499.
[11] Le Cun Y, Bengio Y, Hinton G.Deep learning[J]. Nature (S1476-4687), 2015, 521(7553): 436-444.
[12] Schulz H, Behnke S.Deep learning[J]. KI-Künstliche Intelligenz(S1610-1987), 2012, 26(4): 357-363.
[13] Schmidhuber J.Deep learning in neural networks: An overview[J]. Neural networks, 2015, 61: 85-117.
[14] Krizhevsky A, Sutskever I, Hinton G E.ImageNet Classification with Deep Convolutional Neural Networks[J]. Advances in Neural Information Processing Systems (S1049-5258), 2012, 25(2): 2012.
[15] Massa F, Aubry M, Marlet R.Convolutional Neural Networks for joint object detection and pose estimation: A comparative study[J]. Revista Brasileira De Farmacognosia (S0102-695X ), 2014, 19(2a): 412-417.
[16] Simard P Y, Steinkraus D, Platt J C.Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis[C]//International Conference on Document Analysis and Recognition. IEEE Computer Society, 2003: 958.
[17] Gkioxari G, Hariharan B, Girshick R, et al.R-CNNs for Pose Estimation and Action Detection[J]. Computer Science (S1508-2806), 2014.
[18] Ouyang W, Chu X, Wang X.Multi-source Deep Learning for Human Pose Estimation[C]// IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 2014: 2337-2344.
[19] Smith S M, Brady J M.SUSAN—a new approach to low level image processing[J]. International journal of computer vision (S0920-5691), 1997, 23(1): 45-78.
[20] Su H, Qi C R, Li Y, et al.Render for CNN: Viewpoint Estimation in Images Using CNN s Trained with Rendered 3D Model Views[C]// IEEE International Conference on Computer Vision. IEEE, 2015.
[21] 王春雪. 基于图像的空间目标三维姿态估计研究[D]. 中国科学院大学(工程管理与信息技术学院), 2014.
Wang C L.Three dimensional pose estimation of space target based on image[D]. University of Chinese Academy of Sciences, 2014.
[22] LeCun Y. LeNet-5, convolutional neural networks[J]. URL: http://yann. lecun. com/exdb/lenet, 2015.
[23] Tulsiani S, Malik J.Viewpoints and keypoints[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 1510-1519.
[24] Eslami S M A, Heess N, Williams C K I, et al. The shape boltzmann machine: a strong model of object shape[J]. International Journal of Computer Vision (S0920-5691), 2014, 107(2): 155-176.