Pose Estimation Using Convolutional Neural Network with Synthesis Depth Data

doi:10.16182/j.issn1004731x.joss.201711003

Abstract

Abstract: 3D scenes can be reconstructed more easily and rapidly with depth camera. However, it is difficult to retrieve items in 3D scenes from a single view depth image, especially for the pose estimation. In this paper, we present a method of pose estimation using convolutional neural network with synthesis depth data, which predicts the items' pose in 3D scenes by regression. This is achieved by (i) synthesizing large amount of depth images with different pose for linear regression using 3D model, (ii) designing a class-dependent linear regression framework, which estimates the object's pose from different classes separately, (iii) reforming LeNet-5 model by representing the loss layer as a linear regression form. The proposed algorithm is demonstrated on different data sets and achieves higher accuracy (average error 4.3°) than other algorithms.

Key words: pose estimation, convolutional neural networks, depth images, scene reconstruction

CLC Number:

TP391.4

Wang Song, Liu Fuchang, Huang Ji, Xu Weiwei, Dong Hongwei. Pose Estimation Using Convolutional Neural Network with Synthesis Depth Data[J]. Journal of System Simulation, 2017, 29(11): 2618-2623.

References

[1] Poggio T, Edelman S.A network that learns to recognize 3D objects[J]. Nature (S1476-4687), 1990, 343(6255): 263-266.
[2] Khotanzad A R, Liou J H.Neural network system for 3D object recognition and pose estimation from a single arbitrary 2D view[J]. Proceedings of SPIE - The International Society for Optical Engineering (S0277-786X ), 1992, 1709: 107-118.
[3] Savarese S, Li F F.3D generic object categorization, localization and pose estimation[C]// Proceedings / IEEE International Conference on Computer Vision. IEEE International Conference on Computer Vision. 2007: 1-8.
[4] Fanelli G, Weise T, Gall J, et al.Real Time Head Pose Estimation from Consumer Depth Cameras[M]// Pattern Recognition. Springer Berlin Heidelberg, 2011: 101-110.
[5] Shao T, Xu W, Zhou K, et al.An interactive approach to semantic modeling of indoor scenes with an RGBD camera[J]. Acm Transactions on Graphics (S0730-0301), 2012, 31(6): 439-445.
[6] Lu C P, Hager G D, Mjolsness E.Fast and globally convergent pose estimation from video images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (S0162-8828 ), 2000, 22(6): 610-622.
[7] 冷大炜, 马洪兵, 孙卫东. 基于2D-3D泛轮廓点对应的三维刚体目标的迭代姿态估计[J]. 中国科学院大学学报, 2012,29(6): 821-828.
Leng D W, Ma H B, Sun W D.Rigid object based on general 2D-3D contour point Iterative pose estimation of a 3D correspondence[J]. Journal of Graduate University of Chinese Academy of Sciences, 2012, 29(6): 821-828.
[8] Haralick R M, Joo H, Lee C N, et al.Pose Estimation from Corresponding Point Data[J]. IEEE Transactions on Systems Man & Cybernetics (S0018-9472), 1989, 19(6): 1-84.
[9] 邱丽梅. 基于人脸特征点和线性回归的3D人脸姿态估计方法[J]. 三明学院学报, 2008, 25(4): 390-394.
Qiu L M.3D Face Pose Estimation Based on Face Feature Points and Linear Regression[J]. Journal of Sanming University, 2008, 25(4): 390-394.
[10] 张浩鹏, 姜志国. 基于姿态加权核回归的航天器姿态估计[J]. 北京航空航天大学学报, 2014, 40(4): 494-499.
Zhang H P, Jiang Z G.Spacecraft attitude estimation based on attitude-weighted kernel regression[J]. Beijing Hangkong Hangtian Daxue Xuebao/journal of Beijing University of Aeronautics & Astronautics, 2014, 40(4): 494-499.
[11] Le Cun Y, Bengio Y, Hinton G.Deep learning[J]. Nature (S1476-4687), 2015, 521(7553): 436-444.
[12] Schulz H, Behnke S.Deep learning[J]. KI-Künstliche Intelligenz(S1610-1987), 2012, 26(4): 357-363.
[13] Schmidhuber J.Deep learning in neural networks: An overview[J]. Neural networks, 2015, 61: 85-117.
[14] Krizhevsky A, Sutskever I, Hinton G E.ImageNet Classification with Deep Convolutional Neural Networks[J]. Advances in Neural Information Processing Systems (S1049-5258), 2012, 25(2): 2012.
[15] Massa F, Aubry M, Marlet R.Convolutional Neural Networks for joint object detection and pose estimation: A comparative study[J]. Revista Brasileira De Farmacognosia (S0102-695X ), 2014, 19(2a): 412-417.
[16] Simard P Y, Steinkraus D, Platt J C.Best Practices for Convolutional Neural Networks Applied to Visual Document Analysis[C]//International Conference on Document Analysis and Recognition. IEEE Computer Society, 2003: 958.
[17] Gkioxari G, Hariharan B, Girshick R, et al.R-CNNs for Pose Estimation and Action Detection[J]. Computer Science (S1508-2806), 2014.
[18] Ouyang W, Chu X, Wang X.Multi-source Deep Learning for Human Pose Estimation[C]// IEEE Conference on Computer Vision and Pattern Recognition. IEEE Computer Society, 2014: 2337-2344.
[19] Smith S M, Brady J M.SUSAN—a new approach to low level image processing[J]. International journal of computer vision (S0920-5691), 1997, 23(1): 45-78.
[20] Su H, Qi C R, Li Y, et al.Render for CNN: Viewpoint Estimation in Images Using CNN s Trained with Rendered 3D Model Views[C]// IEEE International Conference on Computer Vision. IEEE, 2015.
[21] 王春雪. 基于图像的空间目标三维姿态估计研究[D]. 中国科学院大学(工程管理与信息技术学院), 2014.
Wang C L.Three dimensional pose estimation of space target based on image[D]. University of Chinese Academy of Sciences, 2014.
[22] LeCun Y. LeNet-5, convolutional neural networks[J]. URL: http://yann. lecun. com/exdb/lenet, 2015.
[23] Tulsiani S, Malik J.Viewpoints and keypoints[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2015: 1510-1519.
[24] Eslami S M A, Heess N, Williams C K I, et al. The shape boltzmann machine: a strong model of object shape[J]. International Journal of Computer Vision (S0920-5691), 2014, 107(2): 155-176.