系统仿真学报 ›› 2018, Vol. 30 ›› Issue (7): 2423-2444.doi: 10.16182/j.issn1004731x.joss.201807001
• 专家约稿 • 下一篇
王珊1,2, 沈旭昆1,2, 赵沁平1
收稿日期:
2018-06-08
出版日期:
2018-07-10
发布日期:
2019-01-08
作者简介:
王珊(1982-), 女, 回族, 博士生, 实验师,研究方向为虚拟现实; 沈旭昆(1965-), 男, 云南, 博士, 教授, 博导, 研究方向为虚拟现实; 赵沁平(1948-), 男, 山西, 博士, 教授, 博导, 中国工程院院士, 研究方向虚拟现实,人工智能。
基金资助:
Wang Shan1,2, Shen Xukun1,2, Zhao Qinping1
Received:
2018-06-08
Online:
2018-07-10
Published:
2019-01-08
摘要: 人脸是人类重要的生物特征,逼真重建三维人脸面部表情有着广泛的应用,多年来一直受到研究人员的密切关注。目前,比较成熟的人脸三维模型获取大多还是依靠实验室昂贵的硬件系统和可控的光照环境,而学术界的研究热点已逐渐过渡到基于日常生活环境和低成本设备的研究。按照输入数据的来源,将现有的研究工作分为两大类:一是基于可控环境的三维人脸获取及重建技术;二是基于非可控环境的三维人脸获取及重建技术。以此为线索对当前技术和方法进行总结分类,并在此基础上探讨了当前研究存在的问题和未来的研究方向。
中图分类号:
王珊, 沈旭昆, 赵沁平. 三维人脸表情获取及重建技术综述[J]. 系统仿真学报, 2018, 30(7): 2423-2444.
Wang Shan, Shen Xukun, Zhao Qinping. Review of 3D Facial Expression Acquisition and Modeling Technology[J]. Journal of System Simulation, 2018, 30(7): 2423-2444.
[1] J Roth, Y Tong, Liu X. Adaptive 3D Face Reconstruction from Unconstrained Photo Collections[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (S0162-8828), 2017, 39(11): 2127-2141. [2] Vicon. Vicon Cara System[EB/OL].[2018-05-16]. https://www.vicon.com/products/camera-systems/cara-1. [3] OpTiTrack. OptiTrack Expression[EB/OL]. [2018-05-16]. http://optitrack.com/products/expression/indepth.html. [4] Guenter B, Grimm C, Wood D, et al.Making faces[C]// Proceedings of the 25th annual conference on Computer graphics and interactive techniques. 1998, ACM. 1998: 55-66. [5] Bickel B, Botsch M, Angst R, et al.Multi-scale capture of facial geometry and motion[J]. ACM Trans. Graph.(S0730-0301), 2007, 26(3): 33. [6] Huang H, Chai J, Tong X, et al.Leveraging motion capture and 3D scanning for high-fidelity facial performance acquisition[J]. ACM Trans. Graph.(S0730-0301), 2011, 30(4): 1-10. [7] Beeler T, Bickel B, Beardsley P, et al.High-quality single-shot capture of facial geometry[J]. ACM Trans. Graph.(S0730-0301), 2010, 29(4): 1-9. [8] Beeler T, Hahn F, Bradley D, et al.High-quality passive facial performance capture using anchor frames[C]// ACM SIGGRAPH 2011 papers. 2011, ACM: Vancouver, British Columbia, Canada. 2011: 1-10. [9] Bradley D, Heidrich W, Popa T, et al.High resolution passive facial performance capture[J]. ACM Trans. Graph.(S0730-0301), 2010, 29(4): 1-10. [10] Zhang L, Snavely N, Curless B, et al.Spacetime faces: high resolution capture for modeling and animation[J]. ACM Trans. Graph.(S0730-0301), 2004, 23(3): 548-558. [11] Wang Y, Huang X, Lee C, et al.High Resolution Acquisition, Learning and Transfer of Dynamic 3-D Facial Expressions[J]. Computer Graphics Forum (S0167-7055), 2004, 23(3): 677-686. [12] Li H, Adams B, Guibas L, et al.Robust single-view geometry and motion reconstruction[J]. ACM Trans. Graph.(S0730-0301), 2009, 28(5): 1-10. [13] Ma W C, Hawkins T, Peers P, et al.Rapid acquisition of specular and diffuse normal maps from polarized spherical gradient illumination[C]//Eurographics Conference on Rendering Techniques, 2007: 183-194. [14] Debevec P, Hawkins T, Tchou C, et al.Acquiring the reflectance field of a human face[C]//Proceedings of the 27th annual conference on Computer graphics and interactive techniques. 2000, ACM Press/Addison- Wesley Publishing Co. 2000: 145-156. [15] Hawkins T, Wenger A, Tchou C, et al.Animatable Facial Reflectance Fields[C]//Rendering Techniques ’04 (Proceedings of the Second Eurographics Symposium on Rendering, 2004: 309-321. [16] Ma W C, Jones A, Chiang J Y, et al.Facial performance synthesis using deformation-driven polynomial displacement maps[J]. ACM Trans. Graph.(S0730-0301), 2008, 27(5): 1-10. [17] Ghosh A, Fyffe G, Tunwattanapong B, et al.Multiview face capture using polarized spherical gradient illumination[J]. ACM Trans. Graph.(S0730-0301), 2011, 30(6): 1-10. [18] Blanz V, T Vetter. A morphable model for the synthesis of 3D faces[C]//Proceedings of the 26th annual conference on Computer graphics and interactive techniques. 1999, ACM Press/Addison-Wesley Publishing Co. 1999: 187-194. [19] Cao C, Weng Y, Zhou S, et al.FaceWarehouse: A 3D Facial Expression Database for Visual Computing[J]. IEEE Transactions on Visualization & Computer Graphics (S1077-2626), 2014, 20(3): 413-425. [20] Cootes T F, Taylor C J.Statistical Models of Appearance for computer vision[C]//Proceedings of SPIE - The International Society for Optical Engineering, 2000. 4322(1): 236-248. [21] Matthews I, Baker S.Active appearance models revisited[J]. International Journal of Computer Vision (S0920-5691) , 2004, 60(2): 135-164. [22] Cristinacce D, Cootes T.Automatic feature localisation with constrained local models[J]. Pattern Recognition (S0031-3203), 2008, 41(10): 3054-3067. [23] Cristinacce D, Cootes T F.Feature Detection and Tracking with Constrained Local Models[C]//British Machine Vision Conference 2006, Edinburgh, UK, 2006. [24] Tulyakov S, Sebe N.Regressing a 3D Face Shape from a Single Image[C]//2015 IEEE International Conference on Computer Vision (ICCV), Santiago Chile: IEEE, 2015: 3748-3755. [25] Tzimiropoulos G.Project-Out Cascaded Regression with an application to face alignment[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015: 3659-3667. [26] Sun Y, Wang X, Tang X.Deep Convolutional Network Cascade for Facial Point Detection[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR, USA: IEEE, 2013: 3476-3483. [27] Wu Y, Wang Z, Ji Q.Facial Feature Tracking Under Varying Facial Expressions and Face Poses Based on Restricted Boltzmann Machines[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland, OR, USA: IEEE, 2013: 3452-3459. [28] Luo P, Wang X, Tang X.Hierarchical face parsing via deep learning[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA: IEEE, 2012: 2480-2487. [29] Zhang J, Shan S, Kan M, et al.Coarse-to-Fine Auto-Encoder Networks (CFAN) for Real-Time Face Alignment[C]//Computer Vision-ECCV 2014. Cham: Springer International Publishing. 2014: 1-16. [30] Hao W L, Xiaoming, Doretto G.Face alignment via boosted ranking model[C]//2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA: IEEE, 2008: 1-8. [31] Pandzic I S, Forchheimer R.MPEG-4 Facial Animation: The Standard, Implementation And Applications[M]. John Wiley & Sons, Inc., 2002: 299. [32] Ekman P, Friesen W V.Facial Action Coding System (FACS): a Technique for the Measurement of Facial Actions[J]. Rivista Di Psichiatria (S2038-2502), 1978, 47(2): 126-38. [33] Chuang E S, Deshpande H, Bregler C.Facial Expression Space Learning[C]//Pacific Conference on Computer Graphics and Applications. Beijing, China: IEEE, 2002: 68. [34] Pighin F, Hecker J, Lischinski D, et al.Synthesizing realistic facial expressions from photographs[C]// ACM SIGGRAPH 2005 Courses. 2005, ACM: Los Angeles, California. 2005. [35] Pyun H, Kim Y, Chae W, et al.An example-based approach for facial expression cloning[C]//ACM SIGGRAPH 2006 Courses. 2006, ACM: Boston, Massachusetts, 2006. [36] Chu B, S Romdhani, Chen L. 3D-Aided Face Recognition Robust to Expression and Pose Variations[C]//IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014: 1907-1914. [37] L Yin, X Chen, Y Sun, et al.A high-resolution 3D dynamic facial expression database[C]//2008 8th IEEE International Conference on Automatic Face Gesture Recognition. Amsterdam, Netherlands: IEEE, 2008: 1-6. [38] 3dMD. 3D Surface Imaging System [EB/OL]. [2018-05-16]. http://www.3dmd.com/. [39] Vlasic D, Brand M, Pfister H, et al.Face transfer with multilinear models[C]//ACM SIGGRAPH 2006 Courses. 2006, ACM: Boston, Massachusetts. 2006. [40] Kemelmacher-Shlizerman I, S M Seitz. Face reconstruction in the wild[J]. IEEE International Conference on Computer Vision, 2011, 58(11): 1746-1753. [41] J Roth, Y Tong, Liu X. Unconstrained 3D face reconstruction[C]//Computer Vision and Pattern Recognition. Boston, MA, USA: IEEE, 2015: 2606-2615. [42] Ruo Z, Ping-Sing T, J.E.C, et al. Shape-from-shading: a survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (S0162-8828), 1999, 21(8): 690-706. [43] Decarlo D, Metaxas D.The integration of optical flow and deformable models with applications to human face shape and motion estimation[C]//Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, USA: IEEE, 1996. [44] Brand M, Bhotika R.Flexible flow for 3D nonrigid racking and shape recovery[C]//Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001. Kauai, HI, USA: IEEE, 2001. [45] Borshukov G, Piponi D, Larsen O, et al.Universal capture: image-based facial animation for "The Matrix Reloaded"[C]//ACM SIGGRAPH 2006 Courses. 2006, ACM: Boston, Massachusetts, 2006. [46] Valgaerts L, Wu C, Bruhn A, et al.Lightweight binocular facial performance capture under uncontrolled lighting[J]. ACM Trans. Graph.(S0730-0301), 2012, 31(6): 1-11. [47] Weise T, Bouaziz S, Li H, et al.Realtime performance-based facial animation[J]. ACM Trans. Graph.(S0730-0301), 2011, 30(4): 1-10. [48] Fyffe G, Jones A, Alexander O, et al.Driving High-Resolution Facial Scans with Video Performance Capture[J]. ACM Trans. Graph.(S0730-0301), 2014, 34(1): 1-14. [49] Garrido P, Valgaert L, Wu C, et al.Reconstructing detailed dynamic face geometry from monocular video[J]. ACM Trans. Graph.(S0730-0301), 2013, 32(6): 1-10. [50] Li H, Yu J, Ye Y, et al., Realtime facial animation with on-the-fly correctives[J]. ACM Trans. Graph.(S0730-0301), 2013, 32(4): 1-10. [51] Bouaziz S, Wang Y, Pauly M.Online modeling for realtime facial animation[J]. ACM Trans. Graph.(S0730-0301), 2013, 32(4): 1-10. [52] Newcombe R A, Izadi S, Hilliges O, et al.KinectFusion: Real-time dense surface mapping and tracking[C]//2011 10th IEEE International Symposium on Mixed and Augmented Reality. Basel, Switzerland: IEEE, 2011. [53] Da F, Sui Y.3D reconstruction of human face based on an improved seeds-growing algorithm[J]. Machine Vision and Applications(S1432-1769), 2011, 22(5): 879-887. [54] Huang C, Hu W, Zhang Y.Robust 3D human face reconstruction by consumer binocular-stereo cameras[C]//Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry. 2012, ACM: Singapore, Singapore. 2012: 271-278. [55] Kemelmacher-Shlizerman I, R Basri. 3D Face Reconstruction from a Single Image Using a Single Reference Face Shape[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence (S0162-8828), 2011, 33(2): 394-405. [56] S F DARPA. Human-ID 3D Face Database[DB/OL]. [2018-05-16]. http://www.csee.usf.edu/sarkar. [57] Wu T, Zhou F, Liao Q.A fast 3D face reconstruction method from a single image using adjustable model[C]// 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Shanghai, China: IEEE, 2016. [58] Hassner T.Viewing Real-World Faces in 3D[C]//2013 IEEE International Conference on Computer Vision. Sydney, NSW, Australia: IEEE, 2013. [59] Booth J, Antonakos E, Ploumpis S, et al.3D Face Morphable Models "In-the-Wild"[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017. [60] Dou P, Shah S K, Kakadiaris I A.End-to-End 3D Face Reconstruction with Deep Neural Networks[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017. [61] Jackson A S, Bulat A, Argyriou V, et al.Large Pose 3D Face Reconstruction from a Single Image via Direct Volumetric CNN Regression[C]//2017 IEEE International Conference on Computer Vision (ICCV), 2017. [62] Tewari A, Zollhofer M, Kim H, et al.MoFA: Model- Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction[C]// 2017 IEEE International Conference on Computer Vision Workshops (ICCVW). Venice, Italy: IEEE, 2017. [63] Tran A T, Hassner T, Masi I, et al.Regressing Robust and Discriminative 3D Morphable Models with a Very Deep Neural Network[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017. [64] Taigman Y, Yang M, Ranzato M, et al.DeepFace: Closing the Gap to Human-Level Performance in Face Verification[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, USA: IEEE, 2014. [65] Richardson E, Sela M, Kimmel R.3D Face Reconstruction by Learning from Synthetic Data[C]// 2016 Fourth International Conference on 3D Vision (3DV). Stanford, CA, USA: IEEE, 2016. [66] Richardson E, Sela M, Or-EI R, et al.Learning Detailed Face Reconstruction from a Single Image[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017. [67] Paysan P, Knothe R, Amberg B, et al.A 3D Face Model for Pose and Illumination Invariant Face Recognition[C]//2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance. Genova, Italy: IEEE, 2009. [68] Sela M, Richardson E, Kimmel R.Unrestricted Facial Geometry Reconstruction Using Image-to-Image Translation[C]//2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017. [69] Kemelmacher-Shlizerman I, S M Seitz. Face reconstruction in the wild[C]//2011 International Conference on Computer Vision. Barcelona, Spain: IEEE, 2011. [70] Roth J, Yiying T, Liu X.Unconstrained 3D face reconstruction[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015. [71] Shi F, Wu H T,Tong X, et al.Automatic acquisition of high-fidelity facial performances using monocular videos[J]. ACM Trans. Graph.(S0730-0301), 2014. 33(6): 1-13. [72] Suwajanakorn S, I Kemelmacher-Shlizerman, S M Seitz. Total Moving Face Reconstruction[C]//European Conference on Computer Vision. Zurich, Switzerland: Springer, Cham, 2014. [73] Cao C, Weng Y, Lin S, et al.3D shape regression for real-time facial animation[J]. ACM Trans. Graph.(S0730-0301), 2013, 32(4): 1-10. [74] Cao C, Hou Q, Zhou K.Displaced dynamic expression regression for real-time facial tracking and animation[J]. ACM Trans. Graph.(S0730-0301), 2014, 33(4): 1-10. [75] Cao C, Bradley D, Zhou K, et al.Real-time high-fidelity facial performance capture[J]. ACM Trans. Graph.(S0730-0301), 2015, 34(4): 1-9. [76] Garrido P, Zollhofer M, Casas D, et al.Reconstruction of Personalized 3D Face Rigs from Monocular Video[J]. ACM Trans. Graph.(S0730-0301), 2016, 35(3): 1-15. [77] Wu C, Bradley D, Gross M, et al.An anatomically- constrained local deformation model for monocular face capture[J]. ACM Trans. Graph.(S0730-0301), 2016, 35(4): 1-12. [78] Ichim A E, Bouaziz S, Pauly M.Dynamic 3D avatar creation from hand-held video input[J]. ACM Trans. Graph.(S0730-0301), 2015, 34(4): 1-14. [79] Yu R, Russell C, Campbell N D F, et al. Direct, Dense, and Deformable: Template-Based Non-rigid 3D Reconstruction from RGB Video[C]//2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile:IEEE, 2015. [80] Thies J, Zollhofer M, Stamminger M, et al.Face2Face: Real-Time Face Capture and Reenactment of RGB Videos[C]/2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016. [81] Piotraschke M, Blanz V.Automated 3D Face Reconstruction from Multiple Images Using Quality Measures[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA: IEEE, 2016. |
[1] | 李智杰, 石昊琦, 李昌华, 张颉. 基于改进遗传算法的影像中心布局优化方法[J]. 系统仿真学报, 2022, 34(6): 1173-1184. |
[2] | 陈斌, 刘悦, 杨亚磊. 基于STN的机场航班过站保障时间协同规划建模[J]. 系统仿真学报, 2022, 34(6): 1196-1207. |
[3] | 杨凯, 陈纯毅, 胡小娟, 于海洋. 蒙卡渲染画面多特征非局部均值滤波降噪算法[J]. 系统仿真学报, 2022, 34(6): 1259-1266. |
[4] | 陈麒, 崔昊杨. 基于改进鸽群层级的无人机集群视觉巡检模型[J]. 系统仿真学报, 2022, 34(6): 1275-1285. |
[5] | 王沐晴, 张磊, 范秀敏, 骆晓萌, 朱文敏. VR外设驱动的虚拟人姿态优化仿真方法[J]. 系统仿真学报, 2022, 34(6): 1296-1303. |
[6] | 陆承, 靳学胜. 基于Steam VR的交互仿真水枪灭火训练系统设计[J]. 系统仿真学报, 2022, 34(6): 1312-1319. |
[7] | 高宏鼐, 付丽疆, 夏倩, 郭亚. 可观测度在光合作用模型性能评估中的应用[J]. 系统仿真学报, 2022, 34(6): 1330-1342. |
[8] | 倪凌佳, 黄晓霞, 李红旮, 张子博. 基于协作式深度强化学习的火灾应急疏散仿真研究[J]. 系统仿真学报, 2022, 34(6): 1353-1366. |
[9] | 蒙盾, 胡卓, 张华军. 基于改进A*算法的多层邮轮疏散系统仿真[J]. 系统仿真学报, 2022, 34(6): 1375-1382. |
[10] | 郭宇飞, 赵康, 海永清. 面向有限元分析的三角网格布尔运算方法[J]. 系统仿真学报, 2022, 34(5): 1003-1014. |
[11] | 吴桐, 王清辉, 徐志佳. 三周期极小曲面多孔材料渗透率尺度特性研究[J]. 系统仿真学报, 2022, 34(5): 1015-1024. |
[12] | 蒋阳升, 王思琛, 高宽, 刘梦, 姚志洪. 混入智能网联车队的混合交通流元胞自动机模型[J]. 系统仿真学报, 2022, 34(5): 1025-1032. |
[13] | 梁江涛, 王慧琴. 基于改进蚁群算法的建筑火灾疏散路径规划研究[J]. 系统仿真学报, 2022, 34(5): 1044-1053. |
[14] | 张其文, 张斌. 基于教学优化算法求解置换流水车间调度问题[J]. 系统仿真学报, 2022, 34(5): 1054-1063. |
[15] | 邢根上, 鲁芳, 李书山, 罗定提. 基于产品体验性的供应链交货模型与仿真研究[J]. 系统仿真学报, 2022, 34(5): 1064-1075. |
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||