基于网格映射的虚拟视点合成

系统仿真学报 ›› 2016, Vol. 28 ›› Issue (10): 2321-2328.

基于网格映射的虚拟视点合成

姚莉^1,2,3, 吴琼颖³, 许辉³

1.计算机网络和信息集成教育部重点实验室(东南大学),南京 211189;
2.计算机软件新技术国家重点实验室(南京大学),南京 210023;
3.东南大学计算机科学与工程学院,南京 211189

收稿日期:2016-05-27 修回日期:2016-07-14 出版日期:2016-10-08 发布日期:2020-08-13
作者简介:姚莉(1977-),女,湖北,博士,副教授,研究方向为计算机图形学与视觉。
基金资助:
国家863计划(2015AA015904),CCF-腾讯犀牛鸟基金(RAGR20150120),NSFC-广东联合基金（第二期）超级计算科学应用研究专项资助和国家超级计算广州中心支持NSFC-广东联合基金

Virtual Viewpoint Synthesis Based on Grid Mapping

Yao Li^1,2,3, Wu Qiongying³, Xu Hui³

1. Key Laboratory of Computer Network and Information Integration (Southeast University) Ministry of Education, Nanjing 211189, China;
2. State Key Laboratory for Novel Software Technology (Nanjing University), nanjing 210032, China;
3. School of Computer Science and Engineering, Southeast University, Nanjing 211189, China

Received:2016-05-27 Revised:2016-07-14 Online:2016-10-08 Published:2020-08-13

摘要/Abstract

摘要： 由于深度不准确、遮挡以及相邻参考视点曝光不同等原因,基于深度图的绘制技术(DIBR)会产生空洞、重影等一系列问题,同时修复空洞需要消耗大量时间,提出基于网格映射的虚拟视点合成方法,以轮廓为基础在图像中构建网格,以网格三角形为映射单元,并通过能量约束进行映射生成虚拟视点图像。利用融合算法将不同视点的图像进行融合,得到最终的虚拟视点图像。算法相比于传统的DIBR中像素为单元的处理方式在合成图像质量相当的前提下具有更快的处理速度。

关键词: 边缘检测, 网格化, 纹理映射, 网格映射

Abstract: Due to imperfect depth maps, occlusion and the illumination difference between reference images, annoying artifacts, such as holes and ghost edges, appearing in the rendering image by DIBR, holes filling being time-consuming, triangle mesh based on the depth contour and maps mesh triangles with energy constrains were created. Image fusion algorithm merged different viewpoint images and got final virtual viewpoint image. While DIBR warped the image with pixels, triangles were mapped with. Compared with the traditional DIBR, the proposed algorithm has a faster processing speed than the traditional pixel based processing method in the same image quality.

Key words: edge detection, grid, texture mapping, grid mapping

中图分类号:

TP391.9

姚莉, 吴琼颖, 许辉. 基于网格映射的虚拟视点合成[J]. 系统仿真学报, 2016, 28(10): 2321-2328.

Yao Li, Wu Qiongying, Xu Hui. Virtual Viewpoint Synthesis Based on Grid Mapping[J]. Journal of System Simulation, 2016, 28(10): 2321-2328.

参考文献 20

[1]	郁理. 基于深度图像绘制的自由视点视频关键技术研究 [D]. 合肥: 中国科学技术大学, 2010.
[2]	Mori Y, Fukushima N, Yendo T, et al.View generation with 3D warping using depth information for FTV[J]. Signal Processing: Image Communication, ISSN:(0923-5965), 2009, 24(1): 65-72.
[3]	Zinger S, Do L, de With P H N. Free-viewpoint depth image based rendering[J]. Journal of Visual Communication and Image Representation, ISSN:(1047-3203), 2010, 21(5): 533-541.
[4]	Oh K J, Yea S, Ho Y S.Hole filling method using depth based in-painting for view synthesis in free viewpoint television and 3-d video[C]// Picture Coding Symposium, 2009, PCS 2009. USA: IEEE, 2009: 1-4.
[5]	Merkle P, Morvan Y, Smolic A, et al.The effects of multiview depth video compression on Multiview rendering[J]. Signal Processing: Image Communication, ISSN:(0923-5965), 2009, 24(1): 73-88.
[6]	Yoon S S, Sohn H, Jung Y J, et al.Inter-view consistent hole filling in view extrapolation for multi-view image generation[C]// Image Processing (ICIP), 2014 IEEE International Conference on. USA: IEEE, 2014: 2883-2887.
[7]	Feng Y M, Li D X, Luo K, et al.Asymmetric bidirectional view synthesis for free viewpoint and three dimensional video[J]. IEEE Transactions on Consumer Electronic, ISSN:(0098-3063), 2009, 55(4): 2349-2355.
[8]	Yao C, Tillo T, Zhao Y, et al.Depth map driven hole filling algorithm exploiting temporal correlation information[J]. IEEE Transactions on Broadcasting, ISSN:(0018-9316), 2014, 60(2): 394-404.
[9]	Müller K, Smolic A, Dix K, et al.Reliability-based generation and view synthesis in layered depth video[C]// Multimedia Signal Processing, 2008 IEEE 10th Workshop on. USA: IEEE, 2008: 34-39.
[10]	Sharma M, Chaudhury S, Lall B, et al.A flexible architecture for multi-view 3DTV based on uncalibrated cameras[J]. Journal of Visual Communication and Image Representation. ISSN:(1047-3203), 2014, 25(4): 599-621.
[11]	Ho T Y, Yang D N, Liao W.Efficient Resource Allocation of Mobile Multi-View 3D Videos with Depth-Image-Based rendering[J]. IEEE Transactions on Mobile Computing. ISSN:(1536-1233), 2015, 14(2): 344-357.
[12]	Xiao J, Hannuksela M M, Tillo T, et al.Scalable bit allocation between texture and depth views for 3-d video streaming over heterogeneous networks[J]. IEEE Transactions on Circuits and Systems for Video Technology. ISSN:(1051-8215), 2015, 25(1): 139-152.
[13]	Chaurasia G, Sorkine O, Drettakis G.Silhouette Aware Warping for Image Based Rendering [C]// Computer Graphics Forum. USA: Blackwell Publishing Ltd, 2011, 30(4): 1223-1232.
[14]	Canny J.A computational approach to edge detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. ISSN:(0162-8828), 1986, 8(6): 679-698.
[15]	Liu F, Gleicher M, Jin H, et al.Content-preserving warps for 3D video stabilization[C]// ACM Transactions on Graphics (TOG). USA: ACM, 2009, 28(3): 44.
[16]	Zhang G X, Cheng M M, Hu S M, et al.A Shape Preserving Approach to Image Resizing [C]// Computer Graphics Forum. USA: Blackwell Publishing Ltd, 2009, 28(7): 1897-1906.
[17]	Wang Y S, Lin H C, Sorkine O, et al.Motion-based video retargeting with optimized crop-andwarp[C]// ACM Transactions on Graphics (TOG). USA: ACM, 2010, 29(4): 90.
[18]	王震. 自由视点立体电视系统的虚拟视点合成技术研究 [D]. 上海: 上海交通大学, 2012.
[19]	Zitnick C L, Kang S B, Uyttendaele M, et al.High-quality video view interpolation using a layered representation[C]// ACM Transactions on Graphics (TOG). USA: ACM, 2004, 23(3): 600-608.
[20]	Do L, Zinger S.Quality improving techniques for free-viewpoint DIBR[C]// IS&T/SPIE Electronic Imaging. USA: International Society for Optics and Photonics, 2010, 7524: 1-4.