Mobile-phone-oriented Stereoscopic Display and Interaction Framework for Cloud-based Virtual Reality 3D Scenes

doi:10.16182/j.issn1004731x.joss.19-VR0477

Abstract

Abstract: A VR application framework based on cloud computing is proposed to overcome the difficulty of interactively generating high quality 3D scene images on mobile phones. We combine the cloud-based rendering and the VR technology. Our framework uses a mobile browser as a client, transmits the image data by the video block stream, processes the interactive data by the anti-jitter algorithm, and has accomplished the web-based lightweight rendering. The framework reduces the cost of data visualization on mobile phone and avoids the need of installing a plug-in package. The experimental results show that our display rate could be improved to 30 frames per second(fps), and the bandwidth could be limited under 80 kb/s. With the proposed framework, we can obtain much better VR Experience than the general mobile applications.

Key words: cloud-based rendering, VR technology, web-based lightweight rendering, video block stream, anti-jitter algorithm, mobile plug-in

CLC Number:

TP391.41

Liu Jiazhe, Chen Chunyi, Hu Xiaojuan, Liang Weidong, Xing Qiwei, Yang Huamin. Mobile-phone-oriented Stereoscopic Display and Interaction Framework for Cloud-based Virtual Reality 3D Scenes[J]. Journal of System Simulation, 2020, 32(7): 1360-1374.

References 25

[1]	Suselbeck R, Schiele G, Becker C, et al.Peer-to-peer support for low-latency Massively Multiplayer Online Games in the cloud[C]. Paris: 2009 8th Annual Workshop on Network and Systems Support for Games (NetGames), 2009: 1-2.
[2]	Ross P E.Cloud Computing's Killer App: Gaming[J]. IEEE Spectrum (S0018-9235), 2009, 46(3): 14.
[3]	Ha J, Jung J, Han B, et al.Mobile Augmented Reality using scalable recognition and tracking[C]. Singapore: IEEE Virtual Reality Conference, 2011: 211-212.
[4]	Huang B, Lin C H, Lee C, et al.Mobile augmented reality based on cloud computing[C]. Chinese Taipei: International Conference on Anti-Counterfeiting, Security and Identification (ASID), 2012: 1-5.
[5]	Cao Y Z.Application Development of Virtual Reality Based on Smart Phone and VR Glasses[C]. Institute of Management Science and Industrial Engineering. Proceedings of 2018 5th International Conference on Electrical & Electronics Engineering and Computer Science (ICEEECS 2018), 2018: 497-502.
[6]	刘小军, 贾金原. 面向手机网页的大规模 WebBIM 场景轻量级实时漫游算法[J]. 中国科学: 信息科学, 2018, 48(3): 274-292.Liu Xiaojun, Jia Jinyuan.Mobile web-based lightweight and real-time roaming algorithm for large-scale WebBIM scenes[J]. Scientia Sinica (Informationis), 2018, 48(3): 274-292.
[7]	刘镇, 刘晓, 梅向东. 面向移动终端的分布并行化渲染[J]. 中国图象图形学报, 2015, 20(9): 1247-1252.Liu Zhen, Liu Xiao, Mei Xiangdong.Distributed parallel rendering method for mobile terminals[J]. Journal of Image and Graphics, 2015, 20(9): 1247-1252.
[8]	Wang S, Dey S.Rendering Adaptation to Address Communication and Computation Constraints in Cloud Mobile Gaming[C]. Miami, Florida, USA: Global Communications Conference, 2010: 1-6.
[9]	Cai W, Shea R, Huang C, et al.A Survey on Cloud Gaming: Future of Computer Games[J]. IEEE Access (S2169-3536), 2016, 4(99): 7605-7620.
[10]	Hong H, Chen D, Huang C, et al.Placing Virtual Machines to Optimize Cloud Gaming Experience[J]. IEEE Transactions on Cloud Computing (S2168-7161), 2015, 3(1): 42-53.
[11]	Schmalstieg D.The remote rendering pipeline-managing geometry and bandwidth in distributed virtual environments[D]. Vienna: Vienna University of Technology, 1997.
[12]	Dey S.Cloud Mobile Media: Opportunities, challenges, and directions[C]. Maui, HI: 2012 International Conference on Computing, Networking and Communications (ICNC), 2012: 929-933.
[13]	Wang S, Dey S.Adaptive Mobile Cloud Computing to Enable Rich Mobile Multimedia Applications[J]. IEEE Transactions on Multimedia (S1520-9210), 2013, 15(4): 870-883.
[14]	Ojala A, Tyrvainen P.Developing Cloud Business Models: A Case Study on Cloud Gaming[J]. IEEE Software (S0740-7459), 2011, 28(4): 42-47.
[15]	Lampe U, Hans R, Steinmetz R.Will mobile cloud gaming work? ﬁndings on latency, energy, and cost[C]. Santa Clara, California, USA: Proceedings of the 2013 IEEE Second International Conference on Mobile Services, 2013: 960-961.
[16]	Xun L, Woo W.The emerging Cloud-Mobile convergence paradigm for next-generation VR systems: Message from the CMCVR 2010 organizers[C]. Boston: In 2010 Cloud-Mobile Convergence for Virtual Reality Workshop (CMCVR 2010), 2010: 1-4.
[17]	Satyanarayanan M.Pervasive computing: vision and challenges[J]. IEEE Personal Communications (S1070-9916), 2001, 8(4): 10-17
[18]	Li Y, Su G, Hui P, et al.Multiple mobile data offloading through delay tolerant networks[C]. New York, NY, USA: Proceedings of the 6th ACM Workshop on Challenged Networks, 2011: 43-48.
[19]	Flores H, Su X, Kostakos V, et al.Large-scale offloading in the Internet of Things[C]. Seattle, Washington, USA: 2011 IEEE International Conference on Pervasive Computing and Communications Workshops, 2017: 479-484.
[20]	Sermpezis P, Spyropoulos T.Offloading on the edge: Performance and cost analysis of local data storage and offloading in HetNets[C]. Jackson Hole, Wyoming, USA: IEEE/IFIP Wireless On-demand Network systems and Services Conference, 2017: 49-56.
[21]	Shi C, Habak K, Pandurangan P, et al.COSMOS: computation offloading as a service for mobile devices[C]. Philadelphia, Pennsylvania, USA: Proceedings of the 15th ACM international symposium on Mobile ad hoc networking and computing, 2014: 287-296.
[22]	Kang Y, Kim H, Kang J.Docker based Computation Off-loading for Video Game based Mobile VR Framework[C]. The Institute of Electrical and Electronics Engineers、IEEE Beijing Section. Proceedings of 2017 IEEE 8th International Conference on Software Engineering and Service Science. The Institute of Electrical and Electronics Engineers, 2017: 143-145.
[23]	Nguyen D, Le T, Lee S, et al.SHVC Tile-Based 360-Degree Video Streaming for Mobile VR: PC Offloading Over mmWave[J]. Sensors (S1424-8220), 2018, 18(11): 3728.
[24]	林定, 黄国新, 徐颖. 一种基于WebVR的网络数据三维树形可视化[J]. 系统仿真学报, 2018, 30(7): 2736-2743, 2752.Lin Ding, Huang Guoxin, Xu Ying.A 3D Tree Visualization of Network Data Based on WebVR[J]. Journal of System Simulation, 2018, 30(7): 2736-2743, 2752.
[25]	Chen C, Liang W, Yang C, et al.Generating stereoscopic videos of realistic 3D scenes with ray tracing[C]. Hefei, China: 5th Conference on Frontiers in Optical Imaging Technology and Applications, 2018(10832): 108320E.