Ultra-Real-Time Visual Simulation System for Multi-View Rendering Tasks

doi:10.16182/j.issn1004731x.joss.21-0248

Abstract

Abstract:

For multi-view rendering tasks, a theoretical analysis of ultra-real-time visual simulation is given in terms of implementation principle and feasibility. Based on the theoretical results, an ultra-real-time visual simulation architecture is designed, which decouples the simulation and rendering computation. A parallel-rendering-based ultra-real-time visual simulation method is proposed to solve the problems of rendering task assignment, simulation world synchronization, and rendering-execution time selection. An ultra-real-time visual simulation system is implemented based on Unreal Engine 4 (UE4), the performance of which is demonstrated on a designated application case per rendering efficiency and ultra-real-time simulation.

Key words: ultra-real-time simulation, visual simulation, multi-view rendering, UE4

CLC Number:

TP391.9

Xunyun Liu, Xinhai Xu, Chengzhang Zhu, Hao Li, Lei Zeng. Ultra-Real-Time Visual Simulation System for Multi-View Rendering Tasks[J]. Journal of System Simulation, 2022, 34(7): 1619-1628.

Figures/Tables 9

Fig.1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

References 14

1	Koenig, Nathan, Howard Andrew. Design and Use Paradigms for Gazebo, An Open-source Multi-Robot Simulator[C]// IEEE/RSJ International Conference on Intelligent Robots and Systems, Sendai, Japan: IEEE, 2004: 2149-2154.
2	Vaughan, Richard. Massively Multi-Robot Simulation in Stage[J]. Swarm Intelligence (S1935-3812), 2008(2): 189-208.
3	Todorov, Emanuel, Erez Tom, and Tassa Yuval. Mujoco: A Physics Engine for Model-based Control[C]// IEEE/RSJ International Conference on Intelligent Robots and Systems. Viamoura-Algarve, Portugal: IEEE, 2012: 5026-5033.
4	Qiu, Weichao, and Yuille Alan. Unrealcv: Connecting Computer Vision to Unreal Engine[C]// European Conference on Computer Vision. Amsterdam, The Netherlands Springer Cham, 2016, 9-16.
5	Martinez-Gonzalez, Pablo, Oprea Sergiu, Garcia-Garcia Alberto, et al. Unrealrox: An Extremely Photorealistic Virtual Reality Environment for Robotics Simulations and Synthetic Data Generation[J]. Virtual Reality (S1359-4338), 2020, 24: 271-288.
6	Shah, Shital, Dey Debadeepta, Lovett Chris, et al. Airsim: High-fidelity Visual and Physical Simulation for Autonomous Vehicles[J]. Field and Service Robotics (S2511-1256). 2018, 5: 621-635.
7	Dosovitskiy, Alexey, Ros German, Codevilla Felipe, et al. CARLA: An Open Urban Driving Simulator[C]//Conference on Robot Learning. Montain view,California, USA: CARLA, 2017: 1-16.
8	Yan, Claudia, Misra Dipendra, Bennnett Andrew, et al. Chalet: Cornell House Agent Learning Environment [M/OL]. arXiv. 2018: 1-5.[2018-01-23]..
9	Ros, German, Sellart Laura, Materzynska Joanna, et al. The Synthia Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes[C]// IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 3234-3243.
10	Savva, Manolis, X. Chang Angel, Dosovitskiy Alexey, et al. MINOS: Multimodal Indoor Simulator for Navigation in Complex Environments [M/OL]// arXiv, 2017: 1-14 [2017-12-11]. .
11	Sanders, Andrew. An Introduction to Unreal Engine 4 [M]. AK Peters/CRC Press, 2016.
12	谭亚新, 罗建华, 范锐, 等. 一种基于作战编成并行仿真方法及其实现[J]. 系统仿真学报, 2019, 31(11): 2350–2355.
	Tan Yaxin, Luo Jianhua, Fan Rui, et al. A Parallel Simulation Method for Combat Organization and Implementation[J]. Journal of System Simulation. 2019, 31(11): 2350–2355.
13	屈洪春, 姚献慧, 尹力, 等. 多 Agent 系统在空间直观仿真建模中的并行化[J]. 系统仿真学报, 2020, 32(3): 446-454.
	Qu Hongchun, Yao Xianhui, Yin Li, et al. Parallel Computing of Multi-Agent System in Spatially-Explicit Modeling and Simulation[J]. Journal of System Simulation. 2020, 32(3): 446-454.
14	曾蕾, 李豪, 林宇斐, 等. 基于异步机制的 Gazebo 仿真优化研究 [J]. 计算机科学, 2020, 47(11A): 593-598.
	Zeng Lei, Li Hao, Lin Yufei, et al. Study on Simulation on Optimization of Gazebo Based on Asynchronous Mechanism[J]. Computer Science. 2020, 47(11A): 593-598.

[1]	Zhang Yang, Gan Qingbo, Yuan Hong, Fu Jiangliang, Sun Chengming. Design of Space-Based Surveillance Distributed Simulation System for Space Targets [J]. Journal of System Simulation, 2020, 32(4): 620-626.
[2]	Xue Jiang, Li Junfeng, Wang Peng, Xiong Shan, Feng Weixia. Design and Realization of UAV Inspection and Evaluation Simulation System [J]. Journal of System Simulation, 2019, 31(9): 1852-1859.
[3]	Xue Wenxia, Lin Bin, Hu Xu, Zhang Jingbo. Dynamic Simulation of Marine Pasture Water Quality Monitoring Network Based on AUG [J]. Journal of System Simulation, 2019, 31(12): 2758-2767.
[4]	Li Xing, Fu Yanfang, Wang Liang, Lu Chengtao. Dynamic Collision Optimization Algorithm Based on Ray Detection [J]. Journal of System Simulation, 2019, 31(11): 2393-2401.
[5]	Guo Jinliang, Zhang Yang, Li Xiaoyan, Xue Yuan. Research on Parametric Modeling of Radar Electronic Warfare Visual Simulation [J]. Journal of System Simulation, 2018, 30(7): 2576-2582.
[6]	Yang Kun, Ge Xiangyu, Ju Pengfei, Bai Jie. Evaluation on Pilot Performance of Detecting Runway Incursion Based on Visual Simulation [J]. Journal of System Simulation, 2018, 30(5): 1967-1973.
[7]	Tong Yuxuan, Lin Bin, Ren Hongxiang, Tao Rui. Design and Implementation of Virtual Visual Simulation Platform for Maritime Wireless Transmission [J]. Journal of System Simulation, 2018, 30(11): 4227-4235.
[8]	Zhang Shaojie, Meng Qingkai. Design of Fault Tolerant Control Visual Simulation System For Flying Wing Aircrafts [J]. Journal of System Simulation, 2018, 30(10): 3732-3738.
[9]	Liu Ruihua, Cao Hui. Visual Simulation of Aircraft Approach and Landing System Based on Beidou [J]. Journal of System Simulation, 2018, 30(10): 3753-3760.
[10]	Han Liang, Gong Xuan, Wang Jiangyun, Du Yanghua. Study on visual Function of Human Computer Interaction in Parachuting Simulator [J]. Journal of System Simulation, 2017, 29(8): 1747-1752.
[11]	Long Mansheng. Visual Simulation of Eroded Rill Evolution Process on Graphics Processing Unit [J]. Journal of System Simulation, 2017, 29(3): 516-523.
[12]	Xie Cui, Xun Jinjin, Zhou Xiaowei, Dong Junyu. Visual Tsunami Simulation and Disaster Risk Assessment [J]. Journal of System Simulation, 2016, 28(10): 2567-2573.
[13]	Qi Juntong, Liu Jinda, Shang Hong, Yang Liying, Mei Sen, Han Jianda. Autonomous Flight Control and Simulation Study of Independent Developed Soft-wing UAV [J]. Journal of System Simulation, 2015, 27(12): 2988-2997.
[14]	Li Zengyan, Li Xiaomin. Modeling Analysis and Visualization Simulation for Leveled Stage of Loitering Munition [J]. Journal of System Simulation, 2015, 27(10): 2258-2263.
[15]	Yang Cheng, Zhou Jianliang, Tang Geshi, Li Xie, Chen Guangming, Liu Shushi. Soft Shadow Map Method for Exploration Scene Simulation of Lunar Rover [J]. Journal of System Simulation, 2015, 27(10): 2600-2606.