Research on VR Experience Comfort Based on Motion Perception

doi:10.16182/j.issn1004731x.joss.21-0966

Abstract

Abstract:

A VR video comfort evaluation model based on motion perception is proposed for viewers who will feel discomfort such as vertigo and nausea after a virtual reality (VR) experience. By performing dense optical flow estimation on stereoscopic VR video and calculating the video frame velocity matrix by analyzing the horizontal and vertical motions in the scene, the frame acceleration feature extraction methods based on frame difference method and based on time domain are proposed. Taking the extracted velocity, acceleration and other motions features as input, a model is established using the support vector regression algorithm, and VR video experience comfort could be evaluated. The experiment shows that our model performs better than other models for introducing time-domain frame acceleration, and the results are closer to subjective evaluation values. The root mean square error (RMSE) is reduced to 7.0, and the Pearson linear correlation coefficient (PLCC) is increased to 0.955 3.

Key words: virtual reality(VR), stereoscopic VR video, optical flow estimate, motion perception, support vector regression, VR experience comfort

CLC Number:

TP391.9

Wei Quan, Chao Wang, Xuena Geng, Cheng Han. Research on VR Experience Comfort Based on Motion Perception[J]. Journal of System Simulation, 2023, 35(1): 169-177.

Figures/Tables 8

Fig. 1

Fig. 2

Fig. 3

Table 1

Fig. 4

Table 2

Table 3

Table 4

References 22

1	Jang K M, Shin Woo Y, Kyoon Lim H. Electrophysiological Changes in the Virtual Reality Sickness: EEG in the VR Sickness[C]// The 25th International Conference on 3D Web Technology. New York: Association for Computing Machinery, 2020: 1-4.
2	Arafat I M, Ferdous S M S, Quarles J. The Effects of Cybersickness on Persons with Multiple Sclerosis[C]//The 22nd ACM Conference on Virtual Reality Software and Technology. New York: Association for Computing Machinery, 2016: 51-59.
3	Von Mammen S, Knote A, Edenhofer S. Cyber Sick but Still Having Fun[C]// The 22nd ACM Conference on Virtual Reality Software and Technology. New York: Association for Computing Machinery, 2016: 325-326.
4	Lee S, Kim S, Kim H G, et al. Assessing Individual VR Sickness Through Deep Feature Fusion of VR Video and Physiological Response[J]. IEEE Transactions on Circuits and Systems for Video Technology (S1051-8215), 2021.
5	Dennison M S, Wisti A Z, D'Zmura M. Use of Physiological Signals to Predict Cybersickness[J]. Displays(S0141-9382), 2016, 44: 42-52.
6	Kim H G, Baddar W J, Lim H, et al. Measurement of Exceptional Motion in VR Video Contents for VR Sickness Assessment Using Deep Convolutional Autoencoder[C]// The 23rd ACM Symposium on Virtual Reality Software and Technology. New York: Association for Computing Machinery, 2017: 1-7.
7	Clifton J, Palmisano S. Effects of Steering Locomotion and Teleporting on Cybersickness and Presence in HMD-based Virtual Reality[J]. Virtual Reality (S1359-4338), 2020, 24(3): 453-468.
8	蔡力, 翁冬冬, 张振亮, 等. 虚实运动一致性对虚拟现实晕动症的影响[J]. 系统仿真学报, 2016, 28(9): 1950-1956.
	Cai Li, Weng Dongdong, Zhang Zhenliang, et al. Impact of Consistency between Visually Perceived Movement and Real Movement on Cybersickness[J]. Journal of System Simulation, 2016, 28(9): 1950-1956.
9	Trutoiu L C, Mohler B J, Schulte-Pelkum J, et al. Circular, Linear, and Curvilinear Vection in a Large-Screen Virtual Environment with Floor Projection[J]. Computers & Graphics (S0097-8493), 2009, 33(1): 47-58.
10	Chang E, Kim H T, Yoo B. Virtual Reality Sickness: a Review of Causes and Measurements[J]. International Journal of Human-Computer Interaction (S1044-7318), 2020, 36(17): 1658-1682.
11	Kim J, Kim W, Ahn S, et al. Virtual Reality Sickness Predictor: Analysis of Visual-Vestibular Conflict and VR Contents[C]// 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). New York: IEEE, 2018: 1-6.
12	Hell S, Argyriou V. Machine Learning Architectures to Predict Motion Sickness Using a Virtual Reality Rollercoaster Simulation Tool[C]// 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). New York: IEEE, 2018: 153-156.
13	Keshavarz B, Riecke B E, Hettinger L J, et al. Vection and Visually Induced Motion Sickness: How Are They Related?[J]. Frontiers in Psychology (S1664-1078), 2015, 6: 472.
14	Padmanaban N, Ruban T, Sitzmann V, et al. Towards a Machine-Learning Approach for Sickness Prediction in 360 Stereoscopic Videos[J]. IEEE Transactions on Visualization and Computer Graphics (S1077-2626), 2018, 24(4): 1594-1603.
15	林丽媛, 侯春萍, 王凯, 等. 基于动态随机点立体图的立体视觉深度运动特性分析[J]. 红外与激光工程, 2015, 44(4): 1365-1369.
	Lin Liyuan, Hou Chunping, Wang Kai, et al. Depth Motion Characteristics Analysis of Stereo Vision Based on Dynamic Random Dot Stereograms[J]. Infrared and Laser Engineering, 2015, 44(4): 1365-1369.
16	Ilg E, Mayer N, Saikia T, et al. Flownet 2.0: Evolution of Optical Flow Estimation with Deep Networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 2462-2470.
17	Kim H G, Lim H T, Lee S, et al. Vrsa net: VR Sickness Assessment Considering Exceptional Motion for 360 VR Video[J]. IEEE Transactions on Image Processing (S1057-7149), 2018, 28(4): 1646-1660.
18	Sharif H, Djeraba C. Exceptional Motion Frames Detection by Means of Spatiotemporal Region of Interest Features[C]// 2009 16th IEEE International Conference on Image Processing (ICIP). New York: IEEE, 2009: 981-984.
19	Lee T M, Yoon J C, Lee I K. Motion Sickness Prediction in Stereoscopic Videos using 3d Convolutional Neural Networks[J]. IEEE Transactions on Visualization and Computer Graphics (S1077-2626), 2019, 25(5): 1919-1927.
20	Kennedy R S, Lane N E, Berbaum K S, et al. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness[J]. The International Journal of Aviation Psychology (S1050-8414), 1993, 3(3): 203-220.
21	Golding J F. Predicting Individual Differences in Motion Sickness Susceptibility by Questionnaire[J]. Personality and Individual Differences (S0191-8869), 2006, 41(2): 237-248.
22	Chang C C, Lin C J. LIBSVM: A Library for Support Vector Machines[J]. ACM Transactions on Intelligent Systems and Technology (TIST)(S2157-6904), 2011, 2(3): 1-27.

视频数据	主观评分	预测评分	误差
Sharks	4.6	10.1	5.5
Glowingdance	7.8	11.8	4.0
Dotsinner	12.8	16.3	3.5
Flyingcar	13.6	17.6	4.0
Oceancoaster	13.9	17.9	4.0
Woodencoaster	15.5	19.5	4.0
Spacevisit	21.1	22.6	1.5
Roadcar	21.6	22.3	0.7
Gardens	22.9	11.7	11.2
Minecraft	23.7	19.7	4.0
Sculptures	24.2	20.2	4.0
Snowplanet	27.2	28	0.8
Ship	31.9	31	0.9
Dotsall	33.9	26.3	7.6
Skyhouse	41.1	38.5	2.6

视频数据	主观评分	评估模型1	评估模型2
Helicoptercrash	4.3	17.0	11.6
Dunerovers	20.8	22.7	20.3
Cartooncoaster	25.4	23.4	24.7
Dotsouter	37.8	19.3	25.9

视频	主观评分	Padmanaban^[14]	3DCNN O+D^[19]	3DCNN O+D+S^[19]	本文模型2
Helicoptercrash	4.3	17.0	14.4	12.8	11.6
Dunerovers	20.8	19.9	24.9	21.0	20.3
Cartooncoaster	25.4	22.1	21.8	25.5	24.7
Dotsouter	37.8	16.1	21.8	23.1	25.9

模型	PLCC	SROCC	RMSE	MAE	P	模型	PLCC	SROCC	RMSE	MAE	P
Padmanaban^[14]	-0.007 8	-0.200 0	12.69	9.65	0.992 2	3DCNN O+S^[19]	0.571 2		10.09
2D CNN^[19]	0.1894		13.00			3DCNN O+D^[19]	0.687 8	0.316 2	9.85	8.45	0.312 2
本文模型1	0.403 1	0.400 0	11.42	6.45	0.596 8	3DCNN O+D+S^[19]	0.845 1	0.800 0	8.49	5.87	0.154 9
3DCNN D+S^[19]	0.407 3		11.00			本文模型 2	0.955 3	1.000 0	7.00	5.10	0.044 7

[1]	Fan Haixia, Chen Xiaohui. A Continuous Non-invasive Blood Pressure Prediction Method Based on Improved SVR Learning [J]. Journal of System Simulation, 2020, 32(9): 1686-1692.
[2]	Xing Zhiwei, Jiang Songyue, Luo Qian, Luo Xiao. Prediction of Flight Taxi-out Time in A Busy Airport Based on LWSVR [J]. Journal of System Simulation, 2020, 32(5): 927-935.
[3]	Wang Dingcheng, Cao Zhili, Chen Beijing, Ni Yujia. Multivariate Time Series Local Support Vector Regression Forecast Methods for Daily Temperature [J]. Journal of System Simulation, 2016, 28(3): 654-660.