基于运动感知的VR体验舒适度研究

doi:10.16182/j.issn1004731x.joss.21-0966

摘要/Abstract

摘要：

针对观众在进行虚拟现实(virtual reality，VR)体验后出现眩晕、恶心等不适的问题，建立一种基于运动感知的VR体验舒适度评估方法。通过对立体VR视频进行稠密光流估计，分析场景中的水平运动和垂直运动计算视频帧速度矩阵，提出了基于帧差法和基于时域的帧加速度特征提取方法，将提取的速度、加速度等运动特征结合支持向量回归算法建立VR体验舒适度评估模型。实验结果表明，基于时域帧加速度特征的评估模型优于基于帧差法的模型，且性能较现有其它模型更接近主观评价值，均方根误差减小到7.0，Pearson相关系数提高到0.955 3。

关键词: 虚拟现实, 立体VR视频, 光流估计, 运动感知, 支持向量回归, VR体验舒适度

Abstract:

A VR video comfort evaluation model based on motion perception is proposed for viewers who will feel discomfort such as vertigo and nausea after a virtual reality (VR) experience. By performing dense optical flow estimation on stereoscopic VR video and calculating the video frame velocity matrix by analyzing the horizontal and vertical motions in the scene, the frame acceleration feature extraction methods based on frame difference method and based on time domain are proposed. Taking the extracted velocity, acceleration and other motions features as input, a model is established using the support vector regression algorithm, and VR video experience comfort could be evaluated. The experiment shows that our model performs better than other models for introducing time-domain frame acceleration, and the results are closer to subjective evaluation values. The root mean square error (RMSE) is reduced to 7.0, and the Pearson linear correlation coefficient (PLCC) is increased to 0.955 3.

Key words: virtual reality(VR), stereoscopic VR video, optical flow estimate, motion perception, support vector regression, VR experience comfort

中图分类号:

TP391.9

权巍, 王超, 耿雪娜, 韩成. 基于运动感知的VR体验舒适度研究[J]. 系统仿真学报, 2023, 35(1): 169-177.

Wei Quan, Chao Wang, Xuena Geng, Cheng Han. Research on VR Experience Comfort Based on Motion Perception[J]. Journal of System Simulation, 2023, 35(1): 169-177.

图/表 8

图1

图2

图 3

表1

图4

表2

表3

表4

参考文献 22

1	Jang K M, Shin Woo Y, Kyoon Lim H. Electrophysiological Changes in the Virtual Reality Sickness: EEG in the VR Sickness[C]// The 25th International Conference on 3D Web Technology. New York: Association for Computing Machinery, 2020: 1-4.
2	Arafat I M, Ferdous S M S, Quarles J. The Effects of Cybersickness on Persons with Multiple Sclerosis[C]//The 22nd ACM Conference on Virtual Reality Software and Technology. New York: Association for Computing Machinery, 2016: 51-59.
3	Von Mammen S, Knote A, Edenhofer S. Cyber Sick but Still Having Fun[C]// The 22nd ACM Conference on Virtual Reality Software and Technology. New York: Association for Computing Machinery, 2016: 325-326.
4	Lee S, Kim S, Kim H G, et al. Assessing Individual VR Sickness Through Deep Feature Fusion of VR Video and Physiological Response[J]. IEEE Transactions on Circuits and Systems for Video Technology (S1051-8215), 2021.
5	Dennison M S, Wisti A Z, D'Zmura M. Use of Physiological Signals to Predict Cybersickness[J]. Displays(S0141-9382), 2016, 44: 42-52.
6	Kim H G, Baddar W J, Lim H, et al. Measurement of Exceptional Motion in VR Video Contents for VR Sickness Assessment Using Deep Convolutional Autoencoder[C]// The 23rd ACM Symposium on Virtual Reality Software and Technology. New York: Association for Computing Machinery, 2017: 1-7.
7	Clifton J, Palmisano S. Effects of Steering Locomotion and Teleporting on Cybersickness and Presence in HMD-based Virtual Reality[J]. Virtual Reality (S1359-4338), 2020, 24(3): 453-468.
8	蔡力, 翁冬冬, 张振亮, 等. 虚实运动一致性对虚拟现实晕动症的影响[J]. 系统仿真学报, 2016, 28(9): 1950-1956.
	Cai Li, Weng Dongdong, Zhang Zhenliang, et al. Impact of Consistency between Visually Perceived Movement and Real Movement on Cybersickness[J]. Journal of System Simulation, 2016, 28(9): 1950-1956.
9	Trutoiu L C, Mohler B J, Schulte-Pelkum J, et al. Circular, Linear, and Curvilinear Vection in a Large-Screen Virtual Environment with Floor Projection[J]. Computers & Graphics (S0097-8493), 2009, 33(1): 47-58.
10	Chang E, Kim H T, Yoo B. Virtual Reality Sickness: a Review of Causes and Measurements[J]. International Journal of Human-Computer Interaction (S1044-7318), 2020, 36(17): 1658-1682.
11	Kim J, Kim W, Ahn S, et al. Virtual Reality Sickness Predictor: Analysis of Visual-Vestibular Conflict and VR Contents[C]// 2018 Tenth International Conference on Quality of Multimedia Experience (QoMEX). New York: IEEE, 2018: 1-6.
12	Hell S, Argyriou V. Machine Learning Architectures to Predict Motion Sickness Using a Virtual Reality Rollercoaster Simulation Tool[C]// 2018 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR). New York: IEEE, 2018: 153-156.
13	Keshavarz B, Riecke B E, Hettinger L J, et al. Vection and Visually Induced Motion Sickness: How Are They Related?[J]. Frontiers in Psychology (S1664-1078), 2015, 6: 472.
14	Padmanaban N, Ruban T, Sitzmann V, et al. Towards a Machine-Learning Approach for Sickness Prediction in 360 Stereoscopic Videos[J]. IEEE Transactions on Visualization and Computer Graphics (S1077-2626), 2018, 24(4): 1594-1603.
15	林丽媛, 侯春萍, 王凯, 等. 基于动态随机点立体图的立体视觉深度运动特性分析[J]. 红外与激光工程, 2015, 44(4): 1365-1369.
	Lin Liyuan, Hou Chunping, Wang Kai, et al. Depth Motion Characteristics Analysis of Stereo Vision Based on Dynamic Random Dot Stereograms[J]. Infrared and Laser Engineering, 2015, 44(4): 1365-1369.
16	Ilg E, Mayer N, Saikia T, et al. Flownet 2.0: Evolution of Optical Flow Estimation with Deep Networks[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2017: 2462-2470.
17	Kim H G, Lim H T, Lee S, et al. Vrsa net: VR Sickness Assessment Considering Exceptional Motion for 360 VR Video[J]. IEEE Transactions on Image Processing (S1057-7149), 2018, 28(4): 1646-1660.
18	Sharif H, Djeraba C. Exceptional Motion Frames Detection by Means of Spatiotemporal Region of Interest Features[C]// 2009 16th IEEE International Conference on Image Processing (ICIP). New York: IEEE, 2009: 981-984.
19	Lee T M, Yoon J C, Lee I K. Motion Sickness Prediction in Stereoscopic Videos using 3d Convolutional Neural Networks[J]. IEEE Transactions on Visualization and Computer Graphics (S1077-2626), 2019, 25(5): 1919-1927.
20	Kennedy R S, Lane N E, Berbaum K S, et al. Simulator Sickness Questionnaire: An Enhanced Method for Quantifying Simulator Sickness[J]. The International Journal of Aviation Psychology (S1050-8414), 1993, 3(3): 203-220.
21	Golding J F. Predicting Individual Differences in Motion Sickness Susceptibility by Questionnaire[J]. Personality and Individual Differences (S0191-8869), 2006, 41(2): 237-248.
22	Chang C C, Lin C J. LIBSVM: A Library for Support Vector Machines[J]. ACM Transactions on Intelligent Systems and Technology (TIST)(S2157-6904), 2011, 2(3): 1-27.

视频数据	主观评分	预测评分	误差
Sharks	4.6	10.1	5.5
Glowingdance	7.8	11.8	4.0
Dotsinner	12.8	16.3	3.5
Flyingcar	13.6	17.6	4.0
Oceancoaster	13.9	17.9	4.0
Woodencoaster	15.5	19.5	4.0
Spacevisit	21.1	22.6	1.5
Roadcar	21.6	22.3	0.7
Gardens	22.9	11.7	11.2
Minecraft	23.7	19.7	4.0
Sculptures	24.2	20.2	4.0
Snowplanet	27.2	28	0.8
Ship	31.9	31	0.9
Dotsall	33.9	26.3	7.6
Skyhouse	41.1	38.5	2.6

视频数据	主观评分	评估模型1	评估模型2
Helicoptercrash	4.3	17.0	11.6
Dunerovers	20.8	22.7	20.3
Cartooncoaster	25.4	23.4	24.7
Dotsouter	37.8	19.3	25.9

视频	主观评分	Padmanaban^[14]	3DCNN O+D^[19]	3DCNN O+D+S^[19]	本文模型2
Helicoptercrash	4.3	17.0	14.4	12.8	11.6
Dunerovers	20.8	19.9	24.9	21.0	20.3
Cartooncoaster	25.4	22.1	21.8	25.5	24.7
Dotsouter	37.8	16.1	21.8	23.1	25.9

模型	PLCC	SROCC	RMSE	MAE	P	模型	PLCC	SROCC	RMSE	MAE	P
Padmanaban^[14]	-0.007 8	-0.200 0	12.69	9.65	0.992 2	3DCNN O+S^[19]	0.571 2		10.09
2D CNN^[19]	0.1894		13.00			3DCNN O+D^[19]	0.687 8	0.316 2	9.85	8.45	0.312 2
本文模型1	0.403 1	0.400 0	11.42	6.45	0.596 8	3DCNN O+D+S^[19]	0.845 1	0.800 0	8.49	5.87	0.154 9
3DCNN D+S^[19]	0.407 3		11.00			本文模型 2	0.955 3	1.000 0	7.00	5.10	0.044 7

[1]	周晓光, 朱鹏, 张源原, 路欢, 周媛. 六自由度跳伞模拟训练系统设计与实现[J]. 系统仿真学报, 2022, 34(6): 1320-1329.
[2]	许世健, 赵丹, 苏铖宇, 王赋攀, 张晓蓉, 王昉, 吴亚东. 面向流场可视化的沉浸式虚拟现实交互系统研究[J]. 系统仿真学报, 2022, 34(5): 1160-1172.
[3]	彭博, 汪强, 青芮冰, 尹立雪, 姜劲枫. 基于光线追踪的实时超声模拟与虚拟现实的集成[J]. 系统仿真学报, 2022, 34(11): 2425-2436.
[4]	邵绪强, 张浩伟, 冯小华. 面向电力变压器虚拟装配的多感官融合交互方法[J]. 系统仿真学报, 2022, 34(10): 2244-2254.
[5]	孙佳文, 任鸿翔, 肖方兵, 蒋效彬. 基于虚拟现实的整船仿真训练平台[J]. 系统仿真学报, 2021, 33(9): 2243-2251.
[6]	魏士松, 周正东, 章栩苓, 毛玲, 贾峻山, 刘传乐. 基于桌面虚拟现实技术的航天器虚拟维修训练系统[J]. 系统仿真学报, 2021, 33(6): 1358-1363.
[7]	樊海霞, 陈小惠. 一种连续无创血压预测的改进向量机学习方法[J]. 系统仿真学报, 2020, 32(9): 1686-1692.
[8]	贾春洋, 邹湘军, 李锦慧, 曾泽钦, 王杰. 虚拟环境下的复杂变速箱交互式装配仿真平台[J]. 系统仿真学报, 2020, 32(9): 1736-1743.
[9]	郭园, 童倩倩, 郑宇铠, 王子琦, 张玉茹, 王党校. 融合多元触觉和沉浸式视觉的可移植VR软件框架[J]. 系统仿真学报, 2020, 32(7): 1385-1392.
[10]	文国军, 夏雨, 王玉丹, 胡自飞, 吴丹. 基于Unity3D的水平定向钻机虚拟实训系统设计[J]. 系统仿真学报, 2020, 32(5): 801-807.
[11]	邢志伟, 姜松岳, 罗谦, 罗晓. 基于LWSVR的繁忙机场航班滑出时间预测[J]. 系统仿真学报, 2020, 32(5): 927-935.
[12]	张宪霞, 章进强, 李致远, 马世伟, 杨帮华. 基于支持向量回归机学习的机械臂视觉反馈模糊控制[J]. 系统仿真学报, 2020, 32(10): 1997-2009.
[13]	刘畅, 贾金原, 陆一凡, 张乾, 赵磊. 光图树：Web3D动态场景全局光照的协同实时渲染[J]. 系统仿真学报, 2019, 31(8): 1591-1604.
[14]	谭力恒, 马强, 蒋秉川, 李锋. 面向头盔式虚拟现实的OsgEarth立体视觉生成方法[J]. 系统仿真学报, 2019, 31(7): 1290-1299.
[15]	祁彬斌, 朱学芳, 范丽鹏. 基于触觉交互的三维弹性物体形变仿真算法[J]. 系统仿真学报, 2019, 31(11): 2296-2305.