沉浸式三维视线追踪算法研究

doi:10.16182/j.issn1004731x.joss.201806004

系统仿真学报 ›› 2018, Vol. 30 ›› Issue (6): 2027-2035.doi: 10.16182/j.issn1004731x.joss.201806004

沉浸式三维视线追踪算法研究

赵新灿, 潘世豪, 王雅萍, 帖云

郑州大学信息工程学院,郑州 450001

收稿日期:2016-04-27 修回日期:2016-08-03 出版日期:2018-06-08 发布日期:2018-06-14
第一作者简介:赵新灿(1972-),男,山东曹县,博士,副教授,研究方向为增强现实、人机交互;潘世豪(1990-),男,河南平顶山,硕士生,研究方向为虚拟现实、人机交互。
基金资助:
国家自然科学基金(U1433106,U1504606),中国博士后科学基金(2016T90679,2015M582201)

Eye Gaze Tracking in 3D Immersive Environments

Zhao Xincan, Pan Shihao, Wang Yaping, Tie Yun

College of Information Engineering, Zhengzhou University, Zhengzhou 450001, China

Received:2016-04-27 Revised:2016-08-03 Online:2018-06-08 Published:2018-06-14

摘要/Abstract

摘要： 针对大型沉浸式虚拟环境中人机交互完全依赖肢体动作且效率低等问题,提出利用三维视线追踪技术得到用户注视点,以实现交互操作,为沉浸式环境提供一种自然、双向的交互手段。创新性地将Leap Motion用于瞳孔位置跟踪,通过被动式光学追踪设备获取使用者的头部运动状态,依据初始标定得到的映射方程来估计使用者大空间范围内自由运动状态下的三维注视点。实验表明,使用者在3.0 m×3.2 m×2.0 m的空间内自由运动时,集成系统对三维注视点的估计频率可达60 Hz,估计误差小于45 mm,为视线追踪在沉浸式虚拟环境中的广泛应用奠定了基础。

关键词: 三维视线追踪, 双视线向量, 沉浸式环境, 自由运动

Abstract: Aiming at the problem that human-computer interaction in immersive environments is entirely dependent on the body movement with low operational efficiency,a 3D eye gaze tracking method is proposed to gain the 3D gaze point of the user. Thus the interactive operation could be achieved, which provides a natural and bidirectional way for interaction. The Leap Motion controller is innovatively used to track the coordinates of eyes and the head position is tracked with the motion capture device. The gaze point under free movement can be estimated based on the mapping equation that is obtained in the initial calibration state. Experimental results show that when user moves freely in the 3.0 m×3.2 m×2.0 m space, the frequency of estimation of 3D gaze point is up to 60 Hz and the error is within 45 mm, which lays a foundation for the wide application of eye gaze tracking in immersive virtual environments.

Key words: 3D gaze tracking, dual gaze vectors, immersive environments, free movement

中图分类号:

TP391.9

赵新灿,潘世豪,王雅萍等 . 沉浸式三维视线追踪算法研究[J]. 系统仿真学报, 2018, 30(6): 2027-2035.

Zhao Xincan,Pan Shihao,Wang Yaping,et al . Eye Gaze Tracking in 3D Immersive Environments[J]. Journal of System Simulation, 2018, 30(6): 2027-2035.

参考文献

[1] Hansen D W, Ji Q.In the eye of the beholder: A survey of models for eyes and gaze[J]. IEEE transactions on pattern analysis and machine intelligence (S0162-8828), 2010, 32(3): 478-500.
[2] Yoo D H, Chung M J.A novel non-intrusive eye gaze estimation using cross-ratio under large head motion[J]. Computer Vision and Image Understanding (S1077-3142), 2005, 98(1): 25-51.
[3] Zhang Z, Cai Q.Improving cross-ratio-based eye tracking techniques by leveraging the binocular fixation constraint[C]// Proceedings of the Symposium on Eye Tracking Research and Applications. ACM, 2014: 267-270.
[4] Shin Y G, Choi K A, Kim S T, et al.A novel 2-D mapping-based remote eye gaze tracking method using two IR light sources[C]// Consumer Electronics (ICCE), 2015 IEEE International Conference on. IEEE, 2015: 190-191.
[5] Zhu Z, Ji Q, Bennett K P.Nonlinear eye gaze mapping function estimation via support vector regression[C]// Pattern Recognition, 2006. ICPR 2006. 18th International Conference on. IEEE, 2006: 1132-1135.
[6] Gneo M, Schmid M, Conforto S, et al.A free geometry model-independent neural eye-gaze tracking system[J]. Journal of neuroengineering and rehabilitation (S1743-0003), 2012, 9(1): 82.
[7] Zhu Z, Ji Q.Novel eye gaze tracking techniques under natural head movement[J]. IEEE Transactions on biomedical engineering (S1558-2531), 2007, 54(12): 2246-2260.
[8] Chen J, Tong Y, Gray W, et al.A robust 3D eye gaze tracking system using noise reduction[C]// Proceedings of the 2008 symposium on Eye tracking research & applications. ACM, 2008: 189-196.
[9] Sun L, Liu Z, Sun M T.Real time gaze estimation with a consumer depth camera[J]. Information Sciences (S0020-0255), 2015, 320: 346-360.
[10] Xiong X, Cai Q, Liu Z, et al.Eye gaze tracking using an RGBD camera: a comparison with a RGB solution[C]// Proceedings of the 2014 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct Publication. ACM, 2014: 1113-1121.
[11] Hou W, Ding Y, Qin X, et al.Evaluation of Cognitive Awareness Based on Dual Task Situation[C]// International Conference on Human Centered Computing. Springer, Cham, 2016: 160-171.
[12] McMurrough C, Conly C, Athitsos V, et al. A mobile, low-cost headset for 3D point of gaze estimation[J]. International Journal of Advanced Computer Science (S1819-9224), 2014, 1(1): 2-5.
[13] Marin G, Dominio F, Zanuttigh P.Hand gesture recognition with leap motion and kinect devices[C]// Image Processing (ICIP), 2014 IEEE International Conference on. IEEE, 2014: 1565-1569.
[14] Weichert F, Bachmann D, Rudak B, et al.Analysis of the accuracy and robustness of the leap motion controller[J]. Sensors (S1424-8220), 2013, 13(5): 6380-6393.
[15] Guna J, Jakus G, Pogačnik M, et al.An analysis of the precision and reliability of the leap motion sensor and its suitability for static and dynamic tracking[J]. Sensors (S1424-8220), 2014, 14(2): 3702-3720.
[16] Shao G, Che M, Zhang B, et al.A novel simple 2D model of eye gaze estimation[C]// Intelligent Human-Machine Systems and Cybernetics (IHMSC), 2010 2nd International Conference on. IEEE, 2010, 1: 300-304.
[17] Li D, Winfield D, Parkhurst D J.Starburst: A hybrid algorithm for video-based eye tracking combining feature-based and model-based approaches[C]// Computer Vision and Pattern Recognition-Workshops, 2005. CVPR Workshops. IEEE Computer Society Conference on. IEEE, 2005: 79-79.
[18] Zhu Z, Ji Q.Eye gaze tracking under natural head movements[C]// Computer Vision and Pattern Recognition, 2005. CVPR 2005. IEEE Computer Society Conference on. IEEE, 2005: 918-923.
[19] Chambers T L, Aglawe A, Reiners D, et al.Real-time simulation for a virtual reality-based MIG welding training system[J]. Virtual Reality (S1359-4338), 2012, 16(1): 45-55.

沉浸式三维视线追踪算法研究

Eye Gaze Tracking in 3D Immersive Environments

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	董志明, 胡忠奇, 戴浩然, 高建成. 基于大语言模型的作战仿真想定自动化生成方法[J]. 系统仿真学报, 2026, 38(5): 1129-1145.
[2]	李校男, 晁涛, 马萍, 杨明, 王玉轩. 基于期望最大化方法的非线性SSM黑箱鲁棒辨识[J]. 系统仿真学报, 2026, 38(5): 1146-1158.
[3]	刘银钢, 马明, 张荣华. 基于大语言模型的兵棋推演动态任务规划[J]. 系统仿真学报, 2026, 38(5): 1187-1204.
[4]	苏泓嘉, 张成, 刘飞. 基于模糊功能依赖网分析的体系效能评估方法[J]. 系统仿真学报, 2026, 38(5): 1224-1238.
[5]	梅华威, 杨鹏慧, 余洋. 计及数据漂移改进PatchTST的超短期光伏功率预测[J]. 系统仿真学报, 2026, 38(5): 1239-1254.
[6]	李权, 苏鹏, 万海英, 张承玺, 何志坚, 倪艺洋, 赵忠盖, 刘飞. 基于多阶段LHS-EPRCC方法的青霉素发酵过程建模[J]. 系统仿真学报, 2026, 38(5): 1255-1276.
[7]	周子聪, 曾俊杰, 胡越, 朱正秋, 尹全军. 基于次优示例引导的兵棋推演多智能体强化学习方法[J]. 系统仿真学报, 2026, 38(5): 1277-1289.
[8]	石敏, 郭诗盛, 王素琴, 李兆歆, 朱登明. 融合物理与几何先验的无抓取标注6-DoF抓取检测方法[J]. 系统仿真学报, 2026, 38(5): 1290-1302.
[9]	姜彦吉, 肖星佚, 董浩, 于淼, 黄金山, 刘大千, 费博雯. 融合点线特征的图关系优化3D车道线检测方法[J]. 系统仿真学报, 2026, 38(5): 1303-1319.
[10]	张鑫, 张平, 张琛, 刘威, 韩博阳. 非均质土壤条件下挖掘阻力计算模型研究[J]. 系统仿真学报, 2026, 38(5): 1320-1332.
[11]	王伟, 刘东, 崔新豪, 李博, 肖依永, 任羿. 复杂项目多级动态挣值管理数字化模型及应用[J]. 系统仿真学报, 2026, 38(5): 1350-1364.
[12]	彭莉峻, 苏庭琪, 刘沛津, 何林, 周协武, 张闽心. 融合人体关键点的实验室PPE规范穿戴检测方法[J]. 系统仿真学报, 2026, 38(5): 1365-1382.
[13]	滕靖, 童文聪, 张中杰, 姚幸, 李君羡. 有轨电车交叉口速度自动引导方法及仿真评价[J]. 系统仿真学报, 2026, 38(5): 1426-1439.
[14]	蒋圣超, 裴云庆, 翟宏营, 吴国键, 高放. 基于块编码绝热量子牛顿‒拉夫逊法的潮流计算[J]. 系统仿真学报, 2026, 38(5): 1453-1465.
[15]	秦浪, 谢嘉成, 乔晓军, 王学文, 肖智杰. 执行器位姿异常的机器人轨迹规划调整方法[J]. 系统仿真学报, 2026, 38(5): 1466-1483.