融合多元触觉和沉浸式视觉的可移植VR软件框架

doi:10.16182/j.issn1004731x.joss.19-VR0440

系统仿真学报 ›› 2020, Vol. 32 ›› Issue (7): 1385-1392.doi: 10.16182/j.issn1004731x.joss.19-VR0440

• 物理效应/模拟器仿真技术 • 上一篇下一篇

融合多元触觉和沉浸式视觉的可移植VR软件框架

郭园¹, 童倩倩³, 郑宇铠¹, 王子琦¹, 张玉茹^{1, 2}, 王党校^{1, 2, 3, *}

1. 虚拟现实技术与系统国家重点实验室,北京航空航天大学,北京 100191;
2. 北京航空航天大学生物医学工程高精尖创新中心,北京 100191;
3. 鹏城实验室,深圳 518055

收稿日期:2019-08-24 修回日期:2019-11-14 出版日期:2020-07-25 发布日期:2020-07-15
作者简介:郭园(1992-),男,山西,博士生,研究方向为人机交互、力反馈手套;童倩倩(1989-),女,河南,博士,研究方向为人机交互、触觉反馈。

An Adaptable VR Software Framework for Collaborative Multi-modal Haptic and Immersive Visual Display

Guo Yuan¹, Tong Qianqian³, Zheng Yukai¹, Wang Ziqi¹, Zhang Yuru^{1, 2}, Wang Dangxiao^{1, 2, 3, *}

1. State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China;
2. Beijing Advanced Innovation Center for Biomedical Engineering, Beihang University, Beijing 100191, China;
3. Peng Cheng Laboratory, Shenzhen 518055, China

Received:2019-08-24 Revised:2019-11-14 Online:2020-07-25 Published:2020-07-15

摘要/Abstract

摘要： 多元触觉和沉浸式视觉反馈的时空一致性配准以及现有VR软件框架中硬件设备的接入便捷性问题,制约了医疗、商务、娱乐等不同行业中视触觉融合场景的快速开发。为解决这一难题,提出融合多元触觉和沉浸式视觉的可移植VR软件框架,根据视触觉反馈精度要求的不同设计了双层和三层架构兼顾视觉渲染、触觉渲染、硬件控制的不同需求。实验结果表明该框架具有支持视-触融合同步反馈、硬件平台可移植、图形场景开发简便和交互场景可快速替换等特点,能够满足不同领域的交互应用需求。

关键词: 虚拟现实, 多元触觉, 人机交互, 视触觉融合, 可移植软件框架

Abstract: The spatiotemporal consistency of multi-modal haptic and immersive visual feedback, as well as the integration of hardware devices in the existing VR software framework, restricts the rapid development of visuo-haptic fusion scenarios in different industries such as medical, business, and entertainment. In order to tackle this problem, an adaptable VR software framework combining multi-modal haptic and immersive visual display is proposed. To meet the different requirements of visual rendering, haptic rendering and hardware control, the double-layer and three-layer architectures are devised according to the different precision requirements of visuo-haptic fusion feedback. Experimental results show that the framework has the characteristics of supporting the synchronous feedback of visuo-haptic fusion, the convenient integration of diverse haptic devices, the simplicity of scene development and the rapid replacement of interactive scene, and it can meet the requirements of different application fields.

Key words: virtual reality, multi-modal haptic, human-machine interaction, visuo-haptic fusion, adaptable software framework

中图分类号:

TP391.9

郭园, 童倩倩, 郑宇铠, 王子琦, 张玉茹, 王党校. 融合多元触觉和沉浸式视觉的可移植VR软件框架[J]. 系统仿真学报, 2020, 32(7): 1385-1392.

Guo Yuan, Tong Qianqian, Zheng Yukai, Wang Ziqi, Zhang Yuru, Wang Dangxiao. An Adaptable VR Software Framework for Collaborative Multi-modal Haptic and Immersive Visual Display[J]. Journal of System Simulation, 2020, 32(7): 1385-1392.

参考文献 17

[1]	Fukumoto M, Sugimura T.Active click: tactile feedback for touch panels[C]// CHI'01 Extended Abstracts on Human Factors in Computing Systems. New York, USA: ACM, 2001: 121-122.
[2]	Yatani K, Truong K N.SemFeel: a user interface with semantic tactile feedback for mobile touch-screen devices[C]// Proceedings of the 22nd annual ACM symposium on User interface software and technology. New York, USA: ACM, 2009: 111-120.
[3]	Sawada H, Takeda Y.Tactile pen for presenting texture sensation from touch screen[C]// 2015 8th International Conference on Human System Interaction (HSI). Warsaw, Poland: IEEE, 2015: 334-339.
[4]	Yang G H, Jin M, Jin Y, et al.T-mobile: Vibrotactile display pad with spatial and directional information for hand-held device[C]// 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. Taipei, China: IEEE, 2010: 5245-5250.
[5]	Sun Z, Miao X, Li X.Design of a bidirectional force feedback dataglove based on pneumatic artificial muscles[C]// 2009 International Conference on Mechatronics and Automation. Changchun, China: IEEE, 2009: 1767-1771.
[6]	Cassar D J, Saliba M A.A force feedback glove based on magnetorheological fluid: preliminary design issues[C]// Melecon 2010-2010 15th IEEE Mediterranean Electrotechnical Conference. Valletta, Malta: IEEE, 2010: 618-623.
[7]	Gu X, Zhang Y, Sun W, et al.Dexmo: An inexpensive and lightweight mechanical exoskeleton for motion capture and force feedback in VR[C]// Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems. New York, USA: ACM, 2016: 1991-1995.
[8]	Choi I, Hawkes E W, Christensen D L, et al.Wolverine: A wearable haptic interface for grasping in virtual reality[C]// 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Daejeon, South Korea: IEEE, 2016: 986-993.
[9]	Wang D, Ohnishi K, Xu W.Multimodal Haptic Display for Virtual Reality: A Survey[J]. IEEE Transactions on Industrial Electronics (S0278-0046), 2020, 67(1): 610-623.
[10]	Mansour N A, El-Bab A M R F, Abdellatif M. Design of a novel multi-modal tactile display device for biomedical applications[C]// 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob). Rome, Italy: IEEE, 2012: 183-188.
[11]	Stanley A A, Okamura A M.Deformable model-based methods for shape control of a haptic jamming surface[J]. IEEE transactions on visualization and computer graphics (S1077-2626), 2016, 23(2): 1029-1041.
[12]	Murakami T, Person T, Fernando C L, et al.Altered touch: miniature haptic display with force, thermal and tactile feedback for augmented haptics[C]// ACM SIGGRAPH 2017 Emerging Technologies. New York, USA: ACM, 2017.
[13]	Choi I, Culbertson H, Miller M R, et al.Grabity: A wearable haptic interface for simulating weight and grasping in virtual reality[C]// Proceedings of the 30th Annual ACM Symposium on User Interface Software and Technology. New York, USA: ACM, 2017: 119-130.
[14]	Choi I, Ofek E, Benko H, et al.Claw: A multifunctional handheld haptic controller for grasping, touching, and triggering in virtual reality[C]// Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. New York, USA: ACM, 2018.
[15]	Otaduy M A, Lin M C.Sensation preserving simplification forhaptic rendering[J]. ACM Transactions on Graphics (S0730-0301), 2003, 22(3): 543-553.
[16]	Wang D, Zhang X, Zhang Y, et al.Configuration-based optimization for six degree-of-freedom haptic rendering for fine manipulation[J]. IEEE Transactions on Haptics (S1939-1412), 2013, 6(2): 167-180.
[17]	Wang D, Shi Y, Liu S, et al.Haptic simulation of organ deformation and hybrid contacts in dental operations[J]. IEEE Transactions on Haptics (S1939-1412), 2014, 7(1): 48-60.

融合多元触觉和沉浸式视觉的可移植VR软件框架

An Adaptable VR Software Framework for Collaborative Multi-modal Haptic and Immersive Visual Display

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献 17

相关文章 15

编辑推荐

Metrics

本文评价

[1]	周晓光, 朱鹏, 张源原, 路欢, 周媛. 六自由度跳伞模拟训练系统设计与实现[J]. 系统仿真学报, 2022, 34(6): 1320-1329.
[2]	许世健, 赵丹, 苏铖宇, 王赋攀, 张晓蓉, 王昉, 吴亚东. 面向流场可视化的沉浸式虚拟现实交互系统研究[J]. 系统仿真学报, 2022, 34(5): 1160-1172.
[3]	孙佳文, 任鸿翔, 肖方兵, 蒋效彬. 基于虚拟现实的整船仿真训练平台[J]. 系统仿真学报, 2021, 33(9): 2243-2251.
[4]	卢柏宏, 赵建军, 刘戈三. 电影虚拟摄影半实物仿真的研究与实现[J]. 系统仿真学报, 2021, 33(8): 1938-1946.
[5]	魏士松, 周正东, 章栩苓, 毛玲, 贾峻山, 刘传乐. 基于桌面虚拟现实技术的航天器虚拟维修训练系统[J]. 系统仿真学报, 2021, 33(6): 1358-1363.
[6]	贾春洋, 邹湘军, 李锦慧, 曾泽钦, 王杰. 虚拟环境下的复杂变速箱交互式装配仿真平台[J]. 系统仿真学报, 2020, 32(9): 1736-1743.
[7]	文国军, 夏雨, 王玉丹, 胡自飞, 吴丹. 基于Unity3D的水平定向钻机虚拟实训系统设计[J]. 系统仿真学报, 2020, 32(5): 801-807.
[8]	薛艳萍, 王学松, 武仲科, 王醒策, 周明全. Myo手势识别方法在古建筑漫游系统中的应用[J]. 系统仿真学报, 2019, 31(9): 1907-1915.
[9]	刘畅, 贾金原, 陆一凡, 张乾, 赵磊. 光图树：Web3D动态场景全局光照的协同实时渲染[J]. 系统仿真学报, 2019, 31(8): 1591-1604.
[10]	谭力恒, 马强, 蒋秉川, 李锋. 面向头盔式虚拟现实的OsgEarth立体视觉生成方法[J]. 系统仿真学报, 2019, 31(7): 1290-1299.
[11]	吴佳佳, 刘箴, 许辉煌, 刘婷婷, 王瑾, 柴艳杰. 面向康复训练的严肃游戏人机交互设计[J]. 系统仿真学报, 2019, 31(5): 909-918.
[12]	祁彬斌, 朱学芳, 范丽鹏. 基于触觉交互的三维弹性物体形变仿真算法[J]. 系统仿真学报, 2019, 31(11): 2296-2305.
[13]	闫丰亭, 史志才, 苏前敏, 董婧, 陈珏, 石蕴玉, 汤显. 物联网与虚拟现实融合的智能火灾营救机制[J]. 系统仿真学报, 2019, 31(11): 2306-2322.
[14]	焦欢欢, 潘志庚, 徐舒畅, 袁庆曙. 基于VR的言语失用症康复训练系统[J]. 系统仿真学报, 2019, 31(11): 2344-2349.
[15]	张波, 孙俊, 商蕾, 徐弘基, 杨诚. 基于HMD的船舶虚拟消防训练系统设计及实现[J]. 系统仿真学报, 2019, 31(1): 43-52.