面向康复训练的严肃游戏人机交互设计

doi:10.16182/j.issn1004731x.joss.18-VR0695

系统仿真学报 ›› 2019, Vol. 31 ›› Issue (5): 909-918.doi: 10.16182/j.issn1004731x.joss.18-VR0695

面向康复训练的严肃游戏人机交互设计

吴佳佳¹, 刘箴^1,*, 许辉煌², 刘婷婷³, 王瑾³, 柴艳杰¹

1. 宁波大学信息科学与工程学院,宁波 315211;
2. 淡江大学资讯工程学系,淡水 25137;
3. 宁波大学科学技术学院,宁波 315032

收稿日期:2018-06-30 修回日期:2018-10-20 出版日期:2019-05-08 发布日期:2019-11-20
作者简介:吴佳佳(1993-),女,安徽合肥,硕士,研究方向为严肃游戏交互设计。
基金资助:
国家自然科学基金(61761166005, U1636111),宁波市科技计划项目(2017C50018,2016D10016),浙江省医药卫生科技计划项目(2019PY077),宁波市医药卫生科技项目(2016A07)

Serious Game Human-Computer Interaction Design for Rehabilitation Training

Wu Jiajia¹, Liu Zhen^1,*, Xu Huihuang², Liu Tingting³, Wang Jin³, Chai Yanjie¹

1. Faculty of Electrical Engineering and Computer Science, Ningbo University, Ningbo 315211, China;
2. Department of Computer Science and Information Engineering, Tamkang University, Tamsui 25137, China;
3. College of Science & Technology Ningbo University, Ningbo 315032, China;

Received:2018-06-30 Revised:2018-10-20 Online:2019-05-08 Published:2019-11-20

摘要/Abstract

摘要： 构建适合用户需求的人机交互设计,能够有效地提升严肃游戏用户的体验感。提出将康复训练过程融入交互式的游戏剧情中,通过建立社区场景和交互任务,对用户的智力和肢体进行康复训练。采用游戏引擎实现了严肃游戏原型系统。初步的实验发现,在漫游交互操作中,对比不同的人机交互方式,鼠标点选漫游方式更便于操作;在任务交互操作中,体感交互便于锻炼肢体运动。通过为虚拟角色设计表情符号,便于用户识别他人的情绪和社交距离。这些研究为进一步开展严肃游戏应用于康复领域提供了一定借鉴。

关键词: 人机交互, 康复训练, 严肃游戏, 情绪识别

Abstract: Constructing a human-computer interaction environment which can effectively enhance users’ experiences of serious game. By integrating the rehabilitation training process into an interactive game story, users can do their mental and body rehabilitation training in the community. The prototype of the game is realized by the game engine. Experimental results showed that using mouse to roam in the system is easier than other interaction method and the somatosensory interaction is convenient for exercising the limb. By designing emoji for virtual characters, it is easy for users to learn other people's emotions and the comfortable distances for others. The results can be references for further studies about serious games in the field of rehabilitation training.

Key words: human-computer interaction, rehabilitation training, serious game, emotional recognition

中图分类号:

TP391.9

吴佳佳, 刘箴, 许辉煌, 刘婷婷, 王瑾, 柴艳杰. 面向康复训练的严肃游戏人机交互设计[J]. 系统仿真学报, 2019, 31(5): 909-918.

Wu Jiajia, Liu Zhen, Xu Huihuang, Liu Tingting, Wang Jin, Chai Yanjie. Serious Game Human-Computer Interaction Design for Rehabilitation Training[J]. Journal of System Simulation, 2019, 31(5): 909-918.

参考文献

[1] Barnes T, Encarnação L M, Shaw C D.Serious Games[J]. IEEE Computer Graphics & Applications(S0272-1716), 2009, 29(2): 18-19.
[2] Song Y, Demirdjian D, Davis R .Continuous body and hand gesture recognition for natural human-computer interaction[J]. ACM Transactions on Interactive Intelligent Systems(S2160-6455), 2012, 2(1): 1-28.
[3] Chan S, Cai Y, Lu A, et al.Virtual reality enhanced pink dolphin game for children with ASD[C]// Asia-Europe Symposium on Simulation & Serious Gaming. ACM, 2016: 215-218.
[4] Marsella S, Gratch J.EMA: A Process Model of Emotional Dynamics[J]. Journal of Cognitive Systems Research (S1389-0417), 2009, 10(1): 70-90.
[5] Pirovano M, Mainetti R, Baud-Bovy G, et al.Intelligent Game Engine for Rehabilitation (IGER)[J]. IEEE Transactions on Computational Intelligence & Ai in Games (S1943-0698), 2016, 8(1): 43-55.
[6] Baur T, Mehlmann G, Damian I, et al.Context-Aware Automated Analysis and Annotation of Social Human-Agent Interactions[J]. Acm Transactions on Interactive Intelligent Systems(S2160-6455), 2015, 5(2): 1-33.
[7] Bozgeyikli L, Bozgeyikli E, Raij A, et al.Vocational Rehabilitation of Individuals with Autism Spectrum Disorder with Virtual Reality[J]. Acm Transactions on Accessible Computing(S1936-7228), 2017, 10(2): 1-25.
[8] Kandalaft M R, Didehbani N, Krawczyk D C, et al.Virtual Reality Social Cognition Training for Young Adults with High-Functioning Autism[J]. Journal of Autism and Developmental Disorders(S1573-3432), 2013, 43(1): 34-44.
[9] Simões M, Bernardes M, Barros F, et al.Virtual Travel Training for Autism Spectrum Disorder: Proof-of- Concept Interventional Study[J]. Jmir Serious Games (S2291-9279), 2018, 6(1): e5.
[10] 梁旭, 王卫群, 侯增广, 等. 康复机器人的人机交互控制方法[J]. 中国科学: 信息科学, 2018, 8(1): 24-46.
Liang Xu, Wang Weiqun, Hou Zengguang, et al.Human-computer interaction control method for rehabilitation robots[J]. Science in China: Information Science, 2018, 8(1) : 24-46.
[11] 杜义浩, 邱石, 谢平, 等. 下肢康复机器人的自适应人机交互控制策略[J]. 自动化学报, 2018, 44(4): 169-176.
Du Yihao, Qiu Shi, Xie Ping, et al.Adaptive Human- Computer Interaction Control Strategy for Lower Limb Rehabilitation Robots[J]. Acta Automatica Sinica, 2018, 44(4) :169-176.
[12] 杨建辉, 刘箴, 陈浩. 基于深度相机的自主虚拟化身情感交互技术[J]. 计算机工程, 2016, 42(6): 293-298.
Yang Jianhui, Liu Zhen, Chen Hao.Autonomous Virtual Body Emotional Interaction Technology Based on Depth Camera[J]. Computer Engineering, 2016, 42(6): 293-298.
[13] 刘箴, 潘志庚. 虚拟人动机驱动的自主情绪模型研究[J]. 中国图象图形学报, 2009, 14(5): 773-781.
Liu Zhen, Pan Zhigeng.Research on the Model of Autonomic Emotion Driven by Virtual Human Motivation[J]. Journal of Image and Graphics, 2009, 14(5): 773-781.
[14] Ochs M, Sabouret N, Corruble V.Simulation of the Dynamics of Nonplayer Characters' Emotions and Social Relations in Games[J]. IEEE Transactions on Computational Intelligence & Ai in Games (S1943-0698), 2009, 1(4): 281-297.
[15] 吴奇, 申寻兵, 傅小兰. 微表情研究及其应用[J]. 心理科学进展, 2010, 18(9): 1359-1368.
Wu Qi, Shen Xunbing, Fu Xiaolan.Micro-expression research and its application[J]. Advances in Psychological Science, 2010, 18(9): 1359-1368.
[16] 拉米亚. 儿童情绪管理与性格培养-我要了解自己(青少年情绪管理手册) [M]. 北京: 化学工业出版社, 2013.
Lamia. Children's Emotion Management and Character Cultivation--I Want to Know About myself (Youth Emotion Management Handbook) [M]. Beijing: Chemical Industry Press, 2013.
[17] Was J, Gudowski B, Matuszyk P J.Social Distances Model of Pedestrian Dynamics[C]// Cellular Automata, International Conference on Cellular Automata, for Research and Industry, Acri 2006, Perpignan, France, September 20-23, 2006, Proceedings. DBLP, 2006: 492-501.
[18] Yoones A Sekhavat.Behavior Trees for Computer Games[J]. International Journal on Artificial Intelligence Tools (S0218-2130), 2017, 26(2): 1-27.
[19] 王洪源. Unity3D人工智能编程精粹 [M]. 北京: 清华大学出版社, 2014.
Wang Hongyuan.The essence of Unity3D artificial intellig- ence programming [M]. Beijing: Tsinghua University Press, 2014.
[20] Marchand E, Uchiyama H, Spindler F.Pose estimation for augmented reality: a hands-on survey[J]. IEEE Trans Vis Comput Graph (S1941-0506), 2016, 22(12): 2633-2651.
[21] Hasan H, Abdul-Kareem S.RETRACTED ARTICLE: Static hand gesture recognition using neural networks[J]. Artificial Intelligence Review (S0269-2821), 2014, 41(2): 147-181.
[22] Jin H, Chen Q, Chen Z, et al.Multi-LeapMotion sensor based demonstration for robotic refine tabletop object manipulation task[J]. CAAI Transactions on Intelligence Technology(S2468-2322), 2016, 1(1): 104-113.
[23] Yao N, Wang K, Cai Y, et al.Application of Somatosensory Sensor Kinect in Man-Machine Interaction Framework of Gesture Recognition[J]. Sensor Letters(S1546-1971), 2015, 13(12): 1050-1054.
[24] Pinelle D, Wong N, Stach T.Heuristic evaluation for games: usability principles for video game design[C]// Sigchi Conference on Human Factors in Computing Systems. ACM. Florence, Italy: CHI 2008 Proceedings·Game Zone, 2008: 1453-1462.

面向康复训练的严肃游戏人机交互设计

Serious Game Human-Computer Interaction Design for Rehabilitation Training

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 11

编辑推荐

Metrics

本文评价

[1]	卢柏宏, 赵建军, 刘戈三. 电影虚拟摄影半实物仿真的研究与实现[J]. 系统仿真学报, 2021, 33(8): 1938-1946.
[2]	郭园, 童倩倩, 郑宇铠, 王子琦, 张玉茹, 王党校. 融合多元触觉和沉浸式视觉的可移植VR软件框架[J]. 系统仿真学报, 2020, 32(7): 1385-1392.
[3]	刘延斌, 庞翔元, 张彦斌, 郭冰菁, 韩建海. 踝关节康复机器人主动训练柔顺控制研究[J]. 系统仿真学报, 2020, 32(1): 54-60.
[4]	薛艳萍, 王学松, 武仲科, 王醒策, 周明全. Myo手势识别方法在古建筑漫游系统中的应用[J]. 系统仿真学报, 2019, 31(9): 1907-1915.
[5]	李峰, 吴智政, 侯伟, 秦晓飞. 下肢康复训练机器人跑步机速度协调控制[J]. 系统仿真学报, 2019, 31(6): 1111-1122.
[6]	杨帮华, 李博. 基于脑机接口的康复训练系统[J]. 系统仿真学报, 2019, 31(2): 174-180.
[7]	朱晓强, 潘虹艺, 徐浩, 宋磊, 朱梦尧, 王向阳. 头戴设备VR环境下光滑加权等距面交互建模[J]. 系统仿真学报, 2018, 30(7): 2459-2464.
[8]	王志文, 张子豪, 贾玉祥, 魏毅, 沈燕飞. 基于语音识别控制的人体运动仿真[J]. 系统仿真学报, 2018, 30(11): 4203-4209.
[9]	韩亮, 公璇, 王江云, 杜阳华. 跳伞模拟器人机交互可视化功能研究[J]. 系统仿真学报, 2017, 29(8): 1747-1752.
[10]	陈浩, 刘箴, 陈田, 刘婷婷, 刘翠娟, 朱宪莹. 一个面向老年人娱乐健身的体感严肃游戏研究[J]. 系统仿真学报, 2016, 28(10): 2586-2592.
[11]	卢浩然, 张宇, 牛苗苗, 税午阳, 周明全. 基于Leap motion的三维文物虚拟拼接方法[J]. 系统仿真学报, 2015, 27(12): 3006-3011.