Serious Game Human-Computer Interaction Design for Rehabilitation Training

doi:10.16182/j.issn1004731x.joss.18-VR0695

Abstract

Abstract: Constructing a human-computer interaction environment which can effectively enhance users’ experiences of serious game. By integrating the rehabilitation training process into an interactive game story, users can do their mental and body rehabilitation training in the community. The prototype of the game is realized by the game engine. Experimental results showed that using mouse to roam in the system is easier than other interaction method and the somatosensory interaction is convenient for exercising the limb. By designing emoji for virtual characters, it is easy for users to learn other people's emotions and the comfortable distances for others. The results can be references for further studies about serious games in the field of rehabilitation training.

Key words: human-computer interaction, rehabilitation training, serious game, emotional recognition

CLC Number:

TP391.9

Wu Jiajia, Liu Zhen, Xu Huihuang, Liu Tingting, Wang Jin, Chai Yanjie. Serious Game Human-Computer Interaction Design for Rehabilitation Training[J]. Journal of System Simulation, 2019, 31(5): 909-918.

References

[1] Barnes T, Encarnação L M, Shaw C D.Serious Games[J]. IEEE Computer Graphics & Applications(S0272-1716), 2009, 29(2): 18-19.
[2] Song Y, Demirdjian D, Davis R .Continuous body and hand gesture recognition for natural human-computer interaction[J]. ACM Transactions on Interactive Intelligent Systems(S2160-6455), 2012, 2(1): 1-28.
[3] Chan S, Cai Y, Lu A, et al.Virtual reality enhanced pink dolphin game for children with ASD[C]// Asia-Europe Symposium on Simulation & Serious Gaming. ACM, 2016: 215-218.
[4] Marsella S, Gratch J.EMA: A Process Model of Emotional Dynamics[J]. Journal of Cognitive Systems Research (S1389-0417), 2009, 10(1): 70-90.
[5] Pirovano M, Mainetti R, Baud-Bovy G, et al.Intelligent Game Engine for Rehabilitation (IGER)[J]. IEEE Transactions on Computational Intelligence & Ai in Games (S1943-0698), 2016, 8(1): 43-55.
[6] Baur T, Mehlmann G, Damian I, et al.Context-Aware Automated Analysis and Annotation of Social Human-Agent Interactions[J]. Acm Transactions on Interactive Intelligent Systems(S2160-6455), 2015, 5(2): 1-33.
[7] Bozgeyikli L, Bozgeyikli E, Raij A, et al.Vocational Rehabilitation of Individuals with Autism Spectrum Disorder with Virtual Reality[J]. Acm Transactions on Accessible Computing(S1936-7228), 2017, 10(2): 1-25.
[8] Kandalaft M R, Didehbani N, Krawczyk D C, et al.Virtual Reality Social Cognition Training for Young Adults with High-Functioning Autism[J]. Journal of Autism and Developmental Disorders(S1573-3432), 2013, 43(1): 34-44.
[9] Simões M, Bernardes M, Barros F, et al.Virtual Travel Training for Autism Spectrum Disorder: Proof-of- Concept Interventional Study[J]. Jmir Serious Games (S2291-9279), 2018, 6(1): e5.
[10] 梁旭, 王卫群, 侯增广, 等. 康复机器人的人机交互控制方法[J]. 中国科学: 信息科学, 2018, 8(1): 24-46.
Liang Xu, Wang Weiqun, Hou Zengguang, et al.Human-computer interaction control method for rehabilitation robots[J]. Science in China: Information Science, 2018, 8(1) : 24-46.
[11] 杜义浩, 邱石, 谢平, 等. 下肢康复机器人的自适应人机交互控制策略[J]. 自动化学报, 2018, 44(4): 169-176.
Du Yihao, Qiu Shi, Xie Ping, et al.Adaptive Human- Computer Interaction Control Strategy for Lower Limb Rehabilitation Robots[J]. Acta Automatica Sinica, 2018, 44(4) :169-176.
[12] 杨建辉, 刘箴, 陈浩. 基于深度相机的自主虚拟化身情感交互技术[J]. 计算机工程, 2016, 42(6): 293-298.
Yang Jianhui, Liu Zhen, Chen Hao.Autonomous Virtual Body Emotional Interaction Technology Based on Depth Camera[J]. Computer Engineering, 2016, 42(6): 293-298.
[13] 刘箴, 潘志庚. 虚拟人动机驱动的自主情绪模型研究[J]. 中国图象图形学报, 2009, 14(5): 773-781.
Liu Zhen, Pan Zhigeng.Research on the Model of Autonomic Emotion Driven by Virtual Human Motivation[J]. Journal of Image and Graphics, 2009, 14(5): 773-781.
[14] Ochs M, Sabouret N, Corruble V.Simulation of the Dynamics of Nonplayer Characters' Emotions and Social Relations in Games[J]. IEEE Transactions on Computational Intelligence & Ai in Games (S1943-0698), 2009, 1(4): 281-297.
[15] 吴奇, 申寻兵, 傅小兰. 微表情研究及其应用[J]. 心理科学进展, 2010, 18(9): 1359-1368.
Wu Qi, Shen Xunbing, Fu Xiaolan.Micro-expression research and its application[J]. Advances in Psychological Science, 2010, 18(9): 1359-1368.
[16] 拉米亚. 儿童情绪管理与性格培养-我要了解自己(青少年情绪管理手册) [M]. 北京: 化学工业出版社, 2013.
Lamia. Children's Emotion Management and Character Cultivation--I Want to Know About myself (Youth Emotion Management Handbook) [M]. Beijing: Chemical Industry Press, 2013.
[17] Was J, Gudowski B, Matuszyk P J.Social Distances Model of Pedestrian Dynamics[C]// Cellular Automata, International Conference on Cellular Automata, for Research and Industry, Acri 2006, Perpignan, France, September 20-23, 2006, Proceedings. DBLP, 2006: 492-501.
[18] Yoones A Sekhavat.Behavior Trees for Computer Games[J]. International Journal on Artificial Intelligence Tools (S0218-2130), 2017, 26(2): 1-27.
[19] 王洪源. Unity3D人工智能编程精粹 [M]. 北京: 清华大学出版社, 2014.
Wang Hongyuan.The essence of Unity3D artificial intellig- ence programming [M]. Beijing: Tsinghua University Press, 2014.
[20] Marchand E, Uchiyama H, Spindler F.Pose estimation for augmented reality: a hands-on survey[J]. IEEE Trans Vis Comput Graph (S1941-0506), 2016, 22(12): 2633-2651.
[21] Hasan H, Abdul-Kareem S.RETRACTED ARTICLE: Static hand gesture recognition using neural networks[J]. Artificial Intelligence Review (S0269-2821), 2014, 41(2): 147-181.
[22] Jin H, Chen Q, Chen Z, et al.Multi-LeapMotion sensor based demonstration for robotic refine tabletop object manipulation task[J]. CAAI Transactions on Intelligence Technology(S2468-2322), 2016, 1(1): 104-113.
[23] Yao N, Wang K, Cai Y, et al.Application of Somatosensory Sensor Kinect in Man-Machine Interaction Framework of Gesture Recognition[J]. Sensor Letters(S1546-1971), 2015, 13(12): 1050-1054.
[24] Pinelle D, Wong N, Stach T.Heuristic evaluation for games: usability principles for video game design[C]// Sigchi Conference on Human Factors in Computing Systems. ACM. Florence, Italy: CHI 2008 Proceedings·Game Zone, 2008: 1453-1462.