基于VR的言语失用症康复训练系统

doi:10.16182/j.issn1004731x.joss.19-FZ0334

系统仿真学报 ›› 2019, Vol. 31 ›› Issue (11): 2344-2349.doi: 10.16182/j.issn1004731x.joss.19-FZ0334

基于VR的言语失用症康复训练系统

焦欢欢^1,3, 潘志庚^2,3,4, 徐舒畅¹, 袁庆曙¹

1. 杭州师范大学信息科学与工程学院,浙江杭州 311121;
2. 杭州师范大学阿里巴巴商学院,浙江杭州 311121;
3. 杭州师范大学虚拟现实与智能系统研究院,浙江杭州 311121;
4. 广州玖的数码科技有限公司,广东广州 510000

收稿日期:2019-05-30 修回日期:2019-07-16 出版日期:2019-11-10 发布日期:2019-12-13
作者简介:焦欢欢(1993-),女,安徽宿州,硕士生,研究方向为虚拟现实与人机交互。
基金资助:
国家重点研发计划(2017YFB1002803),广州创新创业领军团队项目(CXLJTD-201609),浙江省健康智慧厨房系统集成重点实验室开放课题(2017F04)

Rehabilitation Training System of VR-based Speech Apraxia

Jiao Huanhuan^1,3, Pan Zhigeng^2,3,4, Xu Shuchang¹, Yuan Qingshu¹

1. School of Information Science and Engineering, Hangzhou Normal University, Hangzhou 311121, China;
2.Alibaba Business School, Hangzhou Normal University, Hangzhou 311121, China;
3. Virtual Reality and Intelligent Systems Research, Hangzhou Normal University, Hangzhou 311121, China;
4. Guangzhou nine Digital Technology Co., Ltd., Guangzhou 510000, China

Received:2019-05-30 Revised:2019-07-16 Online:2019-11-10 Published:2019-12-13

摘要/Abstract

摘要： 针对传统言语失用症康复训练中患者主动训练积极性逐渐减弱的问题,设计了一种基于虚拟现实的言语失用症康复训练系统。采用虚拟人嘴型动画模拟汉语中26个音节发音嘴型,并用于学习模块。利用孤立字语音识别算法识别26个音节,提高识别的准确率。系统还用训练模块、场景模块辅助患者康复训练。结果表明：系统中的虚拟人发音过程能够逼真地模拟真实发音器官的运动,对提高患者的语言能力具有积极作用。

关键词: 虚拟现实, 言语失用症, 虚拟人嘴型动画, 孤立字语音识别

Abstract: Aiming at the problem that the enthusiasm of active training in patients with traditional speech apraxia rehabilitation is gradually weakened, a virtual reality-based rehabilitation training system for speech apraxia is designed. The system simulates the 26 monosyllabic pronunciation mouths shapes made by virtual human in Chinese and uses it for the learning module. It is proposed to identify 26 monosyllabic words by using the isolated word speech recognition algorithm, which improves the accuracy of speech recognition. The system includes a training module and a scene module for assisting the patient's pronunciation rehabilitation training. The evaluation results show that the virtual human articulation process in the system can realistically simulate the movement of real articulation organs, which plays a positive role in improving the patients' language ability.

Key words: Virtual reality, Speech Aphasia, Virtual mouth animation, Speech recognition of isolated word

中图分类号:

TP391.9

焦欢欢, 潘志庚, 徐舒畅, 袁庆曙. 基于VR的言语失用症康复训练系统[J]. 系统仿真学报, 2019, 31(11): 2344-2349.

Jiao Huanhuan, Pan Zhigeng, Xu Shuchang, Yuan Qingshu. Rehabilitation Training System of VR-based Speech Apraxia[J]. Journal of System Simulation, 2019, 31(11): 2344-2349.

参考文献

[1] 汪洁. 运动性失语症口语表达障碍的语言治疗[J]. 现代康复, 2001(19): 20-22.
Wang Jie.Speech therapy for motor aphasia with speech disorders[J]. Modern Rehabilitation, 2001(19): 20-22.
[2] 王芹, 陶静, 刘斐雯, 等. 失用症的分类及治疗研究现状[J]. 实用医学杂志, 2016, 32(5): 689-692.
Wang Qin, Tao Jing, Liu Feiwei, et al.Classification and treatment of apraxia[J]. Journal of Practical Medicine, 2016, 32(5): 689-692.
[3] 潘翠兰, 毛韶丽. 言语失用症康复训练报告[J]. 现代康复, 2000(6): 851.
Pan Cuilan, Mao Shaoli.A report on rehabilitation training for speech apraxia[J]. Modern Rehabilitation, 2000(6): 851.
[4] Ogourtsova T, Archambault P, Lamontagne A.Post-stroke unilateral spatial neglect: virtual reality-based navigation and detection tasks reveal lateralized and non-lateralized deficits in tasks of varying perceptual and cognitive demands[J]. J Neuroeng Rehabil (S1743-0003), 2018, 15(1): 34.
[5] Maier M, Banuelos N, Ballester B, et al.Conjunctive rehabilitation of multiple cognitive domains for chronic stroke patients in virtual reality[C]. International Conference on Rehabilitation Robotics. New Jersey: IEEE, 2017: 947-952.
[6] 王辉, 吴吉生. 虚拟现实训练对认知障碍的脑卒中偏瘫患者的影响[J]. 中国康复, 2017, 32(4): 299-301.
Wang hui, Wu Jisheng. Effect of virtual reality training on lower extremity function and ADL in hemiplegics with cognitive impairment after stroke[J]. China Rehabilitation, 2017, 32(4): 299-301.
[7] 刘婷婷, 刘箴, 钱平安, 等. 虚拟现实在特殊人群康复中的应用研究[J]. 系统仿真学报, 2018, 30(9): 3229-3237.
Liu Tingting, Liu Zhen, Qian Pingan, et al.Application of virtual reality in rehabilitation of special populations[J]. Journal of System Simulation, 2018, 30(9): 3229-3237.
[8] 祝路, 吴春凤, 叶卉. 综合康复护理在脑卒中后言语障碍患者中的应用效果观察[J]. 中国医学创新, 2019, 16(12): 80-83.
Zhu Lu, Wu Chunfeng, Ye Hui.Application effect of comprehensive rehabilitation nursing in patients with post-stroke speech impairment[J]. Chinese Medical Innovation, 2019, 16(12): 80-83.
[9] 孙咏虹, 吴冰洁. 丰富康复训练与神经可塑性[J]. 中国康复理论与实践, 2010, 16(7): 635-637.
Sun Yonghong, Wu Bingjie.To enrich rehabilitation training and neuroplasticity[J]. Rehabilitation Theory and Practice in China, 2010, 16(7): 635-637.
[10] 王进嘉, 张喆. 虚拟现实技术在医疗领域应用的发展建议[J]. 电信网技术, 2017(2): 14-18.
Wang Jinjia, Zhang Zhe.Development suggestions on application of virtual reality technology in the medical field[J]. Telecommunication Network Technology, 2017(2): 14-18.
[11] Miller P.Learning with a missing sense: what can we learn from the interaction of a deaf child with a turtle[J]. American Annals of the Deaf(S0002-726X), 2009, 154(1): 71-82.
[12] Eden S.Virtual intervention to improve storytelling ability among deaf and hard of hearing children[J]. European Journal of Special Needs Education (S0885-6257), 2014, 29(3): 370-386.
[13] 杨雪, 雷江华. 听障儿童基于虚拟现实技术的干预研究[J]. 昆明学院学报, 2019, 41(2): 39-44.
Yang Xue, Lei Jianghua.A review of virtual reality research into interventions for children with hearing impairment[J]. Journal of Kunming University, 2019, 41(2): 39-44.
[14] Rathinavelu A, Thiagarajan H, Rajkumar A.Three dimensional articulator model for speech acquisition by children with hearing loss[C]. Proceedings of 4th International Conference on Universal Access in Human Computer Interaction. Berlin: Springer, 2007, 4554: 786-794.
[15] Rathinavelu A, Yuvaraj G.Data visualization model for speech articulators[C]. Proceedings of AICERA. New Jersey: IEEE, 2011: 155-159.
[16] Bälter O, Engwall O, Öster A M, et al.Wizard of Oz test of ARTUR: a computer-based speech training system with articulation correction[C]. Proceedings of 7th International ACM SIGACCESS Conference on Computers and Accessibility. New York: ACM, 2005: 36-43.
[17] Moccozet L, Laurent N.Dirichlet free-form deformations and their application to hand simulation[C]. Proceedings of the Computer Animation. New Jersey: IEEE, 1997: 93-102.
[18] Adams J. Direct X.高级动画制作[M]. 刘刚译. 重庆: 重庆大学出社, 2005.
Adams J.Direct X. Advanced animation[M]. Liu Gang, Trans. Chongqing: Press of Chongqing University, 2005.
[19] 周炳良, 邓立新, 洪民江. 一种新的基于DTW的孤立词语音识别算法[J]. 计算机技术与发展, 2018, 28(4): 127-131.
Zhou Bingliang, Deng Lixin, Hong Minjiang.A novel isolated word algorithm of speech recognition based on DTW[J]. Conputer Technology and Development, 2018, 28(4): 127-131.

基于VR的言语失用症康复训练系统

Rehabilitation Training System of VR-based Speech Apraxia

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	周晓光, 朱鹏, 张源原, 路欢, 周媛. 六自由度跳伞模拟训练系统设计与实现[J]. 系统仿真学报, 2022, 34(6): 1320-1329.
[2]	许世健, 赵丹, 苏铖宇, 王赋攀, 张晓蓉, 王昉, 吴亚东. 面向流场可视化的沉浸式虚拟现实交互系统研究[J]. 系统仿真学报, 2022, 34(5): 1160-1172.
[3]	孙佳文, 任鸿翔, 肖方兵, 蒋效彬. 基于虚拟现实的整船仿真训练平台[J]. 系统仿真学报, 2021, 33(9): 2243-2251.
[4]	魏士松, 周正东, 章栩苓, 毛玲, 贾峻山, 刘传乐. 基于桌面虚拟现实技术的航天器虚拟维修训练系统[J]. 系统仿真学报, 2021, 33(6): 1358-1363.
[5]	贾春洋, 邹湘军, 李锦慧, 曾泽钦, 王杰. 虚拟环境下的复杂变速箱交互式装配仿真平台[J]. 系统仿真学报, 2020, 32(9): 1736-1743.
[6]	郭园, 童倩倩, 郑宇铠, 王子琦, 张玉茹, 王党校. 融合多元触觉和沉浸式视觉的可移植VR软件框架[J]. 系统仿真学报, 2020, 32(7): 1385-1392.
[7]	文国军, 夏雨, 王玉丹, 胡自飞, 吴丹. 基于Unity3D的水平定向钻机虚拟实训系统设计[J]. 系统仿真学报, 2020, 32(5): 801-807.
[8]	刘畅, 贾金原, 陆一凡, 张乾, 赵磊. 光图树：Web3D动态场景全局光照的协同实时渲染[J]. 系统仿真学报, 2019, 31(8): 1591-1604.
[9]	谭力恒, 马强, 蒋秉川, 李锋. 面向头盔式虚拟现实的OsgEarth立体视觉生成方法[J]. 系统仿真学报, 2019, 31(7): 1290-1299.
[10]	祁彬斌, 朱学芳, 范丽鹏. 基于触觉交互的三维弹性物体形变仿真算法[J]. 系统仿真学报, 2019, 31(11): 2296-2305.
[11]	闫丰亭, 史志才, 苏前敏, 董婧, 陈珏, 石蕴玉, 汤显. 物联网与虚拟现实融合的智能火灾营救机制[J]. 系统仿真学报, 2019, 31(11): 2306-2322.
[12]	张波, 孙俊, 商蕾, 徐弘基, 杨诚. 基于HMD的船舶虚拟消防训练系统设计及实现[J]. 系统仿真学报, 2019, 31(1): 43-52.
[13]	刘婷婷, 刘箴, 钱平安, 宣荣荣, 王瑾, 柴艳杰. 虚拟现实在特殊人群康复中的应用研究[J]. 系统仿真学报, 2018, 30(9): 3229-3237.
[14]	朱晓强, 潘虹艺, 徐浩, 宋磊, 朱梦尧, 王向阳. 头戴设备VR环境下光滑加权等距面交互建模[J]. 系统仿真学报, 2018, 30(7): 2459-2464.
[15]	黄晨曦, 郝泳涛, 邢浩威, 陈飞. 基于FBS的虚拟血管支架置入模型[J]. 系统仿真学报, 2018, 30(7): 2622-2629.