基于IMU与sEMG混合信号的实时手势分类算法研究

doi:10.16182/j.issn1004731x.joss.21-1117

系统仿真学报 ›› 2023, Vol. 35 ›› Issue (2): 359-371.doi: 10.16182/j.issn1004731x.joss.21-1117

基于IMU与sEMG混合信号的实时手势分类算法研究

王涛¹(), 吴迎年¹^,²^,³(), 杨睿¹, 孙乐音¹

^1.北京信息科技大学自动化学院, 北京 100092
^2.高端装备智能感知与控制北京市国际科技合作基地, 北京 100192
^3.智能物联与协同控制研究所, 北京 100101

收稿日期:2021-11-02 修回日期:2021-12-24 出版日期:2023-02-28 发布日期:2023-02-16
通讯作者: 吴迎年 E-mail:928281098@qq.com;wuyingnian@126.com
作者简介:王涛(1997-)，男，硕士生，研究方向为人机交互与数字孪生控制。E-mail：928281098@qq.com
基金资助:
北京市自然科学基金(4202026);2021年国家级大学生创新创业训练项目(5102110803)

Research on Real-time Gesture Classification Algorithm Based on IMU and sEMG Mixed Signals

Tao Wang¹(), Yingnian Wu¹^,²^,³(), Rui Yang¹, Yueying Sun¹

^1.School of Automation, Beijing Information Science and Technology University, Beijing 100192, China
^2.Intelligent Perception and Control of High-end Equipment Beijing International Science and Technology Cooperation Base, Beijing 100192, China
^3.Intelligent Networked Things and Cooperative Control, Beijing 100101, China

Received:2021-11-02 Revised:2021-12-24 Online:2023-02-28 Published:2023-02-16
Contact: Yingnian Wu E-mail:928281098@qq.com;wuyingnian@126.com

摘要/Abstract

摘要：

为了提高表面肌电信号(surface electromyography，sEMG)的手势分类准确率，通过惯性测量单元(inertial measurement unit，IMU)与采集姿态信号与sEMG的混合信号，提出了GRU-BiLSTM双层网络的实时手势分类算法。第1层门控循环单元(gated recurrent unit，GRU)利用能量组合算子特征对混合信号进行突变点检测，定位运动态数据起始点；第2层双向长短时记忆循环神经网络(Bi-directional long short term memory，BiLSTM)使用能量核相图特征对运动态混合信号进行2个方向10种手势的分类。通过离线模型优化，分类算法识别时间低于40 ms，突变点检测精度88.7%以上，手势分类准确率为85%，信息传输率(information translate rate，ITR)达到89.9 bits/min，与基于机器学习的分类算法相比，在准确率与计算效率上具有优势。

关键词: 表面肌电信号, 惯性测量单元, 门控循环单元, 双向长短时记忆循环神经网络, 手势分类

Abstract:

In order to improve the gesture classification accuracy of surface electromyography (sEMG), the mixed signal of attitude and sEMG is collected by inertial measurement unit (IMU) and EMG sensor, and a GRU-BiLSTM double-layer network real-time gesture classification algorithm is proposed. The first layer of gated recurrent unit (GRU) detects the mutation point of the initial mixed signal though energy combination operator feature and locates the starting point of the dynamic data. The second layer Bi-directional long short term memory (BiLSTM) classifies the motion state mixed signal into 10 gestures in two directions though energy kernel phase map feature. Through offline model optimization, the recognition time of the online GRU-BiLSTM double-layer classifier algorithm is less than 40 ms, the detection accuracy of mutation points is over 88.7%, the accuracy of gesture classification is 85%, and the information transmission rate (ITR) reaches 89.9 bits/min. Compared with machine learning-based classification algorithms, the algorithm has advantages in accuracy and computational efficiency.

Key words: surface electromyography(sEMG), inertial measurement unit(IMU), gated recurrent unit(GRU), Bi-directional long short term memory(BiLSTM), gesture classification

中图分类号:

TP249

王涛, 吴迎年, 杨睿, 孙乐音. 基于IMU与sEMG混合信号的实时手势分类算法研究[J]. 系统仿真学报, 2023, 35(2): 359-371.

Tao Wang, Yingnian Wu, Rui Yang, Yueying Sun. Research on Real-time Gesture Classification Algorithm Based on IMU and sEMG Mixed Signals[J]. Journal of System Simulation, 2023, 35(2): 359-371.

图/表 12

图1

图2

图3

表1

能量核训练特征

能量核相图特征	维度	参数范围
椭圆曲线参数	$α = [a, b, c, d, e] T$	[10, 100]
水平倾角/(°)	$θ$	±2~8
能量核幅值	[smallest_eigenval; smallest_eigenvec]	[10⁵，10⁸]
相位散点率/%	Phase scatter rate	1.8~7.6

表1

图4

图5

表2

图6

图7

表3

图8

表4

参考文献 28

1	张晓俊, 李长勇. 基于深度学习多特征融合的手势分割识别算法[J]. 济南大学学报(自然科学版), 2022, 36(3): 286-291.
	Zhang Xiaojun, Li Changyong. Gesture Segmentation and Recognition Algorithm Based on Deep Learning Multi-feature Fusion[J]. Journal of University of Jinan(Science and Technology), 2022, 36(3): 286-291.
2	张维, 林泽一, 程坚, 等. 动态手势理解与交互综述[J]. 软件学报, 2021, 32(10): 3051-3067.
	Zhang Wei, Lin Zeyi, Cheng Jian, et al. Survey of Dynamic Hand Gesture Understanding and Interaction [J]. Journal of Software, 2021, 32(10): 3051- 3067.
3	刘光达, 董梦坤, 许蓝予, 等. 手臂疲劳时表面肌电信号特征[J]. 科学技术与工程, 2021, 21(25): 10690-10696.
	Liu Guangda, Dong Mengkun, Xu Lanyu, et al. Characteristic of Surface Electromyography Signal in Arm Fatigue[J]. Science Technology and Engineering, 2021, 21(25): 10690-10696.
4	Duan Feng, Yang Yikang. Recognizing Missing Electromyography Signal by Data Split Reorganization Strategy and Weight-Based Multiple Neural Network Voting Method[J]. IEEE Transactions on Neural Networks and Learning Systems(S2162-2388), 2021,33(5): 2070- 2079.
5	Lee S J, Hong K R, Lee T W, et al. A Study on the Reliability Comparison of Median Frequency and Spike Parameter and the Improved Spike Detection Algorithm for the Muscle Fatigue Measurement[J]. Transactions-Korean Institute of Electrical Engineers D(S1975-8359), 2004, 53(5): 380-388.
6	Hossen A, Muthuraman M, Raethjen J, et al. Discrimination of Parkinsonian Tremor from Essential Tremor by Implementation of a Wavelet-Based Soft-Decision Technique on EMG and Accelerometer Signals[J]. Biomedical Signal Processing and Control(S1746-8094), 2010, 5(3): 181-188.
7	Loconsole C, Cascarano G D, Brunetti A, et al. A Model-Free Technique Based on Computer Vision and sEMG for Classification in Parkinson's Disease by Using Computer-Assisted Handwriting Analysis[J]. Pattern Recognition Letters(S0167-8655), 2019, 121: 28-36.
8	Zhou Y, Bi Z, Ji M, et al. A Data-Driven Volitional EMG Extraction Algorithm During Functional Electrical Stimulation with Time Variant Parameters[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering(S1534-4320), 2020, 28(5): 1069-1080.
9	Zhang Guangyi, Ali Etemad. Capsule Attention for Multimodal EEG-EOG Representation Learning with Application to Driver Vigilance Estimation[J] IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 1138-1149.
10	Qi J, Jiang G, Li G, et al. Surface EMG Hand Gesture Recognition System Based on PCA and GRNN[J]. Neural Computing and Applications(S1433-3058), 2020, 32(10): 6343-6351.
11	Rabin N, Kahlon M, Malayev S, et al. Classification of Human Hand Movements Based on EMG signals Using Nonlinear Dimensionality Reduction and Data Fusion Techniques[J]. Expert Systems with Applications(S0957-4174), 2020, 149: 113281.
12	李虹飞, 胡满红. 采摘机器人SEMG手势识别研究—基于RNN循环神经网络[J]. 农机化研究, 2022, 44(5): 212-216.
	Li Hongfei, Hu Manhong. SEMG Gesture Recognition of Picking Robot Based on RNN Cyclic Neural Network[J]. Journal of Agricultural Mechanization Research, 2022, 44(5): 212-216.
13	李自由, 赵新刚, 张弼, 等. 基于表面肌电的意图识别方法在非理想条件下的研究进展[J]. 自动化学报, 2021, 47(5): 955-969.
	Li Ziyou, Zhao Xingang, Zhang Bi, et al. Review of sEMG-Based Motion Intent Recognition Methods in Non-ideal Conditions[J]. Acta Automatica Sinica, 2021, 47(5): 955-969.
14	李建亮, 王从庆, 展文豪, 等. 基于表面肌电信号的人机交互软件设计[J]. 计算机测量与控制, 2021, 29(7): 171-174, 179.
	Li Jianliang, Wang Congqing, Zhan Wenhao, et al. Human-Computer Interaction Software Design Based on Surface EMG Signal[J]. Computer Measurement &Control, 2021, 29(7): 171-174, 179.
15	尤波, 陶守通, 黄玲, 等. 基于MATLAB和ADAMS的肌电假手联合仿真[J]. 系统仿真学报, 2017, 29(5): 957-964.
	You Bo, Tao Shoutong, Huang Ling, et al. Coordinated Simulation of EMG Prosthetic Hand Based on MATLAB and ADAMS[J]. Journal of System Simulation, 2017, 29(5): 957-964.
16	王远明, 张珺, 秦远辉, 等. 基于多特征融合的指挥手势识别方法研究[J]. 系统仿真学报, 2019, 31(2): 346-352.
	Wang Yuanming, Zhang Jun, Qin Yuanhui, et al. Gesture Recognition Method Based on Multi-feature Fusion[J]. Journal of System Simulation, 2019, 31(2): 346-352.
17	宋佳强, 裴晓敏, 赵强, 等. 基于组合能量特征的表面肌电信号手势识别算法[J]. 传感器与微系统, 2020, 39(6): 139-142.
	Song Jiaqiang, Pei Xiaomin, Zhao Qiang, et al. Surface EMG Signal Hand Motion Recognition Algorithm Based on Combinated Energy Characteristics[J]. Transducer and Microsystem Technologies, 2020, 39(6): 139-142.
18	蒋明峰, 王洪. 基于小波分析与神经网络相结合的表面肌电信号识别的研究[J]. 生物医学工程研究, 2005(1): 50-52.
	Jiang Mingfeng, Wang Hong. The Classification of Surface EMG Signal Based on Wavelet Transform and Neural Networks[J]. Journal of Biomedical Engineering Research, 2005(1): 50-52.
19	许留凯, 张克勤, 徐兆红, 等. 基于表面肌电信号能量核相图的卷积神经网络人体手势识别算法[J]. 生物医学工程学杂志, 2021, 38(4): 621-629.
	Xu Liukai, Zhang Keqin, Xu Zhaohong, et al. Convolutional Neural Network Human Gesture Recognition Algorithm Based on Phase Portrait of Surface Electromyography Energy Kernel[J]. Journal of Biomedical Engineering, 2021, 38(4): 621-629.
20	Guo Y, Gravina R, Gu X, et al. EMG-Based Abnormal Gait Detection and Recognition[C]//2020 IEEE International Conference on Human-Machine Systems (ICHMS). Rome: IEEE, 2020: 1-6.
21	冯媛硕, 宋吉江. 基于小波变换的信号特征与突变点检测算法研究[J]. 曲阜师范大学学报(自然科学版), 2015, 41(1): 76-80.
	Feng Yuanshuo, Song Jijiang. Research on the Wavelet-Based Algorithms for Signal Feature and Singularity Detection[J]. Journal of Qufu Normal University(Natural Science Edition), 2015, 41(1): 76-80.
22	张丹, 余义德, 王红萍. 基于时频滤波技术的时间相位信号检测方法[J]. 无线电工程, 2016, 46(12): 31-34, 42.
	Zhang Dan, Xu Yide, Wang Hongping. Time Phase Detection Method of Time-Frequency Filter Technique[J]. Radio Engineering, 2016, 46 (12): 31-34, 42.
23	吴娜. 基于表面肌电信号的双通道实时手势识别系统研究[D].杭州: 浙江大学, 2021.
	Wu Na. Research on Real-Time Gesture Recognition System Based on Dual-Channel sEMGs Signal[D]. Hangzhou: Zhejiang University, 2021.
24	石欣, 范智瑞, 张杰毅, 等. 基于LMS-随机森林的肌电信号下肢动作快速分类[J]. 仪器仪表学报, 2020, 41(6): 218-224.
	Shi Xin, Fan Zhirui, Zhang Jieyi, et al. Rapid Classification of Lower Limb Movements of EMG Signals Based on LMS-Random Forest[J]. Chinese Journal of Scientific Instrument, 2020, 41(6): 218-224.
25	叶科学. 基于表面肌电信号的脊柱侧弯症辅助诊断技术研究[D]. 南京: 南京航空航天大学, 2020.
	Ye Kexue. Research on Scoliosis Assisted Diagnosis Technology Based on sEMG[D]. Nanjing: Nanjing University of Aeronautics and Astronautics, 2020.
26	温倩. 基于表面肌电信号的人体行走模式识别[D]. 天津: 河北工业大学, 2015.
	Wen Qian. Walking Pattern Recognition Based on Surface Electromyography[D]. Tianjin: Hebei University of Technology, 2015.
27	Ketykó I, Kovács F, Varga K Z. Domain Adaptation for Semg-Based Gesture Recognition with Recurrent Neural Networks[C]//International Joint Conference on Neural Networks. Budapest: IEEE, 2019: 1-7.
28	李沿宏, 江茜, 邹可, 等. 融合注意力机制的多流卷积肌电手势识别网络[J]. 计算机应用研究, 2021, 38(11): 3258-3263.
	Li Yanhong, Jiang Qian, Zou Ke, et al. Multi-stream Convolutional Myoelectric Gesture Recognition Networks Fusing Attentional Mechanisms[J]. Application Research of Computers, 2021, 38(11): 3258-3263.

	不同时间窗	GRU平均识别精度/%	平均识别时间/ms	突变点检测误差率/%
水平方向	100 ms时间窗	90.7	2.81	1.44
	200 ms时间窗	92.3	2.96	1.08
	0.1~0.5 s	93.8	3.16	1.26
	0.1~1 s	89.2	3.22	2.71
垂直方向	100 ms时间窗	91.4	2.89	1.53
	200 ms时间窗	91.8	3.06	1.43
	0.1~0.5 s	90.7	3.77	1.76
	0.1~1 s	87.6	3.48	2.01

滑动窗采样输入段/s	BiLSTM网络
滑动窗采样输入段/s	单次动作准确率/%	随机动作准确率/%	平均识别时间/ms
0~0.4	80.4	78.7	28.6
0.1~0.5	86.7	80.4	30.2
0.2~0.6	83.6	77.3	34.8
0.3~0.7	80.7	71.9	28.9
0.4~0.8	78.2	69.3	24.3
0.5~0.9	67.1	57.1	21.1
0.6~1.0	58.6	54.9	20.7

分类方法	识别目标个数N/个	平均识别准确率P/%	识别时间t/s	信息传输率ITR/ (bits/min)
GRU-BiLSTM	20	85.4	2.13	89.90
BiLSTM	20	79.1	2.56	63.15
LMS-随机森林	4	97.3	2.14	49.85
SVM	6	97.5	2.67	52.98
Fisher判别	3	95.0	1.75	42.81
KNN	5	94.0	1.82	61.80
GengNet	12	87.3	3.01	51.76
MSCAnet	52	86.9	2.87	91.92

[1]	孙宁可, 王艳, 纪志成. 基于深度自编码器的电力能耗异常检测方法[J]. 系统仿真学报, 2022, 34(12): 2557-2565.
[2]	孙伟, 王野, 李瑞豹, 杨聃. 基于IMU/GPS的安卓无缝定位系统设计[J]. 系统仿真学报, 2018, 30(9): 3571-3577.
[3]	尤波, 陶守通, 黄玲, 李勃君, 赵汗青. 基于MATLAB和ADAMS的肌电假手联合仿真[J]. 系统仿真学报, 2017, 29(5): 957-964.
[4]	高哲, 李擎, 李超, 刘宁. 行人自主导航定位的IEKF-SWCS方法[J]. 系统仿真学报, 2015, 27(9): 1944-1950.
[5]	张瑒, 许林, 孙广毅. 基于RGB-D摄像机与IMU数据融合的动作捕捉系统[J]. 系统仿真学报, 2015, 27(10): 2582-2588.

基于IMU与sEMG混合信号的实时手势分类算法研究

Research on Real-time Gesture Classification Algorithm Based on IMU and sEMG Mixed Signals

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 28

相关文章 5

编辑推荐

Metrics

本文评价