融合空间信息的运动想象脑电在线分类方法

doi:10.16182/j.issn1004731x.joss.21-0894

摘要/Abstract

摘要：

基于脑电图EEG(electroencephalogram)的脑机接口BCI(brain computer interface)系统可以帮助肢体运动障碍患者进行日常生活和康复训练。由于EEG信号的信噪比低、个体差异大，使得脑电信号的特征提取和分类存在精度和效率不高的问题，进而影响了在线BCI系统的广泛应用。提出一种融合空间信息的CNN(convolution neural network)用于MI(motor imagery)脑电信号的在线分类，结合运动想象ERD/ERS(event related desynchronization/event related synchronization)现象的对侧效应，对通道重新排序后的MI-EEG分别进行横向和纵向卷积，充分利用了MI-EEG中的空间信息，完成MI-EEG信号的实时采集和分类。结果分析表明：该方法具有一定的实时性和有效性，为在线MI-BCI系统的实现提供了基础。

关键词: 脑机接口, 卷积神经网络, 运动想象, 在线分类

Abstract:

EEG-based BCI system can help the daily life and rehabilitation training of limb movement disorders patients. Due to the low signal-to-noise ratio and large individual differences of EEG signals, the accuracy and efficiency of EEG feature extraction and classification are not high, which affects the wide application of online BCI system. A CNN with spatial information is proposed for the online classification of MI-EEG signals. The reordered MI-EEG is convolved horizontally and vertically respectively. With the contralateral effect of motor imagery ERD/ERS phenomenon, the spatial information in MI-EEG is fully utilized to achieve the real-time acquisition and classification of MI-EEG signals. Experimental results show that the proposed method is effectively performed in real time, which provide a basis for the implementation of online MI-BCI system.

Key words: brain-computer interface, convolutional neural network, motor imagery, online classification

中图分类号:

TP391.9

杨丰玮, 陈鹏, 郗凯, 蒲华林, 刘雪垠. 融合空间信息的运动想象脑电在线分类方法[J]. 系统仿真学报, 2023, 35(2): 254-267.

Fengwei Yang, Peng Chen, Kai Xi, Hualin Pu, Xueyin Liu. Online Classification Method for Motor Imagery EEG with Spatial Information[J]. Journal of System Simulation, 2023, 35(2): 254-267.

图/表 15

图1

图2

图3

图4

图5

图6

图7

表1

图8

图9

图10

表2

图11

图12

表3

参考文献 19

1	Wolpaw R, Birbaumer N, Heetderks W J, et al. Brain-computer Interface Technology: a Review of the First International Meeting[J]. IEEE Transactions on Rehabilitation Engineering(S1534-4320), 2000, 8(2): 164-173.
2	何群, 邵丹丹, 王煜文, 等. 基于多特征卷积神经网路的运动想象脑电信号分析及意图识别[J]. 仪器仪表学报, 2020, 41(1): 138-146.
	He Qun, Shao Dandan, Wang Yuwen, et al. Analysis and Intention Recognition of Motor Imagery EEG Signals Based on Multi-feature Convolutional Neural Network [J]. Chinese Journal of Scientific Instrument, 2020, 41(1): 138-146.
3	曹洋, 唐宏伟, 马艳妮, 等. 脑电采集系统的发展及其现代化应用[J]. 新技术新工艺, 2015(11): 83-89.
	Cao Yang, Tang Hongwei, Ma Yanni, et al. The Development and Modern Application of the EEG Signal Acquisition System[J]. New Technology & New Process, 2015(11): 83-89.
4	杨帮华, 李博. 基于脑机接口的康复训练系统[J]. 系统仿真学报, 2019, 31(2): 174-180.
	Yang Banghua, Li Bo. Rehabilitation Training System Based on Brain Computer Interface[J]. Journal of System Simulation, 2019, 31(2): 174-180.
5	陈超, 平尧, 郝斌, 等. 基于脑机接口技术的写字系统建模仿真与实现[J]. 系统仿真学报, 2018, 30(12): 4499-4505.
	Chen Chao, Ping Yao, Hao Bin, et al. Modeling, Simulation and Realization of Writing System Based on BCI Technology[J]. Journal of System Simulation, 2018, 30(12): 4499-4505.
6	Sandhya B, Shendkar C, Mahadevappa M. Single Channel Event Related (de) Synchronization (erd/ers) Analysis of Motor Execution in Stroke Affected Foot Drop Subjects [C]//2014 International Conference on Medical Imaging, m-Health and Emerging Communication Systems(MedCom). IEEE, 2014: 325-328.
7	Krizhevsky A, Sutskever I, Hinton G E. Imagenet Classification with Deep Convolutional Neural Networks[C]//Advances in Neural Information Processing Systems. Lake Tahoe, NV, 2012: 1097-1105.
8	Tabar Y R, Halici U. A Novel Deep Learning Approach for Classification of EEG Motor Imagery Signals[J]. Journal of Neural Engineering(S1741-2560), 2016, 14(1): 016003.
9	Bashivan P, Rish I, Yeasin M, et al. Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks[C]//International Conference on Learning Representations. 2015.
10	Lawhern V J, Solon A J, Waytowich N R, et al. EEGNet: a Compact Convolutional Neural Network for EEG-based Brain-computer Interfaces[J]. Journal of Neural Engineering(S1741-2560), 2018, 15(5): 056013.
11	Hou Y, Zhou L, Jia S, et al. A Novel Approach of Decoding EEG Four-class Motor Imagery Tasks via Scout ESI and CNN[J]. Journal of Neural Engineering (S1741-2560), 2020, 17(1): 016048.
12	杨帮华. 自发脑电脑机接口技术及脑电信号识别方法研究[D]. 上海: 上海交通大学, 2007.
	Yang Banghua. Technology of Brain-computer Interface Based on Spontaneous EEG and Research on Recognition Methods of EEG Signal[D]. Shanghai: Shanghai JiaoTong University, 2007.
13	Al-Saegh A, Dawwd S A, Abdul-Jabbar J M. Deep Learning for Motor Imagery EEG-based Classification: a Review[J]. Biomedical Signal Processing and Control (S1746-8094), 2021, 63: 102172.
14	佟歌. 基于深度学习的运动想象脑电信号分类算法研究[D]. 哈尔滨: 哈尔滨工程大学, 2018.
	Tong Ge. Research on Classification of Motor Imaginary EEG Signals Based on Deep Learning[D]. Harbin: Harbin Engineering University, 2018.
15	杜勇. 基于脑电控制的智能护理轮椅系统研究与设计[D]. 曲阜: 曲阜师范大学, 2019.
	Du Yong. Research and Design of Intelligent Nursing Wheelchair System Based on EEG Control[D]. Qufu: Qufu Normal University, 2019.
16	Amin S U, Alsulaiman M, Muhammad G, et al. Deep Learning for EEG Motor Imagery Classification Based on Multi-layer CNNs Feature Fusion[J]. Future Generation Computer Systems(S0167-739X), 2019, 101: 542-554.
17	深圳市诺比邻科技有限公司. 卷积神经网络的参数优化方法及系统: CN201610795257.X[P]. 2017-01-11.
	Shenzhen Nuobilin Technology Co., Ltd. Parameter Optimization Method and System of Convolutional Neural Network: CN201610795257.X[P]. 2017-01-11
18	王薇. 基于深度学习的脑电分类识别方法[D]. 南京: 南京邮电大学, 2020.
	Wang Wei. Classification and recognition method of EEG Based on deep learning[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2020.
19	Young B M, Nigogosyan Z, Nair V A, et al. Case Report: Post-stroke Interventional BCI Rehabilitation in an Individual with Preexisting Sensorineural Disability[J]. Frontiers in Neuroengineering(S1662-6443), 2014, 7: 18.

层类型	输出通道数	输出特征图尺寸	核大小	步幅	剪枝率	填充大小	参数数量
总计							24 082
输入层	1	32×512	—	—	—	—	0
卷积层1	16	32×256	1×23	1×2	—	0×11	368
批归一化层1	16	32×256	—	—	—	—	32
ELU激活层1	16	32×256	—	—	—	—	0
卷积层2	32	16×256	2×11	2×1	—	0×5	11 264
批归一化层2	32	16×256	—	—	—	—	64
ELU激活层2	32	16×256	—	—	—	—	0
平均池化层1	32	8×64	2×4	2×4	—	0	0
剪枝层1	32	8×64	—	—	0.25	—	0
卷积层3	32	4×64	2×5	2×1	—	0×2	10 240
批归一化层3	32	4×64	—	—	—	—	64
ELU激活层3	32	4×64	—	—	—	—	0
平均池化层2	32	2×16	2×4	2×4	—	0	0
剪枝层2	32	2×16	—	—	0.25	—	0
展平层	—	1024	—	—	—	—	0
全连接层	—	2	—	—	—	—	2 050
Softmax输出层	—	2	—	—	—	—	0

被试	左右手运动想象		左右脚运动想象		坐站运动想象
被试	准确率/%	Kappa系数	准确率/%	Kappa系数	准确率/%	Kappa系数
平均	94.60	0.893 9	94.57	0.892 9	94.58	0.893 1
P01	96.31	0.925 9	95.77	0.915 5	96.31	0.925 8
P02	98.12	0.962 4	96.77	0.933 9	96.18	0.943 6
P03	94.93	0.898 7	95.31	0.906 1	94.37	0.887 3
P04	93.90	0.876 4	95.22	0.904 0	95.69	0.913 8
P05	88.48	0.790 2	90.14	0.802 0	93.55	0.860 9
P06	94.93	0.897 9	96.24	0.924 7	94.74	0.891 9
P07	99.06	0.981 2	96.77	0.935 5	93.43	0.868 1
P08	95.64	0.889 3	96.38	0.947 5	97.59	0.941 7
P09	94.85	0.896 9	96.31	0.926 2	93.48	0.876 1
P10	90.43	0.835 0	89.00	0.780 4	90.91	0.817 6
P11	93.96	0.878 5	92.34	0.846 4	94.13	0.897 3

被试	准确率%	Kappa系数
平均	71.3	0.43
P07	67.5	0.35
P08	75.0	0.50

[1]	张立峰, 王会忍. 基于卷积神经网络及有限元仿真的电容层析成像图像重建[J]. 系统仿真学报, 2022, 34(4): 712-718.
[2]	敖邦乾, 杨莎, 令狐金卿, 叶振环. 基于级联神经网络疲劳驾驶检测系统设计[J]. 系统仿真学报, 2022, 34(2): 323-333.
[3]	王亚茹, 杨凯, 翟永杰, 郭聪彬, 赵文清, 苏杰. 基于人工图像数据扩充的输电线路绝缘子识别[J]. 系统仿真学报, 2022, 34(11): 2337-2347.
[4]	徐佳乐, 张海东, 赵东海, 倪晚成. 基于卷积神经网络的陆战兵棋战术机动策略学习[J]. 系统仿真学报, 2022, 34(10): 2181-2193.
[5]	郭业才, 王庆伟. 基于截断迁移与并行残差网络的调制识别算法[J]. 系统仿真学报, 2022, 34(09): 2009-2018.
[6]	赵瑛, 陆耀, 张健, 梁启弟, 龙炜. 基于深度神经网络的多视角人体动作识别[J]. 系统仿真学报, 2021, 33(5): 1019-1030.
[7]	仝卫国, 庞雪纯, 朱赓宏. 基于卷积神经网络的气液两相流流型识别方法[J]. 系统仿真学报, 2021, 33(4): 883-891.
[8]	马榕, 陈秋瑞, 张晗, 梅铮, 王锐, 魏伟. 基于单目深度估计的低功耗视觉里程计[J]. 系统仿真学报, 2021, 33(12): 3001-3011.
[9]	邓欣, 龙灿, 米建勋, 张博宪, 孙开伟, 王进. 基于多域特征与随机子空间集成的脑电分类[J]. 系统仿真学报, 2020, 32(9): 1787-1798.
[10]	黄欣, 方钰, 顾梦丹. 基于卷积神经网络的X线胸片疾病分类研究[J]. 系统仿真学报, 2020, 32(6): 1188-1194.
[11]	樊名鲁, 王艳, 纪志成. 基于特征生成的轴承不均衡数据故障诊断[J]. 系统仿真学报, 2020, 32(12): 2438-2448.
[12]	王红军, 黎邹邹, 邹湘军. 基于Adaboost与CNN的木材表面缺陷检测[J]. 系统仿真学报, 2019, 31(8): 1636-1645.
[13]	杨帮华, 李博. 基于脑机接口的康复训练系统[J]. 系统仿真学报, 2019, 31(2): 174-180.
[14]	卢裕秋, 孙金玉, 马世伟. 基于深度卷积神经网络的运动目标检测方法[J]. 系统仿真学报, 2019, 31(11): 2275-2280.
[15]	石祥滨, 钟健, 刘翠微, 刘芳, 张德园. 小样本高光谱遥感图像深度学习方法[J]. 系统仿真学报, 2018, 30(7): 2744-2752.