Online Classification Method for Motor Imagery EEG with Spatial Information

doi:10.16182/j.issn1004731x.joss.21-0894

Abstract

Abstract:

EEG-based BCI system can help the daily life and rehabilitation training of limb movement disorders patients. Due to the low signal-to-noise ratio and large individual differences of EEG signals, the accuracy and efficiency of EEG feature extraction and classification are not high, which affects the wide application of online BCI system. A CNN with spatial information is proposed for the online classification of MI-EEG signals. The reordered MI-EEG is convolved horizontally and vertically respectively. With the contralateral effect of motor imagery ERD/ERS phenomenon, the spatial information in MI-EEG is fully utilized to achieve the real-time acquisition and classification of MI-EEG signals. Experimental results show that the proposed method is effectively performed in real time, which provide a basis for the implementation of online MI-BCI system.

Key words: brain-computer interface, convolutional neural network, motor imagery, online classification

CLC Number:

TP391.9

Fengwei Yang, Peng Chen, Kai Xi, Hualin Pu, Xueyin Liu. Online Classification Method for Motor Imagery EEG with Spatial Information[J]. Journal of System Simulation, 2023, 35(2): 254-267.

Figures/Tables 15

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Table 1

Fig. 8

Fig. 9

Fig. 10

Table 2

Fig. 11

Fig. 12

Table 3

References 19

1	Wolpaw R, Birbaumer N, Heetderks W J, et al. Brain-computer Interface Technology: a Review of the First International Meeting[J]. IEEE Transactions on Rehabilitation Engineering(S1534-4320), 2000, 8(2): 164-173.
2	何群, 邵丹丹, 王煜文, 等. 基于多特征卷积神经网路的运动想象脑电信号分析及意图识别[J]. 仪器仪表学报, 2020, 41(1): 138-146.
	He Qun, Shao Dandan, Wang Yuwen, et al. Analysis and Intention Recognition of Motor Imagery EEG Signals Based on Multi-feature Convolutional Neural Network [J]. Chinese Journal of Scientific Instrument, 2020, 41(1): 138-146.
3	曹洋, 唐宏伟, 马艳妮, 等. 脑电采集系统的发展及其现代化应用[J]. 新技术新工艺, 2015(11): 83-89.
	Cao Yang, Tang Hongwei, Ma Yanni, et al. The Development and Modern Application of the EEG Signal Acquisition System[J]. New Technology & New Process, 2015(11): 83-89.
4	杨帮华, 李博. 基于脑机接口的康复训练系统[J]. 系统仿真学报, 2019, 31(2): 174-180.
	Yang Banghua, Li Bo. Rehabilitation Training System Based on Brain Computer Interface[J]. Journal of System Simulation, 2019, 31(2): 174-180.
5	陈超, 平尧, 郝斌, 等. 基于脑机接口技术的写字系统建模仿真与实现[J]. 系统仿真学报, 2018, 30(12): 4499-4505.
	Chen Chao, Ping Yao, Hao Bin, et al. Modeling, Simulation and Realization of Writing System Based on BCI Technology[J]. Journal of System Simulation, 2018, 30(12): 4499-4505.
6	Sandhya B, Shendkar C, Mahadevappa M. Single Channel Event Related (de) Synchronization (erd/ers) Analysis of Motor Execution in Stroke Affected Foot Drop Subjects [C]//2014 International Conference on Medical Imaging, m-Health and Emerging Communication Systems(MedCom). IEEE, 2014: 325-328.
7	Krizhevsky A, Sutskever I, Hinton G E. Imagenet Classification with Deep Convolutional Neural Networks[C]//Advances in Neural Information Processing Systems. Lake Tahoe, NV, 2012: 1097-1105.
8	Tabar Y R, Halici U. A Novel Deep Learning Approach for Classification of EEG Motor Imagery Signals[J]. Journal of Neural Engineering(S1741-2560), 2016, 14(1): 016003.
9	Bashivan P, Rish I, Yeasin M, et al. Learning Representations from EEG with Deep Recurrent-Convolutional Neural Networks[C]//International Conference on Learning Representations. 2015.
10	Lawhern V J, Solon A J, Waytowich N R, et al. EEGNet: a Compact Convolutional Neural Network for EEG-based Brain-computer Interfaces[J]. Journal of Neural Engineering(S1741-2560), 2018, 15(5): 056013.
11	Hou Y, Zhou L, Jia S, et al. A Novel Approach of Decoding EEG Four-class Motor Imagery Tasks via Scout ESI and CNN[J]. Journal of Neural Engineering (S1741-2560), 2020, 17(1): 016048.
12	杨帮华. 自发脑电脑机接口技术及脑电信号识别方法研究[D]. 上海: 上海交通大学, 2007.
	Yang Banghua. Technology of Brain-computer Interface Based on Spontaneous EEG and Research on Recognition Methods of EEG Signal[D]. Shanghai: Shanghai JiaoTong University, 2007.
13	Al-Saegh A, Dawwd S A, Abdul-Jabbar J M. Deep Learning for Motor Imagery EEG-based Classification: a Review[J]. Biomedical Signal Processing and Control (S1746-8094), 2021, 63: 102172.
14	佟歌. 基于深度学习的运动想象脑电信号分类算法研究[D]. 哈尔滨: 哈尔滨工程大学, 2018.
	Tong Ge. Research on Classification of Motor Imaginary EEG Signals Based on Deep Learning[D]. Harbin: Harbin Engineering University, 2018.
15	杜勇. 基于脑电控制的智能护理轮椅系统研究与设计[D]. 曲阜: 曲阜师范大学, 2019.
	Du Yong. Research and Design of Intelligent Nursing Wheelchair System Based on EEG Control[D]. Qufu: Qufu Normal University, 2019.
16	Amin S U, Alsulaiman M, Muhammad G, et al. Deep Learning for EEG Motor Imagery Classification Based on Multi-layer CNNs Feature Fusion[J]. Future Generation Computer Systems(S0167-739X), 2019, 101: 542-554.
17	深圳市诺比邻科技有限公司. 卷积神经网络的参数优化方法及系统: CN201610795257.X[P]. 2017-01-11.
	Shenzhen Nuobilin Technology Co., Ltd. Parameter Optimization Method and System of Convolutional Neural Network: CN201610795257.X[P]. 2017-01-11
18	王薇. 基于深度学习的脑电分类识别方法[D]. 南京: 南京邮电大学, 2020.
	Wang Wei. Classification and recognition method of EEG Based on deep learning[D]. Nanjing: Nanjing University of Posts and Telecommunications, 2020.
19	Young B M, Nigogosyan Z, Nair V A, et al. Case Report: Post-stroke Interventional BCI Rehabilitation in an Individual with Preexisting Sensorineural Disability[J]. Frontiers in Neuroengineering(S1662-6443), 2014, 7: 18.

层类型	输出通道数	输出特征图尺寸	核大小	步幅	剪枝率	填充大小	参数数量
总计							24 082
输入层	1	32×512	—	—	—	—	0
卷积层1	16	32×256	1×23	1×2	—	0×11	368
批归一化层1	16	32×256	—	—	—	—	32
ELU激活层1	16	32×256	—	—	—	—	0
卷积层2	32	16×256	2×11	2×1	—	0×5	11 264
批归一化层2	32	16×256	—	—	—	—	64
ELU激活层2	32	16×256	—	—	—	—	0
平均池化层1	32	8×64	2×4	2×4	—	0	0
剪枝层1	32	8×64	—	—	0.25	—	0
卷积层3	32	4×64	2×5	2×1	—	0×2	10 240
批归一化层3	32	4×64	—	—	—	—	64
ELU激活层3	32	4×64	—	—	—	—	0
平均池化层2	32	2×16	2×4	2×4	—	0	0
剪枝层2	32	2×16	—	—	0.25	—	0
展平层	—	1024	—	—	—	—	0
全连接层	—	2	—	—	—	—	2 050
Softmax输出层	—	2	—	—	—	—	0

被试	左右手运动想象		左右脚运动想象		坐站运动想象
被试	准确率/%	Kappa系数	准确率/%	Kappa系数	准确率/%	Kappa系数
平均	94.60	0.893 9	94.57	0.892 9	94.58	0.893 1
P01	96.31	0.925 9	95.77	0.915 5	96.31	0.925 8
P02	98.12	0.962 4	96.77	0.933 9	96.18	0.943 6
P03	94.93	0.898 7	95.31	0.906 1	94.37	0.887 3
P04	93.90	0.876 4	95.22	0.904 0	95.69	0.913 8
P05	88.48	0.790 2	90.14	0.802 0	93.55	0.860 9
P06	94.93	0.897 9	96.24	0.924 7	94.74	0.891 9
P07	99.06	0.981 2	96.77	0.935 5	93.43	0.868 1
P08	95.64	0.889 3	96.38	0.947 5	97.59	0.941 7
P09	94.85	0.896 9	96.31	0.926 2	93.48	0.876 1
P10	90.43	0.835 0	89.00	0.780 4	90.91	0.817 6
P11	93.96	0.878 5	92.34	0.846 4	94.13	0.897 3

被试	准确率%	Kappa系数
平均	71.3	0.43
P07	67.5	0.35
P08	75.0	0.50

[1]	Lifeng Zhang, Huiren Wang. Image Reconstruction of Electrical Capacitance Tomography Based on Convolutional Neural Network and Finite Element Simulation [J]. Journal of System Simulation, 2022, 34(4): 712-718.
[2]	Bangqian Ao, Sha Yang, Jinqing Linghu, zhenhuan Ye. Design of Fatigue Driving Detection System Based on Cascaded Neural Network [J]. Journal of System Simulation, 2022, 34(2): 323-333.
[3]	Yaru Wang, Kai Yang, Yongjie Zhai, Congbin Guo, Wenqing Zhao, Jie Su. Transmission Line Insulator Recognition Based on Artificial Images Data Expansion [J]. Journal of System Simulation, 2022, 34(11): 2337-2347.
[4]	Jiale Xu, Haidong Zhang, Donghai Zhao, Wancheng Ni. Tactical Maneuver Strategy Learning from Land Wargame Replay Based on Convolutional Neural Network [J]. Journal of System Simulation, 2022, 34(10): 2181-2193.
[5]	Yecai Guo, Qingwei Wang. Modulation Recognition Algorithm Based on Truncated Migration and Parallel ResNet [J]. Journal of System Simulation, 2022, 34(09): 2009-2018.
[6]	Zhao Ying, Lu Yao, Zhang Jian, Liang Qidi, Long Wei. Multi-view Human Action Recognition Based on Deep Neural Network [J]. Journal of System Simulation, 2021, 33(5): 1019-1030.
[7]	Tong Weiguo, Pang Xuechun, Zhu Genghong. Gas-liquid Two-phase Flow Pattern Recognition Method Based on Convolutional Neural Network [J]. Journal of System Simulation, 2021, 33(4): 883-891.
[8]	Huang Xin, Fang Yu, Gu Mengdan. Classification of Chest X-ray Disease Based on Convolutional Neural Network [J]. Journal of System Simulation, 2020, 32(6): 1188-1194.
[9]	Fan Minglu, Wang Yan, Ji Zhicheng. Fault Diagnosis for Bearings of Unbalanced Data Based on Feature Generation [J]. Journal of System Simulation, 2020, 32(12): 2438-2448.
[10]	Wang Hongjun, Li Zouzou, Zou Xiangjun. Wood Surface Defect Detection Based on Adaboost and CNN [J]. Journal of System Simulation, 2019, 31(8): 1636-1645.
[11]	Yang Banghua, Li Bo. Rehabilitation Training System Based on Brain Computer Interface [J]. Journal of System Simulation, 2019, 31(2): 174-180.
[12]	Lu Yuqiu, Sun Jinyu, Ma Shiwei. Moving Object Detection Based on Deep Convolutional Neural Network [J]. Journal of System Simulation, 2019, 31(11): 2275-2280.
[13]	Xia Min, Shen Maoyang, Wang Jianfeng, Wang Yangguang. Cloud Fraction of Satellite Imagery Based On Convolutional Neural Networks [J]. Journal of System Simulation, 2018, 30(5): 1623-1630.
[14]	Chen Chao, Ping Yao, Hao Bin, Xu Rui. Modeling, Simulation and Realization of Writing System Based on BCI Technology [J]. Journal of System Simulation, 2018, 30(12): 4499-4505.
[15]	Feng Jiankui, Jin Jing, Wang Bei, Niu Yugang, Wang Xingyu. Multi-tasks Channel Selection Algorithmic Modeling Based on SVM-RFE [J]. Journal of System Simulation, 2018, 30(12): 4506-4512.