基于自动睡眠分期的多模态残差时空融合模型

doi:10.16182/j.issn1004731x.joss.23-0588

摘要/Abstract

摘要：

高精度的睡眠分期对于正确评定睡眠情况起到了至关重要的作用。针对现有的卷积网络无法获取生理信号拓扑特征的问题，提出了一种基于多模态残差时空融合的睡眠分期算法。利用短时傅里叶变换和自适应图卷积获取时频图像和时空图像，将其转换为高维的特征向量；通过时频特征和时空特征提取模块实现特征信息流的轻量化交互；使用特征增强融合模块融合特征信息，输出睡眠分期结果。结果表明：该模型具有较高的准确率，在ISRUC-S3数据集上整体准确率为85.3%，F1分数为83.8%，Cohen's kappa为81%，N1阶段准确率达到69.81%。ISRUC-S1数据集上的实验证明了模型的普遍性。

关键词: 睡眠分期, 多视图融合, 图卷积网络, 深度学习, 脑电信号

Abstract:

Highly accurate sleep staging plays a crucial role in correctly assessing sleep conditions. Aiming at the problem that the existing convolutional network cannot obtain the topological characteristics of physiological signals, a sleep staging algorithm based on multi-modal residual spatio-temporal fusion is proposed. Time-frequency images and spatio-temporal images are obtained using short-time Fourier transform and adaptive map convolution, which are converted into high-dimensional feature vectors; lightweight interaction of feature information flow is realized through time-frequency feature and spatio-temporal feature extraction modules; the feature enhancement fusion module fuses feature information to outputs sleep staging results. The results show that the model has a high accuracy. On the ISRUC-S3 data set, the overall accuracy is 85.3%, the F1 score is 83.8%, Cohen’s kappa is 81%, and the N1 stage accuracy reaches 69.81%. Experiments on the ISRUC-S1 dataset demonstrate the generality of the model.

Key words: sleep staging, multi-view fusion, graph convolutional network, deep learning, electroencephalogram

中图分类号:

TP391

郭业才,仝爽 . 基于自动睡眠分期的多模态残差时空融合模型[J]. 系统仿真学报, 2024, 36(9): 2065-2074.

Guo Yecai,Tong Shuang . A Multimodal Residual Spatial-temporal Fusion Model Based on Automatic Sleep Classification[J]. Journal of System Simulation, 2024, 36(9): 2065-2074.

图/表 13

图 1

图2

图3

图4

表1

表2

图5

表3

表4

图6

图7

图8

图9

参考文献 26

1	Kohn T P, Kohn J R, Haney N M, et al. The Effect of Sleep on Men's Health[J]. Translational Andrology and Urology, 2020, 9(S2): S178-S185.
2	Guillot Antoine, Thorey Valentin. RobustSleepNet: Transfer Learning for Automated Sleep Staging at Scale[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 1441-1451.
3	Wolpert E A. A Manual of Standardized Terminology, Techniques and Scoring System for Sleep Stages of Human Subjects[J]. Archives of General Psychiatry, 1969, 20(2): 246-247.
4	Xiao Meng, Yan Hong, Song Jinzhong, et al. Sleep Stages Classification Based on Heart Rate Variability and Random Forest[J]. Biomedical Signal Processing and Control, 2013, 8(6): 624-633.
5	Bansal Aayushi, Sharma Rewa, Kathuria Mamta. A Systematic Review on Data Scarcity Problem in Deep Learning: Solution and Applications[J]. ACM Computing Surveys, 2022, 54(10S): 208.
6	Md Mosheyur Rahman, Mohammed Imamul Hassan Bhuiyan, Ahnaf Rashik Hassan. Sleep Stage Classification Using Single-channel EOG[J]. Computers in Biology and Medicine, 2018, 102: 211-220.
7	Jameer Basha A, Saravana Balaji B, Poornima S, et al. Support Vector Machine and Simple Recurrent Network Based Automatic Sleep Stage Classification of Fuzzy Kernel[J]. Journal of Ambient Intelligence and Humanized Computing, 2021, 12(6): 6189-6197.
8	Pei Wei, Li Yan, Siuly S, et al. A Hybrid Deep Learning Scheme for Multi-channel Sleep Stage Classification[J]. Computers, Materials & Continua, 2022, 71(1): 889-905.
9	Phan H, Andreotti F, Cooray N, et al. SeqSleepNet: End-to-end Hierarchical Recurrent Neural Network for Sequence-to-sequence Automatic Sleep Staging[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2019, 27(3): 400-410.
10	Wenpeng Neng, Lu Jun, Xu Lei. CCRRSleepNet: A Hybrid Relational Inductive Biases Network for Automatic Sleep Stage Classification on Raw Single-channel EEG[J]. Brain Sciences, 2021, 11(4): 456.
11	Supratak A, Dong Hao, Wu Chao, et al. DeepSleepNet: A Model for Automatic Sleep Stage Scoring Based on Raw Single-channel EEG[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2017, 25(11): 1998-2008.
12	Perslev Mathias, Darkner Sune, Kempfner Lykke, et al. U-sleep: Resilient High-frequency Sleep Staging[J]. npj Digital Medicine, 2021, 4(1): 72.
13	Jia Z, Lin Y, Wang J, et al. Salientsleepnet: Multimodal Salient Wave Detection Network for Sleep Staging[C]//2021 30th International Joint Conference on Artificial Intelligence (IJCAI). Montreal: Morgan Kaufmann, 2021: 1-10.
14	Kong G, Li C, Peng H, et al. EEG-based Sleep Stage Classification via Neural Architecture Search[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31: 1075-1085.
15	Zhou Jie, Cui Ganqu, Hu Shengding, et al. Graph Neural Networks: A Review of Methods and Applications[J]. AI Open, 2020, 1: 57-81.
16	Zhang Si, Tong Hanghang, Xu Jiejun, et al. Graph Convolutional Networks: A Comprehensive Review[J]. Computational Social Networks, 2019, 6(1): 11.
17	Jia Ziyu, Lin Youfang, Wang Jing, et al. GraphSleepNet: Adaptive Spatial-temporal Graph Convolutional Networks for Sleep Stage Classification[C]//Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence. New York, NY, USA: ACM, 2021: 1324-1330.
18	Jia Ziyu, Lin Youfang, Wang Jing, et al. Multi-view Spatial-temporal Graph Convolutional Networks with Domain Generalization for Sleep Stage Classification[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 1977-1986.
19	Ji Xiaopeng, Li Yan, Wen Peng. Jumping Knowledge Based Spatial-temporal Graph Convolutional Networks for Automatic Sleep Stage Classification[J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30: 1464-1472.
20	Li Miaofen, Liu Youmin, Zhi Shaodan, et al. Short-time Fourier Transform Using Odd Symmetric Window Function[J]. Journal of Dynamics, Monitoring and Diagnostics, 2022, 1(1): 37-45.
21	Zhou Haoran, Feng Yidan, Fang Mingsheng, et al. Adaptive Graph Convolution for Point Cloud Analysis[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2021: 4945-4954.
22	Supratak Akara, Guo Yike. TinySleepNet: An Efficient Deep Learning Model for Sleep Stage Scoring Based on Raw Single-channel EEG[C]//2020 42nd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC). Piscataway, NJ, USA: IEEE, 2020: 641-644.
23	Cai Xiyang, Jia Ziyu, Jiao Zehui. Two-stream Squeeze-and-excitation Network for Multi-modal Sleep Staging[C]//2021 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Piscataway, NJ, USA: IEEE, 2021: 1262-1265.
24	Xu Ziliang, Yang Xuejuan, Sun Jinbo, et al. Sleep Stage Classification Using Time-frequency Spectra from Consecutive Multi-time Points[J]. Frontiers in Neuroscience, 2020, 14: 14.
25	Cai Qing, Gao Zhongke, An Jianpeng, et al. A Graph-temporal Fused Dual-input Convolutional Neural Network for Detecting Sleep Stages from EEG Signals[J]. IEEE Transactions on Circuits and Systems II: Express Briefs, 2021, 68(2): 777-781.
26	Khalighi Sirvan, Sousa Teresa, José Moutinho Santos, et al. ISRUC-sleep: A Comprehensive Public Dataset for Sleep Researchers[J]. Computer Methods and Programs in Biomedicine, 2016, 124: 180-192.

数据集	阶段	阶段数	比例/%	总数
S1	W	20 098	23.05	87 187
	N1	11 062	12.69
	N2	27 511	31.55
	N3	17 251	19.79
	REM	11 265	12.92
S3	W	1 651	19.31	8 549
	N1	1 215	14.21
	N2	2 609	30.52
	N3	2 014	23.56
	REM	1 060	12.40

超参数	值
训练次数	60
训练批次	32
优化器	Adam
学习率	0.000 02
下采样率	0.5
自适应图卷积正则化参数	0.000 5
切比雪夫多项式阶数	9
卷积核大小	3×3

睡眠阶段	预测值					性能指标/%
睡眠阶段	W	N1	N2	N3	REM	Pre	Re
W	1 518	99	24	4	6	89.08	91.94
N1	140	733	215	2	125	69.81	60.33
N2	31	159	2 260	118	38	82.97	86.62
N3	1	1	189	1 822	1	93.58	90.47
REM	8	61	36	1	954	84.88	90.00

数据集	模型	总体性能指标			各阶段F1分数
数据集	模型	A	F₁	K	W	N1	N2	N3	REM
ISRUC-S3	DeepSleep^[11]	78.8	77.9	73.0	88.7	60.2	74.6	85.8	80.8
	U-Sleep^[12]	77.0	76.4	—	90.0	55.0	78.0	74.0	85.0
	SalientSleepNet^[13]	79.9	78.6	74.2	86.0	58.9	79.3	88.6	80.2
	GraphSleepNet^[17]	79.9	78.7	74.1	87.8	57.4	77.6	86.4	84.1
	MSTGCN^[18]	82.1	80.8	76.9	89.4	59.6	80.6	89.0	85.6
	JK-STGCN^[19]	83.1	81.4	78.2	90.0	59.8	82.6	90.1	84.7
	MJ-Sleep	85.3	83.8	81.0	90.5	64.7	84.8	92.0	87.4
ISRUC-S1	DeepSleep^[11]	71.7	69.1	63.8	82.3	46.6	73.8	80.9	62.1
	U-Sleep^[12]	77.0	77.0	—	89.0	52.0	79.0	77.0	88.0
	SalientSleepNet^[13]	81.5	80.1	76.2	89.9	57.0	80.0	87.8	85.7
	GraphSleepNet^[17]	78.6	75.4	72.3	88.4	43.7	77.5	83.8	83.5
	MSTGCN^[18]	80.4	78.5	74.8	88.7	54.5	79.1	87.2	83.2
	JK-STGCN^[19]	82.0	79.8	76.7	89.5	55.0	81.1	88.3	85.0
	MJ-Sleep	83.2	81.9	78.4	90.1	61.5	82.8	90.3	84.8

[1]	刘泽森, 毕盛, 郭传鈜, 王延葵, 董敏. 基于深度学习的机器人局部路径规划方法[J]. 系统仿真学报, 2024, 36(5): 1199-1210.
[2]	韦金阳, 王科平, 杨艺, 费树岷. 基于多重迁移注意力的增量式图像去雾算法[J]. 系统仿真学报, 2024, 36(4): 969-980.
[3]	李晨, 何明, 董晨, 李伟. 基于余弦相似性的定向注意力行为识别模型[J]. 系统仿真学报, 2024, 36(1): 67-82.
[4]	张凤全, 曹铎, 马晓寒, 陈柏君, 张江霄. 一种面向戏曲妆容细节生成的风格迁移网络[J]. 系统仿真学报, 2023, 35(9): 2064-2076.
[5]	苏本跃, 孙满贞, 马庆, 盛敏. 单视角下基于投影子空间视图的动作识别方法[J]. 系统仿真学报, 2023, 35(5): 1098-1108.
[6]	杜宇, 杨新权, 张建华, 袁素春, 肖化超, 袁晶晶. 基于循环谱截面智能分析的混合信号调制识别方法[J]. 系统仿真学报, 2023, 35(1): 146-157.
[7]	张大永, 杨镜宇, 吴曦. 兵棋推演空中任务智能预测方法研究[J]. 系统仿真学报, 2023, 35(1): 212-220.
[8]	柴慧敏, 张勇, 李欣粤, 宋雅楠. 基于深度学习的空中目标威胁评估方法[J]. 系统仿真学报, 2022, 34(7): 1459-1467.
[9]	康旭, 张晓峰. 基于生成对抗神经网络的雷达遥感数据增广方法[J]. 系统仿真学报, 2022, 34(4): 920-927.
[10]	刘家瑞, 杨国田, 王孝伟. 基于孪生深度神经网络的风电机组故障诊断方法[J]. 系统仿真学报, 2022, 34(11): 2348-2358.
[11]	陆荣秀, 张笔豪, 莫振龙. 基于脸部特征和头部姿态的疲劳检测方法[J]. 系统仿真学报, 2022, 34(10): 2279-2292.
[12]	仝卫国, 曾世超, 张立峰, 侯哲, 郭佳跃. 基于深度残差神经网络的电阻层析成像及流型辨识方法[J]. 系统仿真学报, 2022, 34(09): 2028-2036.
[13]	林硕, 安磊, 高治军, 单丹, 尚文利. 结合栈式自编码及长短时记忆的入侵检测研究[J]. 系统仿真学报, 2021, 33(6): 1288-1296.
[14]	程文聪, 史小康, 王志刚. 基于生成对抗网络的仿真卫星云图生成方法[J]. 系统仿真学报, 2021, 33(6): 1297-1306.
[15]	董书琴, 张斌. 面向不平衡数据的网络流量异常检测方法[J]. 系统仿真学报, 2021, 33(3): 679-689.