基于脸部特征和头部姿态的疲劳检测方法

doi:10.16182/j.issn1004731x.joss.21-0583

摘要/Abstract

摘要：

针对疲劳驾驶检测方法存在疲劳特征单一、鲁棒性低和不能因驾驶员不同定制疲劳阈值等问题，提出了一种基于脸部特征和头部姿态的疲劳检测方法。利用HOG(histogram of oriented gradients)特征算子和回归树算法进行人脸检测和人脸关键点定位；通过脸部关键点结合坐标系变换估计头部姿态欧拉角；建立深度残差神经网络模型对眼部疲劳特征进行提取，同时结合眼部、嘴部纵横比和头部姿态欧拉角进行疲劳特征提取；利用眼部、嘴部和头部姿态疲劳特征建立针对不同驾驶员的支持向量机模型对疲劳驾驶进行预警。实验表明：在YawDD和自建疲劳模拟数据集上，该方法均表现出较高的准确率和鲁棒性，在某一疲劳特征检测受阻时依然能进行较好的疲劳预警。

关键词: 疲劳驾驶, 人脸检测, 头部姿态, 深度学习, 支持向量机

Abstract:

Aiming at the of the single fatigue characteristics, low robustness and inability to customize fatigue thresholds for different drivers of fatigue detection methods, a method based on facial features and head posture is proposed. In face detection and face key point positioning HOG feature operator and regression tree algorithm are used. In head posture estimation, head posture Euler angle is estimated by combining the face key points with the coordinate system transformation. In fatigue feature extraction, a deep residual neural network model is established to extract the eye fatigue features, which the eye, mouth aspect ratio and head posture Euler angle. The fatigue characteristics of eyes, mouth and head are used to establish the support vector machine models for different drivers to provide the early fatigue driving warning. Experiments show that on YawDD and self-built fatigue simulation data sets, the method shows high accuracy and robustness, and can provide better fatigue warning when a certain fatigue feature detection is blocked.

Key words: fatigue driving, face detection, head posture, deep learning, support vector machine

中图分类号:

TP391

陆荣秀, 张笔豪, 莫振龙. 基于脸部特征和头部姿态的疲劳检测方法[J]. 系统仿真学报, 2022, 34(10): 2279-2292.

Rongxiu Lu, Bihao Zhang, Zhenlong Mo. Fatigue Detection Method Based on Facial Features and Head Posture[J]. Journal of System Simulation, 2022, 34(10): 2279-2292.

图/表 20

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

表1

图14

表2

表3

图15

图16

表4

参考文献 23

1	Tuncer T, Dogan S, Ertam F, et al. A Dynamic Center and Multi Threshold Point Based Stable Feature Extraction Network for Driver Fatigue Detection Utilizing EEG Signals[J]. Cognitive Neurodynamics(S1871-4080), 2021, 15(7): 223-237.
2	王博石, 吴修诚, 胡馨艺, 等. 基于单通道脑电信号的疲劳检测系统[J]. 计算机科学, 2020, 47(5): 225-229.
	Wang Boshi, Wu Xiucheng, Hu Xinyi, et al. Fatigue Detection System Based on Single Channel EEG Signal [J]. Computer Science, 2020, 47(5): 225-229.
3	Kyehoon, LEE, SUNG-AE, et al. Detecting Driver Fatigue by Steering Wheel Grip Force[J]. International Journal of Contents(S1738-6764), 2016, 12(1): 44-48.
4	Wang M, Guo L, Chen W. Blink Detection Using Adaboost and Contour Circle for Fatigue Recognition[J]. Computers&Electrical Engineering(S0045-7906), 2017, 58: 502-512.
5	Mandal B, Li L, Wang G S, et al. Towards Detection of Bus Driver Fatigue Based on Robust Visual Analysis of Eye State[J]. IEEE Transactions on Intelligent Transportation Systems(S1524-9050), 2017, 18(3): 1-13.
6	Zhao Xuepeng, Meng Chunning, Feng Mingkui, et al. Fatigue Detection Based on Cascade Convolutional Neural Network[J]. Journal of Optoelectronics·Laser (S1005-0086), 2017, 28(5): 497-502.
7	Omidyeganeh M, Shirmohammadi S, Abtahi S, et al. Yawning Detection Using Embedded Smart Cameras[J]. IEEE Transactions on Instrumentation and Measurement (S0018-9456), 2016, 65(3): 570-582.
8	Kazemi V, Sullivan J. One Millisecond Face Alignment with an Ensemble of Regression Trees[C]// IEEE Conference on Computer Vision and Pattern Recognition, Columbus. New York: IEEE Press, 2014: 1867-1874.
9	Zhu X, Ramanan D. Face Detection, Pose Estimation, and Landmark Localization in the wild[C]// IEEE Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2012: 2879-2886.
10	Le V, Brandt J, Lin Z, et al. Interactive Facial Feature Localization[C]// European Conference on Computer Vision. Heidelberg, Berlin: Springer, 2012: 679-692.
11	Belhumeur P N, Jacobs D W, Kriegman D J, et al. Localizing Parts of Faces Using a Consensus of Exemplars[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(S0162-8828), 2013, 35(12): 2930-2940.
12	Sagonas C, Tzimiropoulos G, Zafeiriou S, et al. 300 Faces in-the-wild Challenge: The First Facial Landmark Localization Challenge [C]//IEEE International Conference on Computer Vision Workshops. Sydney: IEEE, 2013: 397-403.
13	Sagonas C, Antonakos E, Tzimiropoulos G, et al. 300Faces in-the-Wild Challenge: Database and Results [J]. Image and vision computing(S0262-8856), 2016, 47: 3-18.
14	Rafacl C, Richard E. Digital Image Processing[M]. Beijing: PHI, 2017.
15	Zhang Z. A Flexible New Technique for Camera Calibration [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(S0162-8828), 2000, 22(11): 1330-1334.
16	Huber P, Hu G, Tena R, et al. A Multiresolution 3D Morphable Face Model and Fitting Framework[C]//11th Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. 2016: 79-86.
17	He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition[C]// IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Las Vegas, NV: IEEE, 2016: 770-778.
18	Yosinski J, Clune J, Bengio Y, et al. How Transferable are Features in Deep Neural Networks?[C]// International Conference on Neural Information Processing Systems. Quebec, Canada: MIT Press, 2014: 3320-3328.
19	Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-scale Image Recognition [C]//International Conference on Learning Representations. 2015.
20	Ferrario V F, Sforza C, Serrao G, et al. Active Range of Motion of the Head and Cervical Spine: A Three-Dimensional Investigation in Healthy Young Adults[J]. Journal of Orthopaedic Research (S1554-527X), 2002, 20(1): 122-129.
21	Song F, Tan X, Chen S, et al. A Literature Survey on Robust and Efficient Eye Localization in Real-Life Scenarios [J]. Pattern Recognition(S0031-3203), 2013, 46(12): 3157-3173.
22	Abtahi S, Omidyeganeh M, Shirmohammadi S, et al. YawDD: A Yawning Detection Dataset[C]// ACM Multimedia Systems (MMSys). Singapore: ACM, 2014: 24-28.
23	汪磊, 孙瑞山. 基于面部特征识别的管制员疲劳监测方法研究[J]. 中国安全科学学报, 2012, 22(7): 66-71.
	Wang Lei, Sun Ruishan. Study on Feature Recognition-based Fatigue Monitoring Method for Air Traffic Contorller[J]. China Safety Science Journal, 2012, 22(7): 66-71.

测试员编号	EAR		MAR
测试员编号	睁眼	闭眼	嘴闭合	说话	哈欠
1	0.274 1	0.085 9	0.372 8	0.640 0	1.521 6
2	0.262 0	0.107 8	0.380 2	0.750 4	1.428 1
3	0.250 3	0.139 4	0.155 2	0.500 3	1.063 1
4	0.291 9	0.159 3	0.295 5	0.867 1	1.160 4
5	0.236 6	0.145 8	0.427 7	0.794 6	1.238 9

视频编号	实际闭眼数	检测闭眼数/准确率%
视频编号	实际闭眼数	单一阈值	平均阈值	SVM模型
1	90	78/86.7	83/92.2	89/98.9
2	157	148/94.3	142/90.4	154/98.1
3	131	97/74.0	120/91.6	129/98.5
4	87	60/69.1	77/88.5	85/97.7
5	106	74/69.8	98/92.5	103/97.2

视频编号	实际哈欠数	检测哈欠数/准确率%
视频编号	实际哈欠数	单一阈值	平均阈值	SVM模型
1	3	2/66.7	3/100	3/100
2	0	0	0	0
3	2	2/100	1/50	2/100
4	5	6/83.3	4/80	6/83.3
5	4	2/50	4/100	4/100

视频编号	眼部			嘴部				头部		预警次数
视频编号	实际闭眼数	检测闭眼数	准确率/%	实际哈欠数	检测哈欠数	实际说话时间/s	检测说话时间/s	实际低头数	检测低头数	预警次数
1	90	89	98.90	3	3	12	9	5	5	8
2	157	155	98.70	0	0	20	18	1	0	0
3	131	130	99.20	2	2	17	13	4	5	7
4	87	85	97.70	5	6	6	8	8	6	13
5	106	104	98.10	4	4	18	17	13	15	21

[1]	柴慧敏, 张勇, 李欣粤, 宋雅楠. 基于深度学习的空中目标威胁评估方法[J]. 系统仿真学报, 2022, 34(7): 1459-1467.
[2]	康旭, 张晓峰. 基于生成对抗神经网络的雷达遥感数据增广方法[J]. 系统仿真学报, 2022, 34(4): 920-927.
[3]	敖邦乾, 杨莎, 令狐金卿, 叶振环. 基于级联神经网络疲劳驾驶检测系统设计[J]. 系统仿真学报, 2022, 34(2): 323-333.
[4]	仝卫国, 曾世超, 张立峰, 侯哲, 郭佳跃. 基于深度残差神经网络的电阻层析成像及流型辨识方法[J]. 系统仿真学报, 2022, 34(09): 2028-2036.
[5]	李晖, 云昊, 岳红丽. 基于SVM逆系统的PMSM云模型整定PID控制[J]. 系统仿真学报, 2021, 33(8): 1846-1855.
[6]	林硕, 安磊, 高治军, 单丹, 尚文利. 结合栈式自编码及长短时记忆的入侵检测研究[J]. 系统仿真学报, 2021, 33(6): 1288-1296.
[7]	程文聪, 史小康, 王志刚. 基于生成对抗网络的仿真卫星云图生成方法[J]. 系统仿真学报, 2021, 33(6): 1297-1306.
[8]	位晶晶, 刘勤明, 叶春明, 李冠林. 基于GA-SVR的小样本数据缺失下的设备故障诊断[J]. 系统仿真学报, 2021, 33(6): 1342-1349.
[9]	邵诚, 孙宏远, 张林. 基于PSO-LSSVM的反舰导弹自控弹道仿真[J]. 系统仿真学报, 2021, 33(4): 918-926.
[10]	董书琴, 张斌. 面向不平衡数据的网络流量异常检测方法[J]. 系统仿真学报, 2021, 33(3): 679-689.
[11]	王霄汉, 张霖, 任磊, 谢堃钰, 王昆玉, 叶飞, 陈真. 基于强化学习的车间调度问题研究简述[J]. 系统仿真学报, 2021, 33(12): 2782-2791.
[12]	王步维, 王敏, 范谦, 王雅男, 章涵文, 乐云亮. 基于深度学习的晶体性质预测研究[J]. 系统仿真学报, 2021, 33(12): 2854-2863.
[13]	冯晓, 张辉, 周蕊, 乔璐, 魏东, 李丹丹, 张玉尧, 郑国清. 基于深度学习和籽粒双面特征的玉米品种识别[J]. 系统仿真学报, 2021, 33(12): 2983-2991.
[14]	刘薛勤, 刘宁, 苏中, 王靖骁, 袁超杰. 基于足底压力感知的智能步态识别方法研究[J]. 系统仿真学报, 2021, 33(11): 2572-2578.
[15]	杜金莲, 李淑飞, 金雪云. 三维烟雾流场超分辨率数据生成网络模型的研究[J]. 系统仿真学报, 2021, 33(10): 2381-2389.