基于级联神经网络疲劳驾驶检测系统设计

doi:10.16182/j.issn1004731x.joss.20-0703

摘要/Abstract

摘要：

算法通过调整输入图片的大小、扩大最小人脸尺寸以及减小检测窗口层间放缩比例等方式，在保证准确率的同时极大的提升了人脸检测速率，其检测效率为原始MTCNN（multitask casaded convolutional networks）的18倍；构建新的卷积神经网络结构模型检测眼、嘴，网络检测的准确率可以达到95.6%。将设计的检测网络与经过改进后的MTCNN进行级联，在已经检测出来的脸部区域继续对眼、嘴进行分类及定位；设置综合性的疲劳检测函数，加强了由于单一特征而出现的虚检及误检等错误检测，准确率可以达到95.7%。

关键词: 卷积神经网络, 人脸检测, 疲劳判定, 级联网络, 边界框

Abstract:

An algorithm is proposed to greatly improves the face detection rate and ensures the accuracy by adjusting the size of input images, expanding the minimum face size, and reducing the scaling ratio between layers of the detection window. The detection efficiency of this algorithm is 18 times higher than that of the original MTCNN. By building a new CNN structure model for the detection of eyes and mouths, we can achieve network detection accuracy of 95.6%. The proposed network is cascaded with the original MTCNN to continue classifying and locating the eyes and mouth in the formerly detected face area. The false detection due to a single feature are improved through the setting of a comprehensive fatigue detection function, and the detection accuracy can reach 95.7%.

Key words: convolutional neural networks(CNN), face detection, fatigue determination, cascaded network, bounding box

中图分类号:

TP277

敖邦乾, 杨莎, 令狐金卿, 叶振环. 基于级联神经网络疲劳驾驶检测系统设计[J]. 系统仿真学报, 2022, 34(2): 323-333.

Bangqian Ao, Sha Yang, Jinqing Linghu, zhenhuan Ye. Design of Fatigue Driving Detection System Based on Cascaded Neural Network[J]. Journal of System Simulation, 2022, 34(2): 323-333.

图/表 18

图1

图2

图3

图4

图5

表1

表2

图6

图7

图8

图9

图10

图11

表3

表4

表5

图12

图13

参考文献 17

1	Krajewski J, Sommer D, Trutschel U, et al. Steering Wheel Behavior based Estimation of Fatigue[C]// Proceedings of the Fifth International Driving Symposium on Human Factors in Driver Assessment Training and Vehicle Design, 2009: 118-124.
2	刘炜煌, 钱锦浩, 姚增伟, 等. 基于多面部特征融合的驾驶员疲劳检测算法[J]. 计算机系统应用, 2018, 27 (10): 181-186.
	Liu Weihuang, Qian Jinhao, Yao Zengwei, et al. Driver Fatigue Detection Algorithm Based on Multi-Facial Feature Fusion [J]. Computer Systems & Applications, 2018, 27 (10): 181-186.
3	陈志勇, 杨佩, 彭力, 等. 基于BP神经网络的驾驶员疲劳监测研究[J].计算机科学, 2015, 42(Z6): 67-69.
	Chen Zhiyong, Yang Pei, Peng Li, et al. Fatigue Driving Monitoring based on BP Neural Network [J]. Computer Science, 2015, 42(Z6): 67-69.
4	Forsman P M, Vila B J, Short R A, et al. Efficient Driver Drowsiness Detection at Moderate Levels of drowsiness[J]. Accident Analysis & Prevention(S0001-4575), 2013, 50: 341-350.
5	Sun Chao, Li Jianhua, Song Yang, et al. Real-time driver fatigue detection based on eye state recognition [C]// Applied Mechanics and Materials. Trans Tech Publications, 2014: 944-952.
6	戴诗琪, 曾智勇.基于深度学习的疲劳驾驶检测算法[J].计算机系统应用, 2018, 27 (7): 113-120.
	Dai Shiqi, Zeng Zhiyong. Fatigue Driving Monitoring based on BP Neural Network [J]. Computer Systems& Applications, 2018, 27 (7): 113-120.
7	顾王欢, 朱煜, 陈旭东, 等. 基于多尺度池化卷积神经网络的疲劳检测方法研究[J]. 计算机应用研究, 2019, 36(11): 3471-3475.
	Gu Wanghuan, Zhu Yu, Chen Xudong, et al. Driver's Fatigue Detection System based on Multi-scale Pooling Convolutional Neural Networks [J]. Application Research of Computers, 2019, 36(11): 3471-3475.
8	耿磊, 袁菲, 肖志涛, 等. 基于面部行为分析的驾驶员疲劳检测方法[J]. 计算机工程, 2018, 44 (1): 274-279.
	Geng Lei, Yuan Fei, Xiao Zhitao, et al. Driver Fatigue Detection Method Based on Facial Behavior Analysis [J]. Computer Engineering, 2018, 44 (1): 274-279.
9	Chuang C H, Huang C S, Ko L W, et al. An EEG-based Perceptual Function Integration Network for Application to Drowsy Driving[J]. Knowledge-Based Systems(S0950-7051), 2015, 80: 143-152.
10	Zheng W L, Gao K, Li G, et al. Vigilance Estimation Using a Wearable EOG Device in Real Driving Environment[J]. IEEE Transactions on Intelligent Transportation Systems(S1524-9050), 2019, 21(1): 170-184.
11	Sun Chao, Li Jianhua, Song Yang, et al. Real-time driver fatigue detection based on eye state recognition[C]// Applied Mechanics and Materials. Trans Tech Publications, 2014: 944-952.
12	Tang Xinxing, Zhou Pengfei, Wang Ping. Real-time Image-based Driver Fatigue Detection and Monitoring System for Monitoring Driver Vigilance[C]// Proc of the 35th Chinese Control Conference (CCC), 2016: 4188-4193.
13	Zhang F, Su J J, Geng L, et al. Driver Fatigue Detection based on Eye State Recognition[C]// International Conference on Machine Vision and Information Technology.Singapore, 2017: 105-110.
14	Yan C, Zhang B L, Coenen F. Driving Posture Recognition by Convolutional Neural Networks[C]// Proceedings of the 11th International Conference on Natural Computation. Zhangjiajie, China, 2015: 680-685.
15	Zhu X M, Zheng W L, Lu B L, et al. EOG-based Drowsiness Detection using Convolutional Neural Networks[C]// International Joint Conference on Neural Networks. Beijing, China, 2014: 128-134.
16	Zhang K, Zhang Z, Li Z, et al. Joint Face Detection and Alignment using Multitask Cascaded Convolutional Networks[J]. IEEE Signal Processing Letters(S1070-9908), 2016, 23(10): 1499-1503.
17	Liu W, Anguelov D, Erhan D, et al. SsD: Single Shot Multi-Box Detector[C]// European Conference on Computer Vision. Springer, Cham, 2016: 21-37.

测试任务	本文时间开销	SSD样本时间开销
左眼检测分类	2.685	6.453
右眼检测分类	2.668	6.386
嘴部检测分类	2.488	6.568

算法	检出准确率/%
文献[6]算法	95.0
文献[7]算法	91.6
文献[8]算法	92.5
本文算法	95.7

任务类别	时间开销/ms
原始图像获取及优化	3.034
MTCNN人脸检测	5.686
EMs-Net眼、嘴提取	2.563
眼部状态返回框	4.569
嘴部状态返回框	3.326
总时间开销	19.177

[1]	郭业才, 王庆伟. 基于截断迁移与并行残差网络的调制识别算法[J]. 系统仿真学报, 2022, 34(9): 2009-2018.
[2]	张立峰, 王会忍. 基于卷积神经网络及有限元仿真的电容层析成像图像重建[J]. 系统仿真学报, 2022, 34(4): 712-718.
[3]	赵瑛, 陆耀, 张健, 梁启弟, 龙炜. 基于深度神经网络的多视角人体动作识别[J]. 系统仿真学报, 2021, 33(5): 1019-1030.
[4]	仝卫国, 庞雪纯, 朱赓宏. 基于卷积神经网络的气液两相流流型识别方法[J]. 系统仿真学报, 2021, 33(4): 883-891.
[5]	马榕, 陈秋瑞, 张晗, 梅铮, 王锐, 魏伟. 基于单目深度估计的低功耗视觉里程计[J]. 系统仿真学报, 2021, 33(12): 3001-3011.
[6]	黄欣, 方钰, 顾梦丹. 基于卷积神经网络的X线胸片疾病分类研究[J]. 系统仿真学报, 2020, 32(6): 1188-1194.
[7]	樊名鲁, 王艳, 纪志成. 基于特征生成的轴承不均衡数据故障诊断[J]. 系统仿真学报, 2020, 32(12): 2438-2448.
[8]	王红军, 黎邹邹, 邹湘军. 基于Adaboost与CNN的木材表面缺陷检测[J]. 系统仿真学报, 2019, 31(8): 1636-1645.
[9]	谷学静, 张凯峰, 郭宇承, 王志良. 基于Unity3D的ASD儿童认知教育系统设计与实现[J]. 系统仿真学报, 2019, 31(5): 893-900.
[10]	卢裕秋, 孙金玉, 马世伟. 基于深度卷积神经网络的运动目标检测方法[J]. 系统仿真学报, 2019, 31(11): 2275-2280.
[11]	石祥滨, 钟健, 刘翠微, 刘芳, 张德园. 小样本高光谱遥感图像深度学习方法[J]. 系统仿真学报, 2018, 30(7): 2744-2752.
[12]	夏旻, 申茂阳, 王舰锋, 王阳光. 基于卷积神经网络的卫星云图云量计算[J]. 系统仿真学报, 2018, 30(5): 1623-1630.
[13]	廖鹰, 易卓, 胡晓峰, 田园, 陶九阳. 基于三维卷积神经网络的战场聚集行为预测[J]. 系统仿真学报, 2018, 30(3): 801-808.
[14]	吴昀璞, 金炜东, 黄颖坤. 基于多域融合CNN的高速列车转向架故障检测[J]. 系统仿真学报, 2018, 30(11): 4492-4497.
[15]	高尚兵, 颉正, 潘志庚, 覃方哲, 李锐. 一种新的路面裂缝自动检测算法[J]. 系统仿真学报, 2017, 29(9): 2009-2015.