Design of Fatigue Driving Detection System Based on Cascaded Neural Network

doi:10.16182/j.issn1004731x.joss.20-0703

Abstract

Abstract:

An algorithm is proposed to greatly improves the face detection rate and ensures the accuracy by adjusting the size of input images, expanding the minimum face size, and reducing the scaling ratio between layers of the detection window. The detection efficiency of this algorithm is 18 times higher than that of the original MTCNN. By building a new CNN structure model for the detection of eyes and mouths, we can achieve network detection accuracy of 95.6%. The proposed network is cascaded with the original MTCNN to continue classifying and locating the eyes and mouth in the formerly detected face area. The false detection due to a single feature are improved through the setting of a comprehensive fatigue detection function, and the detection accuracy can reach 95.7%.

Key words: convolutional neural networks(CNN), face detection, fatigue determination, cascaded network, bounding box

CLC Number:

TP277

Bangqian Ao, Sha Yang, Jinqing Linghu, zhenhuan Ye. Design of Fatigue Driving Detection System Based on Cascaded Neural Network[J]. Journal of System Simulation, 2022, 34(2): 323-333.

Figures/Tables 18

Fig1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Table 1

Table 2

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Table 3

Table 4

Table 5

Fig. 12

Fig. 13

References 17

1	Krajewski J, Sommer D, Trutschel U, et al. Steering Wheel Behavior based Estimation of Fatigue[C]// Proceedings of the Fifth International Driving Symposium on Human Factors in Driver Assessment Training and Vehicle Design, 2009: 118-124.
2	刘炜煌, 钱锦浩, 姚增伟, 等. 基于多面部特征融合的驾驶员疲劳检测算法[J]. 计算机系统应用, 2018, 27 (10): 181-186.
	Liu Weihuang, Qian Jinhao, Yao Zengwei, et al. Driver Fatigue Detection Algorithm Based on Multi-Facial Feature Fusion [J]. Computer Systems & Applications, 2018, 27 (10): 181-186.
3	陈志勇, 杨佩, 彭力, 等. 基于BP神经网络的驾驶员疲劳监测研究[J].计算机科学, 2015, 42(Z6): 67-69.
	Chen Zhiyong, Yang Pei, Peng Li, et al. Fatigue Driving Monitoring based on BP Neural Network [J]. Computer Science, 2015, 42(Z6): 67-69.
4	Forsman P M, Vila B J, Short R A, et al. Efficient Driver Drowsiness Detection at Moderate Levels of drowsiness[J]. Accident Analysis & Prevention(S0001-4575), 2013, 50: 341-350.
5	Sun Chao, Li Jianhua, Song Yang, et al. Real-time driver fatigue detection based on eye state recognition [C]// Applied Mechanics and Materials. Trans Tech Publications, 2014: 944-952.
6	戴诗琪, 曾智勇.基于深度学习的疲劳驾驶检测算法[J].计算机系统应用, 2018, 27 (7): 113-120.
	Dai Shiqi, Zeng Zhiyong. Fatigue Driving Monitoring based on BP Neural Network [J]. Computer Systems& Applications, 2018, 27 (7): 113-120.
7	顾王欢, 朱煜, 陈旭东, 等. 基于多尺度池化卷积神经网络的疲劳检测方法研究[J]. 计算机应用研究, 2019, 36(11): 3471-3475.
	Gu Wanghuan, Zhu Yu, Chen Xudong, et al. Driver's Fatigue Detection System based on Multi-scale Pooling Convolutional Neural Networks [J]. Application Research of Computers, 2019, 36(11): 3471-3475.
8	耿磊, 袁菲, 肖志涛, 等. 基于面部行为分析的驾驶员疲劳检测方法[J]. 计算机工程, 2018, 44 (1): 274-279.
	Geng Lei, Yuan Fei, Xiao Zhitao, et al. Driver Fatigue Detection Method Based on Facial Behavior Analysis [J]. Computer Engineering, 2018, 44 (1): 274-279.
9	Chuang C H, Huang C S, Ko L W, et al. An EEG-based Perceptual Function Integration Network for Application to Drowsy Driving[J]. Knowledge-Based Systems(S0950-7051), 2015, 80: 143-152.
10	Zheng W L, Gao K, Li G, et al. Vigilance Estimation Using a Wearable EOG Device in Real Driving Environment[J]. IEEE Transactions on Intelligent Transportation Systems(S1524-9050), 2019, 21(1): 170-184.
11	Sun Chao, Li Jianhua, Song Yang, et al. Real-time driver fatigue detection based on eye state recognition[C]// Applied Mechanics and Materials. Trans Tech Publications, 2014: 944-952.
12	Tang Xinxing, Zhou Pengfei, Wang Ping. Real-time Image-based Driver Fatigue Detection and Monitoring System for Monitoring Driver Vigilance[C]// Proc of the 35th Chinese Control Conference (CCC), 2016: 4188-4193.
13	Zhang F, Su J J, Geng L, et al. Driver Fatigue Detection based on Eye State Recognition[C]// International Conference on Machine Vision and Information Technology.Singapore, 2017: 105-110.
14	Yan C, Zhang B L, Coenen F. Driving Posture Recognition by Convolutional Neural Networks[C]// Proceedings of the 11th International Conference on Natural Computation. Zhangjiajie, China, 2015: 680-685.
15	Zhu X M, Zheng W L, Lu B L, et al. EOG-based Drowsiness Detection using Convolutional Neural Networks[C]// International Joint Conference on Neural Networks. Beijing, China, 2014: 128-134.
16	Zhang K, Zhang Z, Li Z, et al. Joint Face Detection and Alignment using Multitask Cascaded Convolutional Networks[J]. IEEE Signal Processing Letters(S1070-9908), 2016, 23(10): 1499-1503.
17	Liu W, Anguelov D, Erhan D, et al. SsD: Single Shot Multi-Box Detector[C]// European Conference on Computer Vision. Springer, Cham, 2016: 21-37.

测试任务	本文时间开销	SSD样本时间开销
左眼检测分类	2.685	6.453
右眼检测分类	2.668	6.386
嘴部检测分类	2.488	6.568

算法	检出准确率/%
文献[6]算法	95.0
文献[7]算法	91.6
文献[8]算法	92.5
本文算法	95.7

任务类别	时间开销/ms
原始图像获取及优化	3.034
MTCNN人脸检测	5.686
EMs-Net眼、嘴提取	2.563
眼部状态返回框	4.569
嘴部状态返回框	3.326
总时间开销	19.177

[1]	Li Mingyu, Lin Jiaquan. Lightweight Driver Face Object Detection Algorithm Based on YOLOv8-DF [J]. Journal of System Simulation, 2025, 37(8): 2103-2114.
[2]	Rongxiu Lu, Bihao Zhang, Zhenlong Mo. Fatigue Detection Method Based on Facial Features and Head Posture [J]. Journal of System Simulation, 2022, 34(10): 2279-2292.
[3]	Shu Liang, Zhang Jie, Yang Yanfang, Yang Miao, Wu Ziran. Dynamic Control of Twin-workshop Model of Circuit Breaker Considering Takt Time Constraints [J]. Journal of System Simulation, 2021, 33(6): 1277-1287.
[4]	Gu Xuejing, Zhang Kaifeng, Guo Yucheng, Wang Zhiliang. Design and Implementation of ASD Children’s Cognitive Education System Based on Unity3D [J]. Journal of System Simulation, 2019, 31(5): 893-900.
[5]	Wang Chao, Zhang Zhili, Long Yong, Wang Shaodi. Improved Hybrid Bounding Box Collision Detection Algorithm [J]. Journal of System Simulation, 2018, 30(11): 4236-4243.
[6]	Li Zhao, Jin Yanxia, Qin Zhipeng, Ren Chao. Collision Detection Algorithm of Deformable Object Based on Snake Model Optimization [J]. Journal of System Simulation, 2018, 30(1): 62-68.
[7]	Yuan Peng, Jin Xin, Li Xiaodong, Zhao Geng, Wu Yaming, Ma Mingxin, Tian Yulu, Chen Yingya. Private Face Detection Based on Random Sub-images in Cloud [J]. Journal of System Simulation, 2016, 28(9): 2195-2200.
[8]	Zhang Xiaoli, Yao Junfeng, Huang Ping. 360-Degree Virtual Fitting Based on Kinect [J]. Journal of System Simulation, 2016, 28(10): 2378-2385.
[9]	Tao Zeen, Pu Yuanyuan, Xu Dan, Liu Yiwei. Aesthetic Assessment for Digital Portrait Photos [J]. Journal of System Simulation, 2016, 28(10): 2298-2303.
[10]	Wang Renda, Yin Yong, Xing Shengwei. Research on Skin Color Image Post-processing Methods for Gesture Recognition System [J]. Journal of System Simulation, 2015, 27(10): 2483-2488.
[11]	Jin Yaoli, Wang Shiliang, Xu Gang, Hu Weihua, Wang Yigang. Automatic Hair Contour Extraction Method with Complex Background [J]. Journal of System Simulation, 2015, 27(10): 2305-2309.