Traffic Sign Recognition Model with Long-Tail Distribution Based on YOLOX-Tiny

doi:10.16182/j.issn1004731x.joss.23-0906

Abstract

Abstract:

Accurate recognition of traffic signs plays an important role in the field of intelligent driving. Traffic sign training datasets with long-tail distribution increase the difficulty of traffic sign recognition. A traffic sign recognition model with long-tail distribution based on YOLOX-Tiny was proposed to improve the poor performance of the model trained on long-tail distribution datasets. A long-tail traffic sign dataset was created based on the TT100K_2021 (tsinghua-tencent 100K 2021) dataset. YOLOX-Tiny was chosen as the underlying model by considering picture numbers in datasets, sample distribution, and model size. Equalization loss v2 (EQL v2) was used as classification loss to balance the head and tail of the classifier, and focal loss(FL) was used as target confidence loss to enhance the model's prediction of target confidence. In order to solvethe backpropagation conflicts of feature graphs at different levels on the traditional feature pyramid, enhance the featurereorganization effect, and highlight target feature, up-sampling operator CARAFE, coordinate attention (CA), and CARAFE + adaptively spatial feature fusion modules (CAR-ASFF) were introduced to the neck bidirectional pyramid. The research results show that the improved YOLOX-Tiny model achieves 43.67% and 29.98% respectively in the long-tail traffic sign datasets, namely mAP₅₀ and mAP_50:95. The improved model has higher detection accuracy than other target detection models.

Key words: long-tail distribution, YOLOX, traffic sign recognition, attention mechanism, feature reorganization, multiscale feature fusion

CLC Number:

TP391.9

Wu Yunpeng, Fu Yingxiong, Shen Lijun, Cui Feng. Traffic Sign Recognition Model with Long-Tail Distribution Based on YOLOX-Tiny[J]. Journal of System Simulation, 2024, 36(11): 2503-2516.

Figures/Tables 11

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Table 1

Results of ablation experiments

模型	mAP₅₀	mAP_50:95	$m A P 50 : 95 r$	$m A P 50 : 95 c$	$m A P 50 : 95 f$
YOLOX-Tiny	39.35	26.41	16.64	30.19	40.09
+EQL v2	40.95	27.56	17.49	31.91	39.85
+FL	40.29	27.85	17.10	32.54	40.87
+CARAFE	41.38	27.85	17.48	31.70	41.70
+CA	40.61	27.19	16.91	31.20	40.59
+CAR-ASFF	41.76	28.34	18.95	31.32	41.89
+EQL v2+FL	40.97	28.41	18.16	32.99	40.62
+EQL v2+FL+CARAFE	41.64	28.72	18.49	33.04	41.39
+EQL v2+FL+CARAFE+CA	42.25	29.19	19.6	32.66	42.19
+EQL v2+FL+CARAFE+CA+CAR-ASFF	43.67	29.98	20.69	32.56	44.12

Table 1

Fig. 7

Table 2

Comparison results of performance of each model

模型	Size	mAP₅₀	mAP_50:95	$m A P 50 : 95 r$	$m A P 50 : 95 c$	$m A P 50 : 95 f$
YOLOv5s	416	37.90	26.37	19.20	27.97	38.02
YOLOv7-Tiny	416	40.19	26.77	20.86	27.98	36.58
YOLOv8n	416	34.43	25.46	17.33	29.07	35.17
YOLOX-Tiny Faster RCNN_r50f_CAS Faster RCNN_r50f_SL	416 608 608	39.35 26.40 21.70	26.41 20.13 16.28	16.64 20.70 14.93	30.19 19.95 17.02	40.09 19.29 17.64
本文	416	43.67	29.98	20.69	32.56	44.12

Table 2

Fig. 8

Fig. 9

References 31

1	Xie Qiwei, Hu Xiyuan, Ren Lei, et al. A Binocular Vision Application in IoT: Realtime Trustworthy Road Condition Detection System in Passable Area[J]. IEEE Transactions on Industrial Informatics, 2023, 19(1): 973-983.
2	伍晓晖, 田启川. 交通标志识别方法综述[J]. 计算机工程与应用, 2020, 56(10): 20-26.
	Wu Xiaohui, Tian Qichuan. Survey of Traffic Sign Recognition[J]. Computer Engineering and Applications, 2020, 56(10): 20-26.
3	Tabernik Domen, Skočaj Danijel. Deep Learning for Large-scale Traffic-sign Detection and Recognition[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 21(4): 1427-1440.
4	陈飞, 刘云鹏, 李思远. 复杂环境下的交通标志检测与识别方法综述[J]. 计算机工程与应用, 2021, 57(16): 65-73.
	Chen Fei, Liu Yunpeng, Li Siyuan. Survey of Traffic Sign Detection and Recognition Methods in Complex Environment[J]. Computer Engineering and Applications, 2021, 57(16): 65-73.
5	Jiao Licheng, Zhang Fan, Liu Fang, et al. A Survey of Deep Learning-based Object Detection[J]. IEEE Access, 2019, 7: 128837-128868.
6	罗会兰, 陈鸿坤. 基于深度学习的目标检测研究综述[J]. 电子学报, 2020, 48(6): 1230-1239.
	Luo Huilan, Chen Hongkun. Survey of Object Detection Based on Deep Learning[J]. Acta Electronica Sinica, 2020, 48(6): 1230-1239.
7	刘洪江, 王懋, 刘丽华, 等. 基于深度学习的小目标检测综述[J]. 计算机工程与科学, 2021, 43(8): 1429-1442.
	Liu Hongjiang, Wang Mao, Liu Lihua, et al. A Survey of Small Object Detection Based on Deep Learning[J]. Computer Engineering & Science, 2021, 43(8): 1429-1442.
8	史燕燕, 史殿习, 乔子腾, 等. 小样本目标检测研究综述[J]. 计算机学报, 2023, 46(8): 1753-1780.
	Shi Yanyan, Shi Dianxi, Qiao Ziteng, et al. A Survey on Recent Advances in Few-Shot Object Detection[J]. Chinese Journal of Computers, 2023, 46(8): 1753-1780.
9	Lim Kwangyong, Hong Yongwon, Choi Yeongwoo, et al. Real-time Traffic Sign Recognition Based on a General Purpose GPU and Deep-learning[J]. PLoS One, 2017, 12(3): e0173317.
10	王银, 王飞翔, 孙前来. 多尺度特征融合车辆检测方法[J]. 系统仿真学报, 2022, 34(6): 1219-1229.
	Wang Yin, Wang Feixiang, Sun Qianlai. Vehicle Detection Method Based on Multi Scale Feature Fusion[J]. Journal of System Simulation, 2022, 34(6): 1219-1229.
11	江金洪, 鲍胜利, 史文旭, 等. 基于YOLO v3算法改进的交通标志识别算法[J]. 计算机应用, 2020, 40(8): 2472-2478.
	Jiang Jinhong, Bao Shengli, Shi Wenxu, et al. Improved Traffic Sign Recognition Algorithm Based on YOLO v3 Algorithm[J]. Journal of Computer Applications, 2020, 40(8): 2472-2478.
12	廖璐明, 张伟, 王亚刚. 基于注意力机制的交通标志识别[J]. 小型微型计算机系统, 2021, 42(6): 1262-1266.
	Liao Luming, Zhang Wei, Wang Yagang. Traffic Sign Recognition Based on Attention Mechanism[J]. Journal of Chinese Computer Systems, 2021, 42(6): 1262-1266.
13	潘惠苹, 王敏琴, 张福泉. 基于优化YOLO-V4的交通标志检测识别方法[J]. 计算机科学, 2022, 49(11): 179-184.
	Pan Huiping, Wang Minqin, Zhang Fuquan. Traffic Sign Detection and Recognition Method Based on Optimized YOLO-V4[J]. Computer Science, 2022, 49(11): 179-184.
14	李祥坤, 杨争峰, 曾霞, 等. 一种面向图像识别的神经网络通用扰动生成算法[J]. 系统科学与数学, 2019, 39(12): 1944-1963.
	Li Xiangkun, Yang Zhengfeng, Zeng Xia, et al. An Approach to Generate Universal Adversarial Perturbation for Deep Neural Networks in Image Recognization[J]. Journal of Systems Science and Mathematical Sciences, 2019, 39(12): 1944-1963.
15	刘庆庆, 陈岑, 汪世界. 两个广泛相依随机变量在最值运算下的次指数性[J]. 应用数学学报, 2017, 40(2): 279-286.
	Liu Qingqing, Chen Cen, Wang Shijie. The Subexponentiality for Two Widely Dependent Random Variables Under the Operations of Minimum and Maximum[J]. Acta Mathematicae Applicatae Sinica, 2017, 40(2): 279-286.
16	Shen Li, Lin Zhouchen, Huang Qingming. Relay Backpropagation for Effective Learning of Deep Convolutional Neural Networks[C]//Computer Vision – ECCV 2016. Cham: Springer International Publishing, 2016: 467-482.
17	Drummond C, Holte R C. C4.5, Class Imbalance, and Cost Sensitivity: Why Under-sampling Beats Over-sampling[C]//ICML'2003 Workshop on Learning from Imbalanced Datasets II. Washington: ACM, 2003: 1-8.
18	Wang Tao, Li Yu, Kang Bingyi, et al. The Devil is in Classification: A Simple Framework for Long-tail Instance Segmentation[C]//Computer Vision–ECCV 2020. Cham: Springer International Publishing, 2020: 728-744.
19	Li Yu, Wang Tao, Kang Bingyi, et al. Overcoming Classifier Imbalance for Long-tail Object Detection With Balanced Group Softmax[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 10988-10997.
20	Wang Tong, Zhu Yousong, Zhao Chaoyang, et al. Adaptive Class Suppression Loss for Long-tail Object Detection[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 3102-3111.
21	Tan Jingru, Lu Xin, Zhang Gang, et al. Equalization Loss v2: A New Gradient Balance Approach for Long-tailed Object Detection[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 1685-1694.
22	龚安, 李中浩, 梁辰宏. 夜间多场景的邻近感知实时行人检测算法[J]. 中国图象图形学报, 2023, 28(9): 2693-2705.
	Gong An, Li Zhonghao, Liang Chenhong. NSPDet: Real-time Nearby-aware Pedestrian Detection Algorithm for Multi-scene Surveillance at Night[J]. Journal of Image and Graphics, 2023, 28(9): 2693-2705.
23	Zhu Zhe, Liang Dun, Zhang Songhai, et al. Traffic-sign Detection and Classification in the Wild[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2016: 2110-2118.
24	Lin T Y, Goyal P, Girshick R, et al. Focal Loss for Dense Object Detection[C]//2017 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2017: 2999-3007.
25	Wang Jiaqi, Chen Kai, Xu Rui, et al. CARAFE: Content-aware ReAssembly of FEatures[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 3007-3016.
26	Hou Qibin, Zhou Daquan, Feng Jiashi. Coordinate Attention for Efficient Mobile Network Design[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 13708-13717.
27	冷坤, 秦伦明, 王悉. 基于CA-ASFF-YOLOv4的交通标志识别研究[J]. 计算机工程与应用, 2023, 59(17): 169-177.
	Leng Kun, Qin Lunming, Wang Xi. Research on Traffic Sign Recognition Based on CA-ASFF-YOLOv4[J]. Computer Engineering and Applications, 2023, 59(17): 169-177.
28	Wang C Y, Bochkovskiy Alexey, Liao Hongyuan. YOLOv7: Trainable Bag-of-freebies Sets New State-of-the-art for Real-time Object Detectors[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2023: 7464-7475.
29	Ren Shaoqing, He Kaiming, Girshick R, et al. Faster R-CNN: Towards Real-time Object Detection with Region Proposal Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
30	Wang Jiaqi, Zhang Wenwei, Zhang Yuhang, et al. Seesaw Loss for Long-tailed Instance Segmentation[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 9690-9699.
31	Tan Jingru, Wang Changbao, Li Buyu, et al. Equalization Loss for Long-tailed Object Recognition[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 11659-11668.

[1]	Li Chao, Li Jiabao, Ding Caichang, Ye Zhiwei, Zuo Fangwei. Edge Surveillance Task Offloading and Resource Allocation Algorithm Based on DRL [J]. Journal of System Simulation, 2024, 36(9): 2113-2126.
[2]	Liu Peijin, Fu Xuefeng, Sun Haofeng, He Lin, Liu Shujie. A Highly Robust Target Tracking Algorithm Merging CNN and Transformer [J]. Journal of System Simulation, 2024, 36(8): 1854-1868.
[3]	Lu Yang, Liu Pengfei, Xu Siyuan, Liu Qiwang, Gu Fuqian, Wang Peng. Simulation of Rice Disease Recognition Based on Improved Attention Mechanism Embedded in PR-Net Model [J]. Journal of System Simulation, 2024, 36(6): 1322-1333.
[4]	Liu Jinhui, Chen Mengyuan, Han Pengpeng, Chen Hebao, Zhang Yukun. A Graph Neural Network Visual SLAM Algorithm for Large-angle View Motion [J]. Journal of System Simulation, 2024, 36(5): 1043-1060.
[5]	Zhang Xiliu, Zhang Xiaoling, He Minjun. Research on Vehicle Detection Method Based on Improved YOLOX-s [J]. Journal of System Simulation, 2024, 36(2): 487-496.
[6]	Dong Qingqing, Wu Hao, Qian Wenhua, Kong Fengling. RGB-D Saliency Object Detection Based on Cross-refinement and Circular Attention [J]. Journal of System Simulation, 2023, 35(9): 1931-1947.
[7]	Hao Yu, Jinxia Jiang, Xiaohan Lai, Feng Mei, Qing Wang. Surface Defect Detection of Power Equipment Using Adaptive Receptive Field Network [J]. Journal of System Simulation, 2023, 35(7): 1572-1580.
[8]	Yun Wei, Qi Luo, Yingzhi Zhao. Semantic Segmentation Model Based on Adaptive Fusion and Attention Refinement [J]. Journal of System Simulation, 2023, 35(6): 1226-1234.
[9]	Ding Shi, Xuefeng Yan, Lina Gong, Jingxuan Zhang, Donghai Guan, Mingqiang Wei. Multi-agent Cooperative Combat Simulation in Naval Battlefield with Reinforcement Learning [J]. Journal of System Simulation, 2023, 35(4): 786-796.
[10]	Nan Xiang, Lu Wang, Chongliu Jia, Yuemou Jian, Xiaoxia Ma. Simulation of Occluded Pedestrian Detection Based on Improved YOLO [J]. Journal of System Simulation, 2023, 35(2): 286-299.
[11]	Hong Sun, Yuxiang Zhang, Yuelan Ling. Research on Image Super-resolution Reconstruction Based on Loss Extraction Feedback Attention Network [J]. Journal of System Simulation, 2023, 35(2): 308-317.
[12]	Zhao Weiping, Chen Yu, Xiang Song, Liu Yuanqiang, Wang Chaoyue. Image Semantic Segmentation Algorithm Based on Improved DeepLabv3+ [J]. Journal of System Simulation, 2023, 35(11): 2333-2344.
[13]	Weidong Jin, Shuli Zhang, Peng Tang, Man Zhang. Image Dehazing Network Based on Densely Connected Residual Block and Channel Pixel Attention [J]. Journal of System Simulation, 2022, 34(8): 1663-1673.
[14]	Junjie Qiu, Hong Zheng, Yunhui Cheng. Research on Prediction of Model Based on Multi-scale LSTM [J]. Journal of System Simulation, 2022, 34(7): 1593-1604.
[15]	Yin Wang, Feixiang Wang, Qianlai Sun. Vehicle Detection Method Based on Multi Scale Feature Fusion [J]. Journal of System Simulation, 2022, 34(6): 1219-1229.