Multi-camera Vehicle Recognition Method Based on Feature Robustness Enhancement

doi:10.16182/j.issn1004731x.joss.22-0069

Abstract

Abstract:

Due to factors such as viewpoint changes, complex environments, and pose differences under multiple cameras, the images of the same vehicle in different scenes show huge appearance ambiguity, which brings challenges to vehicle identity matching. In order to solve this problem, a feature robustness enhancement method for vehicle recognition is proposed under the transformer framework. Based on the fact that the structural information of the vehicle is invariant under multiple cameras, a module for enhancing structural information guided by contour features is designed, and a structural feature perception loss is proposed to promote the fusion of structural information in the model. The attribute information of the vehicle is embedded into the transformer framework as a vector, which further alleviates the influence of vehicle pose changes under multiple viewpoints. Experimental results show that the proposed method exhibits superiority and better recognition effect compared with similar methods on VeRi-776 and VehicleID datasets.

Key words: vehicle recognition, dual-stream network, structural information, vector embedding, domain-invariant feature extraction

CLC Number:

TP391.9

Huicheng Luo, Shujuan Wang. Multi-camera Vehicle Recognition Method Based on Feature Robustness Enhancement[J]. Journal of System Simulation, 2023, 35(5): 1059-1074.

Figures/Tables 13

Fig. 1

Fig. 2

Fig. 3

Table 1

Fig. 4

Table 2

Table 3

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

References 35

1	Quispe R, Lan Cuiling, Zeng Wenjun, et al. AttributeNet: Attribute Enhanced Vehicle Re-identification[J]. Neurocomputing (S0925-2312), 2021, 465: 84-92.
2	Tang Zheng, Naphade M, Birchfield S, et al. PAMTRI: Pose-Aware Multi-task Learning for Vehicle Re-identification Using Highly Randomized Synthetic Data[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2019: 211-220.
3	Chen T S, Liu C T, Wu C W, et al. Orientation-Aware Vehicle Re-identification with Semantics-Guided Part Attention Network[C]// Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II. Heidelberg: Springer-Verlag, 2020: 330-346.
4	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale[C]// International Conference on Learning Representations. New York, USA: ICLR, 2021.
5	Chen Jieneng, Lu Yongyi, Yu Qihang, et al. TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation[EB/OL]. (2021-02-08)[2021-10-21]. .
6	Liu Xinchen, Liu Wu, Mei Tao, et al. A Deep Learning-Based Approach to Progressive Vehicle Re-identification for Urban Surveillance[C]// Computer Vision-ECCV 2016. Cham: Springer International Publishing, 2016: 869-884.
7	Liu Hongye, Tian Yonghong, Wang Yaowei, et al. Deep Relative Distance Learning: Tell the Difference between Similar Vehicles[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2016: 2167-2175.
8	Sobel I, Feldman G. A 3x3 Isotropic Gradient Operator for Image Processing[J]. A Talk at the Stanford Artificial Project in. Piscataway, USA: IEEE, 1968: 271-272.
9	Hermans A, Beyer L, Leibe B. In Defense of the Triplet Loss for Person Re-Identification[EB/OL]. (2017-11-21) [2021-10-21]. .
10	Lou Yihang, Bai Yan, Liu Jun, et al. VERI-Wild: A Large Dataset and a New Method for Vehicle Re-identification in the Wild[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 3230-3238.
11	Liu Xinchen, Liu Wu, Mei Tao, et al. PROVID: Progressive and Multimodal Vehicle Reidentification for Large-Scale Urban Surveillance[J]. IEEE Transactions on Multimedia (S1520-9210), 2018, 20(3): 645-658.
12	Zhu Jianqing, Zeng Huanqiang, Lei Zhen, et al. A Shortly and Densely Connected Convolutional Neural Network for Vehicle Re-Identification[C]// 2018 24th International Conference on Pattern Recognition (ICPR). Piscataway, NJ, USA: IEEE, 2018: 3285-3290.
13	Zhou Yi, Shao Ling. Viewpoint-Aware Attentive Multi-View Inference for Vehicle Re-Identification[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 6489-6498.
14	Hou Jinhui, Zeng Huanqiang, Zhu Jianqing, et al. Deep Quadruplet Appearance Learning for Vehicle Re-identification[J]. IEEE Transactions on Vehicular Technology (S0018-9545), 2019, 68(9): 8512-8522.
15	Zhu Yangchun, Zha Zhengjun, Zhang Tianzhu, et al. A Structured Graph Attention Network for Vehicle Re-Identification[C]// 28th ACM International Conference on Multimedia. New York, NY, USA: Association for Computing Machinery, 2020: 646-654.
16	Xu Yue, Jiang Na, Zhang Lei, et al. Multi-Scale Vehicle Re-identification Using Self-Adapting Label Smoothing Regularization[C]// ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ, USA: IEEE, 2019: 2117-2121.
17	Liu Hongye, Tian Yonghong, Wang Yaowei, et al. Deep Relative Distance Learning: Tell the Difference Between Similar Vehicles[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2016: 2167-2175.
18	Liu Xinchen, Liu Wu, Mei Tao, et al. PROVID: Progressive and Multimodal Vehicle Reidentification for Large-Scale Urban Surveillance[J]. IEEE Transactions on Multimedia (S1520-9210), 2018, 20(3): 645-658.
19	Zhu Jianqing, Zeng Huanqiang, Huang Jingchang, et al. Vehicle Re-identification Using Quadruple Directional Deep Learning Features[J]. IEEE Transactions on Intelligent Transportation Systems (S1524-9050), 2020, 21(1): 410-420.
20	Khorramshahi P, Kumar A, Peri N, et al. A Dual-Path Model with Adaptive Attention for Vehicle Re-identification[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2019: 6131-6140.
21	Chu Ruihang, Sun Yifan, Li Yadong, et al. Vehicle Re-identification With Viewpoint-Aware Metric Learning[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2019: 8281-8290.
22	Chen Guangyi, Zhang Tianren, Lu Jiwen, et al. Deep Meta Metric Learning[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2019: 9546-9555.
23	Peng Jinjia, Wang Huibing, Zhao Tongtong, et al. Learning Multi-region Features for Vehicle Re-identification with Context-Based Ranking Method[J]. Neurocomputing (S0925-2312), 2019, 359: 427-437.
24	Qian Jingjing, Jiang Wei, Luo Hao, et al. Stripe-Based and Attribute-Aware Network: a Two-Branch Deep Model for Vehicle Re-identification[J]. Measurement Science and Technology (S0957-0233), 2020, 31(9): 095401.
25	He Bing, Li Jia, Zhao Yifan, et al. Part-Regularized Near-Duplicate Vehicle Re-identification[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 3992-4000.
26	Liu Chenghuan, Huynh D Q, Reynolds M. Urban Area Vehicle Re-Identification with Self-Attention Stair Feature Fusion and Temporal Bayesian Re-Ranking[C]// 2019 International Joint Conference on Neural Networks (IJCNN). Piscataway, NJ, USA: IEEE, 2019: 1-8.
27	Sun Ziruo, Nie Xiushan, Xi Xiaoming, et al. CFVMNet: A Multi-Branch Network for Vehicle Re-identification Based on Common Field of View[C]// 28th ACM International Conference on Multimedia. New York, NY, USA: Association for Computing Machinery, 2020: 3523-3531.
28	He Yanguang, Dong Chenhe, Wei Ying. Combination of Appearance and License Plate Features for Vehicle Re-identification[C]// 2019 IEEE International Conference on Image Processing (ICIP). Piscataway, NJ, USA: IEEE, 2019: 3108-3112.
29	Meng Dechao, Li Liang, Liu Xuejing, et al. Parsing-Based View-Aware Embedding Network for Vehicle Re-identification[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2020: 7101-7110.
30	Zhu Xiangyu, Luo Zhenbo, Fu Pei, et al. VOC-RelD: Vehicle Re-identification based on Vehicle-Orientation-Camera[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway, NJ, USA: IEEE, 2020: 2566-2573.
31	Khorramshahi P, Peri N, Chen Juncheng, et al. The Devil is in the Details: Self-Supervised Attention for Vehicle Re-identification[C]// Computer Vision-ECCV 2020. Cham: Springer International Publishing, 2020: 369-386.
32	Guo Haiyun, Zhu Kuan, Tang Ming, et al. Two-Level Attention Network with Multi-grain Ranking Loss for Vehicle Re-identification[J]. IEEE Transactions on Image Processing (S1057-7149), 2019, 28(9): 4328-4338.
33	Lou Yihang, Bai Yan, Liu Jun, et al. Embedding Adversarial Learning for Vehicle Re-identification[J]. IEEE Transactions on Image Processing (S1057-7149), 2019, 28(8): 3794-3807.
34	Abnar S, Zuidema W. Quantifying Attention Flow in Transformers[C]// 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2020: 4190-4197.
35	李熙莹, 周智豪, 邱铭凯. 基于部件融合特征的车辆重识别算法[J]. 计算机工程, 2019, 45(6): 12-20.
	Li Xiying, Zhou Zhihao, Qiu Mingkai. Vehicle Re-identification Algorithm Based on Component Fusion Feature[J]. Computer Engineering, 2019, 45(6): 12-20.

方法	rank-1	rank-5	mAP
PROVID	81.56	95.11	53.42
SDC-CNN	83.49	92.55	53.45
VAMI+STR	85.92	91.84	61.32
DQAL	88.50	94.41	60.01
QD-LDF	88.50	94.46	61.83
AAVER	88.97	94.70	61.18
SGAT	89.69	-	65.66
VANet	89.78	95.99	66.34
DMML	91.20	96.30	70.10
MSV	91.24	-	65.13
MRM	91.77	95.82	68.55
PAMTRI	92.86	96.97	71.88
SAN	93.30	97.10	72.50
Part Regular	94.30	98.70	74.30
SFF+SAtt+TBR	94.93	97.85	74.11
CFVMNet	95.30	98.40	77.06
App+Lic	95.41	97.38	78.08
PVEN	95.60	98.40	79.50
VOC-ReID	96.30	-	79.70
SAVER	96.40	98.60	79.60
Ours	96.78	98.39	80.56

方法	Test-800		Test-1600		Test-2400
方法	rank-1	mAP	rank-1	mAP	rank-1	mAP
SDC-CNN	56.98	63.52	50.57	80.05	42.92	49.68
VAMI	63.12	-	52.87	75.12	47.34	-
SFF+SAtt	64.50	-	59.12	79.85	54.41	-
TAMR	66.02	67.64	62.90	-	59.69	60.97
QD-DLF	72.32	76.54	70.66	88.90	64.14	68.41
AAVER	74.69	-	68.62	89.95	63.54	-
DQAL	74.74	-	71.01	86.56	68.23	-
MSV	75.10	79.30	71.80	86.1	68.70	73.30
EALN	75.11	77.50	71.78	74.20	69.30	71.00
MRM	76.64	80.02	74.20	77.32	70.86	74.02
SGAT	78.12	81.49	73.98	77.46	71.87	75.35
SAN	79.70	-	78.40	-	75.60	-
SAVER	79.90	-	77.60	-	75.30	-
App+Lic	79.50	82.70	76.90	79.90	74.80	77.70
CFVMNet	81.40	-	77.30	-	74.70	-
Ours	84.03	89.32	78.99	85.39	76.61	83.08

消融方式	rank-1	rank-5	mAP
Baseline	92.73	97.08	69.86
Baseline+SIGM	95.95	98.09	78.57
Baseline+NIEM(only type)	95.11	97.85	75.73
Baseline+NIEM(only direction)	95.36	97.91	76.82
Baseline+NIEM	95.89	98.03	77.84
Baseline+SIGM+NIEM	96.78	98.39	80.56

[1]	Yandong Liu, Gaoxiang Huang, Wen Chen. Improved Social Force Model Based on Enhancing Psych behavioral Heterogeneity [J]. Journal of System Simulation, 2023, 35(5): 1120-1130.
[2]	Yuxuan Dai, Chenggang Cui. Deep Reinforcement Learning-Based Control Strategy for Boost Converter [J]. Journal of System Simulation, 2023, 35(5): 1109-1119.
[3]	Yaqian Fan, Songyuan Yu, Fang Fang. Two-Stage Distributed Robust Optimal Dispatching for a Combined Heat and Power Virtual Power Plant [J]. Journal of System Simulation, 2023, 35(5): 1046-1058.
[4]	Junyang Liu, Shisong Zhu. Auxiliary Decision-Making Method for Flight Support Force Allocation Based on DBN [J]. Journal of System Simulation, 2023, 35(5): 1008-1019.
[5]	Yu Fu, Yao Zhang, Meng Zhao, Mianzhao Wang, Jiangpeng Zheng, Chen Jia, Shengyong Chen. Fixed-Wing UAV Detection Based on Simulated Data Transfer Learning [J]. Journal of System Simulation, 2023, 35(5): 998-1007.
[6]	Yuan Zhang, Haoyu Han, Xie Han, Jiaxu Fu. Point Cloud Registration Method Based on Improved Covariance Matrix Descriptor [J]. Journal of System Simulation, 2023, 35(5): 979-986.
[7]	Chao Fu, Dongyue Wang, Xinxin Li, Min Xue. Simulation Platform for Optimization and Decision Making of Flexible Manufacturing Process of Automatic Production Lines [J]. Journal of System Simulation, 2023, 35(5): 920-935.
[8]	Xiangqian Wang, Puhao Guo, Xiangrui Meng. Simulation and Optimization of Integrated Production Logistics System of Underground Coal Mining, Dressing, and Backfilling [J]. Journal of System Simulation, 2023, 35(5): 907-919.
[9]	Haotian Xu, Long Qin, Junjie Zeng, Yue Hu, Qi Zhang. Research Progress of Opponent Modeling Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2023, 35(4): 671-694.
[10]	Tuo Zhao, Hanqiang Deng, Jialong Gao, Jian Huang. Dynamic Target Assignment of Multiple Unmanned Aerial Vehicles Based on Clustering of Network Nodes [J]. Journal of System Simulation, 2023, 35(4): 695-708.
[11]	Wei Chen, Yan Wang, Zhicheng Ji. Dynamics Modeling and Online Prediction of Energy Consumption of Discrete Manufacturing System [J]. Journal of System Simulation, 2023, 35(4): 760-772.
[12]	Weikai Yang, Yan Wang, Zhicheng Ji. Knowledge Graph-based Process Knowledge Reasoning Method for Intelligent Production System [J]. Journal of System Simulation, 2023, 35(4): 773-785.
[13]	Ding Shi, Xuefeng Yan, Lina Gong, Jingxuan Zhang, Donghai Guan, Mingqiang Wei. Multi-agent Cooperative Combat Simulation in Naval Battlefield with Reinforcement Learning [J]. Journal of System Simulation, 2023, 35(4): 786-796.
[14]	Yongzhang Zhou, Yan Wang, Zhicheng Ji. Bottleneck Drift Fluctuation Analysis of Discrete Remanufacturing System under Disturbance [J]. Journal of System Simulation, 2023, 35(4): 809-821.
[15]	Qiang Chen, Yi Wang, Kangshun Li. Research on Modeling and Scheduling of Virtual Power Plant with Dual Demand Response [J]. Journal of System Simulation, 2023, 35(4): 822-832.