基于特征鲁棒性增强的多摄像头下车辆识别方法

doi:10.16182/j.issn1004731x.joss.22-0069

摘要/Abstract

摘要：

由于多摄像头下的视角变化、复杂环境和姿态差异等因素，同一辆车在不同场景下的图像表现出巨大的外观歧义性，为车辆的身份匹配带来了挑战。为解决这一难题，在transformer框架下提出了一种面向车辆识别的特征鲁棒性增强的方法。基于车辆结构信息是多个摄像头下域不变的信息，设计了一种轮廓特征引导增强结构信息的模块，并提出了一种结构特征感知损失来促进模型对结构信息的融合；通过将车辆的属性信息作为矢量嵌入到transformer框架中，进一步缓解多视角下车辆姿态改变带来的影响。实验结果表明，所提方法在数据集VeRi-776和VehicleID上均展现出对比同类方法的优越性与更好的识别效果。

关键词: 车辆识别, 双流网络, 结构信息, 矢量嵌入, 域不变特征提取

Abstract:

Due to factors such as viewpoint changes, complex environments, and pose differences under multiple cameras, the images of the same vehicle in different scenes show huge appearance ambiguity, which brings challenges to vehicle identity matching. In order to solve this problem, a feature robustness enhancement method for vehicle recognition is proposed under the transformer framework. Based on the fact that the structural information of the vehicle is invariant under multiple cameras, a module for enhancing structural information guided by contour features is designed, and a structural feature perception loss is proposed to promote the fusion of structural information in the model. The attribute information of the vehicle is embedded into the transformer framework as a vector, which further alleviates the influence of vehicle pose changes under multiple viewpoints. Experimental results show that the proposed method exhibits superiority and better recognition effect compared with similar methods on VeRi-776 and VehicleID datasets.

Key words: vehicle recognition, dual-stream network, structural information, vector embedding, domain-invariant feature extraction

中图分类号:

TP391.9

罗慧诚, 汪淑娟. 基于特征鲁棒性增强的多摄像头下车辆识别方法[J]. 系统仿真学报, 2023, 35(5): 1059-1074.

Huicheng Luo, Shujuan Wang. Multi-camera Vehicle Recognition Method Based on Feature Robustness Enhancement[J]. Journal of System Simulation, 2023, 35(5): 1059-1074.

图/表 13

图1

图2

图3

表1

图4

表2

表3

图5

图6

图7

图8

图9

图10

参考文献 35

1	Quispe R, Lan Cuiling, Zeng Wenjun, et al. AttributeNet: Attribute Enhanced Vehicle Re-identification[J]. Neurocomputing (S0925-2312), 2021, 465: 84-92.
2	Tang Zheng, Naphade M, Birchfield S, et al. PAMTRI: Pose-Aware Multi-task Learning for Vehicle Re-identification Using Highly Randomized Synthetic Data[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2019: 211-220.
3	Chen T S, Liu C T, Wu C W, et al. Orientation-Aware Vehicle Re-identification with Semantics-Guided Part Attention Network[C]// Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part II. Heidelberg: Springer-Verlag, 2020: 330-346.
4	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale[C]// International Conference on Learning Representations. New York, USA: ICLR, 2021.
5	Chen Jieneng, Lu Yongyi, Yu Qihang, et al. TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation[EB/OL]. (2021-02-08)[2021-10-21]. .
6	Liu Xinchen, Liu Wu, Mei Tao, et al. A Deep Learning-Based Approach to Progressive Vehicle Re-identification for Urban Surveillance[C]// Computer Vision-ECCV 2016. Cham: Springer International Publishing, 2016: 869-884.
7	Liu Hongye, Tian Yonghong, Wang Yaowei, et al. Deep Relative Distance Learning: Tell the Difference between Similar Vehicles[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2016: 2167-2175.
8	Sobel I, Feldman G. A 3x3 Isotropic Gradient Operator for Image Processing[J]. A Talk at the Stanford Artificial Project in. Piscataway, USA: IEEE, 1968: 271-272.
9	Hermans A, Beyer L, Leibe B. In Defense of the Triplet Loss for Person Re-Identification[EB/OL]. (2017-11-21) [2021-10-21]. .
10	Lou Yihang, Bai Yan, Liu Jun, et al. VERI-Wild: A Large Dataset and a New Method for Vehicle Re-identification in the Wild[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 3230-3238.
11	Liu Xinchen, Liu Wu, Mei Tao, et al. PROVID: Progressive and Multimodal Vehicle Reidentification for Large-Scale Urban Surveillance[J]. IEEE Transactions on Multimedia (S1520-9210), 2018, 20(3): 645-658.
12	Zhu Jianqing, Zeng Huanqiang, Lei Zhen, et al. A Shortly and Densely Connected Convolutional Neural Network for Vehicle Re-Identification[C]// 2018 24th International Conference on Pattern Recognition (ICPR). Piscataway, NJ, USA: IEEE, 2018: 3285-3290.
13	Zhou Yi, Shao Ling. Viewpoint-Aware Attentive Multi-View Inference for Vehicle Re-Identification[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2018: 6489-6498.
14	Hou Jinhui, Zeng Huanqiang, Zhu Jianqing, et al. Deep Quadruplet Appearance Learning for Vehicle Re-identification[J]. IEEE Transactions on Vehicular Technology (S0018-9545), 2019, 68(9): 8512-8522.
15	Zhu Yangchun, Zha Zhengjun, Zhang Tianzhu, et al. A Structured Graph Attention Network for Vehicle Re-Identification[C]// 28th ACM International Conference on Multimedia. New York, NY, USA: Association for Computing Machinery, 2020: 646-654.
16	Xu Yue, Jiang Na, Zhang Lei, et al. Multi-Scale Vehicle Re-identification Using Self-Adapting Label Smoothing Regularization[C]// ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ, USA: IEEE, 2019: 2117-2121.
17	Liu Hongye, Tian Yonghong, Wang Yaowei, et al. Deep Relative Distance Learning: Tell the Difference Between Similar Vehicles[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2016: 2167-2175.
18	Liu Xinchen, Liu Wu, Mei Tao, et al. PROVID: Progressive and Multimodal Vehicle Reidentification for Large-Scale Urban Surveillance[J]. IEEE Transactions on Multimedia (S1520-9210), 2018, 20(3): 645-658.
19	Zhu Jianqing, Zeng Huanqiang, Huang Jingchang, et al. Vehicle Re-identification Using Quadruple Directional Deep Learning Features[J]. IEEE Transactions on Intelligent Transportation Systems (S1524-9050), 2020, 21(1): 410-420.
20	Khorramshahi P, Kumar A, Peri N, et al. A Dual-Path Model with Adaptive Attention for Vehicle Re-identification[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2019: 6131-6140.
21	Chu Ruihang, Sun Yifan, Li Yadong, et al. Vehicle Re-identification With Viewpoint-Aware Metric Learning[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2019: 8281-8290.
22	Chen Guangyi, Zhang Tianren, Lu Jiwen, et al. Deep Meta Metric Learning[C]// 2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2019: 9546-9555.
23	Peng Jinjia, Wang Huibing, Zhao Tongtong, et al. Learning Multi-region Features for Vehicle Re-identification with Context-Based Ranking Method[J]. Neurocomputing (S0925-2312), 2019, 359: 427-437.
24	Qian Jingjing, Jiang Wei, Luo Hao, et al. Stripe-Based and Attribute-Aware Network: a Two-Branch Deep Model for Vehicle Re-identification[J]. Measurement Science and Technology (S0957-0233), 2020, 31(9): 095401.
25	He Bing, Li Jia, Zhao Yifan, et al. Part-Regularized Near-Duplicate Vehicle Re-identification[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 3992-4000.
26	Liu Chenghuan, Huynh D Q, Reynolds M. Urban Area Vehicle Re-Identification with Self-Attention Stair Feature Fusion and Temporal Bayesian Re-Ranking[C]// 2019 International Joint Conference on Neural Networks (IJCNN). Piscataway, NJ, USA: IEEE, 2019: 1-8.
27	Sun Ziruo, Nie Xiushan, Xi Xiaoming, et al. CFVMNet: A Multi-Branch Network for Vehicle Re-identification Based on Common Field of View[C]// 28th ACM International Conference on Multimedia. New York, NY, USA: Association for Computing Machinery, 2020: 3523-3531.
28	He Yanguang, Dong Chenhe, Wei Ying. Combination of Appearance and License Plate Features for Vehicle Re-identification[C]// 2019 IEEE International Conference on Image Processing (ICIP). Piscataway, NJ, USA: IEEE, 2019: 3108-3112.
29	Meng Dechao, Li Liang, Liu Xuejing, et al. Parsing-Based View-Aware Embedding Network for Vehicle Re-identification[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2020: 7101-7110.
30	Zhu Xiangyu, Luo Zhenbo, Fu Pei, et al. VOC-RelD: Vehicle Re-identification based on Vehicle-Orientation-Camera[C]// 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway, NJ, USA: IEEE, 2020: 2566-2573.
31	Khorramshahi P, Peri N, Chen Juncheng, et al. The Devil is in the Details: Self-Supervised Attention for Vehicle Re-identification[C]// Computer Vision-ECCV 2020. Cham: Springer International Publishing, 2020: 369-386.
32	Guo Haiyun, Zhu Kuan, Tang Ming, et al. Two-Level Attention Network with Multi-grain Ranking Loss for Vehicle Re-identification[J]. IEEE Transactions on Image Processing (S1057-7149), 2019, 28(9): 4328-4338.
33	Lou Yihang, Bai Yan, Liu Jun, et al. Embedding Adversarial Learning for Vehicle Re-identification[J]. IEEE Transactions on Image Processing (S1057-7149), 2019, 28(8): 3794-3807.
34	Abnar S, Zuidema W. Quantifying Attention Flow in Transformers[C]// 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA, USA: ACL, 2020: 4190-4197.
35	李熙莹, 周智豪, 邱铭凯. 基于部件融合特征的车辆重识别算法[J]. 计算机工程, 2019, 45(6): 12-20.
	Li Xiying, Zhou Zhihao, Qiu Mingkai. Vehicle Re-identification Algorithm Based on Component Fusion Feature[J]. Computer Engineering, 2019, 45(6): 12-20.

方法	rank-1	rank-5	mAP
PROVID	81.56	95.11	53.42
SDC-CNN	83.49	92.55	53.45
VAMI+STR	85.92	91.84	61.32
DQAL	88.50	94.41	60.01
QD-LDF	88.50	94.46	61.83
AAVER	88.97	94.70	61.18
SGAT	89.69	-	65.66
VANet	89.78	95.99	66.34
DMML	91.20	96.30	70.10
MSV	91.24	-	65.13
MRM	91.77	95.82	68.55
PAMTRI	92.86	96.97	71.88
SAN	93.30	97.10	72.50
Part Regular	94.30	98.70	74.30
SFF+SAtt+TBR	94.93	97.85	74.11
CFVMNet	95.30	98.40	77.06
App+Lic	95.41	97.38	78.08
PVEN	95.60	98.40	79.50
VOC-ReID	96.30	-	79.70
SAVER	96.40	98.60	79.60
Ours	96.78	98.39	80.56

方法	Test-800		Test-1600		Test-2400
方法	rank-1	mAP	rank-1	mAP	rank-1	mAP
SDC-CNN	56.98	63.52	50.57	80.05	42.92	49.68
VAMI	63.12	-	52.87	75.12	47.34	-
SFF+SAtt	64.50	-	59.12	79.85	54.41	-
TAMR	66.02	67.64	62.90	-	59.69	60.97
QD-DLF	72.32	76.54	70.66	88.90	64.14	68.41
AAVER	74.69	-	68.62	89.95	63.54	-
DQAL	74.74	-	71.01	86.56	68.23	-
MSV	75.10	79.30	71.80	86.1	68.70	73.30
EALN	75.11	77.50	71.78	74.20	69.30	71.00
MRM	76.64	80.02	74.20	77.32	70.86	74.02
SGAT	78.12	81.49	73.98	77.46	71.87	75.35
SAN	79.70	-	78.40	-	75.60	-
SAVER	79.90	-	77.60	-	75.30	-
App+Lic	79.50	82.70	76.90	79.90	74.80	77.70
CFVMNet	81.40	-	77.30	-	74.70	-
Ours	84.03	89.32	78.99	85.39	76.61	83.08

消融方式	rank-1	rank-5	mAP
Baseline	92.73	97.08	69.86
Baseline+SIGM	95.95	98.09	78.57
Baseline+NIEM(only type)	95.11	97.85	75.73
Baseline+NIEM(only direction)	95.36	97.91	76.82
Baseline+NIEM	95.89	98.03	77.84
Baseline+SIGM+NIEM	96.78	98.39	80.56

[1]	刘延东, 黄高翔, 陈文. 基于增强心理行为异质性的改进社会力模型[J]. 系统仿真学报, 2023, 35(5): 1120-1130.
[2]	戴宇轩, 崔承刚. 基于深度强化学习的Boost变换器控制策略[J]. 系统仿真学报, 2023, 35(5): 1109-1119.
[3]	范雅倩, 于松源, 房方. 热电联产虚拟电厂两阶段分布鲁棒优化调度[J]. 系统仿真学报, 2023, 35(5): 1046-1058.
[4]	刘君阳, 朱世松. 基于DBN的飞行保障力量配置辅助决策方法[J]. 系统仿真学报, 2023, 35(5): 1008-1019.
[5]	付玉, 张垚, 赵萌, 王绵沼, 郑江鹏, 贾晨, 陈胜勇. 基于仿真数据迁移学习的固定翼无人机检测[J]. 系统仿真学报, 2023, 35(5): 998-1007.
[6]	张元, 韩浩宇, 韩燮, 付嘉旭. 基于改进的协方差矩阵描述子的点云配准方法[J]. 系统仿真学报, 2023, 35(5): 979-986.
[7]	付超, 王冬越, 李鑫鑫, 薛旻. 自动化生产线柔性制造过程优化决策仿真平台[J]. 系统仿真学报, 2023, 35(5): 920-935.
[8]	王向前, 郭璞昊, 孟祥瑞. 采选充一体化煤炭生产物流系统仿真及优化[J]. 系统仿真学报, 2023, 35(5): 907-919.
[9]	徐浩添, 秦龙, 曾俊杰, 胡越, 张琪. 基于深度强化学习的对手建模方法研究综述[J]. 系统仿真学报, 2023, 35(4): 671-694.
[10]	赵拓, 邓汉强, 高佳隆, 黄健. 基于网络节点聚类的多无人机动态目标分配[J]. 系统仿真学报, 2023, 35(4): 695-708.
[11]	陈威, 王艳, 纪志成. 离散制造系统能耗动态建模与在线预测[J]. 系统仿真学报, 2023, 35(4): 760-772.
[12]	杨伟凯, 王艳, 纪志成. 面向知识图谱的智能生产系统工艺知识推理方法[J]. 系统仿真学报, 2023, 35(4): 773-785.
[13]	石鼎, 燕雪峰, 宫丽娜, 张静宣, 关东海, 魏明强. 强化学习驱动的海战场多智能体协同作战仿真算法[J]. 系统仿真学报, 2023, 35(4): 786-796.
[14]	周勇樟, 王艳, 纪志成. 扰动下离散再制造系统瓶颈漂移波动性分析[J]. 系统仿真学报, 2023, 35(4): 809-821.
[15]	陈强, 王意, 李康顺. 双重需求响应的虚拟电厂建模与调度研究[J]. 系统仿真学报, 2023, 35(4): 822-832.