面向城市复杂环境视觉地点识别算法研究

doi:10.16182/j.issn1004731x.joss.25-0031

摘要/Abstract

摘要：

针对复杂城市环境中交通流量、人群密度等动态因素降低视觉地点识别(visual place recognition，VPR)算法精度的问题，提出一种融合语义引导的视觉地点识别(semantic-guided VPR，SG-VPR)算法。设计语义引导特征抑制模块，通过构建语义引导模块与特征抑制层弱化动态物体干扰，精准提取关键静态特征；改进传统三元损失函数，提出自适应三元损失函数（adaptive triplet marginal loss，ATML），依据样本分布自适应调整边距，解决固定边距策略导致的次优解收敛问题，提升特征区分能力。实验结果表明：SG-VPR算法在Pittsburgh250k和Tokyo24/7城市复杂环境公开数据集上的表现优于现有方法，在动态干扰场景下的视觉地点识别性能得到提升。

关键词: 城市复杂环境, 视觉地点识别, 深度学习, 语义引导, 特征抑制, 自适应三元损失函数

Abstract:

Dynamic factors such as traffic flow and crowd density in complex urban environments reduce the accuracy of visual place recognition (VPR) algorithms. To solve these problems, a semantic-guided visual place recognition (SG-VPR) algorithm was proposed. A semantic-guided feature suppression module was designed. A semantic-guided module and feature suppression layer were constructed to reduce the dynamic object interference and more accurately extract the key static features. An adaptive triplet margin loss function (ATML) was proposed by improving the traditional triplet margin loss. The margins were adaptively adjusted according to the sample distribution, solving the problem of suboptimal solution convergence caused by a fixed margin strategy and improving the feature differentiation ability. Experimental results show that the SG-VPR outperforms existing methods on the Pittsburgh250k and Tokyo24/7 public datasets with complex urban environments, especially in coping with dynamic interference scenarios, which significantly improves the performance of the VPR algorithm.

Key words: complex urban environments, visual place recognition, deep learning, semantic guidance, feature suppression, adaptive triplet marginal loss(ATML)

中图分类号:

TP391.9

刘沛津,张闽心,何林等 . 面向城市复杂环境视觉地点识别算法研究[J]. 系统仿真学报, 2026, 38(3): 818-828.

Liu Peijin,Zhang Minxin,He Lin,et al . Research on Visual Place Recognition Algorithms for Complex Urban Environments[J]. Journal of System Simulation, 2026, 38(3): 818-828.

图/表 9

图1

图2

表1

图3

图4

表2

表3

图5

表4

参考文献 30

[1]	侯毅. 基于深度卷积神经网络的移动机器人视觉地点识别[D]. 长沙: 国防科技大学, 2017.
	Hou Yi. Visual Place Recognition with Deep Convolutional Neural Networks for Mobile Robots[D]. Changsha: National University of Defense Technology, 2017.
[2]	曾凡祥. 复杂环境下鲁棒实时目标跟踪技术研究[D]. 北京: 北京邮电大学, 2017.
	Zeng Fanxiang. Robust and Real-time Visual Tracking Under Complex Scenarios[D]. Beijing: Beijing University of Posts and Telecommunications, 2017.
[3]	王博. 基于卷积神经网络的视觉地点识别算法研究[D]. 广州: 华南理工大学, 2021.
	Wang Bo. Research on Visual Place Recognition Algorithm Based on Convolution Neural Network[D]. Guangzhou: South China University of Technology, 2021.
[4]	Jégou Hervé, Perronnin Florent, Douze Matthijs, et al. Aggregating Local Image Descriptors into Compact Codes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9): 1704-1716.
[5]	Sánchez Jorge, Perronnin Florent, Mensink Thomas, et al. Image Classification with the Fisher Vector: Theory and Practice[J]. International Journal of Computer Vision, 2013, 1053: 222-245.
[6]	Arandjelović Relja, Gronat Petr, Torii Akihiko, et al. NetVLAD: CNN Architecture for Weakly Supervised Place Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6): 1437-1451.
[7]	Tan Yanhai, Ji Peng, Zhang Yunzhou, et al. Learning Robust Representation and Sequence Constraint for Retrieval-based Long-term Visual Place Recognition[J]. Engineering Applications of Artificial Intelligence, 2024, 138, Part B: 109425.
[8]	Li Zhenyu, Xu Pengjie. Pyramid Transformer-based Triplet Hashing for Robust Visual Place Recognition[J]. Computer Vision and Image Understanding, 2024, 249: 104167
[9]	Kim H J, Dunn E, Frahm J M. Learned Contextual Feature Reweighting for Image Geo-localization[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 3251-3260.
[10]	Peng Guohao, Zhang Jun, Li Heshan, et al. Attentional Pyramid Pooling of Salient Visual Residuals for Place Recognition[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 865-874.
[11]	Hausler Stephen, Garg Sourav, Xu Ming, et al. Patch-NetVLAD: Multi-scale Fusion of Locally-global Descriptors for Place Recognition[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 14136-14147.
[12]	Wang Ruotong, Shen Yanqing, Zuo Weiliang, et al. TransVPR: Transformer-based Place Recognition with Multi-level Attention Aggregation[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 13638-13647.
[13]	Sunderhauf Niko, Shirazi Sareh, Jacobson Adam, et al. Place Recognition with ConvNet Landmarks: Viewpoint-robust, Condition-robust, Training-free[C]// Robotics: Science and Systems XI. Rome, Italy: Robotics: Science and Systems Foundation, 2015: 1-10.
[14]	Kalantidis Y, Mellina C, Osindero S. Cross-dimensional Weighting for Aggregated Deep Convolutional Features[C]//Computer Vision – ECCV 2016 Workshops. Cham: Springer International Publishing, 2016: 685-701.
[15]	刘沛津, 刘淑婕, 何林, 等. 基于平行全维动态注意力机制的视觉地点识别方法[J]. 液晶与显示, 2024, 39(9): 1233-1242.
	Liu Peijin, Liu Shujie, He Lin, et al. Visual Place Recognition Method Based on Parallel Omni-dimensional Dynamic Attention Mechanism[J]. Chinese Journal of Liquid Crystals and Displays, 2024, 39(9): 1233-1242.
[16]	Wang Xiaolong, Girshick R, Gupta A, et al. Non-local Neural Networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7794-7803.
[17]	Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation Networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
[18]	Schroff F, Kalenichenko D, Philbin J. FaceNet: A Unified Embedding for Face Recognition and Clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2015: 815-823.
[19]	Xiong Yu, Xu Shixiong, Meng Gaofeng. Distance-ranking-based Weighted Triplet Loss for Visual Place Recognition[C]//2023 2nd International Conference on Artificial Intelligence, Human-Computer Interaction and Robotics (AIHCIR). Piscataway: IEEE, 2023: 94-100.
[20]	Leyva-Vallina María, Strisciuglio Nicola, Petkov Nicolai. Generalized Contrastive Optimization of Siamese Networks for Place Recognition[EB/OL]. (2023-04-20) [2024-12-20]. .
[21]	Simonyan Karen, Zisserman Andrew. Very Deep Convolutional Networks for Large-scale Image Recognition[EB/OL]. (2015-04-10) [2024-12-17].
[22]	孔德磊, 方正, 李昊佳, 等. 基于事件的端到端视觉位置识别弱监督网络架构[J]. 机器人, 2022, 44(5): 613-625.
	Kong Delei, Fang Zheng, Li Haojia, et al. An End-to-end Weakly Supervised Network Architecture for Event-based Visual Place Recognition[J]. Robot, 2022, 44(5): 613-625.
[23]	Chen L C, Zhu Yukun, Papandreou G, et al. Encoder-decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 833-851.
[24]	Cordts Marius, Omran Mohamed, Ramos Sebastian, et al. The Cityscapes Dataset for Semantic Urban Scene Understanding[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2016: 3213-3223.
[25]	Torii Akihiko, Arandjelović Relja, Sivic Josef, et al. 24/7 Place Recognition by View Synthesis[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2015: 1808-1817.
[26]	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization[J]. International Journal of Computer Vision, 2020, 128(2): 336-359.
[27]	Gan Wenjian, Zhou Yang, Hu Xiaofei, et al. Convolutional MLP Orthogonal Fusion of Multiscale Features for Visual Place Recognition[J]. Scientific Reports, 2024, 14(1): 11756.
[28]	Lowry Stephanie, Sünderhauf Niko, Newman P, et al. Visual Place Recognition: A Survey[J]. IEEE Transactions on Robotics, 2016, 32(1): 1-19.
[29]	Cai Yingfeng, Zhao Junqiao, Cui Jiafeng, et al. Patch-NetVLAD+: Learned Patch Descriptor and Weighted Matching Strategy for Place Recognition[C]//2022 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI). Piscataway: IEEE, 2022: 1-8.
[30]	Zheng Rui, Jiang Zuo, Ye Yibin, et al. An Adaptive Network Fusing Light Detection and Ranging Height-sliced Bird's-eye View and Vision for Place Recognition[J]. Engineering Applications of Artificial Intelligence, 2024, 137, Part B: 109230.

算法	Pittsburgh250K(test)				Tokyo 24/7
算法	R@1	R@5	R@10	R@1	R@5	R@10
NetVLAD	86.2	93.2	95.1	67.9	77.3	80.5
CRN	85.6	93.6	95.5	75.2	83.8	87.7
APPSVR	88.8	95.6	96.3	77.1	85.7	89.5
Patch-NetVLAD	88.4	95.7	96.4	77.1	84.4	87.0
SG-VPR	90.7	96.3	96.1	79.2	85.7	89.4

模型	参数量×10⁶	R@1/%	时间/s
基线	14.7	57.1	0.04
SG-VPR	16.2	65.4	0.06

方法	Pittsburgh250K(test)			Tokyo 24/7
方法	R@1	R@5	R@10	R@1	R@5	R@10
P	88.4	95.7	96.4	77.1	84.4	87.0
P+ATML	89.5	96.0	96.5	78.3	85.0	88.3
P+语义引导的特征抑制模块	90.2	96.1	96.8	78.8	85.3	88.3
P+语义引导的特征抑制模块+ATML	90.7	96.3	96.1	79.2	85.7	89.4