Research on Visual Place Recognition Algorithms for Complex Urban Environments

doi:10.16182/j.issn1004731x.joss.25-0031

Abstract

Abstract:

Dynamic factors such as traffic flow and crowd density in complex urban environments reduce the accuracy of visual place recognition (VPR) algorithms. To solve these problems, a semantic-guided visual place recognition (SG-VPR) algorithm was proposed. A semantic-guided feature suppression module was designed. A semantic-guided module and feature suppression layer were constructed to reduce the dynamic object interference and more accurately extract the key static features. An adaptive triplet margin loss function (ATML) was proposed by improving the traditional triplet margin loss. The margins were adaptively adjusted according to the sample distribution, solving the problem of suboptimal solution convergence caused by a fixed margin strategy and improving the feature differentiation ability. Experimental results show that the SG-VPR outperforms existing methods on the Pittsburgh250k and Tokyo24/7 public datasets with complex urban environments, especially in coping with dynamic interference scenarios, which significantly improves the performance of the VPR algorithm.

Key words: complex urban environments, visual place recognition, deep learning, semantic guidance, feature suppression, adaptive triplet marginal loss(ATML)

CLC Number:

TP391.9

Liu Peijin, Zhang Minxin, He Lin, Sun Yige, Su Tingqi. Research on Visual Place Recognition Algorithms for Complex Urban Environments[J]. Journal of System Simulation, 2026, 38(3): 818-828.

Figures/Tables 9

Fig. 1

Fig. 2

Table 1

Fig. 3

Fig. 4

Table 2

Table 3

Fig. 5

Table 4

References 30

[1]	侯毅. 基于深度卷积神经网络的移动机器人视觉地点识别[D]. 长沙: 国防科技大学, 2017.
	Hou Yi. Visual Place Recognition with Deep Convolutional Neural Networks for Mobile Robots[D]. Changsha: National University of Defense Technology, 2017.
[2]	曾凡祥. 复杂环境下鲁棒实时目标跟踪技术研究[D]. 北京: 北京邮电大学, 2017.
	Zeng Fanxiang. Robust and Real-time Visual Tracking Under Complex Scenarios[D]. Beijing: Beijing University of Posts and Telecommunications, 2017.
[3]	王博. 基于卷积神经网络的视觉地点识别算法研究[D]. 广州: 华南理工大学, 2021.
	Wang Bo. Research on Visual Place Recognition Algorithm Based on Convolution Neural Network[D]. Guangzhou: South China University of Technology, 2021.
[4]	Jégou Hervé, Perronnin Florent, Douze Matthijs, et al. Aggregating Local Image Descriptors into Compact Codes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(9): 1704-1716.
[5]	Sánchez Jorge, Perronnin Florent, Mensink Thomas, et al. Image Classification with the Fisher Vector: Theory and Practice[J]. International Journal of Computer Vision, 2013, 1053: 222-245.
[6]	Arandjelović Relja, Gronat Petr, Torii Akihiko, et al. NetVLAD: CNN Architecture for Weakly Supervised Place Recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6): 1437-1451.
[7]	Tan Yanhai, Ji Peng, Zhang Yunzhou, et al. Learning Robust Representation and Sequence Constraint for Retrieval-based Long-term Visual Place Recognition[J]. Engineering Applications of Artificial Intelligence, 2024, 138, Part B: 109425.
[8]	Li Zhenyu, Xu Pengjie. Pyramid Transformer-based Triplet Hashing for Robust Visual Place Recognition[J]. Computer Vision and Image Understanding, 2024, 249: 104167
[9]	Kim H J, Dunn E, Frahm J M. Learned Contextual Feature Reweighting for Image Geo-localization[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 3251-3260.
[10]	Peng Guohao, Zhang Jun, Li Heshan, et al. Attentional Pyramid Pooling of Salient Visual Residuals for Place Recognition[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 865-874.
[11]	Hausler Stephen, Garg Sourav, Xu Ming, et al. Patch-NetVLAD: Multi-scale Fusion of Locally-global Descriptors for Place Recognition[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 14136-14147.
[12]	Wang Ruotong, Shen Yanqing, Zuo Weiliang, et al. TransVPR: Transformer-based Place Recognition with Multi-level Attention Aggregation[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 13638-13647.
[13]	Sunderhauf Niko, Shirazi Sareh, Jacobson Adam, et al. Place Recognition with ConvNet Landmarks: Viewpoint-robust, Condition-robust, Training-free[C]// Robotics: Science and Systems XI. Rome, Italy: Robotics: Science and Systems Foundation, 2015: 1-10.
[14]	Kalantidis Y, Mellina C, Osindero S. Cross-dimensional Weighting for Aggregated Deep Convolutional Features[C]//Computer Vision – ECCV 2016 Workshops. Cham: Springer International Publishing, 2016: 685-701.
[15]	刘沛津, 刘淑婕, 何林, 等. 基于平行全维动态注意力机制的视觉地点识别方法[J]. 液晶与显示, 2024, 39(9): 1233-1242.
	Liu Peijin, Liu Shujie, He Lin, et al. Visual Place Recognition Method Based on Parallel Omni-dimensional Dynamic Attention Mechanism[J]. Chinese Journal of Liquid Crystals and Displays, 2024, 39(9): 1233-1242.
[16]	Wang Xiaolong, Girshick R, Gupta A, et al. Non-local Neural Networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7794-7803.
[17]	Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation Networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
[18]	Schroff F, Kalenichenko D, Philbin J. FaceNet: A Unified Embedding for Face Recognition and Clustering[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2015: 815-823.
[19]	Xiong Yu, Xu Shixiong, Meng Gaofeng. Distance-ranking-based Weighted Triplet Loss for Visual Place Recognition[C]//2023 2nd International Conference on Artificial Intelligence, Human-Computer Interaction and Robotics (AIHCIR). Piscataway: IEEE, 2023: 94-100.
[20]	Leyva-Vallina María, Strisciuglio Nicola, Petkov Nicolai. Generalized Contrastive Optimization of Siamese Networks for Place Recognition[EB/OL]. (2023-04-20) [2024-12-20]. .
[21]	Simonyan Karen, Zisserman Andrew. Very Deep Convolutional Networks for Large-scale Image Recognition[EB/OL]. (2015-04-10) [2024-12-17].
[22]	孔德磊, 方正, 李昊佳, 等. 基于事件的端到端视觉位置识别弱监督网络架构[J]. 机器人, 2022, 44(5): 613-625.
	Kong Delei, Fang Zheng, Li Haojia, et al. An End-to-end Weakly Supervised Network Architecture for Event-based Visual Place Recognition[J]. Robot, 2022, 44(5): 613-625.
[23]	Chen L C, Zhu Yukun, Papandreou G, et al. Encoder-decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 833-851.
[24]	Cordts Marius, Omran Mohamed, Ramos Sebastian, et al. The Cityscapes Dataset for Semantic Urban Scene Understanding[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2016: 3213-3223.
[25]	Torii Akihiko, Arandjelović Relja, Sivic Josef, et al. 24/7 Place Recognition by View Synthesis[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2015: 1808-1817.
[26]	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization[J]. International Journal of Computer Vision, 2020, 128(2): 336-359.
[27]	Gan Wenjian, Zhou Yang, Hu Xiaofei, et al. Convolutional MLP Orthogonal Fusion of Multiscale Features for Visual Place Recognition[J]. Scientific Reports, 2024, 14(1): 11756.
[28]	Lowry Stephanie, Sünderhauf Niko, Newman P, et al. Visual Place Recognition: A Survey[J]. IEEE Transactions on Robotics, 2016, 32(1): 1-19.
[29]	Cai Yingfeng, Zhao Junqiao, Cui Jiafeng, et al. Patch-NetVLAD+: Learned Patch Descriptor and Weighted Matching Strategy for Place Recognition[C]//2022 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems (MFI). Piscataway: IEEE, 2022: 1-8.
[30]	Zheng Rui, Jiang Zuo, Ye Yibin, et al. An Adaptive Network Fusing Light Detection and Ranging Height-sliced Bird's-eye View and Vision for Place Recognition[J]. Engineering Applications of Artificial Intelligence, 2024, 137, Part B: 109230.

算法	Pittsburgh250K(test)				Tokyo 24/7
算法	R@1	R@5	R@10	R@1	R@5	R@10
NetVLAD	86.2	93.2	95.1	67.9	77.3	80.5
CRN	85.6	93.6	95.5	75.2	83.8	87.7
APPSVR	88.8	95.6	96.3	77.1	85.7	89.5
Patch-NetVLAD	88.4	95.7	96.4	77.1	84.4	87.0
SG-VPR	90.7	96.3	96.1	79.2	85.7	89.4

模型	参数量×10⁶	R@1/%	时间/s
基线	14.7	57.1	0.04
SG-VPR	16.2	65.4	0.06

方法	Pittsburgh250K(test)			Tokyo 24/7
方法	R@1	R@5	R@10	R@1	R@5	R@10
P	88.4	95.7	96.4	77.1	84.4	87.0
P+ATML	89.5	96.0	96.5	78.3	85.0	88.3
P+语义引导的特征抑制模块	90.2	96.1	96.8	78.8	85.3	88.3
P+语义引导的特征抑制模块+ATML	90.7	96.3	96.1	79.2	85.7	89.4