Visual Relocalization Method Combining Region Classification and Local Feature Enhancement

doi:10.16182/j.issn1004731x.joss.25-0830

Abstract

Abstract:

Visual relocalization tasks have important application value in fields such as digital twin and augmented reality. The current mainstream methods still face challenges such as mismatch between coordinate regression scale and receptive field and insufficient attention to local information. A visual relocalization method that combines region classification and local feature enhancement is proposed. The coordinate regression problem in large space is transformed into a multi-region classification problem and a coordinate regression problem inside a small scene, which significantly reduces the un-certainty of coordinate regression and makes the network globally have a large receptive field. A conditioning layer using deep feature fusion introduces the results of the upper classification layer into the lower network. Feature learning and fusion within a local region through the graph attention mechanism allows the network to learn both global and local feature information, which combined with the hierarchical regression framework, improves the stability of relocalization. Comparative experiments and analyses of the proposed method with mainstream visual relocalization methods are conducted on a publicly available multi-scene dataset. The experimental results show that the visual relocalization method proposed in this paper achieves more precise relocalization results with higher relocalization accuracy.

Key words: visual relocalization, coordinate regression, region classification, space division, graph attention

CLC Number:

TP391.9

Wang Yining, Liu Yanli, Xing Guanyu. Visual Relocalization Method Combining Region Classification and Local Feature Enhancement[J]. Journal of System Simulation, 2026, 38(1): 45-57.

Figures/Tables 8

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Comparison of experimental results on the 7-Scenes dataset

方法	指标	chess	fire	heads	office	pumpkin	kitchen	stairs
AS^[7]	$Δ t m i d$	2.6	2.3	1.1	4.0	6.5	5.3	3.8
	$Δ R m i d$	0.9	1.02	0.8	1.2	1.7	1.7	1.1
	Acc.	86.2	86.6	95.7	65.9	32.5	46.2	68.1
Hloc^[30]	$Δ t m i d$	2.4	1.8	0.9	2.6	4.4	4.0	5.1
	$Δ R m i d$	0.77	0.75	0.59	0.77	1.15	1.38	1.46
	Acc.	94.2	93.7	99.7	83.2	55.10	61.2	49.4
PixLoc^[8]	$Δ t m i d$	2	2	1	3	4	3	5
	$Δ R m i d$	0.80	0.73	0.82	0.82	1.21	1.20	1.30
	Acc.	—	—	—	—	—	—	—
DFNet^[31]	$Δ t m i d$	5	17	6	8	10	22	16
	$Δ R m i d$	1.88	6.45	3.64	2.48	2.78	5.45	3.29
	Acc.	—	—	—	—	—	—	—
DSAC++^[20]	$Δ t m i d$	2	2	1	3	4	4	9
	$Δ R m i d$	0.5	0.9	0.8	0.7	1.1	1.1	2.6
	Acc.	—	—	—	—	—	—	—
DSAC*^[21]	$Δ t m i d$	1.8	1.7	1.0	2.7	3.9	3.9	3.5
	$Δ R m i d$	0.59	0.77	0.66	0.79	1.05	1.24	0.93
	Acc.	97.1	94.5	98.8	83.9	62.0	65.5	78.0
HSCNet^[2]	$Δ t m i d$	2	2	1	3	4	4	3
	$Δ R m i d$	0.7	0.9	0.9	0.8	1.0	1.2	0.8
	Acc.	97.5	96.7	100	86.5	59.9	65.5	87.5
NBE+ SLD^[32]	$Δ t m i d$	2.2	1.8	0.9	3.2	5.6	5.3	5.5
	$Δ R m i d$	0.75	0.73	0.68	0.91	1.55	1.52	1.41
	Acc.	93.7	94.1	96.6	74.8	44.6	45.7	44.6
SC-wLS^[25]	$Δ t m i d$	2.9	5.1	2.6	6.3	8.4	9.9	17.9
	$Δ R m i d$	0.78	1.04	2.00	0.93	1.28	1.60	3.61
	Acc.	—	—	—	—	—	—	—
本文方法	$Δ t m i d$	2.1	2.2	1.1	2.6	4.0	3.9	3.2
	$Δ R m i d$	0.71	0.86	0.83	0.76	1.01	1.21	0.87
	Acc.	97.2	97.3	100	87.3	59.0	66.3	83.7

Table 1

Fig. 5

Table 2

Results of ablation experiments

方法	fire			office
方法	$Δ t m i d$	$Δ R m i d$	Acc.	$Δ t m i d$	$Δ R m i d$	Acc.
本文方法	2.2	0.86	97.3	2.6	0.76	87.3
无特征优化	2.1	0.91	96.5	2.9	0.82	86.6
无调节机制	2.7	0.95	82.4	2.9	1.02	78.2
无区域分类	2.4	0.89	94.9	3.1	0.91	81.3

Table 2

Table 3

Comparison of results by different number of region categories

类别数目	kitchen
类别数目	$Δ t m i d$	$Δ R m i d$	Acc.
$10 × 10$	4.1	1.22	64.7
$25 × 25$	3.9	1.21	66.3
$50 × 50$	4.1	1.20	66.4

Table 3

References 32

[1]	Hu Kewei, Chen Zheng, Kang Hanwen, et al. 3D Vision Technologies for a Self-developed Structural External Crack Damage Recognition Robot[J]. Automation in Construction, 2024, 159: 105262.
[2]	Li X, Wang S, Zhao Y, et al. Hierarchical Scene Coordinate Classification and Regression for Visual Localization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 11983-11992.
[3]	Bui B T, Bui H H, Tran D T, et al. D2S: Representing Sparse Descriptors and 3D Coordinates for Camera Relocalization[J]. IEEE Robotics and Automation Letters, 2024, 9(12): 11449-11456.
[4]	Shotton J, Glocker B, Zach C, et al. Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2013: 2930-2937.
[5]	Li Qing, Zhu Jiasong, Liu Jun, et al. 3D Map-guided Single Indoor Image Localization Refinement[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 161: 13-26.
[6]	Akai N. Mobile Robot Localization Considering Uncertainty of Depth Regression From Camera Images[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 1431-1438.
[7]	Sattler T, Leibe B, Kobbelt L. Efficient & Effective Prioritized Matching for Large-scale Image-based Localization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(9): 1744-1756.
[8]	Sarlin P E, Unagar A, Larsson M, et al. Back to the Feature: Learning Robust Camera Localization from Pixels to Pose[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 3246-3256.
[9]	Wang Bing, Chen Changhao, Cui Zhaopeng, et al. P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 15984-15993.
[10]	Sarlin P E, Detone D, Malisiewicz T, et al. Superglue: Learning Feature Matching with Graph Neural Networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 4937-4946.
[11]	Lindenberger P, Sarlin P E, Pollefeys M. LightGlue: Local Feature Matching at Light Speed[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2023: 17581-17592.
[12]	Sun Jiaming, Shen Zehong, Wang Yuang, et al. LoFTR: Detector-free Local Feature Matching with Transformers[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 8918-8927.
[13]	Kendall A, Grimes M, Cipolla R. PoseNet: A Convolutional Network for Real-time 6-DOF Camera Relocalization[C]//2015 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2015: 2938-2946.
[14]	Szegedy C, Liu Wei, Jia Yangqing, et al. Going Deeper with Convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2015: 1-9.
[15]	KendalL A, Cipolla R. Geometric Loss Functions for Camera Pose Regression with Deep Learning[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 6555-6564.
[16]	Brahmbhatt S, Gu Jinwei, Kim K, et al. Geometry-aware Learning of Maps for Camera Localization[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2616-2625.
[17]	Wang Bing, Chen Changhao, Lu Xiaoxuan, et al. AtLoc: Attention Guided Camera Localization[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 10393-10401.
[18]	Sattler T, Zhou Qunjie, Pollefeys M, et al. Understanding the Limitations of CNN-based Absolute Camera Pose Regression[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 3297-3307.
[19]	Brachmann E, Krull A, Nowozin S, et al. DSAC-differentiable RANSAC for Camera Localization[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 2492-2500.
[20]	Brachmann E, Rother C. Learning Less is More-6D Camera Localization via 3D Surface Regression[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 4654-4662.
[21]	Brachmann E, Rother C. Visual Camera Re-localization from RGB and RGB-D Images Using DSAC[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9): 5847-5865.
[22]	Zhou Lei, Luo Zixin, Shen Tianwei, et al. KFNet: Learning Temporal Camera Relocalization Using Kalman Filtering[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 4918-4927.
[23]	Guan Peiyu, Cao Zhiqiang, Yu Junzhi, et al. Scene Coordinate Regression Network with Global Context-guided Spatial Feature Transformation for Visual Relocalization[J]. IEEE Robotics and Automation Letters, 2021, 6(3): 5737-5744.
[24]	Ruan Jiahao, He Li, Guan Yisheng, et al. Combining Scene Coordinate Regression and Absolute Pose Regression for Visual Relocalization[C]//2023 IEEE International Conference on Robotics and Automation (ICRA). Piscataway: IEEE, 2023: 11749-11755.
[25]	Wu Xin, Zhao Hao, Li Shunkai, et al. SC-wLS: Towards Interpretable Feed-forward Camera Re-localization[C]//Computer Vision – ECCV 2022. Cham: Springer Nature Switzerland, 2022: 585-601.
[26]	Huang Zhaoyang, Zhou Han, Li Yijin, et al. VS-Net: Voting with Segmentation for Visual Localization[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 6097-6107.
[27]	Clevert D A, Unterthiner T, Hochreiter S. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)[EB/OL]. (2016-02-22) [2025-07-18]. .
[28]	Perez E, Strub F, De Vries H, et al. FiLM: Visual Reasoning with a General Conditioning Layer[C]//Proceedings of the Thirty-second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto: AAAI Press, 2018: 3942-3951.
[29]	Wang Yisen, Ma Xingjun, Chen Zaiyi, et al. Symmetric Cross Entropy for Robust Learning with Noisy Labels[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 322-330.
[30]	Sarlin P E, Cadena C, Siegwart R, et al. From Coarse to Fine: Robust Hierarchical Localization at Large Scale[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 12708-12717.
[31]	Chen Shuai, Li Xinghui, Wang Zirui, et al. DFNet: Enhance Absolute Pose Regression with Direct Feature Matching[C]//Computer Vision – ECCV 2022. Cham: Springer Nature Switzerland, 2022: 1-17.
[32]	Do T, Miksik O, Degol J, et al. Learning to Detect Scene Landmarks for Camera Localization[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 11122-11132.