结合区域分类和局部特征增强的视觉重定位方法

doi:10.16182/j.issn1004731x.joss.25-0830

摘要/Abstract

摘要：

视觉重定位在机器人、数字孪生、增强现实等领域具有重要应用价值。针对目前主流视觉重定位方法在实际应用中面临坐标回归尺度与感受野不匹配，局部信息关注不足等问题，提出基于神经网络的结合区域分类和局部特征增强的视觉重定位方法。将大规模场景的坐标回归问题转变为多区域的分类问题与小型场景的坐标回归问题，显著降低了坐标回归的不确定性，使网络全局具有较大的感受野。使用深度特征融合的调节层将上层分类的结果引入下层网络。通过图注意力机制进行局部区域内的特征学习与融合，使网络可以同时学习全局与局部的特征信息，结合分层次的回归框架，提升了重定位的稳定性。在公开多场景数据集上进行对比实验和分析，结果表明：所提视觉重定位方法取得了更为精确的重定位结果与更高的重定位准确率。

关键词: 视觉重定位, 坐标回归, 区域分类, 空间分割, 图注意力

Abstract:

Visual relocalization tasks have important application value in fields such as digital twin and augmented reality. The current mainstream methods still face challenges such as mismatch between coordinate regression scale and receptive field and insufficient attention to local information. A visual relocalization method that combines region classification and local feature enhancement is proposed. The coordinate regression problem in large space is transformed into a multi-region classification problem and a coordinate regression problem inside a small scene, which significantly reduces the un-certainty of coordinate regression and makes the network globally have a large receptive field. A conditioning layer using deep feature fusion introduces the results of the upper classification layer into the lower network. Feature learning and fusion within a local region through the graph attention mechanism allows the network to learn both global and local feature information, which combined with the hierarchical regression framework, improves the stability of relocalization. Comparative experiments and analyses of the proposed method with mainstream visual relocalization methods are conducted on a publicly available multi-scene dataset. The experimental results show that the visual relocalization method proposed in this paper achieves more precise relocalization results with higher relocalization accuracy.

Key words: visual relocalization, coordinate regression, region classification, space division, graph attention

中图分类号:

TP391.9

王宜宁,刘艳丽,邢冠宇 . 结合区域分类和局部特征增强的视觉重定位方法[J]. 系统仿真学报, 2026, 38(1): 45-57.

Wang Yining,Liu Yanli,Xing Guanyu . Visual Relocalization Method Combining Region Classification and Local Feature Enhancement[J]. Journal of System Simulation, 2026, 38(1): 45-57.

图/表 8

图1

图2

图3

图4

表1

7-Scenes数据集上对比实验结果

方法	指标	chess	fire	heads	office	pumpkin	kitchen	stairs
AS^[7]	$Δ t m i d$	2.6	2.3	1.1	4.0	6.5	5.3	3.8
	$Δ R m i d$	0.9	1.02	0.8	1.2	1.7	1.7	1.1
	Acc.	86.2	86.6	95.7	65.9	32.5	46.2	68.1
Hloc^[30]	$Δ t m i d$	2.4	1.8	0.9	2.6	4.4	4.0	5.1
	$Δ R m i d$	0.77	0.75	0.59	0.77	1.15	1.38	1.46
	Acc.	94.2	93.7	99.7	83.2	55.10	61.2	49.4
PixLoc^[8]	$Δ t m i d$	2	2	1	3	4	3	5
	$Δ R m i d$	0.80	0.73	0.82	0.82	1.21	1.20	1.30
	Acc.	—	—	—	—	—	—	—
DFNet^[31]	$Δ t m i d$	5	17	6	8	10	22	16
	$Δ R m i d$	1.88	6.45	3.64	2.48	2.78	5.45	3.29
	Acc.	—	—	—	—	—	—	—
DSAC++^[20]	$Δ t m i d$	2	2	1	3	4	4	9
	$Δ R m i d$	0.5	0.9	0.8	0.7	1.1	1.1	2.6
	Acc.	—	—	—	—	—	—	—
DSAC*^[21]	$Δ t m i d$	1.8	1.7	1.0	2.7	3.9	3.9	3.5
	$Δ R m i d$	0.59	0.77	0.66	0.79	1.05	1.24	0.93
	Acc.	97.1	94.5	98.8	83.9	62.0	65.5	78.0
HSCNet^[2]	$Δ t m i d$	2	2	1	3	4	4	3
	$Δ R m i d$	0.7	0.9	0.9	0.8	1.0	1.2	0.8
	Acc.	97.5	96.7	100	86.5	59.9	65.5	87.5
NBE+ SLD^[32]	$Δ t m i d$	2.2	1.8	0.9	3.2	5.6	5.3	5.5
	$Δ R m i d$	0.75	0.73	0.68	0.91	1.55	1.52	1.41
	Acc.	93.7	94.1	96.6	74.8	44.6	45.7	44.6
SC-wLS^[25]	$Δ t m i d$	2.9	5.1	2.6	6.3	8.4	9.9	17.9
	$Δ R m i d$	0.78	1.04	2.00	0.93	1.28	1.60	3.61
	Acc.	—	—	—	—	—	—	—
本文方法	$Δ t m i d$	2.1	2.2	1.1	2.6	4.0	3.9	3.2
	$Δ R m i d$	0.71	0.86	0.83	0.76	1.01	1.21	0.87
	Acc.	97.2	97.3	100	87.3	59.0	66.3	83.7

表1

图5

表2

消融实验结果

方法	fire			office
方法	$Δ t m i d$	$Δ R m i d$	Acc.	$Δ t m i d$	$Δ R m i d$	Acc.
本文方法	2.2	0.86	97.3	2.6	0.76	87.3
无特征优化	2.1	0.91	96.5	2.9	0.82	86.6
无调节机制	2.7	0.95	82.4	2.9	1.02	78.2
无区域分类	2.4	0.89	94.9	3.1	0.91	81.3

表2

表3

不同区域分类数目的重定位结果比较

类别数目	kitchen
类别数目	$Δ t m i d$	$Δ R m i d$	Acc.
$10 × 10$	4.1	1.22	64.7
$25 × 25$	3.9	1.21	66.3
$50 × 50$	4.1	1.20	66.4

表3

参考文献 32

[1]	Hu Kewei, Chen Zheng, Kang Hanwen, et al. 3D Vision Technologies for a Self-developed Structural External Crack Damage Recognition Robot[J]. Automation in Construction, 2024, 159: 105262.
[2]	Li X, Wang S, Zhao Y, et al. Hierarchical Scene Coordinate Classification and Regression for Visual Localization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020: 11983-11992.
[3]	Bui B T, Bui H H, Tran D T, et al. D2S: Representing Sparse Descriptors and 3D Coordinates for Camera Relocalization[J]. IEEE Robotics and Automation Letters, 2024, 9(12): 11449-11456.
[4]	Shotton J, Glocker B, Zach C, et al. Scene Coordinate Regression Forests for Camera Relocalization in RGB-D Images[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2013: 2930-2937.
[5]	Li Qing, Zhu Jiasong, Liu Jun, et al. 3D Map-guided Single Indoor Image Localization Refinement[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 161: 13-26.
[6]	Akai N. Mobile Robot Localization Considering Uncertainty of Depth Regression From Camera Images[J]. IEEE Robotics and Automation Letters, 2022, 7(2): 1431-1438.
[7]	Sattler T, Leibe B, Kobbelt L. Efficient & Effective Prioritized Matching for Large-scale Image-based Localization[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(9): 1744-1756.
[8]	Sarlin P E, Unagar A, Larsson M, et al. Back to the Feature: Learning Robust Camera Localization from Pixels to Pose[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 3246-3256.
[9]	Wang Bing, Chen Changhao, Cui Zhaopeng, et al. P2-Net: Joint Description and Detection of Local Features for Pixel and Point Matching[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 15984-15993.
[10]	Sarlin P E, Detone D, Malisiewicz T, et al. Superglue: Learning Feature Matching with Graph Neural Networks[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 4937-4946.
[11]	Lindenberger P, Sarlin P E, Pollefeys M. LightGlue: Local Feature Matching at Light Speed[C]//2023 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2023: 17581-17592.
[12]	Sun Jiaming, Shen Zehong, Wang Yuang, et al. LoFTR: Detector-free Local Feature Matching with Transformers[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 8918-8927.
[13]	Kendall A, Grimes M, Cipolla R. PoseNet: A Convolutional Network for Real-time 6-DOF Camera Relocalization[C]//2015 IEEE International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2015: 2938-2946.
[14]	Szegedy C, Liu Wei, Jia Yangqing, et al. Going Deeper with Convolutions[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2015: 1-9.
[15]	KendalL A, Cipolla R. Geometric Loss Functions for Camera Pose Regression with Deep Learning[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 6555-6564.
[16]	Brahmbhatt S, Gu Jinwei, Kim K, et al. Geometry-aware Learning of Maps for Camera Localization[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2616-2625.
[17]	Wang Bing, Chen Changhao, Lu Xiaoxuan, et al. AtLoc: Attention Guided Camera Localization[C]//Proceedings of the AAAI Conference on Artificial Intelligence. Palo Alto: AAAI Press, 2020: 10393-10401.
[18]	Sattler T, Zhou Qunjie, Pollefeys M, et al. Understanding the Limitations of CNN-based Absolute Camera Pose Regression[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 3297-3307.
[19]	Brachmann E, Krull A, Nowozin S, et al. DSAC-differentiable RANSAC for Camera Localization[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 2492-2500.
[20]	Brachmann E, Rother C. Learning Less is More-6D Camera Localization via 3D Surface Regression[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 4654-4662.
[21]	Brachmann E, Rother C. Visual Camera Re-localization from RGB and RGB-D Images Using DSAC[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(9): 5847-5865.
[22]	Zhou Lei, Luo Zixin, Shen Tianwei, et al. KFNet: Learning Temporal Camera Relocalization Using Kalman Filtering[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 4918-4927.
[23]	Guan Peiyu, Cao Zhiqiang, Yu Junzhi, et al. Scene Coordinate Regression Network with Global Context-guided Spatial Feature Transformation for Visual Relocalization[J]. IEEE Robotics and Automation Letters, 2021, 6(3): 5737-5744.
[24]	Ruan Jiahao, He Li, Guan Yisheng, et al. Combining Scene Coordinate Regression and Absolute Pose Regression for Visual Relocalization[C]//2023 IEEE International Conference on Robotics and Automation (ICRA). Piscataway: IEEE, 2023: 11749-11755.
[25]	Wu Xin, Zhao Hao, Li Shunkai, et al. SC-wLS: Towards Interpretable Feed-forward Camera Re-localization[C]//Computer Vision – ECCV 2022. Cham: Springer Nature Switzerland, 2022: 585-601.
[26]	Huang Zhaoyang, Zhou Han, Li Yijin, et al. VS-Net: Voting with Segmentation for Visual Localization[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2021: 6097-6107.
[27]	Clevert D A, Unterthiner T, Hochreiter S. Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)[EB/OL]. (2016-02-22) [2025-07-18]. .
[28]	Perez E, Strub F, De Vries H, et al. FiLM: Visual Reasoning with a General Conditioning Layer[C]//Proceedings of the Thirty-second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto: AAAI Press, 2018: 3942-3951.
[29]	Wang Yisen, Ma Xingjun, Chen Zaiyi, et al. Symmetric Cross Entropy for Robust Learning with Noisy Labels[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 322-330.
[30]	Sarlin P E, Cadena C, Siegwart R, et al. From Coarse to Fine: Robust Hierarchical Localization at Large Scale[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 12708-12717.
[31]	Chen Shuai, Li Xinghui, Wang Zirui, et al. DFNet: Enhance Absolute Pose Regression with Direct Feature Matching[C]//Computer Vision – ECCV 2022. Cham: Springer Nature Switzerland, 2022: 1-17.
[32]	Do T, Miksik O, Degol J, et al. Learning to Detect Scene Landmarks for Camera Localization[C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2022: 11122-11132.