Image Feature Point Matching Algorithm Based on Attention and Hierarchical Features

doi:10.16182/j.issn1004731x.joss.25-0476

Abstract

Abstract:

Feature point detection and matching is one of the core technologies in the field of intelligent driving. Aiming at the lack of consistency and continuity of feature points extracted by the existing algorithms, as well as the problem of easily ignoring the contextual semantic information when matching, this paper proposes an image feature point matching algorithm based on attention and hierarchical features (AHMF). In the feature point detection stage, differential interaction attention module (DIAM) is proposed to enhance the model's attention to the salient regions so as to improve the robustness of the feature points; further introduction of hierarchical feature enhancement module (HFEM), which extracts the hierarchical information in the feature map through the semantic feature enhancement branch and the cross-scale feature enhancement branch to improve the discriminative nature of the descriptors; in the feature point matching stage, the traditional mutual neighbor matching (MNN) algorithm is optimized, and a multi-frame image feature point matching method to screen out stable feature points. Experimental results show that compared with D2-Net, R2D2 and other algorithms, the proposed algorithm has a higher matching accuracy on the two public datasets, has a smaller number of parameters, and shows a good anti-noise interference ability.

Key words: feature point detection, keypoints, attention mechanism, feature point matching, semantic information

CLC Number:

TP391.41

Chen Na, Bai Jiajia, Zhou Qiyin, Li Jialin. Image Feature Point Matching Algorithm Based on Attention and Hierarchical Features[J]. Journal of System Simulation, 2025, 37(11): 2839-2852.

Figures/Tables 15

Fig. 1

Fig. 2

Table 1

Fig. 3

Table 2

Fig. 4

Fig. 5

Table 3

Fig. 6

Table 4

Table 5

Fig. 7

Table 6

Table 7

Fig. 8

References 32

[1]	Yin Yihao, Feng Xin, Wu Haoming. Learning for Graph Matching Based Multi-object Tracking in Auto Driving[J]. Journal of Physics: Conference Series, 2021, 1871(1): 012152.
[2]	Xie Qiwei, Hu Xiyuan, Ren Lei, et al. A Binocular Vision Application in IoT: Realtime Trustworthy Road Condition Detection System in Passable Area[J]. IEEE Transactions on Industrial Informatics, 2023, 19(1): 973-983.
[3]	You Hairong, Xie Yang. Automatic Driving Image Matching via Random Sample Consensus (RANSAC) and Spectral Clustering (SC) with Monocular Camera[J]. Review of Scientific Instruments, 2024, 95(8): 085113.
[4]	张旭, 李亚利, 陈晨, 等. 嵌入式驾驶员状态检测算法的实现与优化[J]. 自动化学报, 2012, 38(12): 2014-2022.
	Zhang Xu, Li Yali, Chen Chen, et al. Realization and Optimization of Embedded Driver Status Detection System[J]. Acta Automatica Sinica, 2012, 38(12): 2014-2022.
[5]	Xie Qiwei, Chen Xi, Zhang Liming, et al. A Robust and Efficient Video Anti-shaking Algorithm for Low-end Smartphone Platforms[J]. IEEE Transactions on Consumer Electronics, 2019, 65(1): 1-10.
[6]	宁小娟, 李洁茹, 高凡, 等. 基于最佳几何约束和RANSAC的特征匹配算法[J]. 系统仿真学报, 2022, 34(4): 727-734.
	Ning Xiaojuan, Li Jieru, Gao Fan, et al. A Feature Matching Algorithm Based on Optimal Geometric Constraints and RANSAC[J]. Journal of System Simulation, 2022, 34(4): 727-734.
[7]	潘衡岳, 王爱平, 程志全, 等. 结合图像特征和几何特征的级联图像匹配方法[J]. 系统仿真学报, 2012, 24(1): 113-116.
	Pan Hengyue, Wang Aiping, Cheng Zhiquan, et al. Cascade Object Matching by Combining Image Feature and Geometry Feature[J]. Journal of System Simulation, 2012, 24(1): 113-116.
[8]	Lowe David G. Distinctive Image Features from Scale-invariant Keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[9]	Mikolajczyk K, Schmid C. Performance Evaluation of Local Descriptors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2005, 27(10): 1615-1630.
[10]	Rublee E, Rabaud V, Konolige K, et al. ORB: An Efficient Alternative to SIFT or SURF[C]//2011 International Conference on Computer Vision. Piscataway: IEEE, 2011: 2564-2571.
[11]	Dong Jing, Hu Maoqing, Lu Jiazhen, et al. Affine Template Matching Based on Multi-scale Dense Structure Principal Direction[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 31(6): 2125-2132.
[12]	Revaud Jerome, Weinzaepfel Philippe, Harchaoui Zaid, et al. DeepMatching: Hierarchical Deformable Dense Matching[J]. International Journal of Computer Vision, 2016, 120(3): 300-323.
[13]	Korman Simon, Reichman Daniel, Tsur Gilad, et al. FasT-match: Fast Affine Template Matching[C]//2013 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2013: 2331-2338.
[14]	Ye Yuanxin, Shen Li. HOPC: A Novel Similarity Metric Based on Geometric Structural Properties for Multi-modal Remote Sensing Image Matching[J]. ISPRS Annals of the Photogrammetry Remote Sensing and Spatial Information Sciences, 2016, III-1: 9-16.
[15]	葛泉波, 李凯, 张兴国. 基于多关键点检测加权融合的无人机相对位姿估计算法[J]. 自动化学报, 2024, 50(7): 1402-1416.
	Ge Quanbo, Li Kai, Zhang Xingguo. Relative Pose Estimation Algorithm for Unmanned Aerial Vehicles Based on Weighted Fusion of Multiple Keypoint Detection[J]. Acta Automatica Sinica, 2024, 50(7): 1402-1416.
[16]	Xie Qiwei, Long Qian, Li Jianping, et al. Application of Intelligence Binocular Vision Sensor: Mobility Solutions for Automotive Perception System[J]. IEEE Sensors Journal, 2024, 24(5): 5578-5592.
[17]	Xie Qiwei, Liu Ranran, Sun Zhao, et al. A Flexible Free-space Detection System Based on Stereo Vision[J]. Neurocomputing, 2022, 485: 252-262.
[18]	辛瑞, 张霄力, 彭侠夫, 等. UFormer: 基于Transformer和U-Net结构的端到端特征点景象匹配算法[J]. 计算机科学, 2023, 50(增2): 334-339.
	Xin Rui, Zhang Xiaoli, Peng Xiafu, et al. UFormer: An End-to-end Feature Point Scene Matching Algorithm Based on Transformer and U-Net[J]. Computer Science, 2023, 50(S2): 334-339.
[19]	Ono Yuki, Trulls Eduard, Fua Pascal, et al. LF-Net: Learning Local Features from Images[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2018: 6237-6247.
[20]	DeTone Daniel, Malisiewicz T, Rabinovich A. SuperPoint: Self-supervised Interest Point Detection and Description[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway: IEEE, 2018: 337-33712.
[21]	Dusmanu Mihai, Rocco Ignacio, Pajdla Tomas, et al. D2-Net: A Trainable CNN for Joint Description and Detection of Local Features[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2019: 8084-8093.
[22]	Luo Zixin, Zhou Lei, Bai Xuyang, et al. ASLFeat: Learning Local Features of Accurate Shape and Localization[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 6588-6597.
[23]	Zhao Xiaoming, Wu Xingming, Miao Jinyu, et al. ALIKE: Accurate and Lightweight Keypoint Detection and Descriptor Extraction[J]. IEEE Transactions on Multimedia, 2023, 25: 3101-3112.
[24]	黄妍妍, 盖绍彦, 达飞鹏. 三分支空间变换注意力机制的图像匹配算法[J]. 系统工程与电子技术, 2023, 45(11): 3363-3373.
	Huang Yanyan, Ge Shaoyan, Feipeng Da. Image Matching Algorithm Based on Attention Mechanism of Three Branch Spatial Transformation[J]. Systems Engineering and Electronics, 2023, 45(11): 3363-3373.
[25]	Potje Guilherme, Cadar Felipe, Araujo André, et al. XFeat: Accelerated Features for Lightweight Image Matching[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2024: 2682-2691.
[26]	Sutton R S, McAllester D, Singh S, et al. Policy Gradient Methods for Reinforcement Learning with Function Approximation[C]//Proceedings of the 13th International Conference on Neural Information Processing Systems. Cambridge: MIT Press, 1999: 1057-1063.
[27]	Mikolajczyk K, Tuytelaars T, Schmid C, et al. A Comparison of Affine Region Detectors[J]. International Journal of Computer Vision, 2005, 65(1): 43-72.
[28]	Balntas V, Lenc K, Vedaldi A, et al. HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2017: 3852-3861.
[29]	Wilson K, Snavely N. Robust Global Translations with 1DSfM[C]//Computer Vision – ECCV 2014. Cham: Springer International Publishing, 2014: 61-75.
[30]	崔建国, 孙长库, 李玉鹏, 等. 基于SURF的快速图像匹配改进算法[J]. 仪器仪表学报, 2022, 43(8): 47-53.
	Cui Jianguo, Sun Changku, Li Yupeng, et al. An Improved Algorithm for Fast Image Matching Based on SURF[J]. Chinese Journal of Scientific Instrument, 2022, 43(8): 47-53.
[31]	Revaud Jerome, Weinzaepfel Philippe, César De Souza, et al. R2D2: Repeatable and Reliable Detector and Descriptor[C]//Proceedings of the 33rd International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2019: 12414-12424.
[32]	Potje Guilherme, Cadar Felipe, Araujo André, et al. Enhancing Deformable Local Features by Jointly Learning to Detect and Describe Keypoints[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2023: 1306-1315.

模型	Bikes	Wall	Leuven	Trees	UBC	模型	Bikes	Wall	Leuven	Trees	UBC
D2-Net	63.45	58.60	86.96	50.55	71.03	XFeat	90.77	25.19	91.36	53.42	93.33
R2D2	86.59	55.73	90.65	53.04	88.72	ALIKE-t	83.93	38.72	73.83	20.98	78.59
DALF	65.33	47.85	83.24	35.99	87.72	AHMF	91.80	52.47	92.35	54.51	86.34

模型	MMA@0.5	MMA@1.5	MMA@3	MHA@0.5	MHA@1.5	MHA@3	模型	MMA@0.5	MMA@1.5	MMA@3	MHA@0.5	MHA@1.5	MHA@3
D2-Net	4.24	19.54	42.51	1.48	15.93	41.85	XFeat	13.89	36.72	52.52	15.00	37.59	61.85
R2D2	17.54	45.34	70.04	11.67	42.30	67.56	ALIKE-t	19.28	47.44	60.52	23.33	51.11	70.74
DALF	12.96	41.57	62.29	15.56	42.59	67.96	AHMF	21.29	52.08	74.03	19.07	40.74	60.19

场景	MMA@0.5	MMA@1	MMA@1.5	MMA@2	MMA@2.5	MMA@3
2frames	50.55	90.59	97.26	98.13	98.36	98.47
3frames	51.20	91.90	98.57	99.34	99.49	99.56

场景	var=100	var=150	var=200	pu=0.01	pu=0.02	pu=0.03
2frames	97.31	96.19	95.18	88.80	83.89	80.71
3frames	98.28	97.56	96.83	93.87	90.23	87.59

亚像素细化模块	HFEM	DIAM	MS/%	Param/K
×	×	×	31.73	336.26
√	×	×	32.01	336.26
√	√	×	39.03	512.38
√	√	√	42.20	512.39