A Method for Road Extraction Using Masked Image Modeling and Contrastive Learning

doi:10.16182/j.issn1004731x.joss.24-1083

Abstract

Abstract:

Aiming at the occlusion problem of road extraction from remote sensing images, a road extraction method combining MIM and CL is proposed, the model training process includes a masked pre-training stage and a contrast training stage. The masked pre-training stage mainly carries out mask image reconstruction, and trains the model to recover the whole image from some areas that are randomly occluded. The comparison training stage is mainly for the prediction error and low confidence regions to learn the comparison, to narrow the distance between the features of the same category and increase the distance between the features of different categories. The experimental results verify the effectiveness and usability of this method.

Key words: remote sensing image, road extraction, masked image modeling, contrastive learning, image reconstruction

CLC Number:

TP751.1

Wu Jiangjiang, Li Zhenghong, Sha Zhichao, Chen Hao, Peng Shuang, Du Chun, Li Jun. A Method for Road Extraction Using Masked Image Modeling and Contrastive Learning[J]. Journal of System Simulation, 2025, 37(4): 922-932.

Figures/Tables 10

Fig. 1

Fig. 2

Fig. 3

Table 1

Fig. 4

Table 2

Fig. 5

Table 3

Fig. 6

Fig. 7

References 21

1	Bong D B L, Lai K C, Joseph A. Automatic Road Network Recognition and Extraction for Urban Planning[J]. International Journal of Applied Science, Engineering and Technology, 2009, 5(1): 209-215.
2	Manandhar Prajowal, Prashanth Reddy Marpu, Aung Zeyar. Deep Learning Approach to Update Road Network Using VGI Data[C]//2018 International Conference on Signal Processing and Information Security (ICSPIS). Piscataway: IEEE, 2018: 1-4.
3	Sravanthi Peddinti A, Arjun Singh Chouhan, Asisa Kumar Panigrahy. Road Extraction Using Aerial Images for Future Navigation[J]. Materials Today: Proceedings, 2021, 47, Part 17: 6306-6308.
4	Kahraman I, Karas I R, Akay A E. Road Extraction Techniques from Remote Sensing Images: A Review[C]//The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Geospatial and Disaster Risk Management. Germany: Copernicus Publications 2018: 339-342.
5	Liu Pengfei, Wang Qing, Yang Gaochao, et al. Survey of Road Extraction Methods in Remote Sensing Images Based on Deep Learning[J]. PFG-Journal of Photogrammetry, Remote Sensing and Geoinformation Science, 2022, 90(2): 135-159.
6	Jaiswal A, Babu A R, Zadeh M Z, et al. A Survey on Contrastive Self-supervised Learning[J]. Technologies, 2020, 9(1): 2.
7	Xu Yingxiao, Chen Hao, Du Chun, et al. MSACon: Mining Spatial Attention-based Contextual Information for Road Extraction[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60: 1-17.
8	Wang Yooseung, Seo Junghoon, Jeon Taegyun. NL-LinkNet: Toward Lighter but More Accurate Road Extraction with Nonlocal Operations[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 1-5.
9	Mei Jie, Li Roujing, Gao Wang, et al. CoANet: Connectivity Attention Network for Road Extraction from Satellite Imagery[J]. IEEE Transactions on Image Processing, 2021, 30: 8540-8552.
10	Li Ruirui, Gao Bochuan, Xu Qizhi. Gated Auxiliary Edge Detection Task for Road Extraction with Weight-balanced Loss[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(5): 786-790.
11	Chen Hao, Li Zhenghong, Wu Jiangjiang, et al. SemiRoadExNet: A Semi-supervised Network for Road Extraction from Remote Sensing Imagery via Adversarial Learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 198: 169-183.
12	Ding Lei, Bruzzone Lorenzo. DiResNet: Direction-aware Residual Network for Road Extraction in VHR Remote Sensing Images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(12): 10243-10254.
13	Zhang Chaoning, Zhang Chenshuang, Song Junha, et al. A Survey on Masked Autoencoder for Self-supervised Learning in Vision and Beyond[EB/OL]. (2022-07-30) [2024-02-24]. .
14	Li Siyuan, Wu Di, Wu Fang, et al. Architecture-agnostic Masked Image Modeling—From ViT back to CNN[EB/OL]. (2023-06-02) [2024-02-24]. .
15	Demir I, Koperski K, Lindenbaum D, et al. DeepGlobe 2018: A Challenge to Parse the Earth Through Satellite Images[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway: IEEE, 2018: 172-181.
16	Mnih V. Machine Learning for Aerial Image Labeling[M]. Toronto: University of Toronto, 2013.
17	Chen L C, Zhu Yukun, Papandreou G, et al. Encoder-decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 833-851.
18	Zhou Lichen, Zhang Chuang, Wu Ming. D-LinkNet: LinkNet with Pretrained Encoder and Dilated Convolution for High Resolution Satellite Imagery Road Extraction[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway: IEEE, 2018: 192-1924.
19	Chaurasia A, Culurciello E. LinkNet: Exploiting Encoder Representations for Efficient Semantic Segmentation[C]//2017 IEEE Visual Communications and Image Processing (VCIP). Piscataway: IEEE, 2017: 1-4.
20	Chen Sibao, Ji Yuxin, Tang Jin, et al. DBRANet: Road Extraction by Dual-branch Encoder and Regional Attention Decoder[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 1-5.
21	Yang Zhigang, Zhou Daoxiang, Yang Ying, et al. Road Extraction from Satellite Imagery by Road Context and Full-stage Feature[J]. IEEE Geoscience and Remote Sensing Letters, 2023, 20: 1-5.

方法	P	R	F₁	IoU
DeepLab v3+	0.793 5	0.738 2	0.764 9	0.619 2
D-LinkNet	0.745 3	0.773 2	0.759 0	0.611 6
NL-LinkNet	0.739 4	0.779 9	0.759 1	0.611 8
RoadExNet	0.738 6	0.783 8	0.760 5	0.613 6
DBRANet	0.747 6	0.780 7	0.763 8	0.617 8
RCFSNet	0.765 7	0.772 4	0.769 0	0.624 7
本文	0.768 4	0.803 0	0.790 8	0.654 0

方法	P	R	F₁	IoU
DeepLab v3+	0.839 1	0.711 0	0.769 7	0.625 7
D-LinkNet	0.791 4	0.768 1	0.779 6	0.638 7
NL-LinkNet	0.804 5	0.758 1	0.780 6	0.640 2
RoadExNet	0.803 8	0.759 8	0.781 2	0.640 9
DBRANet	0.788 3	0.779 5	0.783 9	0.644 6
RCFSNet	0.792 5	0.775 9	0.784 1	0.644 9
本文	0.786 5	0.785 9	0.786 2	0.647 7

数据集	方法	P	R	F₁	IoU
DeepGlobe Road Extraction	Without MIM&CL	0.761 2	0.783 9	0.772 4	0.629 2
	Without MIM	0.758 4	0.789 4	0.773 6	0.630 8
	Without CL	0.766 1	0.782 4	0.774 2	0.631 6
	Ours	0.768 4	0.803 0	0.790 8	0.654 0
Massachusetts Road	Without MIM&CL	0.798 1	0.755 2	0.776 1	0.634 1
	Without MIM	0.798 6	0.765 8	0.781 9	0.641 9
	Without CL	0.803 1	0.761 8	0.781 9	0.641 9
	Ours	0.786 5	0.785 9	0.786 2	0.647 7

[1]	Li Jie, Liu Yang, Li Liang, Su Bengan, Wei Jialong, Zhou Guangda, Shi Yanmin, Zhao Zhen. Remote Sensing Small Object Detection Based on Cross-stage Two-branch Feature Aggregation [J]. Journal of System Simulation, 2025, 37(4): 1025-1040.
[2]	Hong Sun, Yuxiang Zhang, Yuelan Ling. Research on Image Super-resolution Reconstruction Based on Loss Extraction Feedback Attention Network [J]. Journal of System Simulation, 2023, 35(2): 308-317.
[3]	Xin Wang, Mingjun Xu, Jian Xiao, Lizhong Xu. Water Body Extraction from High Resolution Remote Sensing Images Based on Fused Visual Word Bags [J]. Journal of System Simulation, 2022, 34(5): 1033-1043.
[4]	Lifeng Zhang, Huiren Wang. Image Reconstruction of Electrical Capacitance Tomography Based on Convolutional Neural Network and Finite Element Simulation [J]. Journal of System Simulation, 2022, 34(4): 712-718.
[5]	Weiguo Tong, Shichao Zeng, Lifeng Zhang, Zhe Hou, Jiayue Guo. Electrical Resistance Tomography and Flow Pattern Identification Method Based on Deep Residual Neural Network [J]. Journal of System Simulation, 2022, 34(09): 2028-2036.
[6]	Tong Weiguo, Pang Xuechun, Zhu Genghong. Gas-liquid Two-phase Flow Pattern Recognition Method Based on Convolutional Neural Network [J]. Journal of System Simulation, 2021, 33(4): 883-891.
[7]	Chen Yu, Xia Zongji, Zhou Yujia. Modified Sparse Quasi - Newton Algorithm for Electrical Capacitance Tomography System [J]. Journal of System Simulation, 2019, 31(5): 819-827.
[8]	Shi Xiangbin, Zhong Jian, Liu Cuiwei, Liu Fang, Zhang Deyuan. Deep Learning Method for Hyperspectral Remote Sensing Images with Small Samples [J]. Journal of System Simulation, 2018, 30(7): 2744-2752.
[9]	Ma Min, Guo Qi, Yan Chaoqi. ECT Image Reconstruction Algorithm Based on Generalized Regularization [J]. Journal of System Simulation, 2017, 29(8): 1851-1858.
[10]	Xia Jingtao, Wang Qunshu, Ma Jiming, Hei Dongwei, Sheng Liang, Wei Fuli, Song Guzhou. Design and Simulation of Regional Differently Filtering Method of FBP Algorithm for Multilayer Spherical Shell [J]. Journal of System Simulation, 2016, 28(12): 3061-3066.
[11]	Liu Bin, Chen Xiangning, Guo Lianpeng. Study on 3D Battlefield Construction Based on UAV Remote Sensing Images [J]. Journal of System Simulation, 2015, 27(6): 1268-1273.
[12]	Wang Chenhao. Light Illumination Simulation Approach for Remote Sensing Image Based on Geometric Mapping [J]. Journal of System Simulation, 2015, 27(3): 579-583.