Research on Improved Feature Pyramid Algorithm Integrating Border Supervision Strategy

doi:10.16182/j.issn1004731x.joss.21-0529

Abstract

Abstract:

Aiming at the inaccurate boundary division in semantic segmentation and the existence of multi-scale targets, an improved feature pyramid algorithm fused with boundary supervision strategies is proposed. By fusing the boundary supervision strategy and the improved feature pyramid algorithm, the problems of inaccurate boundary division and the existence of multi-scale targets are sloved respectively, and an attention mechanism is added in the upsampling process to further improve the segmentation effect. The experimental results show that the algorithm can reach 58.69% and 78.59% MIOU (mean intersection over union) indicators on the Camvid and PASCAL VOC2012 data sets respectively, and has a good performance in the segmentation effect.

Key words: image semantic segmentation, border supervision, feature pyramid, attention upsampling

CLC Number:

TP391

Hong Sun, Yuelan Ling, Yuxiang Zhang. Research on Improved Feature Pyramid Algorithm Integrating Border Supervision Strategy[J]. Journal of System Simulation, 2022, 34(10): 2119-2129.

Figures/Tables 16

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Table 1

Table 2

Fig. 8

Fig. 9

Table 3

Fig. 10

Fig. 11

Table 4

Fig. 12

References 34

1	Hao S J, Zhou Y, Guo Y R. A Brief Survey on Semantic Segmentation with Deep Learning[J]. Neurocomputing (S0925-2312), 2020, 406: 302-321.
2	青晨, 禹晶, 肖创柏, 等. 深度卷积神经网络图像语义分割研究进展[J]. 中国图像图形学报, 2020, 25(6): 1069-1090.
	Chen Qing, Yu Jing, Xiao Chuangbai, et al. Deep Convolutional Neural Network for Semantic Image Segmentation[J]. Journal of Image and Graphics, 2020, 25(6): 1069-1090.
3	Siam M, Elkerdawy S, Jagersand M, et al. Deep Semantic Segmentation for Automated Driving: Taxonomy, Roadmap and Challenges[C]// The IEEE International Conference on Intelligent Transportation Systems. Piscataway, NJ: IEEE Press, 2017: 1-8.
4	Chen L, Bentley P, Mori K, et al. DRINet for Medical Image Segmentation[J]. IEEE Transactions on Medical Imaging(S0278-0062), 2018, 37(11): 2453-2462.
5	顾菊平, 程天宇, 华亮, 等. 面向脊柱生物建模的图像分割与配准研究综述[J]. 系统仿真学报, 2019, 31(2): 167-173.
	Gu Juping, Cheng Tianyu, Hua Liang, et al. Overview of Image Segmentation and Registration for Spine Biological Modeling[J]. Journal of System Simulation, 2019, 31(2): 167-173.
6	Huan L X, Zheng X W, Tang S J, et al. Learning Deep Cross-Scale Feature Propagation for Indoor Semantic Segmentation[J]. ISPRS Journal of Photogrammetry and Remote Sensing(S0924-2716), 2021, 176: 42-53.
7	Long J, Shelhamer E, Darrell T. Fully Convolutional Networks for Semantic Segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2015: 3431-3440.
8	Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[EB/OL]. (2015-04-10) [2021-05-15]. .
9	He K M, Zhang X Y, Ren S Q, et al. Deep Residual Learning for Image Recognition[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2016: 770-778.
10	Badrinarayanan V, Kendall A, Cipolla R. Segnet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(S0162-8828), 2017, 39(12): 2481-2495.
11	Chen L C, Papandreou G, Kokkinos I, et al. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFS [EB/OL]. (2014-12-22) [2021-05-15]. .
12	Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFS[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(S0162-8828), 2018, 40(4): 834-848.
13	Chen L C, Papandreou G, Schroff F, et al. Rethinking Atrous Convolution for Semantic Image Segmentation [EB/OL]. (2017-12-05) [2021-05-15]. .
14	Chen L C, Zhu Y K, Papandreou G, et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]//The European Conference on Computer Vision. Munich: Springer, 2018: 833-851.
15	Yamashita T, Furukawa H, Fujiyoshi H. Multiple Skip Connections of Dilated Convolution Network for Semantic Segmentation[C]//The IEEE International Conference on Image Processing. Piscataway. NJ: IEEE Press, 2018: 1593-1597.
16	Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions [EB/OL]. (2016-04-30) [2021-05-15]. .
17	陈洪云, 孙作雷, 孔薇. 融合深度神经网络和空洞卷积的语义图像分割研究[J]. 小型微型计算机系统, 2020, 41(1): 166-170.
	Chen Hongyun, Sun Zuolei, Kong Wei. Semantic Image Segmentation Based on Fusion of Deep Neural Networks and Atrous Convolution[J]. Journal of Chinese Computer Systems, 2020, 41(1): 166-170.
18	Zhao H S, Shi J P, Qi X J, et al. Pyramid Scene Parsing Network[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2017: 6230-6239.
19	韩彬彬, 张月婷, 潘宗序, 等. 残差密集空间金字塔网络的城市遥感图像分割[J]. 中国图像图形学报, 2020, 25(12): 2656-2664.
	Han Binbin, Zhang Yueting, Pan Zongxu, et al. Residual Dense Spatial Pyramid Network for Urbanremote Sensing Image Segmentation[J]. Journal of Image and Graphics, 2020, 25(12): 2656-2664.
20	Li H C, Xiong P F, An J, et al. Pyramid Attention Network for Semantic Segmentation [EB/OL]. (2018-11-25) [2021-05-10]. .
21	He J J, Deng Z Y, Qiao Y. Dynamic Multi-Scale Filters for Semantic Segmentation[C]//The IEEE International Conference on Computer Vision. Piscataway. NJ: IEEE Press, 2019: 3561-3571.
22	Hu J, Shen L, Sun G. Squeeze-and-Excitation Networks [C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2018: 7132-7141.
23	Woo S, Park J, Lee J Y, et al. CBAM: Convolutional Block Attention Module[C]//The European Conference on Computer Vision. Munich: Springer, 2018: 3-19.
24	Fu J, Liu J, Tian H J, et al. Dual Attention Network for Scene Segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2019: 3146-3154.
25	Zhong Z L, Lin Z Q, Bidart R, et al. Squeeze-and-Attention Network for Semantic Segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2020: 13062-13071.
26	Li X, Zhong Z S, Wu J L, et al. Expectation- Maximization Attention Networks for Semantic Segmentation[C]//The IEEE International Conference on Computer Vison. Piscataway, NJ: IEEE Press, 2019: 9166-9175.
27	Guo M H, Liu Z N, Mu T J, et al. Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks[EB/OL]. (2021-05-05) [2021-07-13]. .
28	Zhao Y C, Lin F S, Liu S G, et al. Constrained-Focal-Loss Based Deep Learning for Segmentation of Spores[J]. IEEE Access(S2169-3536), 2019, 7: 165029-165038.
29	Wu H S, Liang C X, Liu M S, et al. Optimized HRNet for Image Semantic Segmentation[J]. Expert Systems With Applications(S0957-4174), 2021, 174: 114532.
30	Yu C Q, Wang J B, Peng C, et al. Learning a Discriminative Feature Network for Semantic Segmentation[C]//The IEEE Conference Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2018: 1857-1866.
31	Takikawa T, Acuna D, Jampani V, et al. Gated-SCNN: Gated Shape CNNs for Semantic Segmentation[C]//The IEEE International Conference on Computer Vision. Piscataway, NJ: IEEE Press, 2019: 5228-5237.
32	Bertasius G, Shi J B, Torresani L. Semantic Segmentation with Boundary Neural Fields[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2016: 3602-3610.
33	Ma N N, Zhang X Y, Zheng H T, et al. ShuffleNetV2: Practical Guidelines for Efficient CNN Architecture Design [EB/OL]. (2018-07-30) [2021-05-23]. .
34	Sandler M, Howard A, Zhu M L, et al. MobileNetV2: Inverted Residuals and Linear Bottlenecks[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2018: 4510-4520.

方法	评价指标 MIOU/%	参数量 /MB
未添加特征金字塔及边界监督模块	54.34	52.47
特征金字塔(全局平均池化)	56.07	63.40
特征金字塔(X方向平均池化)	56.19	63.42
特征金字塔(Y方向平均池化)	56.28	63.42
特征金字塔(XY方向平均池化)	56.62	63.51
特征金字塔(XY方向平均池化)+ 边界监督模块	58.69	64.05

方法	评价指标MIOU/%
注意力上采样模块4	57.17
注意力上采样模块(3, 4)	57.49
注意力上采样模块(2, 3, 4)	58.22
注意力上采样模块(1, 2, 3, 4)	58.69

模型	评价指标MIOU/%
FCN^[7]	52.38
SegNet^[10]	53.33
PANet^[20]	54.75
DeepLabv3^[13]	55.63
PSPNet^[18]	55.95
EMANet^[26]	57.26
DMNet^[21]	57.50
Gated-SCNN^[31]	57.76
Ours	58.69

模型	评价指标MIOU/%
FCN	58.36
SegNet	57.38
PSPNet	72.92
PANet	73.21
DeepLabv3	75.65
DMNet	75.59
Gated-SCNN	77.61
EMANet	77.96
Ours	78.59

[1]	Chen Lijia, Wang Kai, Li Shigang, Tian Yanfei. Fast Maritime Simulator Scene Modeling Method Based on Aerial Images [J]. Journal of System Simulation, 2021, 33(7): 1565-1573.
[2]	Liu Wang, Sun Jinyu, Ma Shiwei. A Temporal Action Detection Algorithm Based on Spatio-Temporal Feature Pyramid Network [J]. Journal of System Simulation, 2019, 31(11): 2382-2387.