融合边界监督策略的改进特征金字塔算法研究

doi:10.16182/j.issn1004731x.joss.21-0529

摘要/Abstract

摘要：

针对语义分割中存在的边界划分不够准确及存在多尺度目标等问题，提出了一种融合边界监督策略的改进特征金字塔算法。通过融合的边界监督策略和改进的特征金字塔算法分别解决边界划分不准确和存在多尺度目标的问题，并且在上采样过程中加入注意力机制，进一步提升分割效果。实验结果表明：该算法分别在Camvid和PASCAL VOC2012两个数据集上取得了58.69%和78.59%的平均交并比(mean intersection over union， MIOU)指标，在分割效果上有较好的表现。

关键词: 图像语义分割, 边界监督, 特征金字塔, 注意力上采样

Abstract:

Aiming at the inaccurate boundary division in semantic segmentation and the existence of multi-scale targets, an improved feature pyramid algorithm fused with boundary supervision strategies is proposed. By fusing the boundary supervision strategy and the improved feature pyramid algorithm, the problems of inaccurate boundary division and the existence of multi-scale targets are sloved respectively, and an attention mechanism is added in the upsampling process to further improve the segmentation effect. The experimental results show that the algorithm can reach 58.69% and 78.59% MIOU (mean intersection over union) indicators on the Camvid and PASCAL VOC2012 data sets respectively, and has a good performance in the segmentation effect.

Key words: image semantic segmentation, border supervision, feature pyramid, attention upsampling

中图分类号:

TP391

孙红, 凌岳览, 张玉香. 融合边界监督策略的改进特征金字塔算法研究[J]. 系统仿真学报, 2022, 34(10): 2119-2129.

Hong Sun, Yuelan Ling, Yuxiang Zhang. Research on Improved Feature Pyramid Algorithm Integrating Border Supervision Strategy[J]. Journal of System Simulation, 2022, 34(10): 2119-2129.

图/表 16

图1

图2

图3

图4

图5

图6

图7

表1

表2

图8

图9

表3

图10

图11

表4

图12

参考文献 34

1	Hao S J, Zhou Y, Guo Y R. A Brief Survey on Semantic Segmentation with Deep Learning[J]. Neurocomputing (S0925-2312), 2020, 406: 302-321.
2	青晨, 禹晶, 肖创柏, 等. 深度卷积神经网络图像语义分割研究进展[J]. 中国图像图形学报, 2020, 25(6): 1069-1090.
	Chen Qing, Yu Jing, Xiao Chuangbai, et al. Deep Convolutional Neural Network for Semantic Image Segmentation[J]. Journal of Image and Graphics, 2020, 25(6): 1069-1090.
3	Siam M, Elkerdawy S, Jagersand M, et al. Deep Semantic Segmentation for Automated Driving: Taxonomy, Roadmap and Challenges[C]// The IEEE International Conference on Intelligent Transportation Systems. Piscataway, NJ: IEEE Press, 2017: 1-8.
4	Chen L, Bentley P, Mori K, et al. DRINet for Medical Image Segmentation[J]. IEEE Transactions on Medical Imaging(S0278-0062), 2018, 37(11): 2453-2462.
5	顾菊平, 程天宇, 华亮, 等. 面向脊柱生物建模的图像分割与配准研究综述[J]. 系统仿真学报, 2019, 31(2): 167-173.
	Gu Juping, Cheng Tianyu, Hua Liang, et al. Overview of Image Segmentation and Registration for Spine Biological Modeling[J]. Journal of System Simulation, 2019, 31(2): 167-173.
6	Huan L X, Zheng X W, Tang S J, et al. Learning Deep Cross-Scale Feature Propagation for Indoor Semantic Segmentation[J]. ISPRS Journal of Photogrammetry and Remote Sensing(S0924-2716), 2021, 176: 42-53.
7	Long J, Shelhamer E, Darrell T. Fully Convolutional Networks for Semantic Segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2015: 3431-3440.
8	Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-Scale Image Recognition[EB/OL]. (2015-04-10) [2021-05-15]. .
9	He K M, Zhang X Y, Ren S Q, et al. Deep Residual Learning for Image Recognition[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2016: 770-778.
10	Badrinarayanan V, Kendall A, Cipolla R. Segnet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(S0162-8828), 2017, 39(12): 2481-2495.
11	Chen L C, Papandreou G, Kokkinos I, et al. Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFS [EB/OL]. (2014-12-22) [2021-05-15]. .
12	Chen L C, Papandreou G, Kokkinos I, et al. DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFS[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(S0162-8828), 2018, 40(4): 834-848.
13	Chen L C, Papandreou G, Schroff F, et al. Rethinking Atrous Convolution for Semantic Image Segmentation [EB/OL]. (2017-12-05) [2021-05-15]. .
14	Chen L C, Zhu Y K, Papandreou G, et al. Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation[C]//The European Conference on Computer Vision. Munich: Springer, 2018: 833-851.
15	Yamashita T, Furukawa H, Fujiyoshi H. Multiple Skip Connections of Dilated Convolution Network for Semantic Segmentation[C]//The IEEE International Conference on Image Processing. Piscataway. NJ: IEEE Press, 2018: 1593-1597.
16	Yu F, Koltun V. Multi-Scale Context Aggregation by Dilated Convolutions [EB/OL]. (2016-04-30) [2021-05-15]. .
17	陈洪云, 孙作雷, 孔薇. 融合深度神经网络和空洞卷积的语义图像分割研究[J]. 小型微型计算机系统, 2020, 41(1): 166-170.
	Chen Hongyun, Sun Zuolei, Kong Wei. Semantic Image Segmentation Based on Fusion of Deep Neural Networks and Atrous Convolution[J]. Journal of Chinese Computer Systems, 2020, 41(1): 166-170.
18	Zhao H S, Shi J P, Qi X J, et al. Pyramid Scene Parsing Network[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2017: 6230-6239.
19	韩彬彬, 张月婷, 潘宗序, 等. 残差密集空间金字塔网络的城市遥感图像分割[J]. 中国图像图形学报, 2020, 25(12): 2656-2664.
	Han Binbin, Zhang Yueting, Pan Zongxu, et al. Residual Dense Spatial Pyramid Network for Urbanremote Sensing Image Segmentation[J]. Journal of Image and Graphics, 2020, 25(12): 2656-2664.
20	Li H C, Xiong P F, An J, et al. Pyramid Attention Network for Semantic Segmentation [EB/OL]. (2018-11-25) [2021-05-10]. .
21	He J J, Deng Z Y, Qiao Y. Dynamic Multi-Scale Filters for Semantic Segmentation[C]//The IEEE International Conference on Computer Vision. Piscataway. NJ: IEEE Press, 2019: 3561-3571.
22	Hu J, Shen L, Sun G. Squeeze-and-Excitation Networks [C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2018: 7132-7141.
23	Woo S, Park J, Lee J Y, et al. CBAM: Convolutional Block Attention Module[C]//The European Conference on Computer Vision. Munich: Springer, 2018: 3-19.
24	Fu J, Liu J, Tian H J, et al. Dual Attention Network for Scene Segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2019: 3146-3154.
25	Zhong Z L, Lin Z Q, Bidart R, et al. Squeeze-and-Attention Network for Semantic Segmentation[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2020: 13062-13071.
26	Li X, Zhong Z S, Wu J L, et al. Expectation- Maximization Attention Networks for Semantic Segmentation[C]//The IEEE International Conference on Computer Vison. Piscataway, NJ: IEEE Press, 2019: 9166-9175.
27	Guo M H, Liu Z N, Mu T J, et al. Beyond Self-Attention: External Attention Using Two Linear Layers for Visual Tasks[EB/OL]. (2021-05-05) [2021-07-13]. .
28	Zhao Y C, Lin F S, Liu S G, et al. Constrained-Focal-Loss Based Deep Learning for Segmentation of Spores[J]. IEEE Access(S2169-3536), 2019, 7: 165029-165038.
29	Wu H S, Liang C X, Liu M S, et al. Optimized HRNet for Image Semantic Segmentation[J]. Expert Systems With Applications(S0957-4174), 2021, 174: 114532.
30	Yu C Q, Wang J B, Peng C, et al. Learning a Discriminative Feature Network for Semantic Segmentation[C]//The IEEE Conference Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2018: 1857-1866.
31	Takikawa T, Acuna D, Jampani V, et al. Gated-SCNN: Gated Shape CNNs for Semantic Segmentation[C]//The IEEE International Conference on Computer Vision. Piscataway, NJ: IEEE Press, 2019: 5228-5237.
32	Bertasius G, Shi J B, Torresani L. Semantic Segmentation with Boundary Neural Fields[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2016: 3602-3610.
33	Ma N N, Zhang X Y, Zheng H T, et al. ShuffleNetV2: Practical Guidelines for Efficient CNN Architecture Design [EB/OL]. (2018-07-30) [2021-05-23]. .
34	Sandler M, Howard A, Zhu M L, et al. MobileNetV2: Inverted Residuals and Linear Bottlenecks[C]//The IEEE Conference on Computer Vision and Pattern Recognition. Washington, DC: IEEE Computer Society, 2018: 4510-4520.

方法	评价指标 MIOU/%	参数量 /MB
未添加特征金字塔及边界监督模块	54.34	52.47
特征金字塔(全局平均池化)	56.07	63.40
特征金字塔(X方向平均池化)	56.19	63.42
特征金字塔(Y方向平均池化)	56.28	63.42
特征金字塔(XY方向平均池化)	56.62	63.51
特征金字塔(XY方向平均池化)+ 边界监督模块	58.69	64.05

方法	评价指标MIOU/%
注意力上采样模块4	57.17
注意力上采样模块(3, 4)	57.49
注意力上采样模块(2, 3, 4)	58.22
注意力上采样模块(1, 2, 3, 4)	58.69

模型	评价指标MIOU/%
FCN^[7]	52.38
SegNet^[10]	53.33
PANet^[20]	54.75
DeepLabv3^[13]	55.63
PSPNet^[18]	55.95
EMANet^[26]	57.26
DMNet^[21]	57.50
Gated-SCNN^[31]	57.76
Ours	58.69

模型	评价指标MIOU/%
FCN	58.36
SegNet	57.38
PSPNet	72.92
PANet	73.21
DeepLabv3	75.65
DMNet	75.59
Gated-SCNN	77.61
EMANet	77.96
Ours	78.59

[1]	陈立家, 王凯, 李世刚, 田延飞. 基于航空影像的航海模拟器视景快速建模方法[J]. 系统仿真学报, 2021, 33(7): 1565-1573.
[2]	刘望, 孙金玉, 马世伟. 基于时空特征金字塔网络的动作时序检测方法[J]. 系统仿真学报, 2019, 31(11): 2382-2387.