基于损失提取反馈注意网络的图像超分辨率重建研究

doi:10.16182/j.issn1004731x.joss.21-0986

摘要/Abstract

摘要：

自SRCNN(super-resolution convolutional neural network)将卷积神经网络用于超分辨率图像重建领域以来，人们通过大量的研究证明了使用深度学习的方法能够提高重建图像的效果。针对图像超分辨率网络中参数过多以及图像特征利用不充分导致可用的高频信息较少等问题，提出了一种基于损失提取策略的反馈注意网络(loss extraction feedback attention network，LEFAN)，以循环的方式对参数进行复用，同时增加对低分辨率图像特征的重用，以捕获更多的高频信息，对重建过程中造成的损失进行提取并融合到最终的超分辨率图像中。实验结果表明：算法在实现多次利用低分辨率图像的基础上，对潜在的损失进行提取并融合到最终的超分辨率图像中，可以获得较好的图像重建效果。

关键词: 反馈机制, 注意力机制, 损失提取, 超分辨率图像重建

Abstract:

Since the first application of convolutional neural network to the field of super-resolution image reconstruction (super-resolution convolutional neural network, SRCNN), a large number of studies have proved that deep learning can improve the effect of image reconstruction. Aiming at the too many parameters in the image super-resolution network and the insufficient utilization of image features resulting in less available high-frequency information, a loss extraction feedback attention network (LEFAN) is proposed to reuse parameters in a circular way and increase the reuse of low-resolution image features to capture more high-frequency information. The loss caused in the reconstruction process is extracted and fused into the final super-resolution image. The experimental results show that the algorithm can obtain a better image reconstruction effect by extracting the potential loss and fusing it into the final super-resolution image on the basis of the multiple utilization of low-resolution images.

Key words: feedback mechanism, attention mechanism, loss extraction, super-resolution image reconstruction

中图分类号:

TP391

孙红, 张玉香, 凌岳览. 基于损失提取反馈注意网络的图像超分辨率重建研究[J]. 系统仿真学报, 2023, 35(2): 308-317.

Hong Sun, Yuxiang Zhang, Yuelan Ling. Research on Image Super-resolution Reconstruction Based on Loss Extraction Feedback Attention Network[J]. Journal of System Simulation, 2023, 35(2): 308-317.

图/表 13

图1

图2

图3

图4

图5

图6

图7

图8

表1

表2

图9

图10

图11

参考文献 23

1	李新利, 邹昌铭, 杨国田, 等. 基于生成式对抗网络的发票图像超分辨率研究[J]. 系统仿真学报, 2021, 33(6): 1307-1314.
	Li Xinli, Zou Changming, Yang Guotian, et al. Research of Super-resolution Processing of Invoice Image Based on Generative Adversarial Network[J]. Journal of System Simulation, 2021, 33(6): 1307-1314.
2	胡蕾, 王足根, 陈田, 等. 一种改进的SRGAN红外图像超分辨率重建算法[J].系统仿真学报, 2021, 33(9): 2109-2118.
	Hu Lei, Wang Zugen, Chen Tian, et al. An Improved SRGAN Infrared Image Super-Resolution Reconstruction Algorithm[J]. Journal of System Simulation, 2021, 33(9): 2109-2118.
3	Dong C, Loy C C, He K, et al. Image Super-Resolution Using Deep Convolutional Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(S0162-8828), 2015, 38(2): 295-307.
4	Dong Chao, Chen Change Loy, Tang Xiaoou. Accelerating the Super-Resolution Convolutional Neural Network[C]//European Conference on Computer Vision. Las Vegas, USA: Springer, 2016: 391-407.
5	Shi W, Caballero J, Huszar F, et al. Real-time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016: 1874-1883.
6	He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Las Vegas, NV: IEEE, 2016: 770-778.
7	Kim J, Kwon Lee J, Mu Lee K. Accurate Image Super-Resolution Using Very Deep Convolutional Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE, 2016: 1646-1654.
8	Lim B, Son S, Kim H, et al. Enhanced Deep Residual Networks for Single Image Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, HI: IEEE, 2017: 136-144.
9	Tai Ying, Yang Jian, Liu Xiaoming. Image Super-Resolution Via Deep Recursive Residual Network[C]//IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 3147-3155.
10	Zhang Y, Li K, Li K, et al. Image Super-Resolution Using Very Deep Residual Channel Attention Networks[C]//European Conference on Computer Vision(ECCV). Munich, Germany: ECCV, 2018: 286-301.
11	Haris M, Shakhnarovich G, Ukita N. Deep Back-Projection Networks for Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 1664-1673.
12	Li Z, Yang J, Liu Z, et al. Feedback Network for Image Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 3867-3876.
13	Krizhevsky Alex, Sutskever Ilya, Geoffrey E Hinton. ImageNet Classification with Deep Convolutional Neural Networks[C]//Annual Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, United States, 2012: 1106-1114.
14	程德强, 郭昕, 陈亮亮, 等. 多通道递归残差网络的图像超分辨率重建[J]. 中国图象图形学报, 2021, 26(3): 605-618.
	Cheng Deqiang, Guo Xin, Chen Liangliang, et al. Image Super-resolution Reconstruction from Multi-channel Recursive Residual Network[J]. Journal of Image and Graphics, 2021, 26(3): 605-618.
15	应自炉, 龙祥. 多尺度密集残差网络的单幅图像超分辨率重建[J]. 中国图象图形学报, 2019, 24(3): 410-419.
	Ying Zilu, Long Xiang. Single-image Super-resolution Construction Based on Multi-scale Dense Residual Network[J]. Journal of Image and Graphics, 2019, 24(3): 410-419.
16	Hu J, Shen L, Sun G. Squeeze-and-Excitation Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 7132-7141.
17	雷鹏程, 刘丛, 唐坚刚, 等. 分层特征融合注意力网络图像超分辨率重建[J]. 中国图象图形学报, 2020, 25(9): 1773-1786.
	Lei Pengcheng, Liu Cong, Tang Jiangang, et al. Hierarchical Feature Fusion Attention Network for Image Super-resolution Reconstruction[J]. Journal of Image and Graphics, 2020, 25(9): 1773-1786.
18	Li Q, Li Z, Lu L, et al. Gated Multiple Feedback Network for Image Super-Resolution[C]//30th British Machine Vision Conference 2019, Cardiff, UK: BMVA Press, 2019: 188-205.
19	施举鹏, 李静, 陈琰, 等. DFAN:一种基于深度反馈注意力网络的图像超分辨率方法[J]. 小型微型计算机系统, 2021, 42(6): 1206-1212.
	Shi Jupeng, Li Jing, Cheng Yan, et al. DFAN: An Image Super-resolution Method Based on Depth Feedback Attention Network[J]. Journal of Chinese Computer Systems, 2021, 42(6): 1206-1212.
20	Bevilacqua M, Roumy A, Guillemot C, et al. Low-Complexity Singleimage Super-Resolution Based on Nonnegative Neighbor Embedding[C]//2012 British Machine Vision Conference. Durham: BMVA Press, 2012: 135.
21	Zeyde R, Elad M, Protter M. On Single Image Scale-up Using Sparserepresentations[C]//7th International Conference on Curves and Surfaces. Berlin: Springer, 2010: 711-730.
22	Martin D, Fowlkes C, Tal D, et al. A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics[C]//2001 Eighth IEEE International Conference on Computer Vision. Canada: IEEE, 2001: 416-423.
23	Huang J B, Singh A, Ahuja N. Single Image Super-Resolution from Transformed Self-Exemplars[C]//2015 Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2015: 5197-5206.

模型名称	缩放因子	混合域注意力机制	损失提取策略	PSNR/dB	模型名称	缩放因子	混合域注意力机制	损失提取策略	PSNR/dB
I	×2	×	×	37.72	III	×2	×	√	37.84
	×3	×	×	33.95		×3	×	√	34.33
	×4	×	×	31.80		×4	×	√	32.25
II	×2	√	×	38.03	IV	×2	√	√	38.15
	×3	√	×	34.52		×3	√	√	34.74
	×4	√	×	32.31		×4	√	√	32.48

数据集	缩放因子	Bicubic (PSNR/SSIM)	SRCNN (PSNR/SSIM)	VDSR (PSNR/SSIM)	DRRN (PSNR/SSIM)	EDSR (PSNR/SSIM)	SRFBN (PSNR/SSIM)	本文算法 (PSNR/SSIM)
Set 5	×2	33.66/0.930	36.66/0.954	37.53/0.959	37.74/0.959	38.11/0.960	38.11/0.961	38.15/0.961
	×3	30.39/0.868	32.75/0.909	33.67/0.921	34.03/0.924	34.65/0.928	34.70/0.929	34.74/0.929
	×4	28.42/0.810	30.48/0.863	31.35/0.883	31.68/0.888	32.46/0.897	32.47/0.8983	32.480.899
Set 14	×2	30.24/0.869	33.45/0.907	33.05/0.913	33.23/0.913	33.92/0.920	33.82/0.9196	33.92/0.919
	×3	27.55 /0.774	29.30 /0.822	29.78 /0.83	29.96/0.835	30.52/0.846	30.51/0.8461	30.56/0.846
	×4	26.00/0.703	27.50/0.751	28.02/0.768	28.21/0.773	28.80/0.787	28.81/0.7868	28.85/0.788
BSD100	×2	29.56/0.843	31.36/0.888	31.90/0.896	32.05/0.897	32.32/0.901	32.29/0.901	32.32/0.92
	×3	27.21 /0.739	28.41/0.786	28.83 /0.799	28.95/0.800	29.25/0.809	29.24/0.808	29.26/0.809
	×4	25.96/0.668	26.90/0.710	27.29/0.073	27.38/0.728	27.71/0.742	27.72/0.741	27.71/0.741
Urban100	×2	26.88/0.840	29.50/0.895	30.77/0.914	31.23/0.919	32.93/0.935	32.62/0.9328	32.72/0.934
	×3	24.46 /0.735	26.24/0.799	27.14 /0.829	27.53/0.837	28.80/0.865	28.73/0.8641	28.81/0.864
	×4	23.14/0.658	24.52/0.722	25.18/0.754	25.44/0.764	26.64/0.803	26.60/0.802	26.65/0.803

[1]	向南, 王璐, 贾崇柳, 蹇越谋, 马小霞. 改进YOLO的遮挡行人检测仿真[J]. 系统仿真学报, 2023, 35(2): 286-299.
[2]	金炜东, 张述礼, 唐鹏, 张曼. 基于稠密残差块与通道像素注意力的图像去雾网络[J]. 系统仿真学报, 2022, 34(8): 1663-1673.
[3]	邱俊杰, 郑红, 程云辉. 基于多尺度LSTM预测模型研究[J]. 系统仿真学报, 2022, 34(7): 1593-1604.
[4]	王银, 王飞翔, 孙前来. 多尺度特征融合车辆检测方法[J]. 系统仿真学报, 2022, 34(6): 1219-1229.
[5]	尹诗, 侯国莲, 迟岩, 弓林娟, 胡晓东. 风电机组发电机前轴承健康度预测方法及实现[J]. 系统仿真学报, 2021, 33(6): 1323-1333.
[6]	徐胜, 冯文宇, 刘志诚, 涂鑫涛, 费敏锐, 张堃. 基于机器视觉的复杂环境下精确手势识别算法研究[J]. 系统仿真学报, 2021, 33(10): 2460-2469.
[7]	车力, 康凤举, 侯学理. 大规模三维场景模型层级组合优化方法[J]. 系统仿真学报, 2017, 29(9): 2073-2080.
[8]	辛琦, 周晓. 基于动态收益矩阵的成长网络合作演化仿真研究[J]. 系统仿真学报, 2017, 29(2): 319-325.
[9]	孔亦思, 胡晓峰, 朱丰, 陶九阳. 战场态势感知中的注意力机制探析[J]. 系统仿真学报, 2017, 29(10): 2233-2241.