Research on Image Super-resolution Reconstruction Based on Loss Extraction Feedback Attention Network

doi:10.16182/j.issn1004731x.joss.21-0986

Abstract

Abstract:

Since the first application of convolutional neural network to the field of super-resolution image reconstruction (super-resolution convolutional neural network, SRCNN), a large number of studies have proved that deep learning can improve the effect of image reconstruction. Aiming at the too many parameters in the image super-resolution network and the insufficient utilization of image features resulting in less available high-frequency information, a loss extraction feedback attention network (LEFAN) is proposed to reuse parameters in a circular way and increase the reuse of low-resolution image features to capture more high-frequency information. The loss caused in the reconstruction process is extracted and fused into the final super-resolution image. The experimental results show that the algorithm can obtain a better image reconstruction effect by extracting the potential loss and fusing it into the final super-resolution image on the basis of the multiple utilization of low-resolution images.

Key words: feedback mechanism, attention mechanism, loss extraction, super-resolution image reconstruction

CLC Number:

TP391

Hong Sun, Yuxiang Zhang, Yuelan Ling. Research on Image Super-resolution Reconstruction Based on Loss Extraction Feedback Attention Network[J]. Journal of System Simulation, 2023, 35(2): 308-317.

Figures/Tables 13

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 1

Table 2

Fig. 9

Fig. 10

Fig. 11

References 23

1	李新利, 邹昌铭, 杨国田, 等. 基于生成式对抗网络的发票图像超分辨率研究[J]. 系统仿真学报, 2021, 33(6): 1307-1314.
	Li Xinli, Zou Changming, Yang Guotian, et al. Research of Super-resolution Processing of Invoice Image Based on Generative Adversarial Network[J]. Journal of System Simulation, 2021, 33(6): 1307-1314.
2	胡蕾, 王足根, 陈田, 等. 一种改进的SRGAN红外图像超分辨率重建算法[J].系统仿真学报, 2021, 33(9): 2109-2118.
	Hu Lei, Wang Zugen, Chen Tian, et al. An Improved SRGAN Infrared Image Super-Resolution Reconstruction Algorithm[J]. Journal of System Simulation, 2021, 33(9): 2109-2118.
3	Dong C, Loy C C, He K, et al. Image Super-Resolution Using Deep Convolutional Networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence(S0162-8828), 2015, 38(2): 295-307.
4	Dong Chao, Chen Change Loy, Tang Xiaoou. Accelerating the Super-Resolution Convolutional Neural Network[C]//European Conference on Computer Vision. Las Vegas, USA: Springer, 2016: 391-407.
5	Shi W, Caballero J, Huszar F, et al. Real-time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016: 1874-1883.
6	He K, Zhang X, Ren S, et al. Deep Residual Learning for Image Recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR). Las Vegas, NV: IEEE, 2016: 770-778.
7	Kim J, Kwon Lee J, Mu Lee K. Accurate Image Super-Resolution Using Very Deep Convolutional Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV: IEEE, 2016: 1646-1654.
8	Lim B, Son S, Kim H, et al. Enhanced Deep Residual Networks for Single Image Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops. Honolulu, HI: IEEE, 2017: 136-144.
9	Tai Ying, Yang Jian, Liu Xiaoming. Image Super-Resolution Via Deep Recursive Residual Network[C]//IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI: IEEE, 2017: 3147-3155.
10	Zhang Y, Li K, Li K, et al. Image Super-Resolution Using Very Deep Residual Channel Attention Networks[C]//European Conference on Computer Vision(ECCV). Munich, Germany: ECCV, 2018: 286-301.
11	Haris M, Shakhnarovich G, Ukita N. Deep Back-Projection Networks for Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 1664-1673.
12	Li Z, Yang J, Liu Z, et al. Feedback Network for Image Super-Resolution[C]//IEEE Conference on Computer Vision and Pattern Recognition. Long Beach, USA: IEEE, 2019: 3867-3876.
13	Krizhevsky Alex, Sutskever Ilya, Geoffrey E Hinton. ImageNet Classification with Deep Convolutional Neural Networks[C]//Annual Conference on Neural Information Processing Systems. Lake Tahoe, Nevada, United States, 2012: 1106-1114.
14	程德强, 郭昕, 陈亮亮, 等. 多通道递归残差网络的图像超分辨率重建[J]. 中国图象图形学报, 2021, 26(3): 605-618.
	Cheng Deqiang, Guo Xin, Chen Liangliang, et al. Image Super-resolution Reconstruction from Multi-channel Recursive Residual Network[J]. Journal of Image and Graphics, 2021, 26(3): 605-618.
15	应自炉, 龙祥. 多尺度密集残差网络的单幅图像超分辨率重建[J]. 中国图象图形学报, 2019, 24(3): 410-419.
	Ying Zilu, Long Xiang. Single-image Super-resolution Construction Based on Multi-scale Dense Residual Network[J]. Journal of Image and Graphics, 2019, 24(3): 410-419.
16	Hu J, Shen L, Sun G. Squeeze-and-Excitation Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, USA: IEEE, 2018: 7132-7141.
17	雷鹏程, 刘丛, 唐坚刚, 等. 分层特征融合注意力网络图像超分辨率重建[J]. 中国图象图形学报, 2020, 25(9): 1773-1786.
	Lei Pengcheng, Liu Cong, Tang Jiangang, et al. Hierarchical Feature Fusion Attention Network for Image Super-resolution Reconstruction[J]. Journal of Image and Graphics, 2020, 25(9): 1773-1786.
18	Li Q, Li Z, Lu L, et al. Gated Multiple Feedback Network for Image Super-Resolution[C]//30th British Machine Vision Conference 2019, Cardiff, UK: BMVA Press, 2019: 188-205.
19	施举鹏, 李静, 陈琰, 等. DFAN:一种基于深度反馈注意力网络的图像超分辨率方法[J]. 小型微型计算机系统, 2021, 42(6): 1206-1212.
	Shi Jupeng, Li Jing, Cheng Yan, et al. DFAN: An Image Super-resolution Method Based on Depth Feedback Attention Network[J]. Journal of Chinese Computer Systems, 2021, 42(6): 1206-1212.
20	Bevilacqua M, Roumy A, Guillemot C, et al. Low-Complexity Singleimage Super-Resolution Based on Nonnegative Neighbor Embedding[C]//2012 British Machine Vision Conference. Durham: BMVA Press, 2012: 135.
21	Zeyde R, Elad M, Protter M. On Single Image Scale-up Using Sparserepresentations[C]//7th International Conference on Curves and Surfaces. Berlin: Springer, 2010: 711-730.
22	Martin D, Fowlkes C, Tal D, et al. A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics[C]//2001 Eighth IEEE International Conference on Computer Vision. Canada: IEEE, 2001: 416-423.
23	Huang J B, Singh A, Ahuja N. Single Image Super-Resolution from Transformed Self-Exemplars[C]//2015 Computer Vision and Pattern Recognition. Piscataway, NJ, USA: IEEE, 2015: 5197-5206.

模型名称	缩放因子	混合域注意力机制	损失提取策略	PSNR/dB	模型名称	缩放因子	混合域注意力机制	损失提取策略	PSNR/dB
I	×2	×	×	37.72	III	×2	×	√	37.84
	×3	×	×	33.95		×3	×	√	34.33
	×4	×	×	31.80		×4	×	√	32.25
II	×2	√	×	38.03	IV	×2	√	√	38.15
	×3	√	×	34.52		×3	√	√	34.74
	×4	√	×	32.31		×4	√	√	32.48

数据集	缩放因子	Bicubic (PSNR/SSIM)	SRCNN (PSNR/SSIM)	VDSR (PSNR/SSIM)	DRRN (PSNR/SSIM)	EDSR (PSNR/SSIM)	SRFBN (PSNR/SSIM)	本文算法 (PSNR/SSIM)
Set 5	×2	33.66/0.930	36.66/0.954	37.53/0.959	37.74/0.959	38.11/0.960	38.11/0.961	38.15/0.961
	×3	30.39/0.868	32.75/0.909	33.67/0.921	34.03/0.924	34.65/0.928	34.70/0.929	34.74/0.929
	×4	28.42/0.810	30.48/0.863	31.35/0.883	31.68/0.888	32.46/0.897	32.47/0.8983	32.480.899
Set 14	×2	30.24/0.869	33.45/0.907	33.05/0.913	33.23/0.913	33.92/0.920	33.82/0.9196	33.92/0.919
	×3	27.55 /0.774	29.30 /0.822	29.78 /0.83	29.96/0.835	30.52/0.846	30.51/0.8461	30.56/0.846
	×4	26.00/0.703	27.50/0.751	28.02/0.768	28.21/0.773	28.80/0.787	28.81/0.7868	28.85/0.788
BSD100	×2	29.56/0.843	31.36/0.888	31.90/0.896	32.05/0.897	32.32/0.901	32.29/0.901	32.32/0.92
	×3	27.21 /0.739	28.41/0.786	28.83 /0.799	28.95/0.800	29.25/0.809	29.24/0.808	29.26/0.809
	×4	25.96/0.668	26.90/0.710	27.29/0.073	27.38/0.728	27.71/0.742	27.72/0.741	27.71/0.741
Urban100	×2	26.88/0.840	29.50/0.895	30.77/0.914	31.23/0.919	32.93/0.935	32.62/0.9328	32.72/0.934
	×3	24.46 /0.735	26.24/0.799	27.14 /0.829	27.53/0.837	28.80/0.865	28.73/0.8641	28.81/0.864
	×4	23.14/0.658	24.52/0.722	25.18/0.754	25.44/0.764	26.64/0.803	26.60/0.802	26.65/0.803

[1]	Nan Xiang, Lu Wang, Chongliu Jia, Yuemou Jian, Xiaoxia Ma. Simulation of Occluded Pedestrian Detection Based on Improved YOLO [J]. Journal of System Simulation, 2023, 35(2): 286-299.
[2]	Weidong Jin, Shuli Zhang, Peng Tang, Man Zhang. Image Dehazing Network Based on Densely Connected Residual Block and Channel Pixel Attention [J]. Journal of System Simulation, 2022, 34(8): 1663-1673.
[3]	Junjie Qiu, Hong Zheng, Yunhui Cheng. Research on Prediction of Model Based on Multi-scale LSTM [J]. Journal of System Simulation, 2022, 34(7): 1593-1604.
[4]	Yin Wang, Feixiang Wang, Qianlai Sun. Vehicle Detection Method Based on Multi Scale Feature Fusion [J]. Journal of System Simulation, 2022, 34(6): 1219-1229.
[5]	Yin Shi, Hou Guolian, Chi Yan, Gong Linjuan, Hu Xiaodong. Prediction Method for Health Degree of Front Bearing of Wind Turbine Generator and Implementation [J]. Journal of System Simulation, 2021, 33(6): 1323-1333.
[6]	Yang Weilong, Xu Kai, Xie Xu, Sun Lin. Research on CGF-oriented Virtual Human Perceptual Attention Model [J]. Journal of System Simulation, 2021, 33(2): 262-270.
[7]	Che Li, Kang Fengju, Hou Xueli. Novel Hierarchical Level of Detail Combinatorial Optimization Method for Large-scale 3D Scene [J]. Journal of System Simulation, 2017, 29(9): 2073-2080.
[8]	Xin Qi, Zhou Xiao. Simulation of Cooperative Evolution of Growing Network Based on Dynamic Payoff Matrices [J]. Journal of System Simulation, 2017, 29(2): 319-325.
[9]	Kong Yisi, Hu Xiaofeng, Zhu Feng, Tao Jiuyang. Attention Mechanism in Battlefield Situation Awareness [J]. Journal of System Simulation, 2017, 29(10): 2233-2241.