基于多重迁移注意力的增量式图像去雾算法

doi:10.16182/j.issn1004731x.joss.22-1538

摘要/Abstract

摘要：

为提高深度神经网络去雾算法对增补数据集的处理能力，并使网络差异化处理重要程度不同的图像特征以提高网络去雾能力，提出一种基于多重迁移注意力的增量式去雾算法。通过自编码器形式的教师注意力生成网络提取标签和雾霾的多重注意力，作为特征迁移媒介网络的标签约束网络训练，形成与教师注意力尽可能相近的迁移媒介注意力，并将其融入学生去雾网络的特征中，提高学生去雾网络的去雾能力；通过增量式训练方法提高学生去雾网络对增补数据集的处理能力。结果表明：所提算法对ITS、OTS以及真实雾图上皆具有较好的处理能力，在保证去雾图像像素结构完整、颜色不失真的同时具有较好的去雾效果，算法处理后的图像在主观视觉效果和客观评价指标上皆优于对比算法。

关键词: 深度学习, 图像去雾, 迁移注意力, 增量式训练

Abstract:

In order to improve the processing ability of the depth-neural network dehazing algorithm to the supplementary data set, and to make the network differently process the image features of different importance to improve the dehazing ability of the network, an incremental dehazing algorithm based on multiple migration of attention is proposed. The teacher's attention generation network in the form of Encoder-Decoder extracts the multiple attention of labels and haze, which is used it as the label of the characteristic migration media network to constrain the network training to form the migration media attention as close as possible to the teacher's attention. The attention is integrated into the characteristics of the student's dehazing network to improve the dehazing ability of the student's dehazing network. The incremental training method is used to improve the processing ability of students' dehazing network to the supplementary data set. The results show that the proposed algorithm has good processing ability on ITS, OTS and real hazy images, and has good dehazing effect while ensuring the integrity of pixel structure and color distortion of the dehazing image. The image processed by the algorithm is superior to the contrast algorithm in subjective visual effect and objective evaluation index.

Key words: deep learning, image dehazing, transfer attention, incremental training

中图分类号:

TP391.9

韦金阳,王科平,杨艺等 . 基于多重迁移注意力的增量式图像去雾算法[J]. 系统仿真学报, 2024, 36(4): 969-980.

Wei Jinyang,Wang Keping,Yang Yi,et al . Incremental Image Dehazing Algorithm Based on Multiple Transfer Attention[J]. Journal of System Simulation, 2024, 36(4): 969-980.

图/表 16

图1

图2

图3

图4

图5

图6

表1

图7

图8

表2

图9

图10

表3

表4

图11

图12

参考文献 22

1	金炜东, 胡燕花, 唐鹏, 等. 基于结构化森林的前向车载视频图像的细节强化[J]. 系统仿真学报, 2018, 30(12): 4602-4609, 4617.
	Jin Weidong, Hu Yanhua, Tang Peng, et al. Detailed Enhancement of Forward Vehicle Video Images Based on Structured Forest[J]. Journal of System Simulation, 2018, 30(12): 4602-4609, 4617.
2	杨凯, 陈纯毅, 胡小娟, 等. 蒙卡渲染画面多特征非局部均值滤波降噪算法[J]. 系统仿真学报, 2022, 34(6): 1259-1266.
	Yang Kai, Chen Chunyi, Hu Xiaojuan, et al. Denoising Algorithm Based on Multi-feature Non-local Mean Filtering for Monte Carlo Rendered Images[J]. Journal of System Simulation, 2022, 34(6): 1259-1266.
3	He Kaiming, Sun Jian, Tang Xiaoou. Single Image Haze Removal Using Dark Channel Prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353.
4	麻文刚, 张亚东, 郭进. 基于景深先验引导与环境光优化的图像去雾[J]. 电子学报, 2022, 50(7): 1708-1721.
	Ma Wengang, Zhang Yadong, Guo Jin. Image Dehazing Based on Priori Guidance of Depth of Field and Optimization of Ambient Light[J]. Acta Electronica Sinica, 2022, 50(7): 1708-1721.
5	吕建威, 钱锋, 韩昊男, 等. 结合光源分割和线性图像深度估计的夜间图像去雾[J]. 中国光学, 2022, 15(1): 34-44.
	Jianwei Lü, Qian Feng, Han Haonan, et al. Nighttime Image Dehazing with a New Light Segmentation Method and a Linear Image Depth Estimation Model[J]. Chinese Optics, 2022, 15(1): 34-44.
6	姜雨彤, 杨忠琳, 朱梦琪, 等. 适应性双通道先验的图像去雾方法[J]. 光学精密工程, 2022, 30(10): 1246-1262.
	Jiang Yutong, Yang Zhonglin, Zhu Mengqi, et al. Image Dehazing Method Based on Adaptive Bi-channel Priors[J]. Optics and Precision Engineering, 2022, 30(10): 1246-1262.
7	张楠楠, 李志伟, 郭新军, 等. 使用改进型大气散射模型的双阶段图像修复[J]. 光学精密工程, 2022, 30(18): 2267-2279.
	Zhang Nannan, Li Zhiwei, Guo Xinjun, et al. Two-stage Image Restoration Using Improved Atmospheric Scattering Model[J]. Optics and Precision Engineering, 2022, 30(18): 2267-2279.
8	Narasimhan S G, Nayar S K. Vision and the Atmosphere[J]. International Journal of Computer Vision, 2002, 48(3): 233-254.
9	Qu Yanyun, Chen Yizi, Huang Jingying, et al. Enhanced Pix2pix Dehazing Network[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2019: 8152-8160.
10	金炜东, 张述礼, 唐鹏, 等. 基于稠密残差块与通道像素注意力的图像去雾网络[J]. 系统仿真学报, 2022, 34(8): 1663-1673.
	Jin Weidong, Zhang Shuli, Tang Peng, et al. Image Dehazing Network Based on Densely Connected Residual Block and Channel Pixel Attention[J]. Journal of System Simulation, 2022, 34(8): 1663-1673.
11	Wang Keping, Duan Yumeng, Yang Yi. Single Image Dehazing Algorithm Based on Pyramid Mutil-scale Transposed Convolutional Network[J]. Systems Science & Control Engineering, 2021, 9(S1): 150-160.
12	Wang Keping, Yang Yi, Li Bingfeng, et al. Uneven Image Dehazing by Heterogeneous Twin Network[J]. IEEE Access, 2020, 8: 118485-118496.
13	Castro Francisco M, Marín-Jiménez Manuel J, Guil Nicolás, et al. End-to-end Incremental Learning[C]//Computer Vision-ECCV 2018. Cham: Springer International Publishing, 2018: 241-257.
14	Girshick R. Fast R-CNN[C]//2015 IEEE International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2015: 1440-1448.
15	Li Boyi, Ren Wenqi, Fu Dengpan, et al. Benchmarking Single Image Dehazing and Beyond[J]. IEEE Transactions on Image Processing, 2019, 28(1): 492-505.
16	Zeiler M D. Adadelta: An Adaptive Learning Rate Method[J]. Computer Ence, 2012, 27(12): 1-6.
17	Cai Bolun, Xu Xiangmin, Jia Kui, et al. DehazeNet: An End-to-end System for Single Image Haze Removal[J]. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198.
18	Li Boyi, Peng Xiulian, Wang Zhangyang, et al. AOD-net: All-in-one Dehazing Network[C]//2017 IEEE International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2017: 4780-4788.
19	Chen Dongdong, He Mingming, Fan Qingnan, et al. Gated Context Aggregation Network for Image Dehazing and Deraining[C]//2019 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, NJ, USA: IEEE, 2019: 1375-1383.
20	Qin Xu, Wang Zhilin, Bai Yuanchao, et al. FFA-net: Feature Fusion Attention Network for Single Image Dehazing[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 11908-11915.
21	Wang Zhou, Bovik A C, Sheikh H R, et al. Image Quality Assessment: From Error Visibility to Structural Similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
22	Huynh-Thu Q, Ghanbari M. Scope of Validity of PSNR in Image/Video Quality Assessment[J]. Electronics Letters, 2008, 44(13): 800-801.

算法	SSIM	PSNR/dB
DCP	0.81	17.04
DehazeNet	0.84	20.55
AOD-Net	0.80	18.42
GCANet	0.89	27.50
FFA-Net	0.73	14.36
所提算法(增量前) 所提算法(增量后)	0.90 0.90	27.52 26.74

算法	SSIM	PSNR/dB
DCP DehazeNet AOD-Net	0.83 0.83 0.83	16.94 22.25 19.38
GCANet	0.89	22.74
FFA-Net	0.87	22.09
所提算法(增量前) 所提算法(增量后)	0.85 0.90	21.91 26.68

图像	DCP	DehazeNet	AOD-Net	GCANet	FFA-Net	本文
1	978	2 404	2 035	3 034	2 610	3 106
2	2 828	4 171	2 717	1 510	3 530	4 259
3	1 418	1 325	1 023	1 193	627	2 280
4	1 701	3 404	3 775	4 833	2 151	5 854

图像	DCP	DehazeNet	AOD-Net	GCANet	FFA-Net	本文
1	12.30	12.19	12.04	12.91	11.36	13.00
2	11.71	12.06	11.40	11.80	10.72	12.86
3	11.63	11.65	11.21	11.48	10.32	12.42
4	11.00	11.84	11.86	12.50	11.30	12.55

[1]	李晨, 何明, 董晨, 李伟. 基于余弦相似性的定向注意力行为识别模型[J]. 系统仿真学报, 2024, 36(1): 67-82.
[2]	张凤全, 曹铎, 马晓寒, 陈柏君, 张江霄. 一种面向戏曲妆容细节生成的风格迁移网络[J]. 系统仿真学报, 2023, 35(9): 2064-2076.
[3]	杜宇, 杨新权, 张建华, 袁素春, 肖化超, 袁晶晶. 基于循环谱截面智能分析的混合信号调制识别方法[J]. 系统仿真学报, 2023, 35(1): 146-157.
[4]	张大永, 杨镜宇, 吴曦. 兵棋推演空中任务智能预测方法研究[J]. 系统仿真学报, 2023, 35(1): 212-220.
[5]	金炜东, 张述礼, 唐鹏, 张曼. 基于稠密残差块与通道像素注意力的图像去雾网络[J]. 系统仿真学报, 2022, 34(8): 1663-1673.
[6]	柴慧敏, 张勇, 李欣粤, 宋雅楠. 基于深度学习的空中目标威胁评估方法[J]. 系统仿真学报, 2022, 34(7): 1459-1467.
[7]	康旭, 张晓峰. 基于生成对抗神经网络的雷达遥感数据增广方法[J]. 系统仿真学报, 2022, 34(4): 920-927.
[8]	刘家瑞, 杨国田, 王孝伟. 基于孪生深度神经网络的风电机组故障诊断方法[J]. 系统仿真学报, 2022, 34(11): 2348-2358.
[9]	陆荣秀, 张笔豪, 莫振龙. 基于脸部特征和头部姿态的疲劳检测方法[J]. 系统仿真学报, 2022, 34(10): 2279-2292.
[10]	仝卫国, 曾世超, 张立峰, 侯哲, 郭佳跃. 基于深度残差神经网络的电阻层析成像及流型辨识方法[J]. 系统仿真学报, 2022, 34(09): 2028-2036.
[11]	林硕, 安磊, 高治军, 单丹, 尚文利. 结合栈式自编码及长短时记忆的入侵检测研究[J]. 系统仿真学报, 2021, 33(6): 1288-1296.
[12]	程文聪, 史小康, 王志刚. 基于生成对抗网络的仿真卫星云图生成方法[J]. 系统仿真学报, 2021, 33(6): 1297-1306.
[13]	董书琴, 张斌. 面向不平衡数据的网络流量异常检测方法[J]. 系统仿真学报, 2021, 33(3): 679-689.
[14]	王霄汉, 张霖, 任磊, 谢堃钰, 王昆玉, 叶飞, 陈真. 基于强化学习的车间调度问题研究简述[J]. 系统仿真学报, 2021, 33(12): 2782-2791.
[15]	王步维, 王敏, 范谦, 王雅男, 章涵文, 乐云亮. 基于深度学习的晶体性质预测研究[J]. 系统仿真学报, 2021, 33(12): 2854-2863.