基于生成对抗网络的图像自增强去雾算法

doi:10.16182/j.issn1004731x.joss.22-1551

摘要/Abstract

摘要：

针对现有去雾模型使用合成有雾图像数据集训练后容易出现过拟合的问题，提出了一种融合生成对抗网络的图像自增强去雾算法。在结合两个生成对抗网络的同时估计图像的深度信息。第一个GAN利用清晰图像学习图像加雾过程，将其生成的有雾图像作为第二个GAN的输入，指导第二个GAN如何正确去雾。为了减少图像处理前后的差异，利用一致性损失函数来优化两个网络。在图像加雾部分添加场景深度估计模块，并对散射因子进行随机采样，实现图像自增强功能，更加真实地模拟现实世界中不同浓度的雾气。该算法无需使用合成有雾图像数据集的成对信息，进一步避免过拟合问题。实验结果表明：所提算法能够取得较好的去雾效果，在主观视觉质量和客观评价指标上均有良好表现，优于同类算法。

关键词: 图像处理, 机器视觉, 生成对抗网络, 光学模型, 图像去雾

Abstract:

To solve the problem that existing dehazing models are prone to over fitting after training with synthetic hazy image data sets, an image self-enhancement dehazing algorithm is proposed in combination with generative adversarial network. The depth information of an image is estimated while combining two Generative Adversarial Networks. The first GAN uses a clear image to learn the process of image hazing, and then adopts the hazed image generated by it as the input of the second GAN to guide the second GAN to correct dehazing. In order to reduce the difference before and after image processing, the consistency loss function is applied to optimize two network parts. In addition, the scene depth estimation module is added to the image hazing part, and the scattering factor is randomly sampled to achieve the image self enhancement function, which can more realistically simulate fog of different concentrations in real world. The algorithm does not need to use the paired information of synthetic hazy image dataset to further avoid over fitting. Experimental results show that the proposed algorithm can achieve better dehazing effect, which has good performance in subjective visual quality and objective evaluation indicators, superior to similar algorithms.

Key words: image processing, machine vision, generative adversarial network, optical model, image dehaze

中图分类号:

TP391.4

刘万军,程裕茜,曲海成 . 基于生成对抗网络的图像自增强去雾算法[J]. 系统仿真学报, 2024, 36(5): 1093-1106.

Liu Wanjun,Cheng Yuqian,Qu Haicheng . Image Self-enhancement De-hazing Algorithm Combined with Generative Adversarial Network[J]. Journal of System Simulation, 2024, 36(5): 1093-1106.

图/表 15

图1

图2

图3

图4

图5

表1

一致性损失系数比较结果

$λ c y c$	PSNR	SSIM
0.5	28.365 7	0.954 9
1.0	28.773 2	0.974 5
1.5	27.485 2	0.926 2

表1

图6

表2

表3

表4

图7

图8

图9

图10

表5

参考文献 25

1	Liu Shuchao, Shi Huajun, Guo Zhan. Research on Defog Algorithm of Single Remote Sensing Image Based on Deep Learning[C]//CIBDA 2022; 3rd International Conference on Computer Information and Big Data Applications. Piscataway, NJ, USA: IEEE, 2022: 1-5.
2	金炜东, 张述礼, 唐鹏, 等. 基于稠密残差块与通道像素注意力的图像去雾网络[J]. 系统仿真学报, 2022, 34(8): 1663-1673.
	Jin Weidong, Zhang Shuli, Tang Peng, et al. Image Dehazing Network Based on Densely Connected Residual Block and Channel Pixel Attention[J]. Journal of System Simulation, 2022, 34(8): 1663-1673.
3	史瑞雪, 高保禄, 乔颖婧. 基于融合策略的改进Retinex低照度图像增强算法[J]. 计算机测量与控制, 2021, 29(4): 159-164.
	Shi Ruixue, Gao Baolu, Qiao Yingjing. Improved Retinex Low-light Image Enhancement Algorithm Based on Fusion Strategy[J]. Computer Measurement & Control, 2021, 29(4): 159-164.
4	郑凤仙, 王夏黎, 何丹丹, 等. 单幅图像去雾算法研究综述[J]. 计算机工程与应用, 2022, 58(3): 1-14.
	Zheng Fengxian, Wang Xiali, He Dandan, et al. Survey of Single Image Defogging Algorithm[J]. Computer Engineering and Applications, 2022, 58(3): 1-14.
5	Sengupta Debapriya, Biswas Arindam, Gupta Phalguni. Non-linear Weight Adjustment in Adaptive Gamma Correction for Image Contrast Enhancement[J]. Multimedia Tools and Applications, 2021, 80(3): 3835-3862.
6	Liang Jian, Ju Haijuan, Ren Liyong, et al. Generalized Polarimetric Dehazing Method Based on Low-pass Filtering in Frequency Domain[J]. Sensors, 2020, 20(6): 1729.
7	Fan Di, Guo Xinyun, Lu Xiao, et al. Image Defogging Algorithm Based on Sparse Representation[J]. Complexity, 2020, 2020: 6835367.
8	Tang Qunfang, Yang Jie, He Xiangjian, et al. Nighttime Image Dehazing Based on Retinex and Dark Channel Prior Using Taylor Series Expansion[J]. Computer Vision and Image Understanding, 2021, 202: 103086.
9	陈永, 郭红光, 艾亚鹏. 基于多尺度卷积神经网络的单幅图像去雾方法[J]. 光学学报, 2019, 39(10): 141-150.
	Chen Yong, Guo Hongguang, Ai Yapeng. Single Image Dehazing Method Based on Multi-scale Convolution Neural Network[J]. Acta Optica Sinica, 2019, 39(10): 141-150.
10	Mccartney E J, Hall Freeman F Jr. Optics of the Atmosphere: Scattering by Molecules and Particles[J]. Physics Today, 1977, 30(5): 76-77.
11	He Kaiming, Sun Jian, Tang Xiaoou. Single Image Haze Removal Using Dark Channel Prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341-2353.
12	Agarwal Ishank. Single Image Dehazing Using NN-dehaze Filter[C]//International Conference on Innovative Computing and Communications. Singapore: Springer Singapore, 2022: 701-711.
13	袁小平, 陈艳宇, 石慧. 基于雾线暗通道先验改进的图像去雾算法[J]. 激光与光电子学进展, 2022, 59(8): 171-178.
	Yuan Xiaoping, Chen Yanyu, Shi Hui. Improved Image Dehazing Algorithm Based on Haze-line and Dark Channel Prior[J]. Laser & Optoelectronics Progress, 2022, 59(8): 171-178.
14	Cai Bolun, Xu Xiangmin, Jia Kui, et al. DehazeNet: An End-to-end System for Single Image Haze Removal[J]. IEEE Transactions on Image Processing, 2016, 25(11): 5187-5198.
15	Li Boyi, Peng Xiulian, Wang Zhangyang, et al. AOD-net: All-in-one Dehazing Network[C]//2017 IEEE International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2017: 4780-4788.
16	仲伟峰, 赵晶. 基于生成对抗网络的图像去雾算法[J]. 激光与光电子学进展, 2022, 59(4): 329-337.
	Zhong Weifeng, Zhao Jing. Image Defogging Algorithm Based on Generative Adversarial Network[J]. Laser & Optoelectronics Progress, 2022, 59(4): 329-337.
17	Chen Dongdong, He Mingming, Fan Qingnan, et al. Gated Context Aggregation Network for Image Dehazing and Deraining[C]//2019 IEEE Winter Conference on Applications of Computer Vision (WACV). Piscataway, NJ, USA: IEEE, 2019: 1375-1383.
18	Zhu Hongyuan, Peng Xi, Chandrasekhar Vijay, et al. DehazeGAN: When Image Dehazing Meets Differential Programming[C]//Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence. California: IJCAI, 2018: 1234-1240.
19	Chen Zeyuan, Wang Yangchao, Yang Yang, et al. PSD: Principled Synthetic-to-real Dehazing Guided by Physical Priors[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2021: 7176-7185.
20	Tan Mingxing, Le Q. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks[C]//Proceedings of the 36th International Conference on Machine Learning. Chia Laguna Resort, Sardinia, Italy: PMLR, 2019: 6105-6114.
21	Lin Guosheng, Milan Anton, Shen Chunhua, et al. RefineNet: Multi-path Refinement Networks for High-resolution Semantic Segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ, USA: IEEE, 2017: 5168-5177.
22	Ronneberger Olaf, Fischer Philipp, Brox Thomas. U-net: Convolutional Networks for Biomedical Image Segmentation[C]//Medical Image Computing and Computer-Assisted Intervention-ICCAI 2015. Cham: Springer International Publishing, 2015: 234-241.
23	Lim Bee, Son Sanghyun, Kim Heewon, et al. Enhanced Deep Residual Networks for Single Image Super-resolution[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Piscataway, NJ, USA: IEEE, 2017: 1132-1140.
24	Simonyan K, Zisserman A. Very Deep Convolutional Networks for Large-scale Image Recognition[J]. (2015-04-10) [2022-10-09]. .
25	Mao Xudong, Li Qing, Xie Haoran, et al. Least Squares Generative Adversarial Networks[C]//2017 IEEE International Conference on Computer Vision (ICCV). Piscataway, NJ, USA: IEEE, 2017: 2813-2821.

算法	PSNR	SSIM
De-hazeNet	27.235 4	0.836 2
FFANet	22.356 3	0.839 4
GCANet	27.557 3	0.969 4
AODNet	25.652 8	0.876 2
本文	28.773 2	0.974 5

算法	PSNR	SSIM
De-hazeNet	28.568 5	0.940 1
FFANet	23.343 8	0.865 9
GCANet	27.162 5	0.956 3
AODNet	26.979 3	0.935 4
本文	28.362 1	0.970 3

算法	t/s
De-hazeNet	3.41
FFANet	1.38
GCANet	3.49
AODNet	0.68
本文	1.32

算法	PSNR	SSIM
MODEL-1	25.420 3	0.741 9
MODEL-2	23.485 2	0.716 4
本文	28.773 2	0.974 5

[1]	韦金阳, 王科平, 杨艺, 费树岷. 基于多重迁移注意力的增量式图像去雾算法[J]. 系统仿真学报, 2024, 36(4): 969-980.
[2]	张凤全, 曹铎, 马晓寒, 陈柏君, 张江霄. 一种面向戏曲妆容细节生成的风格迁移网络[J]. 系统仿真学报, 2023, 35(9): 2064-2076.
[3]	刘书刚, 张林坤, 杜昊东, 王洪涛. 雾天条件下改进YOLOv4的目标检测[J]. 系统仿真学报, 2023, 35(8): 1681-1691.
[4]	焦嵩鸣, 姚鑫, 丁辉, 钟宇飞. 适应于环境空间变化的激光雷达SLAM建图方法[J]. 系统仿真学报, 2023, 35(8): 1788-1798.
[5]	焦嵩鸣, 丁辉, 钟宇飞, 姚鑫, 郑家豪. 一种基于SiamRPN的无人机目标跟踪及控制算法[J]. 系统仿真学报, 2023, 35(6): 1372-1380.
[6]	蔡兴泉, 李治均, 奚梦瑶, 孙海燕. 基于神经网络的手绘服饰图纹上色及风格迁移[J]. 系统仿真学报, 2023, 35(3): 604-615.
[7]	金炜东, 张述礼, 唐鹏, 张曼. 基于稠密残差块与通道像素注意力的图像去雾网络[J]. 系统仿真学报, 2022, 34(8): 1663-1673.
[8]	曹建芳, 贾一鸣, 闫敏敏, 田晓东. 稳定增强生成对抗网络在壁画的超分辨率重建[J]. 系统仿真学报, 2022, 34(5): 1076-1089.
[9]	袁彬淦, 钟铭恩, 倪晶鑫. 线束端子压接后外观缺陷的视觉检测算法研究[J]. 系统仿真学报, 2022, 34(5): 1152-1159.
[10]	周维, 刘宇翔, 廖广平, 马鑫. 结合交并比损失的孪生网络目标跟踪算法研究[J]. 系统仿真学报, 2022, 34(09): 1956-1967.
[11]	吴曦, 孟祥林, 杨镜宇. 下一代战略博弈推演系统研究[J]. 系统仿真学报, 2021, 33(9): 2017-2024.
[12]	程文聪, 史小康, 王志刚. 基于生成对抗网络的仿真卫星云图生成方法[J]. 系统仿真学报, 2021, 33(6): 1297-1306.
[13]	冯晓, 张辉, 周蕊, 乔璐, 魏东, 李丹丹, 张玉尧, 郑国清. 基于深度学习和籽粒双面特征的玉米品种识别[J]. 系统仿真学报, 2021, 33(12): 2983-2991.
[14]	张青, 邹湘军, 林桂潮, 孙艳辉. 草莓重量和形状图像特征提取与在线分级方法[J]. 系统仿真学报, 2019, 31(1): 7-9.
[15]	周斌斌, 高尚兵, 潘志庚, 王亮亮, 王洪阳. 基于稀疏取样和梯度分布特征的车标识别[J]. 系统仿真学报, 2017, 29(9): 2035-2042.