上海交通大学学报

上海交通大学学报 >

2022 , Vol. 56 >Issue 2: 134 - 142

DOI: https://doi.org/10.16183/j.cnki.jsjtu.2021.075

利用生成对抗网络实现水下图像增强

展开
  • 上海交通大学 电子信息与电气工程学院, 上海 200240
李钰(1995-),女,山东省潍坊市人,硕士生,主要从事模式识别研究.

收稿日期: 2020-03-11

  网络出版日期: 2022-03-03

基金资助

国家自然科学基金(61633017);国家自然科学基金(61773264);国家自然科学基金(61801295);上海交通大学“深蓝计划”资助项目(SL2020MS011);上海交通大学“深蓝计划”资助项目(SL2020MS015)

收起

Underwater Image Enhancement Based on Generative Adversarial Networks

Expand
  • School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China

Received date: 2020-03-11

  Online published: 2022-03-03

Fold

摘要

提出一种基于生成对抗模型的水下图像修正与增强算法.该算法将多尺度内核应用于改进的残差模块中,以此构建生成器,实现多感受野特征信息的提取与融合;判别器设计考虑了全局信息与局部细节的关系,建立了全局-区域双判别结构,能够保证整体风格与边缘纹理的一致性;最后,根据人类视觉感官系统设计了无监督损失函数,此部分无需参考图像进行约束,同时其与对抗损失和内容损失一起进行联合优化,能够得到更优的色彩和结构表现.在多个数据集上进行实验分析表明,此算法能较好地修正色偏、对比度,保护细节信息不丢失,在主客观指标上都优于典型对比算法.

关键词： 水下图像增强; 生成对抗网络; 残差结构; 无监督学习

本文引用格式

李钰, 杨道勇, 刘玲亚, 王易因 . 利用生成对抗网络实现水下图像增强[J]. 上海交通大学学报, 2022 , 56(2) : 134 -142 . DOI: 10.16183/j.cnki.jsjtu.2021.075

Abstract

This paper proposes an underwater image correction and enhancement algorithm based on generative adversarial networks. In this algorithm, the multi-scale kernel is applied to the improved residual module to construct a generator, which realizes the extraction and fusion of multiple receptive fields feature information. The discriminator design considers the relationship between global information and local details, and establishes a global-region dual discriminator structure, which can ensure the consistency of overall style and edge texture. An unsupervised loss function based on human visual sensory system is proposed. Reference image constraints are not required, and the confrontation loss and the content loss are jointly optimized to obtain better color and structure performance. Experimental evaluations on multiple data sets show that the proposed algorithm can better correct color deviation and contrast, protect details from loss, and is superior to typical algorithms in subjective and objective indexes.

Key words： underwater image enhancement; generative adversarial networks; residual structure; unsupervised learning

参考文献

[1] WHITCOMB L, YOERGER D R, SINGH H, et al. Advances in underwater robot vehicles for deep ocean exploration: Navigation, control, and survey operations[M]//Robotics Research. New York, USA: Springer International Publishing, 2000: 439-448.
[2] LU H M, LI Y J, SERIKAWA S. Computer vision for ocean observing[M]//Artificial Intelligence and Computer Vision. New York, USA: Springer International Publishing, 2016: 1-16.
[3] GAUDRON J O, SURRE F, SUN T, et al. Long period grating-based optical fibre sensor for the underwater detection of acoustic waves[J]. Sensors and Actuators A: Physical, 2013, 201:289-293.
[4] YAN Z, MA J, TIAN J W, et al. A gravity gradient differential ratio method for underwater object detection[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(4):833-837.
[5] WANG Y, SONG W, FORTINO G, et al. An experimental-based review of image enhancement and image restoration methods for underwater imaging[J]. IEEE Access, 2019, 7:140233-140251.
[6] YANG M, HU J T, LI C Y, et al. An in-depth survey of underwater image enhancement and restoration[J]. IEEE Access, 2019, 7:123638-123657.
[7] LI C Y, GUO C L, REN W Q, et al. An underwater image enhancement benchmark dataset and beyond[J]. IEEE Transactions on Image Processing, 2020, 29:4376-4389.
[8] HE K M, SUN J, TANG X O. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12):2341-2353.
[9] DREWS JR P, DO NASCIMENTO E, MORAES F, et al. Transmission estimation in underwater single images[C]// 2013 IEEE International Conference on Computer Vision Workshops. Sydney, NSW, Australia: IEEE, 2013: 825-830.
[10] WANG Y, ZHANG J, CAO Y, et al. A deep CNN method for underwater image enhancement[C]// 2017 IEEE International Conference on Image Processing (ICIP). Beijing, China: IEEE, 2017: 1382-1386.
[11] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11):139-144.
[12] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]// 2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017: 2242-2251.
[13] ISLAM M J, XIA Y Y, SATTAR J. Fast underwater image enhancement for improved visual perception[J]. IEEE Robotics and Automation Letters, 2020, 5(2):3227-3234.
[14] LI Y, YU W B, LIU L Y, et al. SRM-net: An effective end-to-end neural network for single image dehazing[C]// Proceedings of the 3rd International Conference on Video and Image Processing. New York, NY, USA: ACM, 2019: 74-78.
[15] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL].(2016 -04-30)[2020-10-13].https://arxiv.org/abs/1511.07122.
[16] IIZUKA S, SIMO-SERRA E, ISHIKAWA H. Globally and locally consistent image completion[J]. ACM Transactions on Graphics, 2017, 36(4):1-14.
[17] RADFORD A, METZ L, CHINTALA S. Unsupervised representation learning with deep convolutional generative adversarial networks[EB/OL].(2015 -11-15)[2020-10-13]. https://www.researchgate.net/publication/284476553_Unsupervised_Representation_Learning_with_Deep_Convolutional_Generative_Adversarial_Networks.
[18] BARBOSA W V, AMARAL H G B, ROCHA T L, et al. Visual-quality-driven learning for underwater vision enhancement[C]// 2018 25th IEEE International Conference on Image Processing (ICIP). Athens, Greece: IEEE, 2018: 3933-3937.
[19] YAN Y F. Color image quality measures and retrieval[D]. New Jersey, USA: New Jersey Insititute of Techonlogy, 2006.
[20] RUDERMAN D L, CRONIN T W, CHIAO C C. Statistics of cone responses to natural images: Implications for visual coding[J]. Journal of the Optical Society of America A, 1998, 15(8):2036.
[21] JOHNSON J, ALAHI A, LI F F. Perceptual losses for real-time style transfer and super-resolution[C]// Computer Vision-ECCV 2016. Amsterdam, The Netherlands: ECCV, 2016: 694-711.
[22] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].(2014 -09-02) [2020-10-13]. https://www.researchgate.net/publication/319770291_Very_Deep_Convolutional_Networks_for_Large-Scale_Image_Recognition.
[23] GHANI A S A, ISA N A M. Underwater image quality enhancement through composition of dual-intensity images and Rayleigh-stretching[C]// 2014 IEEE Fourth International Conference on Consumer Electronics Berlin (ICCE-Berlin). Berlin, Germany: IEEE, 2014: 219-220.
[24] HUANG D M, WANG Y, SONG W, et al. Shallow-water image enhancement using relative global histogram stretching based on adaptive parameter acquisition[C]//International Conference on Multimedia Modeling. Bangkok, Thailand: MMM, 2018: 453-465.
[25] ANCUTI C, ANCUTI C O, HABER T, et al. Enhancing underwater images and videos by fusion [C]//2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA: IEEE, 2012: 81-88.
[26] ANCUTI C O, ANCUTI C, DE VLEESCHOUWER C, et al. Color balance and fusion for underwater image enhancement[J]. IEEE Transactions on Image Processing, 2018, 27(1):379-393.
[27] LI C Y, ANWAR S, PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement[J]. Pattern Recognition, 2020, 98:107038.
[28] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]//2017 IEEE International Conference on Computer Vision (ICCV). Venice, Italy: IEEE, 2017: 2242-2251.
Options
文章导航

/

〈 〉