用于水下图像增强的全局密集双分支级联网络

doi:10.1007/s12204-024-2735-y

摘要/Abstract

摘要： 近年来，随着海洋资源开发的兴起，水下图像增强技术备受关注。针对现有网络特征提取不充分和增强结果存在去雾不彻底和色偏校正不准确的问题，提出了一种基于全局密集双分支级联网络和空域灰度变换的水下图像增强方法。全局密集双分支级联网络一方面可以在减少信息缩减的同时，放大全局维度交互特征；另一方面通过获得不同尺度的空间信息来提取空间特征，从而实现更丰富的特征提取。空域灰度变换操作可以在提高对比度的同时对图像进行颜色校正，使得图像视觉效果更好。在训练完成后，可以端到端的对水下图像进行推理。实验结果表明，本文的模型在EUVP数据集上效果最好；与第二名相比，本文的模型在PSNR、SSIM、UIQM、UCIQE和CCF方面分别获得了3.371、0.06、0.716、0.024和2.527 dB的提升。和其他代表性方法相比，所提网络在处理水下图像严重色偏、低光照、细节丢失方面取得了显著的视觉效果提升。

关键词: 多尺度密集块, 全局注意力机制, 级联网络, 空域灰度变换

Abstract: In recent years, underwater image enhancement techniques has received a wide range of attention from related researchers with the rise of marine resource exploitation. As the existing network feature extraction is not sufficient and the enhancement results have the problems of incomplete defogging and inaccurate color bias correction, in this paper, an underwater image enhancement method based on global dense two-branch cascade network and spatial domain grayscale transformation is proposed. The global dense two-branch cascade network can amplify the global dimensional interaction features while reducing information reduction on the one hand, and extract spatial features by obtaining spatial information at different scales to achieve richer feature extraction on the other hand; the spatial domain grayscale transformation operation can improve the contrast while color correcting the image, which makes the image visual effect better. After the training is completed, an end-to-end inference can be performed on the underwater images. The experimental results show that this paper’s model works best on the EUVP dataset, and compared with the second best, this paper’s model obtains 3.371, 0.06, 0.716, 0.024, and 1.727 improvements in PSNR, SSIM, UIQM, UCIQE, and CCF, respectively. Compared with other representative methods, the proposed network achieves significant visual enhancement in dealing with severe color bias, low light, and detail loss in underwater images.

中图分类号:

TP391.4

. 用于水下图像增强的全局密集双分支级联网络[J]. J Shanghai Jiaotong Univ Sci, 2026, 31(2): 458-474.

Wang Yan, Wang Likang, Zhang Jinfeng, Fan Xianghui. Global Dense Two-Branch Cascade Network for Underwater Image Enhancement[J]. J Shanghai Jiaotong Univ Sci, 2026, 31(2): 458-474.

参考文献

[1] HUMMEL R. Image enhancement by histogram transformation [J]. Computer Graphics and Image Processing, 1977, 6(2): 184-195.

[2] BUCHSBAUM G. A spatial processor model for object colour perception [J]. Journal of the Franklin Institute, 1980, 310(1): 1-26.

[3] PIZER S M, JOHNSTON R E, ERICKSEN J P, et al. Contrast-limited adaptive histogram equalization: Speed and effectiveness [C]// First Conference on Visualization in Biomedical Computing. Atlanta: IEEE, 2002: 337-345.

[4] LIU Y C, CHAN W H, CHEN Y Q. Automatic white balance for digital still camera [J]. IEEE Transactions on Consumer Electronics, 1995, 41(3): 460-466.

[5] SINGH G, JAGGI N, VASAMSETTI S, et al. Underwater image/video enhancement using wavelet based color correction (WBCC) method [C]//2015 IEEE Underwater Technology. Chennai: IEEE, 2015: 1-5.

[6] ANCUTI C, ANCUTI C O, HABER T, et al. Enhancing underwater images and videos by fusion [C]//2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012: 81-88.

[7] FU X Y, ZHUANG P X, HUANG Y, et al. A retinex-based enhancing approach for single underwater image [C]//2014 IEEE International Conference on Image Processing. Paris: IEEE, 2014: 4572-4576.

[8] MCGLAMERY B L. A computer model for underwater camera systems [C]// Proceedings of SPIE, 1980, 0208: 221-231.

[9] CARLEVARIS-BIANCO N, MOHAN A, EUSTICE R M. Initial results in underwater single image dehazing [C]//OCEANS 2010 MTS/IEEE SEATTLE. Seattle: IEEE, 2010: 1-8.

[10] DREWS P L J Jr, NASCIMENTO E R, BOTELHO S S C, et al. Underwater depth estimation and image restoration based on single images [J]. IEEE Computer Graphics and Applications, 2016, 36(2): 24-35.

[11] PENG Y T, COSMAN P C. Underwater image restoration based on image blurriness and light absorption [J]. IEEE Transactions on Image Processing, 2017, 26(4): 1579-1594.

[12] SONG W, WANG Y, HUANG D M, et al. A rapid scene depth estimation model based on underwater light attenuation prior for underwater image restoration [M]// Advances in Multimedia Information Processing – PCM 2018. Cham: Springer, 2018: 678-688.

[13] LIU R S, FAN X, HOU M J, et al. Learning aggregated transmission propagation networks for haze removal and beyond [J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(10): 2973-2986.

[14] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks [C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2242-2251.

[15] LI C Y, GUO J C, GUO C L. Emerging from water: Underwater image color correction based on weakly supervised color transfer [J]. IEEE Signal Processing Letters, 2018, 25(3): 323-327.

[16] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 1646-1654.

[17] LIU P, WANG G Y, QI H, et al. Underwater image enhancement with a deep residual framework [J]. IEEE Access, 2019, 7: 94614-94629.

[18] ISLAM M J, XIA Y Y, SATTAR J. Fast underwater image enhancement for improved visual perception [J]. IEEE Robotics and Automation Letters, 2020, 5(2): 3227-3234.

[19] LI C Y, ANWAR S, PORIKLI F. Underwater scene prior inspired deep underwater image and video enhancement [J]. Pattern Recognition, 2020, 98: 107038.

[20] CHEN X L, ZHANG P, QUAN L W, et al. Underwater image enhancement based on deep learning and image formation model [DB/OL]. (2021-01-04). http://arxiv.org/abs/2101.00991

[21] NAIK A, SWARNAKAR A, MITTAL K. Shallow-UWnet: Compressed model for underwater image enhancement (student abstract) [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(18): 15853-15854.

[22] YAN K, LIANG L Y, ZHENG Z Q, et al. Medium transmission map matters for learning to restore real-world underwater images [J]. Applied Sciences, 2022, 12(11): 5420.

[23] LI C Y, ANWAR S, HOU J H, et al. Underwater image enhancement via medium transmission-guided multi-color space embedding [J]. IEEE Transactions on Image Processing, 2021, 30: 4985-5000.

[24] ZHANG W D, WANG Y D, LI C Y. Underwater image enhancement by attenuated color channel correction and detail preserved contrast enhancement [J]. IEEE Journal of Oceanic Engineering, 2022, 47(3): 718-735.

[25] ZHUANG P X, WU J M, PORIKLI F, et al. Underwater image enhancement with hyper-laplacian reflectance priors [J]. IEEE Transactions on Image Processing, 2022, 31: 5442-5455.

[26] ZHANG W D, JIN S L, ZHUANG P X, et al. Underwater image enhancement via piecewise color correction and dual prior optimized contrast enhancement [J]. IEEE Signal Processing Letters, 1809, 30: 229-233.

[27] ZHANG W D, ZHUANG P X, SUN H H, et al. Underwater image enhancement via minimal color loss and locally adaptive contrast enhancement [J]. IEEE Transactions on Image Processing, 2022, 31: 3997-4010.

[28] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module [M]// Computer vision – ECCV 2018. Cham: Springer, 2018: 3-19.

[29] PARIS S, DURAND F. A fast approximation of the bilateral filter using a signal processing approach [M]//Computer vision – ECCV 2006. Berlin, Heidelberg: Springer, 2006: 568-580.

[30] ZHAO H, GALLO O, FROSIO I, et al. Loss functions for image restoration with neural networks [J]. IEEE Transactions on Computational Imaging, 2017, 3(1): 47-57.

[31] LI C Y, GUO C L, REN W Q, et al. An underwater image enhancement benchmark dataset and beyond [J]. IEEE Transactions on Image Processing, 2020, 29: 4376-4389.

[32] JAHIDUL ISLAM M, LUO P G, SATTAR J. Simultaneous enhancement and super-resolution of underwater imagery for improved visual perception [DB/DL]. (2020-02-04). https://arxiv.org/abs/2002.01155

[33] AVCIBAS I, SANKUR B, SAYOOD K. Statistical evaluation of image quality measures [J]. Journal of Electronic Imaging, 2002, 11(2): 206-223.

[34] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: From error visibility to structural similarity [J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.

[35] PANETTA K, GAO C, AGAIAN S. Human-visual-system-inspired underwater image quality measures [J]. IEEE Journal of Oceanic Engineering, 2016, 41(3): 541-551.

[36] YANG M, SOWMYA A. An underwater color image quality evaluation metric [J]. IEEE Transactions on Image Processing, 2015, 24(12): 6062-6071.

[37] WANG Y, LI N, LI Z Y, et al. An imaging-inspired no-reference underwater color image quality assessment metric [J]. Computers & Electrical Engineering, 2018, 70: 904-913.