Data Augmentation of Ship Wakes in SAR Images Based on Improved CycleGAN

doi:10.1007/s12204-024-2746-8

Abstract

Abstract: The study on ship wakes of synthetic aperture radar (SAR) images holds great importance in detecting ship targets in the ocean. In this study, we focus on the issues of low quantity and insufficient diversity in ship wakes of SAR images, and propose a method of data augmentation of ship wakes in SAR images based on the improved cycle-consistent generative adversarial network (CycleGAN). The improvement measures mainly include two aspects: First, to enhance the quality of the generated images and guarantee a stable training process of the model, the least-squares loss is employed as the adversarial loss function; Second, the decoder of the generator is augmented with the convolutional block attention module (CBAM) to address the issue of missing details in the generated ship wakes of SAR images at the microscopic level. The experiment findings indicate that the improved CycleGAN model generates clearer ship wakes of SAR images, and outperforms the traditional CycleGAN models in both subjective and objective aspects.

Key words: synthetic aperture radar (SAR), ship wake, data augmentation, cycle-consistent generative adversarial network (CycleGAN), attention mechanism

摘要： 合成孔径雷达（SAR）图像的舰船尾迹研究在海洋船舶目标的探测中具有重要意义。本研究针对SAR图像舰船尾迹数据样本数量少和多样性不足的问题，提出一种基于改进的循环一致性生成对抗网络（CycleGAN）的SAR图像舰船尾迹数据增强方法。改进措施主要包括两个方面：第一，采用最小二乘损失作为对抗损失函数，提高了生成图像的质量，稳定了模型的训练过程；第二，在生成器的解码器中嵌入卷积块注意力模块（CBAM），在微观层面上解决了生成的SAR图像舰船尾迹中信息丢失的问题。实验结果表明，改进后的CycleGAN模型能生成更清晰的SAR图像舰船尾迹样本，在主观和客观方面都优于传统的CycleGAN模型。

关键词: 合成孔径雷达（SAR），舰船尾迹，数据增强，循环一致性生成对抗网络（CycleGAN），注意力机制

CLC Number:

TP391.4

YAN Congqiang^1,2 (鄢丛强), GUO Zhengyun^3,4 (郭正玉), CAI Yunze^1,2∗∗ (蔡云泽). Data Augmentation of Ship Wakes in SAR Images Based on Improved CycleGAN[J]. J Shanghai Jiaotong Univ Sci, 2024, 29(4): 702-711.

References

[1] WANG J F, LIU X Z. Development in SAR movingtarget detection [J]. Journal of Shanghai Jiao Tong University, 2018, 52(10): 1273-1279 (in Chinese).
[2] RIZAEV I, KARAKUS? O, HOGAN S J, et al. The effect of sea state on ship wake detectability in simulated sar imagery [C]//2020 IEEE International Conference on Image Processing. Abu Dhabi: IEEE, 2020: 3478-3482.
[3] ZHAO T, WANG S T, NIU L, et al. Detection algorithm of ship wake in SAR images [J]. Journal of Shanghai Jiao Tong University, 2020, 54(12): 1259-1268 (in Chinese).
[4] LIU Y F, ZHAO J, QIN Y. A novel technique for ship wake detection from optical images [J]. Remote Sensing of Environment, 2021, 258: 112375.
[5] DEL PRETE R, GRAZIANO M D, RENGA A. RetinaNet: A deep learning architecture to achieve a robust wake detector in SAR images [C]//2021 IEEE 6th International Forum on Research and Technology for Society and Industry. Naples: IEEE, 2021: 171-176.
[6] DEL PRETE R, GRAZIANO M D, RENGA A. First results on wake detection in SAR images by deep learning [J]. Remote Sensing, 2021, 13(22): 4573.
[7] DING K Y, YANG J F, LIN H, et al. Towards real-time detection of ships and wakes with lightweight deep learning model in Gaofen-3 SAR images [J]. Remote Sensing of Environment, 2023, 284: 113345.
[8] LI Y, YANG D Y, LIU L Y, et al. Underwater image enhancement based on generative adversarial networks [J]. Journal of Shanghai Jiao Tong University, 2022, 56(2): 134-142 (in Chinese).
[9] BAO X J, PAN Z X, LIU L. SAR image simulation by generative adversarial networks [C]//5th China High Resolution Earth Observation Conference. Xi’an: China High Resolution Earth Observation, 2018: 1-14(in Chinese).
[10] HUANG Q N, ZHU W G, LI Y G. Review on SAR data expansion based on GAN [J]. Journal of Ordnance Equipment Engineering, 2021, 42(11): 31-38 (in Chinese).
[11] TANG Z Y, ZOU X H, LI P F, et al. A few-shots OFDM target augmented identification method based on transfer learning [J]. Journal of Shanghai Jiao Tong University, 2022, 56(12): 1666-1674 (in Chinese).
[12] LI Q, HUANGFU Y B, LI J Y, et al. UConvTrans: A dual-flow cardiac image segmentation network by global and local information integration [J]. Journal of Shanghai Jiao Tong University, 2023, 57(5): 570-581(in Chinese).
[13] RIZAEV I G, KARAKUS? O, HOGAN S J, et al. Modeling and SAR imaging of the sea surface: A review of the state-of-the-art with simulations [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 187: 120-140.
[14] WANG L T, ZHANG M, WEI P B. SAR model of the ship internal wake based on CFD simulation [C]//2018 IEEE International Conference on Computational Electromagnetics. Chengdu: IEEE, 2018: 1-2.
[15] ZHANG M, WANG L T. A coupled simulation of synthetic aperture radar image of the narrow ship wake [C]//2019 IEEE Asia-Pacific Microwave Conference. Singapore: IEEE, 2019: 1352-1353.
[16] GONG B, WANG Y J, CUI L, et al. On the ship wake simulation for multi-frequncy and mutli-polarization SAR imaging [C]//2018 China International SAR Symposium. Shanghai: IEEE, 2018: 1-6.
[17] WANG J J, GUO L X, WEI Y W, et al. Study on ship kelvin wake detection in numerically simulated SAR images [J]. Remote Sensing, 2023, 15(4): 1089.
[18] ZHANG M, LI J X. Numerical simulation and analyses of SAR images from ship wakes [C]//2019 International Applied Computational Electromagnetics Society Symposium. Miami: IEEE, 2019: 1-2.
[19] ZHANG M, FAN W N, LI J X. Numerical simulation and analyses of SAR images from moving ships over a sea surface [C]//2018 IEEE International Symposium on Antennas and Propagation & USNC URSI National Radio Science Meeting. Boston: IEEE, 2018: 573-574.
[20] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks [J]. Communications of the ACM, 2020, 63(11): 139-144.
[21] QI S Y, ZANG Y J, LYU G Y, et al. Research on air target image generation algorithm based on generative adversarial networks [J]. Air & Space Defense, 2021, 4(2): 67-73 (in Chinese).
[22] LU Q L, YE W. A survey of generative adversarial network applications for SAR image processing [J]. Telecommunication Engineering, 2020, 60(1): 121-128 (in Chinese).
[23] ZHU J Y, PARK T, ISOLA P, et al. Unpaired imageto-image translation using cycle-consistent adversarial networks [C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2242-2251.
[24] BAI J B, YANG Y, ZHANG W S. A simulation of the synthetic aperture radar image based on improved CycleGAN [J]. Journal of University of Science and Technology of China, 2020, 50(8): 1181-1186 (in Chinese).
[25] LIU L, LEI B. Can SAR images and optical images transfer with each other? [C]// 2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia: IEEE, 2018: 7019-7022. [26] IIZUKA S, SIMO-SERRA E, ISHIKAWA H. Globally and locally consistent image completion [J]. ACM Transactions on Graphics, 2017, 36(4): 107.
[27] MAO X D, LI Q, XIE H R, et al. Least squares generative adversarial networks [C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2813-2821.
[28] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module [M]// Computer vision– ECCV 2018. Cham: Springer, 2018: 3-19.
[29] HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[30] HU J, SHEN L, ALBANIE S, et al. Gather-excite: Exploiting feature context in convolutional neural networks [C]//32nd Conference on Neural Information Processing Systems. Montr′eal: NIPS, 2018: 1-11.
[31] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[32] XIA G S, BAI X, DING J, et al. DOTA: A large-scale dataset for object detection in aerial images [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 3974-3983.
[33] SUN X, WANG P J, YAN Z Y, et al. FAIR1M: A benchmark dataset for fine-grained object recognition in high-resolution remote sensing imagery [J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 184: 116-130.
[34] LIU Z K, WANG H Z, WENG L B, et al. Ship rotated bounding box space for ship extraction from high-resolution optical satellite images with complex backgrounds [J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(8): 1074-1078.
[35] ZHANG Y L, YUAN Y, FENG Y C, et al. Hierarchical and robust convolutional neural network for very high-resolution remote sensing object detection [J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(8): 5535-5548.
[36] LI J W, QU C W, SHAO J Q, et al. Dataset and performance analysis of ship detection methods based on deep learning [C]// 5th China High Resolution Earth Observation Conference. Xi’an: China High Resolution Earth Observation, 2018: 1-23 (in Chinese).
[37] RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation [M]//Medical image computing and computerassisted intervention – MICCAI 2015. Cham: Springer, 2015: 234-241.