Segmentation of the retinal vessels in the fundus is crucial for diagnosing ocular diseases. Retinal vessel images often suffer from category imbalance and large scale variations. This ultimately results in incomplete vessel segmentation and poor continuity. In this study, we propose CT-MFENet to address the aforementioned issues. First, the use of context transformer (CT) allows for the integration of contextual feature information, which helps establish the connection between pixels and solve the problem of incomplete vessel continuity. Second, multi-scale dense residual networks are used instead of traditional CNN to address the issue of inadequate local feature extraction when the model encounters vessels at multiple scales. In the decoding stage, we introduce a local-global fusion module. It enhances the localization of vascular information and reduces the semantic gap between high- and low-level features. To address the class imbalance in retinal images, we propose a hybrid loss function that enhances the segmentation ability of the model for topological structures. We conducted experiments on the publicly available DRIVE, CHASEDB1, STARE, and IOSTAR datasets. The experimental results show that our CT-MFENet performs better than most existing methods, including the baseline U-Net.
[1] ABRAMOFF M D, GARVIN M K, SONKA M. Retinal imaging and image analysis [J]. IEEE Reviews in Biomedical Engineering, 2010, 3: 169-208.
[2] SHENG B, LI P, MO S, et al. Retinal vessel segmentation using minimum spanning superpixel tree detector [J]. IEEE Trans Cybern, 2019, 49(7): 2707-2719.
[3] JIN Q G, MENG Z P, PHAM T D, et al. DUNet: A deformable network for retinal vessel segmentation [J]. Knowledge-Based Systems, 2019, 178: 149-162.
[4] MOU L, CHEN L, CHENG J, et al. Dense dilated network with probability regularized walk for vessel detection [J]. IEEE Trans Med Imaging, 2020, 39(5): 1392-1403.
[5] WU H, WANG W, ZHONG J, et al. SCS-net: A scale and context sensitive network for retinal vessel segmentation [J]. Med Image Anal, 2021, 70: 102025.
[6] GALDRAN A, ANJOS A, DOLZ J, et al. State-of-the-art retinal vessel segmentation with minimalistic models [J]. Scientific Reports, 2022, 12(1): 6174.
[7] CHAUDHURI S, CHATTERJEE S, KATZ N, et al. Detection of blood vessels in retinal images using two-dimensional matched filters [J]. IEEE Transactions on Medical Imaging, 1989, 8(3): 263-269.
[8] CAN A L, SHEN H, TURNER J N, et al. Rapid automated tracing and feature extraction from retinal fundus images using direct exploratory algorithms [J]. IEEE Transactions on Information Technology in Biomedicine, 1999, 3(2): 125-138.
[9] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// 31st Conference on Neural Information Processing Systems. Long Beach: NIPS, 2017: 1-11.
[10] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale [DB/OL]. (2020-10-22). https://arxiv.org/abs/2010.11929
[11] RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation [M]// Medical image computing and computer-assisted intervention – MICCAI 2015. Cham: Springer, 2015: 234-241.
[12] CHEN J N, LU Y Y, YU Q H, et al. TransUNet: Transformers make strong encoders for medical image segmentation [DB/OL]. (2021-02-08). http://arxiv.org/abs/2102.04306
[13] WU Y C, XIA Y, SONG Y, et al. Multiscale network followed network model for retinal vessel segmentation [M]//Medical image computing and computer assisted intervention – MICCAI 2018. Cham: Springer, 2018: 119-126.
[14] OKTAY O, SCHLEMPER J, LE FOLGOC L, et al. Attention U-net: Learning where to look for the pancreas [DB/OL]. (2018-04-11). http://arxiv.org/abs/1804.03999
[15] YAN Z Q, YANG X, CHENG K T. Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation [J]. IEEE Transactions on Biomedical Engineering, 2018, 65(9): 1912-1923.
[16] LI X, CHEN H, QI X, et al. H-DenseUNet: Hybrid densely connected UNet for liver and tumor segmentation from CT volumes [J]. IEEE Trans Medical Imaging, 2018, 37(12): 2663-2674.
[17] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition [C]//2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[18] GHIASI G, LIN T Y, LE Q V. DropBlock: A regularization method for convolutional networks [C]// 32nd International Conference on Neural Information Processing Systems. Montréal: NIPS, 2018: 10750-10760.
[19] HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[20] STAAL J, ABRAMOFF M D, NIEMEIJER M, et al. Ridge-based vessel segmentation in color images of the retina [J]. IEEE Transactions on Medical Imaging, 2004, 23(4): 501-509.
[21] OWEN C G, RUDNICKA A R, MULLEN R, et al. Measuring retinal vessel tortuosity in 10-year-old children: Validation of the computer-assisted image analysis of the retina (CAIAR) program [J]. Investigative Opthalmology & Visual Science, 2009, 50(5): 2004.
[22] WANG B, QIU S, HE H G. Dual encoding U-net for retinal vessel segmentation [M]// medical image computing and computer assisted intervention – MICCAI 2019. Cham: Springer, 2019: 84-92.
[23] HOOVER A D, KOUZNETSOVA V, GOLDBAUM M. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response [J]. IEEE Transactions on Medical Imaging, 2000, 19(3): 203-210.
[24] ZHANG J, DASHTBOZORG B, BEKKERS E, et al. Robust retinal vessel segmentation via locally adaptive derivative frames in orientation scores [J]. IEEE Transactions on Medical Imaging, 2016, 35(12): 2631-2644.
[25] ZUIDERVELD K. Contrast limited adaptive histogram equalization [M]//Graphics gems. Amsterdam: Elsevier, 1994: 474-485.
[26] WANG S H, LI L, ZHUANG X H. AttU-NET: Attention U-net for brain tumor segmentation [M]// Brainlesion: Glioma, multiple sclerosis, stroke and traumatic brain injuries. Cham: Springer, 2022: 302-311.
[27] LI L Z, VERMA M, NAKASHIMA Y, et al. IterNet: Retinal image segmentation utilizing structural redundancy in vessel networks [C]//2020 IEEE Winter Conference on Applications of Computer Vision. Snowmass: IEEE, 2020: 3645-3654.
[28] HUANG H M, LIN L F, TONG R F, et al. UNet 3: A full-scale connected UNet for medical image segmentation [C]// 2020 IEEE International Conference on Acoustics, Speech and Signal Processing. Barcelona: IEEE, 2020: 1055-1059.
[29] SAHA TCHINDA B, TCHIOTSOP D, NOUBOM M, et al. Retinal blood vessels segmentation using classical edge detection filters and the neural network [J]. Informatics in Medicine Unlocked, 2021, 23: 100521.
[30] JIANG Y, LIU W, WU C, et al. Multi-scale and multi-branch convolutional neural network for retinal image segmentation [J]. Symmetry, 2021, 13(3): 365.
[31] ZHANG Y, HE M, CHEN Z, et al. Bridge-Net: Context-involved U-net with patch-based loss weight mapping for retinal blood vessel segmentation [J]. Expert Systems with Applications, 2022, 195: 116526.
[32] LI J Y, GAO G, LIU Y H, et al. MAGF-Net: A multiscale attention-guided fusion network for retinal vessel segmentation [J]. Measurement, 2023, 206: 112316.
[33] LI Z, LI B Y, ZHANG J, et al. MFCTrans-net: A multi-scale fusion and channel transformer net for retinal vessel segmentation [C]//Fourteenth International Conference on Graphics and Image Processing. Nanjing: SPIE, 2023: 537-545.
[34] LI X, JIANG Y C, LI M L, et al. Lightweight attention convolutional neural network for retinal vessel image segmentation [J]. IEEE Transactions on Industrial Informatics, 2021, 17(3): 1958-1967.
