Engineering & Al for Health

MCFF-Net: Multi-Scale Contextual Feature Fusion Network for Polyp Segmentation

Expand
  • Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming 650500, China; 2. Yunnan Provincial Key Laboratory of Artificial Intelligence, Kunming 650500, China

Received date: 2024-05-20

  Accepted date: 2024-08-05

  Online published: 2024-12-10

Abstract

In clinical practice, technologies for automatically segmenting polyps might greatly improve the speed and accuracy of medical diagnosis, significantly reducing the chances of patients developing colorectal cancer. Nevertheless, the segmentation of polyps presents a considerable challenge due to their low contrast, presence of occlusion, and intricate background disturbances. In this paper, we propose an innovative approach for segmenting polyps, called multi-scale contextual feature fusion network for polyp segmentation (MCFF-Net). Specifically, MCFF-Net includes three modules: feature enhancement module (FEM), global-local feature fusion module (GLFF), and cross-scale feature fusion module (CFFM). Among them, FEM is used to improve the features at each scale. Building on this, the CFFM enables comprehensive interaction among the information across various scales, enhancing the diversity of the feature scales extracted. Moreover, the GLFF fuses global-local features to generate a rough object localization map. This map further modulates the fusion features from the CFFM, facilitating more precise segmentation. On five publicly available datasets, our approach not only segments various polyp types but also surpasses existing methods in terms of accuracy and generalizability.

Cite this article

MA Lei, YAN Jiangkai, SHAO Dangguo, LI Jingtao, WANG Jiawei, YAN Yukun . MCFF-Net: Multi-Scale Contextual Feature Fusion Network for Polyp Segmentation[J]. Journal of Shanghai Jiaotong University(Science), 2026 , 31(5) : 1237 -1249 . DOI: 10.1007/s12204-024-2791-3

References

[1] SUNG H, FERLAY J, SIEGEL R L, et al. Global cancer statistics 2020: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries[J]. CA: A Cancer Journal for Clinicians, 2021, 71(3): 209-249.
[2] SIEGEL R L, MILLER K D, GODING SAUER A, et al. Colorectal cancer statistics, 2020 [J]. CA: A Cancer Journal for Clinicians, 2020, 70(3): 145-164.
[3] RAHIM T, USMAN M A, SHIN S Y. A survey on contemporary computer-aided tumor, polyp, and ulcer detection methods in wireless capsule endoscopy imaging [J]. Computerized Medical Imaging and Graphics, 2020, 85: 101767.
[4] SÁNCHEZ-PERALTA L F, PAGADOR J B, SÁNCHEZ-MARGALLO F M. Artificial intelligence for colorectal polyps in colonoscopy [M]//Artificial intelligence in medicine. Cham: Springer, 2022: 967-981.
[5] RONNEBERGER O, FISCHER P, BROX T. U-net: Convolutional networks for biomedical image segmentation [M]// Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. Cham: Springer, 2015: 234-241.
[6] OTSU N. A threshold selection method from gray-level histograms [J]. IEEE Transactions on Systems, Man, and Cybernetics, 1979, 9(1): 62-66.
[7] Wang K, Zhang X, Zhang X, et al. EANet: Iterative edge attention network for medical image segmentation [J]. Pattern Recognition, 2022, 127: 108636.
[8] ZHANG Y J. An overview of image and video segmentation in the last 40 years [M]// Advances in Image and Video Segmentation. Hershey: IGI Global, 2006: 1-16.
[9] ACHANTA R, SHAJI A, SMITH K, et al. SLIC superpixels compared to state-of-the-art superpixel methods [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(11): 2274-2282.
[10] FAN D P, JI G P, ZHOU T, et al. PraNet: parallel reverse attention network for polyp segmentation [M]// Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Cham: Springer, 2020: 263-273.
[11] ZHANG R, LI G, LI Z, et al. Adaptive context selection for polyp segmentation [M]// Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. Cham: Springer, 2020: 253-262.
[12] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need [C]// 31st Conference on Neural Information Processing Systems. Long Beach: NIPS, 2017: 1-11. 
[13] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale 16 words: Transformers for image recognition at scale [DB/OL]. (2020-10-20). https://arxiv.org/abs/2010.11929
[14] DONG B, WANG W H, FAN D P, et al. Polyp-PVT: Polyp segmentation with pyramid vision transformers [DB/OL]. (2021-08-16). https://arxiv.org/abs/2108.06932
[15] LIU X G, SONG S. Attention combined pyramid vision transformer for polyp segmentation [J]. Biomedical Signal Processing and Control, 2024, 89: 105792.
[16] XIAO B, HU J W, LI W S, et al. CTNet: Contrastive transformer network for polyp segmentation [J]. IEEE Transactions on Cybernetics, 2024, 54(9): 5040-5053.
[17] WANG W H, XIE E Z, LI X, et al. PVT v2: Improved baselines with Pyramid Vision Transformer [J]. Computational Visual Media, 2022, 8(3): 415-424.
[18] LIU S T, HUANG D, WANG Y H. Receptive field block net for accurate and fast object detection [M]//Computer vision – ECCV 2018. Cham: Springer, 2018: 404-419. 
[19] WANG X L, GIRSHICK R, GUPTA A, et al. Non-local neural networks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7794-7803.
[20] ZHAO H S, SHI J P, QI X J, et al. Pyramid scene parsing network [C]//2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6230-6239.
[21] YU C Q, WANG J B, PENG C, et al. BiSeNet: bilateral segmentation network for real-time semantic segmentation [M]// Computer Vision – ECCV 2018. Cham: Springer International Publishing, 2018: 334-349.
[22] HU J, SHEN L, SUN G. Squeeze-and-excitation networks [C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 7132-7141.
[23] LIU H J, LIU F Q, FAN X Y, et al. Polarized self-attention: Towards high-quality pixel-wise regression [DB/OL]. (2021-07-02). https://arxiv.org/abs/2107.00782
[24] PARK T, LIU M Y, WANG T C, et al. Semantic image synthesis with spatially-adaptive normalization [C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 2332-2341.
[25] WEI J, WANG S H, HUANG Q M. F³Net: Fusion, feedback and focus for salient object detection [J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7): 12321-12328.
[26] JHA D, SMEDSRUD P H, RIEGLER M A, et al. Kvasir-SEG: A segmented polyp dataset [M]// MultiMedia Modeling. Cham: Springer, 2020: 451-462.
[27] SILVA J, HISTACE A, ROMAIN O, et al. Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer [J]. International Journal of Computer Assisted Radiology and Surgery, 2014, 9(2): 283-293.
[28] TAJBAKHSH N, GURUDU S R, LIANG J M. Automated polyp detection in colonoscopy videos using shape and context information [J]. IEEE Transactions on Medical Imaging, 2016, 35(2): 630-644.
[29] BERNAL J, SÁNCHEZ F J, FERNÁNDEZ-ESPARRACH G, et al. WM-DOVA maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians [J]. Computerized Medical Imaging and Graphics, 2015, 43: 99-111.
[30] VÁZQUEZ D, BERNAL J, SÁNCHEZ F J, et al. A benchmark for endoluminal scene segmentation of colonoscopy images [J]. Journal of Healthcare Engineering, 2017, 2017: 4037190.
[31] MILLETARI F, NAVAB N, AHMADI S A. V-net: Fully convolutional neural networks for volumetric medical image segmentation [C]//2016 Fourth International Conference on 3D Vision. Stanford: IEEE, 2016: 565-571.
[32] MARGOLIN R, ZELNIK-MANOR L, TAL A. How to evaluate foreground maps [C]//2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus: IEEE, 2014: 248-255.
[33] FAN D P, CHENG M M, LIU Y, et al. Structure-measure: A new way to evaluate foreground maps [C]//2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 4558-4567.
[34] FAN D P, JI G P, QIN X B, et al. Cognitive vision inspired object segmentation metric and loss function [J]. Scientia Sinica Informationis, 2021, 51(9): 1475-1479 (in Chinese).
[35] LOSHCHILOV I, HUTTER F. Decoupled weight decay regularization [DB/OL]. (2017-11-14). https://arxiv.org/abs/1711.05101
[36] ZHOU Z, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: A nested U-Net architecture for medical image segmentation [M]//Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer, 2018: 3-11.
[37] ZHAO X, ZHANG L, LU H. Automatic polyp segmentation via multi-scale subtraction network [M]//Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. Cham: Springer, 2021: 120-130.
[38] WEI J, HU Y, ZHANG R, et al. Shallow attention network for polyp segmentation [M]//Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. Cham: Springer, 2021: 699-708.【自助C缺】
[39] LOU A G, GUAN S Y, KO H, et al. CaraNet: Context axial reverse attention network for segmentation of small medical objects [C]//Medical Imaging 2022: Image Processing. San Diego: SPIE, 2022: 81-92.
[40] WANG J F, HUANG Q M, TANG F L, et al. Stepwise feature fusion: Local guides global [M]// Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. Cham: Springer, 2022: 110-120.
[41] ZHOU T, ZHOU Y, HE K L, et al. Cross-level feature aggregation network for polyp segmentation [J]. Pattern Recognition, 2023, 140: 109555.
[42] DAI Y M, GIESEKE F, OEHMCKE S, et al. Attentional feature fusion [C]//2021 IEEE Winter Conference on Applications of Computer Vision. Waikoloa: IEEE, 2021: 3559-3568.
[43] GAO S H, CHENG M M, ZHAO K, et al. Res2Net: A new multi-scale backbone architecture [J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 43(2): 652-662.
Outlines

/

〈 〉