Application of Deep Learning Method on Ischemic Stroke Lesion Segmentation

doi:10.1007/s12204-021-2273-9

摘要/Abstract

Abstract: Although deep learning methods have been widely applied in medical image lesion segmentation, it is still challenging to apply them for segmenting ischemic stroke lesions, which are different from brain tumors in lesion characteristics, segmentation difficulty, algorithm maturity, and segmentation accuracy. Three main stages are used to describe the manifestations of stroke. For acute ischemic stroke, the size of the lesions is similar to that of brain tumors, and the current deep learning methods have been able to achieve a high segmentation accuracy. For sub-acute and chronic ischemic stroke, the segmentation results of mainstream deep learning algorithms are still unsatisfactory as lesions in these stages are small and diffuse. By using three scientific search engines including CNKI, Web of Science and Google Scholar, this paper aims to comprehensively understand the state-of-the-art deep learning algorithms applied to segmenting ischemic stroke lesions. For the first time, this paper discusses the current situation, challenges, and development directions of deep learning algorithms applied to ischemic stroke lesion segmentation in different stages. In the future, a system that can directly identify different stroke stages and automatically select the suitable network architecture for the stroke lesion segmentation needs to be proposed.

Key words: ischemic stroke, deep learning, brain image segmentation

中图分类号:

. [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 99-111.

ZHANG Yue (张月), LIU Shijie (刘世界), LI Chunlai (李春来), WANG Jianyu (王建宇). Application of Deep Learning Method on Ischemic Stroke Lesion Segmentation[J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 99-111.

参考文献 96

[1]	WANG Y, LIU H, LIU Y, et al. Deep learning frameworkfor hemorrhagic stroke segmentation and detection[C]//International Conference on Biological Informationand Biomedical Engineering. Shanghai: VDE,2018: 78-83.
[2]	DOYLE K P, SIMON R P, STENZEL-POORE M P.Mechanisms of ischemic brain damage [J]. Neuropharmacology,2008, 55(3): 310-318.
[3]	NALL R. What are the different types of strokes?[EB/OL]. (2018-09-20) [2020-08-01]. https://www.healthline.com/health/stroke-types.
[4]	MAIER O, MENZE B H, VON DER GABLENTZ J,et al. ISLES 2015- A public evaluation benchmark for ischemic stroke lesion segmentation from multispectralMRI [J]. Medical Image Analysis, 2017, 35: 250-269.
[5]	REKIK I, ALLASSONNI`ERE S, CARPENTER T K,et al. Medical image analysis methods in MR/CTimagedacute-subacute ischemic stroke lesion: Segmentation,prediction and insights into dynamic evolutionsimulation models. A critical appraisal [J].NeuroImage: Clinical, 2012, 1(1): 164-178.
[6]	LIEW S L, ANGLIN J M, BANKS N W, et al. A large,open source dataset of stroke anatomical brain imagesand manual lesion segmentations [J]. Scientific Data,2018, 5: 180011.
[7]	ZAHARCHUK G, EL MOGY I S, FISCHBEIN N J,et al. Comparison of arterial spin labeling and bolusperfusion-weighted imaging for detecting mismatch inacute stroke [J]. Stroke, 2012, 43(7): 1843-1848.
[8]	WANG G, SONG T, DONG Q, et al. Automatic ischemicstroke lesion segmentation from computed tomographyperfusion images by image synthesis andattention-based deep neural networks [J]. Medical ImageAnalysis, 2020, 65: 101787.
[9]	MEZZAPESA D M, PETRUZZELLIS M, LUCIVEROV, et al. Multimodal MR examination in acute ischemicstroke [J]. Neuroradiology, 2006, 48(4): 238-246.
[10]	GAO C. Research on ischemic stroke lesion segmentationbased on deep learning [D]. Nanchang: NanchangUniversity, 2019 (in Chinese).
[11]	GILLEBERT C R, HUMPHREYS G W, MANTINID. Automated delineation of stroke lesions using brainCT images [J]. NeuroImage: Clinical, 2014, 4: 540-548.
[12]	DONAHUE J, WINTERMARK M. Perfusion CT andacute stroke imaging: Foundations, applications, andliterature review [J]. Journal of Neuroradiology, 2015,42(1): 21-29.
[13]	KAMNITSAS K, LEDIG C, NEWCOMBE V F J, etal. Efficient multi-scale 3D CNN with fully connectedCRF for accurate brain lesion segmentation [J]. MedicalImage Analysis, 2017, 36: 61-78.
[14]	LIU L, CHEN S, ZHANG F, et al. Deep convolutionalneural network for automatically segmenting acute ischemicstroke lesion in multi-modality MRI [J]. NeuralComputing and Applications, 2020, 32: 6545-6558
[15]	POLMAN C H, REINGOLD S C, EDAN G, et al.Diagnostic criteria for multiple sclerosis: 2005 revisionsto the “McDonald Criteria” [J]. Annals of Neurology,2005, 58(6): 840-846.
[16]	TOMITA N, JIANG S, MAEDER M E, et al. Automaticpost-stroke lesion segmentation on MR imagesusing 3D residual convolutional neural network[J]. NeuroImage: Clinical, 2020, 27: 102276.
[17]	ZHANG L, SONG R, WANG Y, et al. Ischemic strokelesion segmentation using multi-plane information fusion[J]. IEEE Access, 2020, 8: 45715-45725.
[18]	ZHANG L, TANNO R, XU M C, et al. Disentanglinghuman error from the ground truth in segmentationof medical images [DB/OL]. (2020-07-31) [2020-08-01].https://arxiv.org/abs/2007.15963.
[19]	LITJENS G, KOOI T, BEJNORDI B E, et al. A surveyon deep learning in medical image analysis [J].Medical Image Analysis, 2017, 42: 60-88.
[20]	LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[21]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E.ImageNet classification with deep convolutional neuralnetworks [J]. Communications of the ACM, 2017,60(6): 84-90.
[22]	RONNEBERGER O, FISCHER P, BROX T. U-net:Convolutional networks for biomedical image segmentation[M]//Medical image computing and computerassistedintervention-MICCAI 2015. Cham: Springer,2015: 234-241.
[23]	GHAFFARI M, SOWMYA A, OLIVER R. Automatedbrain tumor segmentation using multimodalbrain scans: A survey based on models submitted tothe BraTS 2012–2018 challenges [J]. IEEE Reviews inBiomedical Engineering, 2020, 13: 156-168.
[24]	WADHWA A, BHARDWAJ A, SINGH VERMA V.A review on brain tumor segmentation of MRI images[J]. Magnetic Resonance Imaging, 2019, 61: 247-259.
[25]	SARITHA S, AMUTHA PRABHA N. A comprehensivereview: Segmentation of MRI images—brain tumor[J]. International Journal of Imaging Systems andTechnology, 2016, 26(4): 295-304.
[26]	ZHAO X, WU Y, SONG G, et al. A deep learningmodel integrating FCNNs and CRFs for brain tumorsegmentation [J]. Medical Image Analysis, 2018, 43:98-111.
[27]	BEN NACEUR M, AKIL M, SAOULI R, et al.Fully automatic brain tumor segmentation with deeplearning-based selective attention using overlappingpatches and multi-class weighted cross-entropy [J].Medical Image Analysis, 2020, 63: 101692.
[28]	JEONG J, LEI Y, SHU H K, et al. Brain tumor segmentationusing 3D mask R-CNN for dynamic susceptibilitycontrast enhanced perfusion imaging [J]. Proceedingsof SPIE, 2020: 11317: 1131720.
[29]	ZHOU C, DING C, WANG X, et al. One-pass multitasknetworks with cross-task guided attention forbrain tumor segmentation [J]. IEEE Transactions onImage Processing, 2020, 29: 4516-4529.
[30]	LONG J, SHELHAMER E, DARRELL T. Fullyconvolutional networks for semantic segmentation[C]//2015 IEEE Conference on Computer Vision andPattern Recognition (CVPR). Piscataway, NJ, USA:IEEE, 2015: 3431-3440.
[31]	MILLETARI F, NAVAB N, AHMADI S A. V-net:Fully convolutional neural networks for volumetricmedical image segmentation [C]//2016 Fourth InternationalConference on 3D Vision (3DV). Piscataway,NJ: IEEE, 2016: 565-571.
[32]	WINZECK S, HAKIM A, MCKINLEY R, et al. Isles2016 and 2017-benchmarking ischemic stroke lesionoutcome prediction based on multispectral MRI [J].Frontiers in Neurology, 2018, 9: 679.
[33]	ISLES: ISLES challenge 2018: Ischemic stroke lesionsegmentation [EB/OL]. (2018-12-05) [2020-08-01].http://www.isles-challenge.org/.
[34]	PEREIRA S, PINTO A, AMORIM J, et al. Adaptivefeature recombination and recalibration for semanticsegmentation with fully convolutional networks [J].IEEE Transactions on Medical Imaging, 2019, 38(12):2914-2925.
[35]	ZHANG R, ZHAO L, LOU W, et al. Automatic segmentationof acute ischemic stroke from DWI using3-D fully convolutional DenseNets [J]. IEEE Transactionson Medical Imaging, 2018, 37(9): 2149-2160.
[36]	KUMAR A, UPADHYAY N, GHOSAL P, et al.CSNet: A new DeepNet framework for ischemic strokelesion segmentation [J]. Computer Methods and Programsin Biomedicine, 2020, 193: 105524.
[37]	CL` ERIGUES A, VALVERDE S, BERNAL J, et al.Acute and sub-acute stroke lesion segmentation frommultimodal MRI [J]. Computer Methods and Programsin Biomedicine, 2020, 194: 105521.
[38]	WOO I, LEE A, JUNG S C, et al. Fully automaticsegmentation of acute ischemic lesions on diffusionweightedimaging using convolutional neural networks:Comparison with conventional algorithms [J]. KoreanJournal of Radiology, 2019, 20(8): 1275-1284.
[39]	WANG P, GAO C, ZHU L, et al. Segmentation algorithmof ischemic stroke lesion based on 3D deepresidual network and cascade U-Net [J]. Computer Applications,2019, 39(11): 3274-3279 (in Chinese).
[40]	LIU L, WU F, WANG J. Efficient multi-kernel DCNNwith pixel dropout for stroke MRI segmentation [J].Neurocomputing, 2019, 350: 117-127.
[41]	CHEN L, BENTLEY P, RUECKERT D. Fully automaticacute ischemic lesion segmentation in DWI usingconvolutional neural networks [J]. NeuroImage: Clinical,2017, 15: 633-643.
[42]	WINZECK S, MOCKING S J, BEZERRA R, et al.Ensemble of convolutional neural networks improvesautomated segmentation of acute ischemic lesions usingmultiparametric diffusion-weighted MRI [J]. AmericanJournal of Neuroradiology, 2019, 40(6): 938-945.
[43]	HE K, GKIOXARI G, DOLL′AR P, et al. Mask R-CNN[C]//2017 IEEE International Conference on ComputerVision (ICCV ). Piscataway, NJ: IEEE, 2017:2980-2988.
[44]	MANJ ′ON J V, COUP′E P, RANIGA P, et al. MRIwhite matter lesion segmentation using an ensemble ofneural networks and overcomplete patch-based voting[J]. Computerized Medical Imaging and Graphics, 2018,69: 43-51.
[45]	RAJAN R, SATHISH R, SHEET D. Significanceof residual learning and boundaryweighted loss in ischaemic stroke lesion segmentation[DB/OL]. (2019-08-13) [2020-08-01].https://arxiv.org/abs/1908.04840.
[46]	SATHISH R, RAJAN R, VUPPUTURI A, et al. Adversariallytrained convolutional neural networks forsemantic segmentation of ischaemic stroke lesion usingmultisequence magnetic resonance imaging [C]//201941st Annual International Conference of the IEEE Engineeringin Medicine and Biology Society (EMBC).Piscataway, NJ: IEEE, 2019: 1010-1013.
[47]	LUCAS C, KEMMLING A, MAMLOUK A M, etal. Multi-scale neural network for automatic segmentationof ischemic strokes on acute perfusion images[C]//2018 IEEE 15th International Symposiumon Biomedical Imaging (ISBI 2018). Piscataway, NJ:IEEE, 2018: 1118-1121.
[48]	ISLAM M, REN H. Class balanced PixelNet for neurologicalimage segmentation [C]//Proceedings of the2018 6th International Conference on Bioinformaticsand Computational Biology. New York, NY: ACM,2018: 83-87.
[49]	SIMONYAN K, ZISSERMAN A. Very deepconvolutional networks for large-scale imagerecognition [DB/OL]. (2015-04-10) [2020-08-01].https://arxiv.org/abs/1409.1556.
[50]	P′EREZ MALLA C U, VALD′ES HERN′ANDEZ M DC, RACHMADI M F, et al. Evaluation of enhancedlearning techniques for segmenting ischaemic stroke lesionsin brain magnetic resonance perfusion images usinga convolutional neural network scheme [J]. Frontiersin Neuroinformatics, 2019, 13: 33.
[51]	HU X, LUO W, HU J, et al. Brain SegNet: 3D localrefinement network for brain lesion segmentation [J].BMC Medical Imaging, 2020, 20(1): 17.
[52]	CHOI Y, KWON Y, PAIK M C, et al. Ischemicstroke lesion segmentation with convolutional neuralnetworks for small data [EB/OL]. [2020-08-01].http://www.isles-challenge.org/ISLES2017/articles/choi.pdf.
[53]	CL` ERIGUES A, VALVERDE S, BERNAL J, et al.Acute ischemic stroke lesion core segmentation in CTperfusion images using fully convolutional neural networks[J]. Computers in Biology and Medicine, 2019,115: 103487.
[54]	B¨O HME L, MADESTA F, SENTKER T, et al.Combining good old random forest and DeepLabv3+for ISLES 2018 CT-based stroke segmentation[M]//Brainlesion: Glioma, multiple sclerosis, strokeand traumatic brain injuries. Cham: Springer, 2019:335-342.
[55]	CHOUDHURY A R, VANGURI R, JAMBAWALIKARS R, et al. Segmentation of brain tumors usingDeepLabv3+ [M]//Brainlesion: Glioma, multiplesclerosis, stroke and traumatic brain injuries. Cham:Springer, 2019: 154-167.
[56]	KERVADEC H, BOUCHTIBA J, DESROSIERS C, etal. Boundary loss for highly unbalanced segmentation[J]. Medical Image Analysis, 2021, 67: 101851.
[57]	LIU P. Stroke lesion segmentation with 2D novel CNNpipeline and novel loss function [M]//Brainlesion:Glioma, multiple sclerosis, stroke and traumatic braininjuries. Cham: Springer, 2019: 253-262.
[58]	FUCHIGAMI T, AKAHORI S, OKATANI T, et al. Ahyperacute stroke segmentation method using 3D UNetintegrated with physicians’ knowledge for NCCT[J]. Proceedings of SPIE, 2020, 11314: 113140G.
[59]	GAO F, TAO M, LI X, et al. Accurate segmentation ofstroke in CT image based on deep learning [J]. Journalof Jilin University (Engineering and Technology Edition),2020, 50(2): 678-684 (in Chinese).
[60]	MAIER O, SCHR¨ODER C, FORKERT N D, et al.Classifiers for ischemic stroke lesion segmentation:A comparison study [J]. Plos One, 2015, 10(12):e0145118.
[61]	LIU Z, CAO C, DING S, et al. Towards clinical diagnosis:Automated stroke lesion segmentation on multispectralMR image using convolutional neural network[J]. IEEE Access, 2018, 6: 57006-57016.
[62]	LIU L, KURGAN L, WU F, et al. Attention convolutionalneural network for accurate segmentation andquantification of lesions in ischemic stroke disease [J].Medical Image Analysis, 2020, 65: 101791.
[63]	KARTHIK R, GUPTA U, JHA A, et al. A deep supervisedapproach for ischemic lesion segmentation frommultimodal MRI using Fully Convolutional Network[J]. Applied Soft Computing, 2019, 84: 105685.
[64]	ZHOU Y, HUANG W, DONG P, et al. D-UNet:A dimension-fusion U shape network for chronicstroke lesion segmentation [J]. IEEE/ACM Transactionson Computational Biology and Bioinformatics,2021, 18(3): 940-950.
[65]	XUE Y, FARHAT F G, BOUKRINA O, et al. A multipath2.5 dimensional convolutional neural network systemfor segmenting stroke lesions in brain MRI images[J]. NeuroImage: Clinical, 2020, 25: 102118.
[66]	HUI H, ZHANG X, LI F, et al. A partitioning-stackingprediction fusion network based on an improved attentionU-Net for stroke lesion segmentation [J]. IEEEAccess, 2020, 8: 47419-47432.
[67]	YANG H, HUANG W, QI K, et al. CLCI-Net: Crosslevelfusion and context inference networks for lesionsegmentation of chronic stroke [M]//Medical imagecomputing and computer assisted intervention-MICCAI 2019. Cham: Springer, 2019: 266-274.
[68]	QI K, YANG H, LI C, et al. X-net: brainstroke lesion segmentation based on depthwiseseparable convolution and long-range dependencies[C]//International Conference on Medical Image Computingand Computer-Assisted Intervention. Cham:Springer, 2019: 247-255.
[69]	LIU X, YANG H, QI K, et al. MSDF-Net: Multi-scaledeep fusion network for stroke lesion segmentation [J].IEEE Access, 2019, 7: 178486-178495.
[70]	RODERICK D D CWR,WANG KM. Using cascadednetworks for post-stroke lesion [EB/OL]. [2020-08-01]. http://cs230.stanford.edu/projects spring 2018/reports/8288136.pdf.
[71]	WANG Y, WANG H, CHEN S, et al. A 3Dcross-hemisphere neighborhood difference Convnet forchronic stroke lesion segmentation [C]//2019 IEEE InternationalConference on Image Processing (ICIP).Piscataway, NJ: IEEE, 2019: 1545-1549.
[72]	WANG Y, KATSAGGELOS A K, WANG X, et al.A deep symmetry convnet for stroke lesion segmentation[C]//2016 IEEE International Conference on ImageProcessing (ICIP). Piscataway, NJ: IEEE, 2016:111-115.
[73]	HAVAEI M, DAVY A, WARDE-FARLEY D, et al.Brain tumor segmentation with Deep Neural Networks[J]. Medical Image Analysis, 2017, 35: 18-31.
[74]	PEREIRA S, PINTO A, ALVES V, et al. Brain tumorsegmentation using convolutional neural networks inMRI images [J]. IEEE Transactions on Medical Imaging,2016, 35(5): 1240-1251.
[75]	LIU Z, CAO C, DING S, et al. Towards clinical diagnosis:Automated stroke lesion segmentation on multispectralMR image using convolutional neural network[J]. IEEE Access, 2018, 6: 57006-57016.
[76]	GONZ′ALEZ R G, HIRSCH J A, KOROSHETZ WJ, et al. Acute ischemic stroke [M]. Berlin/Heidelberg:Springer, 2006.
[77]	MOSTAPHA M, STYNER M. Role of deep learningin infant brain MRI analysis [J]. Magnetic ResonanceImaging, 2019, 64: 171-189.
[78]	YI X, WALIA E, BABYN P. Generative adversarialnetwork in medical imaging: A review [J]. Medical ImageAnalysis, 2019, 58: 101552.
[79]	ISAAC J S, KULKARNI R. Super resolution techniquesfor medical image processing [C]//2015 InternationalConference on Technologies for SustainableDevelopment (ICTSD). Piscataway, NJ: IEEE, 2015:1-6.
[80]	MAHAPATRA D, BOZORGTABAR B, GARNAVIR. Image super-resolution using progressive generativeadversarial networks for medical image analysis[J]. Computerized Medical Imaging and Graphics, 2019,71: 30-39.
[81]	BRIA A, MARROCCO C, TORTORELLA F. Addressingclass imbalance in deep learning for small lesiondetection on medical images [J]. Computers in Biologyand Medicine, 2020, 120: 103735.
[82]	ANDO S, HUANG C Y. Deep over-sampling frameworkfor classifying imbalanced data [M]//Machinelearning and knowledge discovery in databases. Cham:Springer, 2017: 770-785.
[83]	WONG S C, GATT A, STAMATESCU V, et al.Understanding data augmentation for classification:When to warp? [C]//2016 International Conferenceon Digital Image Computing: Techniques and Applications(DICTA). Piscataway, NJ: IEEE, 2016: 1-6.
[84]	FRID-ADAR M, DIAMANT I, KLANG E, et al.GAN-based synthetic medical image augmentation forincreased CNN performance in liver lesion classification[J]. Neurocomputing, 2018, 321: 321-331.
[85]	WANG S, ZHOU M, LIU Z, et al. Central focused convolutionalneural networks: Developing a data-drivenmodel for lung nodule segmentation [J]. Medical ImageAnalysis, 2017, 40: 172-183.
[86]	GUO H, LI Y, SHANG J, et al. Learning from classimbalanceddata: Review of methods and applications[J]. Expert Systems with Applications, 2017, 73: 220-239.
[87]	KAKAR M, OLSEN D R. Automatic segmentationand recognition of lungs and lesion from CT scans of thorax [J]. Computerized Medical Imaging and Graphics,2009, 33(1): 72-82.
[88]	NARAYANA P A, CORONADO I, SUJIT S J, et al.Are multi-contrast magnetic resonance images necessaryfor segmenting multiple sclerosis brains A largecohort study based on deep learning [J]. Magnetic ResonanceImaging, 2020, 65: 8-14.
[89]	KIM Y, LEE J, YU I, et al. Evaluation of diffusion lesionvolume measurements in acute ischemic stroke usingencoder-decoder convolutional network [J]. Stroke,2019, 50(6): 1444-1451.
[90]	LORENZO P R, NALEPA J, BOBEK-BILLEWICZB, et al. Segmenting brain tumors from FLAIR MRIusing fully convolutional neural networks [J]. ComputerMethods And Programs In Biomedicine, 2019,176: 135-148.
[91]	XU B, CHAI Y, GALARZA C M, et al. Orchestralfully convolutional networks for small lesion segmentationin brain MRI [C]//2018 IEEE 15th InternationalSymposium on Biomedical Imaging (ISBI 2018). Piscataway,NJ: IEEE, 2018: 889-892.
[92]	KRIVOV E, KOSTJUCHENKO V, DALECHINA A,et al. Tumor delineation for brain radiosurgery bya ConvNet and non-uniform patch generation [M]//Patch-based techniques in medical imaging. Cham:Springer, 2018: 122-129.
[93]	GUERRERO R, QIN C, OKTAY O, et al. White matterhyperintensity and stroke lesion segmentation anddifferentiation using convolutional neural networks [J].NeuroImage: Clinical, 2018, 17: 918-934.
[94]	LI H, PARIKH N A, WANG J, et al. Objective andautomated detection of diffuse white matter abnormalityin preterm infants using deep convolutional neuralnetworks [J]. Frontiers in Neuroscience, 2019, 13: 610.
[95]	FANG M, DONG D, SUN R, et al. Using multi-tasklearning to improve diagnostic performance of convolutionalneural networks [J]. Proceedings of SPIE, 2019,10950: 109501V.
[96]	SAMALA R K, CHAN H P, HADJIISKI L M, et al.Multi-task transfer learning deep convolutional neuralnetwork: Application to computer-aided diagnosisof breast cancer on mammograms [J]. Physics inMedicine & Biology, 2017, 62(23): 8894-8908.