J Shanghai Jiaotong Univ Sci

Journal of Shanghai Jiaotong University(Science) >

2022 , Vol. 27 >Issue 2: 176 - 189

DOI: https://doi.org/10.1007/s12204-021-2383-4

Medicine-Engineering Interdisciplinary Research

SeRN: A Two-Stage Framework of Registration for Semi-Supervised Learning for Medical Images

Expand
  • (1. School of Naval Architecture, Ocean and Civil Engineering, Shanghai Jiao Tong University, Shanghai 200240, China;2. School of Data Science, Fudan University, Shanghai 200433, China; 3. College of Physics and Information Engineering, Fuzhou University, Fuzhou 350117, China)

Received date: 2021-06-25

  Online published: 2022-05-02

Fold

Abstract

Significant breakthroughs in medical image registration have been achieved using deep neural networks (DNNs). However, DNN-based end-to-end registration methods often require large quantities of data or adequate annotations for training. To leverage the intensity information of abundant unlabeled images, unsupervised registration methods commonly employ intensity-based similarity measures to optimize the network parameters.However, finding a sufficiently robust measure can be challenging for specific registration applications. Weakly supervised registration methods use anatomical labels to estimate the deformation between images. High-level structural information in label images is more reliable and practical for estimating the voxel correspondence of anatomic regions of interest between images, whereas label images are extremely difficult to collect. In this paper, we propose a two-stage semi-supervised learning framework for medical image registration, which consists of unsupervised and weakly supervised registration networks. The proposed semi-supervised learning framework is trained with intensity information from available images, label information from a relatively small number of labeled images and pseudo-label information from unlabeled images. Experimental results on two datasets (cardiac and abdominal images) demonstrate the efficacy and efficiency of this method in intra- and inter-modality medical image registrations, as well as its superior performance when a vast amount of unlabeled data and a small set of annotations are available. Our code is publicly available at https://github.com/jdq818/SeRN.

Key words： medical image registration; semi-supervised learning; intra-modality; inter-modality

Cite this article

JIA Dengqiang* (贾灯强), LUO Xinzhe (罗鑫喆), DING Wangbin (丁王斌),HUANG Liqin (黄立勤), ZHUANG Xiahai (庄吓海) . SeRN: A Two-Stage Framework of Registration for Semi-Supervised Learning for Medical Images[J]. Journal of Shanghai Jiaotong University(Science), 2022 , 27(2) : 176 -189 . DOI: 10.1007/s12204-021-2383-4

References

[1] SOTIRAS A, DAVATZIKOS C, PARAGIOS N. Deformable medical image registration: A survey [J].IEEE Transactions on Medical Imaging, 2013, 32(7):1153-1190. [2] VIERGEVER M A, MAINTZ J B A, KLEIN S, et al.A survey of medical image registration-under review [J]. Medical Image Analysis, 2016, 33: 140-144. [3] CAO X, YANG J, ZHANG J, et al. Deformable image registration based on similarity-steered CNN regression[M]//Medical image computing and computer assisted intervention - MICCAI 2017. Cham: Springer,2017: 300-308. [4] KREBS J, MANSI T, DELINGETTE H, et al. Robust non-rigid registration through agent-based action learning [M]//Medical image computing and computer assisted intervention - MICCAI 2017. Cham: Springer,2017: 344-352. [5] ROH′EM M, DATAR M, HEIMANN T, et al. SVFNet: SVF-Net: Learning deformable image registration using shape matching [M]//Medical image computing and computer assisted intervention - MICCAI 2017. Cham: Springer, 2017: 266-274. [6] YANG X, KWITT R, STYNER M, et al. Quicksilver: Fast predictive image registration - A deep learning approach [J]. NeuroImage, 2017, 158: 378-396. [7] DE VOS B D, BERENDSEN F F, VIERGEVER M A, et al. A deep learning framework for unsupervised affine and deformable image registration [J]. Medical Image Analysis, 2019, 52: 128-143. [8] BALAKRISHNAN G, ZHAO A, SABUNCU M R, et al. VoxelMorph: A learning framework for deformable medical image registration [J]. IEEE Transactions on Medical Imaging, 2019, 38(8): 1788-1800. [9] DALCA A V, BALAKRISHNAN G, GUTTAG J, et al. Unsupervised learning of probabilistic diffeomorphic registration for images and surfaces [J]. Medical Image Analysis, 2019, 57: 226-236. [10] HU Y, MODAT M, GIBSON E, et al. Weaklysupervised convolutional neural networks for multimodal image registration [J]. Medical Image Analysis,2018, 49: 1-13. [11] LUO X, ZHUANG X. MvMM-RegNet: A new image registration framework based on multivariate mixture model and neural network estimation [M]//Medical image computing and computer assisted intervention - MICCAI 2020. Cham: Springer, 2020: 149-159. [12] CHEPLYGINA V, DE BRUIJNE M, PLUIM J P W.Not-so-supervised: A survey of semi-supervised, multiinstance,and transfer learning in medical image analysis[J]. Medical Image Analysis, 2019, 54: 280-296. [13] ZHU Z, CAO Y, QIN C, et al. Joint affine and deformable three-dimensional networks for brain MRI registration [J]. Medical Physics, 2021, 48(3): 1182-1196. [14] ESTIENNE T, VAKALOPOULOU M,CHRISTODOULIDIS S, et al. U-ReSNet: Ultimate coupling of registration and segmentation with deep nets [M]//Medical image computing and computer assisted intervention - MICCAI 2019. Cham:Springer, 2019: 310-319. [15] XU Z, NIETHAMMER M. DeepAtlas: Joint semisupervised learning of image registration and segmentation[M]//Medical image computing and computer assisted intervention - MICCAI 2019. Cham: Springer,2019: 420-429. [16] PLUIM J P W, MAINTZ J B A, VIERGEVER M A. Mutual-information-based registration of medical images: A survey [J]. IEEE Transactions on Medical Imaging, 2003, 22(8): 986-1004. [17] ARSIGNY V, COMMOWICK O, PENNEC X, et al.A Log-Euclidean framework for statistics on diffeomorphisms[M]//Medical image computing and computer assisted intervention - MICCAI 2006. Berlin: Springer,2006: 924-931. [18] ASHBURNER J. A fast diffeomorphic image registration algorithm [J]. NeuroImage, 2007, 38(1): 95-113. [19] AVANTS B B, EPSTEIN C L, GROSSMAN M, et al. Symmetric diffeomorphic image registration with cross-correlation: Evaluating automated labeling of elderly and neurodegenerative brain [J]. Medical Image Analysis, 2008, 12(1): 26-41. [20] MILLETARI F, NAVAB N, AHMADI S A. V-Net:Fully convolutional neural networks for volumetric medical image segmentation [C]//2016 Fourth International Conference on 3D Vision (3DV). Stanford: IEEE, 2016: 565-571. [21] SUDRE C H, LI W, VERCAUTEREN T, et al. Generalised Dice overlap as a deep learning loss function for highly unbalanced segmentations [M]// Deep learning in medical image analysis and multimodal learning for clinical decision support. Cham: Springer, 2017: 240-248. [22] PEREYRA G, TUCKER G, CHOROWSKI J, et al. Regularizing neural networks by penalizing confident output distributions [EB/OL]. (2017-01-23).https://arxiv.org/abs/1701.06548. [23] ZHUANG X, SHEN J. Multi-scale patch and multimodality atlases for whole heart segmentation of MRI[J]. Medical Image Analysis, 2016, 31: 77-87. [24] ZHUANG X, LI L, PAYER C, et al. Evaluation of algorithms for Multi-Modality Whole Heart Segmentation:An open-access grand challenge [J]. Medical Image Analysis, 2019, 58: 101537. [25] KAVUR A E, GEZER N S, BARIS? M, et al. CHAOS Challenge - combined (CT-MR) healthy abdominal organ segmentation [J]. Medical Image Analysis, 2021,69: 101950. [26] SANDK¨UHLER R, JUD C, ANDERMATT S, et al. AirLab: Autograd image registration laboratory[EB/OL]. (2020-03-02). https://arxiv.org/abs/1806.09907. [27] DING W, LI L, ZHUANG X, et al. Cross-modality multi-atlas segmentation using deep neural networks[M]//Medical image computing and computer assisted intervention - MICCAI 2020. Cham: Springer, 2020:233-242. [28] KINGMA D, BA J. Adam: A method for stochastic optimization [C]//3rd International Conference on Learning Representations. San Diego: Ithaca, 2014: 1-15 [29] WILCOXON F. Individual comparisons by ranking methods [J]. Biometrics Bulletin, 1945, 1(6): 80-83.
Options
Outlines

/

〈 〉