Journal of Shanghai Jiao Tong University ›› 2023, Vol. 57 ›› Issue (5): 570-581.doi: 10.16183/j.cnki.jsjtu.2022.088
Special Issue: 《上海交通大学学报》2023年“生物医学工程”专题
• Biomedical Engineering • Previous Articles Next Articles
LI Qing1,2, HUANGFU Yubin1, LI Jiangyun1,2(), YANG Zhifang1, CHEN Peng3, WANG Zihan1
Received:
2022-03-31
Revised:
2022-05-19
Accepted:
2022-05-24
Online:
2023-05-28
Published:
2023-06-02
Contact:
LI Jiangyun
E-mail:leejy@ustb.edu.cn.
CLC Number:
LI Qing, HUANGFU Yubin, LI Jiangyun, YANG Zhifang, CHEN Peng, WANG Zihan. UConvTrans:A Dual-Flow Cardiac Image Segmentation Network by Global and Local Information Integration[J]. Journal of Shanghai Jiao Tong University, 2023, 57(5): 570-581.
Add to citation manager EndNote|Ris|BibTeX
URL: https://xuebao.sjtu.edu.cn/EN/10.16183/j.cnki.jsjtu.2022.088
Tab.1
Ablation experiment results of FCTB
方法 | Fuse Trans to Conv | Fuse Conv to Trans | DSC /% | |||
---|---|---|---|---|---|---|
平均 | RV | Myo | LV | |||
Only Trans | — | — | 83.75 | 80.75 | 82.48 | 88.02 |
Only Conv | — | — | 87.60 | 86.64 | 86.17 | 89.98 |
Trans+Conv | × | × | 88.61 | 86.70 | 87.72 | 91.40 |
Trans+Conv | × | √ | 88.76 | 87.52 | 87.06 | 91.69 |
Trans+Conv | √ | × | 89.25 | 87.08 | 88.31 | 92.38 |
Trans+Conv | √ | √ | 89.38 | 87.12 | 88.44 | 92.57 |
Tab.3
Comparison of proposed method and advanced methods on validation set
方法 | DSC/% | 参数量×10-6 | 计算量×10-9 | |||
---|---|---|---|---|---|---|
平均 | RV | Myo | LV | |||
U-Net[ | 88.25 | 86.91 | 87.17 | 90.65 | 34.53 | 65.55 |
Attention U-Net[ | 88.52 | 86.78 | 86.93 | 91.84 | 37.88 | 66.62 |
SwinUNet[ | 89.26 | 86.62 | 88.72 | 92.44 | 27.17 | 6.14 |
TransUNet[ | 89.47 | 87.04 | 88.51 | 92.85 | 105.32 | 38.57 |
UConvTrans (C=32,D=32) | 89.38 | 87.12 | 88.44 | 92.57 | 3.65 | 5.03 |
UConvTrans (C=32,D=64) | 89.60 | 88.08 | 88.30 | 92.41 | 10.59 | 12.74 |
[1] |
CHEN C, QIN C, QIU H, et al. Deep learning for cardiac image segmentation: A review[J]. Frontiers in Cardiovascular Medicine, 2020, 7: 25.
doi: 10.3389/fcvm.2020.00025 pmid: 32195270 |
[2] | 刘畅, 林楠, 曹仰杰, 等. Seg-CapNet: 心脏MRI图像分割神经网络模型[J]. 中国图象图形学报, 2021, 26(2): 452-463. |
LIU Chang, LIN Nan, CAO Yangjie, et al. Seg-CapNet: Neural network model for the cardiac MRI segmentation[J]. Journal of Image and Graphics, 2021, 26(2): 452-463. | |
[3] | 李江昀, 赵义凯, 薛卓尔, 等. 深度神经网络模型压缩综述[J]. 工程科学学报, 2019, 41(10): 1229-1239. |
LI Jiangyun, ZHAO Yikai, XUE Zhuoer, et al. A survey of model compression for deep neural networks[J]. Chinese Journal of Engineering, 2019, 41(10): 1229-1239. | |
[4] | 田娟秀, 刘国才, 谷珊珊, 等. 医学图像分析深度学习方法研究与挑战[J]. 自动化学报, 2018, 44(3): 401-424. |
TIAN Juanxiu, LIU Guocai, GU Shanshan, et al. Deep learning in medical image analysis and its challenges[J]. Acta Automatica Sinica, 2018, 44(3): 401-424. | |
[5] | 章云港, 杨剑锋, 易本顺. 低剂量CT图像去噪的改进型残差编解码网络[J]. 上海交通大学学报, 2019, 53(8): 983-989. |
ZHANG Yungang, YANG Jianfeng, YI Benshun. Improved residual encoder-decoder network for low-dose CT image denoising[J]. Journal of Shanghai Jiao Tong University, 2019, 53(8): 983-989. | |
[6] | 郑德重, 杨媛媛, 黄浩哲, 等. 基于距离置信度分数的多模态融合分类网络[J]. 上海交通大学学报, 2022, 56(1): 89-100. |
ZHENG Dezhong, YANG Yuanyuan, HUANG Hao-zhe, et al. Multimodal fusion classification network based on distance confidence score[J]. Journal of Shanghai Jiao Tong University, 2022, 56(1): 89-100. | |
[7] | RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation[C] //Medical Image Computing and Computer-Assisted Intervention-MICCAI 2015. Munich, Germany: Springer, 2015: 234-241. |
[8] |
LI J C, YU Z L, GU Z H, et al. Dilated-inception net: Multi-scale feature aggregation for cardiac right ventricle segmentation[J]. IEEE Transactions on Biomedical Engineering, 2019, 66(12): 3499-3508.
doi: 10.1109/TBME.10 URL |
[9] | CHENG F, CHEN C, WANG Y, et al. Learning directional feature maps for cardiac MRI segmentation[C] //International Conference on Medical Image Computing and Computer-Assisted Intervention. Lima, Peru: Springer, 2020: 108-117. |
[10] | 罗恺锴, 王婷, 叶芳芳. 引入注意力机制和多视角融合的脑肿瘤MR图像U-Net分割模型[J]. 中国图象图形学报, 2021, 26(9): 2208-2218. |
LUO Kaikai, WANG Ting, YE Fangfang. U-Net segmentation model of brain tumor MR image based on attention mechanism and multi-view fusion[J]. Journal of Image and Graphics, 2021, 26(9): 2208-2218. | |
[11] |
王瑞豪, 刘哲, 宋余庆. 结合切片上下文信息的多阶段胰腺定位与分割[J]. 电子学报, 2021, 49(4): 706-715.
doi: 10.12263/DZXB.20200101 |
WANG Ruihao, LIU Zhe, SONG Yuqing. Multi-stage pancreas localization and segmentation combined with slices context information[J]. Acta Electronica Sinica, 2021, 49(4): 706-715.
doi: 10.12263/DZXB.20200101 |
|
[12] | YU H, ZHA S, HUANGFU Y B, et al. Dual attention U-Net for multi-sequence cardiac MR images segmentation[C]//Myocardial Pathology Segmentation Combining Multi-Sequence CMR Challenge. Lima, Peru: Springer, 2020: 118-127. |
[13] | WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 7794-7803. |
[14] | VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems. Long Beach, CA, USA: MIT, 2017: 5998. |
[15] | DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[C]//International Conference on Learning Representations. Vienna: Springer, 2021: 1-21. |
[16] | ZHENG S X, LU J C, ZHAO H S, et al. Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville, TN, USA. IEEE, 2021: 6881-6890. |
[17] | CHEN J, LU Y, YU Q, et al. TransUNet: Transformers make strong encoders for medical image segmentation[EB/OL]. (2021-02-08) [2021-12-20]. https://arxiv.org/abs/2102.04306. |
[18] | 李耀仟, 李才子, 刘瑞强, 等. 面向手术器械语义分割的半监督时空Transformer 网络[J]. 软件学报, 2021, 33(4): 1501-1515. |
LI Yaoqian, LI Caizi, LIU Ruiqiang, et al. Semi-supervised spatiotemporal Transformer networks for semantic segmentation of surgical instrument[J]. Journal of Software, 2021, 33(4): 1501-1515. | |
[19] | CAO H, WANG Y, CHEN J, et al. Swin-Unet: Unet-like pure Transformer for medical image segmentation[EB/OL]. (2021-05-12) [2021-12-20]. https://arxiv.org/abs/2105.05537. |
[20] | LIU Z, LIN Y, CAO Y, et al. Swin Transformer: Hierarchical vision Transformer using shifted windows[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision. Virtual, Online: IEEE, 2021: 10012-10022. |
[21] |
BERNARD O, LALANDE A, ZOTTI C, et al. Deep learning techniques for automatic MRI cardiac multi-structures segmentation and diagnosis: Is the problem solved?[J]. IEEE Transactions on Medical Imaging, 2018, 37(11): 2514-2525.
doi: 10.1109/TMI.2018.2837502 pmid: 29994302 |
[22] | HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 770-778. |
[23] | BAUMGARTNER C F, KOCH L M, POLLEFEYS M, et al. An exploration of 2D and 3D deep learning techniques for cardiac MR image segmentation[C]//International Workshop on Statistical Atlases and Computational Models of the Heart. Quebec City, QC, Canada: Springer, 2017: 111-119. |
[24] |
KHENED M, KOLLERATHU V A, KRISHNAMURTHI G. Fully convolutional multi-scale residual DenseNets for cardiac segmentation and automated cardiac diagnosis using ensemble of classifiers[J]. Medical Image Analysis, 2019, 51: 21-45.
doi: S1361-8415(18)30848-X pmid: 30390512 |
[25] | OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-Net: Learning where to look for the pancreas[EB/OL]. (2018-05-20) [2021-12-20]. https://arxiv.org/abs/1804.03999. |
[26] | ISENSEE F, JAEGER P F, FULL P M, et al. Automatic cardiac disease assessment on cine-MRI via time-series segmentation and domain specific features[C]//International Workshop on Statistical Atlases and Computational Models of the Heart.Quebec City, QC, Canada: Springer, 2017: 120-129. |
[27] |
SIMANTIRIS G, TZIRITAS G. Cardiac MRI segmentation with a dilated CNN incorporating domain-specific constraints[J]. IEEE Journal of Selected Topics in Signal Processing, 2020, 14(6): 1235-1243.
doi: 10.1109/JSTSP.4200690 URL |
[28] |
GIRUM K B, CRÉHANGE G, LALANDE A. Learning with context feedback loop for robust medical image segmentation[J]. IEEE Transactions on Medical Imaging, 2021, 40(6): 1542-1554.
doi: 10.1109/TMI.2021.3060497 pmid: 33606627 |
[29] | ZOTTI C, LUO Z, HUMBERT O, et al. GridNet with automatic shape prior registration for automatic MRI cardiac segmentation[C]//International Workshop on Statistical Atlases and Computational Models of the Heart. Quebec City, QC, Canada: Springer, 2017: 73-81. |
[1] | ZHAN Ke, ZHU Renchuan. A CNN-LSTM Ship Motion Extreme Value Prediction Model [J]. Journal of Shanghai Jiao Tong University, 2023, 57(8): 963-971. |
[2] | FU Jiawei∗ (傅家威), ZHAO Xu (赵 旭). Action-aware Encoder-Decoder Network for Pedestrian Trajectory Prediction [J]. J Shanghai Jiaotong Univ Sci, 2023, 28(1): 20-27. |
[3] | ZENG Guozhi, WEI Ziqing, YUE Bao, DING Yunxiao, ZHENG Chunyuan, ZHAI Xiaoqiang. Energy Consumption Prediction of Office Buildings Based on CNN-RNN Combined Model [J]. Journal of Shanghai Jiao Tong University, 2022, 56(9): 1256-1261. |
[4] | JIANG Zhiguo (蒋志国), CHANG Qing∗ (常 青). USSL Net: Focusing on Structural Similarity with Light U-Structure for Stroke Lesion Segmentation [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(4): 485-497. |
[5] | ZHAO Yong, SU Dan. Rogue Wave Prediction Based on Four Combined Long Short-Term Memory Neural Network Models [J]. Journal of Shanghai Jiao Tong University, 2022, 56(4): 516-522. |
[6] | TUNG Hao (董昊), ZHENG Chao (郑超), MAO Xinsheng(毛新生), QIAN Dahong (钱大宏). Multi-Lead ECG Classification via an Information-Based Attention Convolutional Neural Network [J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 55-69. |
[7] | ZHAN Zhu (占竹), ZHANG Wenjun (张文俊), CHEN Xia (陈霞), WANG Jun (汪军) . Objective Evaluation of Fabric Flatness Grade Based on Convolutional Neural Network [J]. J Shanghai Jiaotong Univ Sci, 2021, 26(4): 503-510. |
[8] | XU Jiangchang (许江长), HE Shamin (何莎敏), YU Dedong (于德栋), WU Yiqun (吴轶群), CHEN Xiaojun, (陈晓军). Automatic Segmentation Method for Cone-Beam Computed Tomography Image of the Bone Graft Region within Maxillary Sinus Based on the Atrous Spatial Pyramid Convolution Network [J]. J Shanghai Jiaotong Univ Sci, 2021, 26(3): 298-305. |
[9] | JIANG Yudi, HU Hui, YIN Yuehong. Unsupervised Transfer Learning for Remaining Useful Life Prediction of Elevator Brake [J]. Journal of Shanghai Jiao Tong University, 2021, 55(11): 1408-1416. |
[10] | XUE Rongrong, WANG Zhiwu, YAN Guozheng, ZHUANG Haoyu. Noise Reduction Method for Intestinal Image Acquired by Intestinal Robot [J]. Journal of Shanghai Jiao Tong University, 2021, 55(10): 1303-1309. |
[11] | ZHAO Yong (赵勇), MENG Yang (孟杨), YU Pengyao (于鹏垚), WANG Tianlin (王天霖), SU Shaojuan (苏绍娟). Prediction of Fluid Force Exerted on Bluff Body by Neural Network Method [J]. Journal of Shanghai Jiao Tong University (Science), 2020, 25(2): 186-192. |
[12] | FU Ling (傅玲), MA Jingchen (马璟琛), CHEN Yizhi (琛奕志), LARSSON Rasmus, ZHAO Jun *(赵俊). Automatic Detection of Lung Nodules Using 3D Deep Convolutional Neural Networks [J]. Journal of Shanghai Jiao Tong University (Science), 2019, 24(4): 517-523. |
[13] | CHEN Yimin (陈一民), LU Rongrong (陆蓉蓉), ZOU Yibo (邹一波), ZHANG Yanhui (张燕辉). Branch-Activated Multi-Domain Convolutional Neural Network for Visual Tracking [J]. Journal of Shanghai Jiao Tong University (Science), 2018, 23(3): 360-. |
[14] | LI Yangyang,SHI Licheng,WAN Weibing,ZHAO Qunfei. A Convolutional Neural Network-Based Method for 3D Object Detection [J]. Journal of Shanghai Jiaotong University, 2018, 52(1): 7-12. |
Viewed | ||||||
Full text |
|
|||||
Abstract |
|
|||||