上海交通大学学报

上海交通大学学报 >

2022 , Vol. 56 >Issue 8: 1078 - 1088

DOI: https://doi.org/10.16183/j.cnki.jsjtu.2021.113

生物医学工程

基于LSTM与注意力结构的肺结节多特征抽取方法

展开
  • 1.中国科学院上海技术物理研究所 医学影像信息学实验室,上海 200080
    2.中国科学院大学,北京 100049
    3.中国人民解放军总医院, 北京 100089
倪扬帆(1993- ),男,江苏省无锡市人,博士生,主要研究方向为深度学习在医学影像方面的应用.

收稿日期: 2021-04-14

  网络出版日期: 2022-08-26

基金资助

科技部重点研发计划(2019YFC0118803)

收起

Multi-Feature Extraction of Pulmonary Nodules Based on LSTM and Attention Structure

Expand
  • 1. Laboratory for Medical Imaging Informatics, Shanghai Institution of Technical Physics, Chinese Academy of Sciences, Shanghai 200080, China
    2. University of Chinese Academy of Sciences, Beijing 100049, China
    3. Chinese PLA General Hospital, Beijing 100089, China

Received date: 2021-04-14

  Online published: 2022-08-26

Fold

摘要

对肺结节的形状特征、边缘特征和内部特征进行准确分类,能够辅助影像科医生的日常诊断工作,提高影像报告的书写效率.针对这一问题,提出一种基于长短时记忆(LSTM)结构与注意力结构的多任务分类模型.该模型通过注意力机制融合各个任务间的共享特征,提高当前任务的特征抽取效果.LSTM结构分类器能够有效地筛选任务间的共享特征,提高模型的信息传递效率.实验表明,相较于传统多任务结构,所提模型在公开数据集LIDC-IDRI上能够取得更好的多特征分类效果,辅助医生快捷地获取肺结节特征信息.

关键词： 肺结节; 注意力结构; 长短时记忆网络; 多任务分类

本文引用格式

倪扬帆, 杨媛媛, 谢哲, 郑德重, 王卫东 . 基于LSTM与注意力结构的肺结节多特征抽取方法[J]. 上海交通大学学报, 2022 , 56(8) : 1078 -1088 . DOI: 10.16183/j.cnki.jsjtu.2021.113

Abstract

The accurate classification of shape, edge, and internal features of pulmonary nodules can not only assist the radiologists in their daily diagnosis, but also improve the writing efficiency of imaging reports. This paper proposes a multi-task classification model based on long-short term memory (LSTM) and attention structure, which merges the shared features among different classification tasks through attention mechanism to improve the feature extraction performance of the current task. The classifier based on temporal sequence LSTM structure can effectively screen the shared features and improve the efficiency of information transmission in the multi-task model. Experimental results show that compared with the traditional multi-task structure, the proposed model can achieve better classification results on the public dataset LIDC-IDRI, and assist doctors to quickly obtain nodule characteristics.

Key words： pulmonary nodule; attention structure; long-short term memory(LSTM)network; multi-task classification

参考文献

[1] VANDENHENDE S, GEORGOULIS S, VAN GANSBEKE W, et al. Multi-task learning for dense prediction tasks: A survey[DB/OL]. (2021-01-26) [2021-03-01]. https://ieeexplore.ieee.org/abstract/document/9336293.
[2] HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]// 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 2980-2988.
[3] LIU S, QI L, QIN H F, et al. Path aggregation network for instance segmentation[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 8759-8768.
[4] REN S Q, HE K M, GIRSHICK R, et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(6): 1137-1149.
[5] LIU W, ANGUELOV D, ERHAN D, et al. SSD: Single shot MultiBox detector[M]// Computer vision-ECCV 2016. Cham, Switzerland: Springer International Publishing, 2016: 21-37.
[6] DUNNICK N R, LANGLOTZ C P. The radiology report of the future: A summary of the 2007 intersociety conference[J]. Journal of the American College of Radiology, 2008, 5(5): 626-629.
[7] KOKKINOS I. UberNet:Training a universal convolutional neural network for low-, mid-, and high-level vision using diverse datasets and limited memory[C]// 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 5454-5463.
[8] MISRA I, SHRIVASTAVA A, GUPTA A, et al. Cross-stitch networks for multi-task learning[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA: IEEE, 2016: 3994-4003.
[9] SHEN S W, HAN S X, ABERLE D R, et al. An interpretable deep hierarchical semantic convolutional neural network for lung nodule malignancy classification[J]. Expert Systems With Applications, 2019, 128: 84-95.
[10] AMYAR A, MODZELEWSKI R, LI H, et al. Multi-task deep learning based CT imaging analysis for COVID-19 pneumonia: Classification and segmentation[J]. Computers in Biology and Medicine, 2020, 126: 104037.
[11] CHEN Z V. BADRINARAYANAN V, LEE C Y, et al. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks[C]// International Conference on Machine Learning. Stockholm, Sweden: PMLR, 2018: 794-803.
[12] LIU S K, JOHNS E, DAVISON A J. End-to-end multi-task learning with attention[C]// 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019: 1871-1880.
[13] GUO M, HAQUE A, HUANG D A, et al. Dynamic task prioritization for multitask learning[M]// Computer vision-ECCV 2018. Cham, Switzerland: Springer International Publishing, 2018: 282-299.
[14] JALALI A, SANGHAVI S, RUAN C, et al. A dirty model for multi-task learning[J]. Advances in Neural Information Processing Systems, 2010, 23: 964-972.
[15] ANDO R K, ZHANG T, BARTLETT P. A framework for learning predictive structures from multiple tasks and unlabeled data[J]. Journal of Machine Learning Research, 2005, 6: 1817-1853.
[16] CHEN J H, LIU J, YE J P. Learning incoherent sparse and low-rank patterns from multiple tasks[J]. ACM Transactions on Knowledge Discovery from Data, 2012, 5(4): 1-31.
[17] JACOBS R A, JORDAN M I, NOWLAN S J, et al. Adaptive mixtures of local experts[J]. Neural Computation, 1991, 3(1): 79-87.
[18] SHAZEER N, MIRHOSEINI A, MAZIARZ K, et al. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer[DB/OL]. (2017-01-23) [2021-03-01]. https://arxiv.org/abs/1701.06538.
[19] MA J Q, ZHAO Z, YI X Y, et al. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts[C]// Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York, NY, USA: ACM, 2018: 1930-1939.
[20] CHEN S H, QIN J, JI X, et al. Automatic scoring of multiple semantic attributes with multi-task feature leverage: A study on pulmonary nodules in CT images[J]. IEEE Transactions on Medical Imaging, 2017, 36(3): 802-814.
[21] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]// 2017 IEEE International Conference on Computer Vision. Venice, Italy: IEEE, 2017: 2999-3007.
[22] ARMATO S G I, MCLENNAN G, BIDAUT L, et al. The lung image database consortium (LIDC) and image database resource initiative (IDRI): A completed reference database of lung nodules on CT scans[J]. Medical Physics, 2011, 38(2): 915-931.
[23] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, Nevada, USA: IEEE, 2016: 770-778.
[24] MCNITT-GRAY M F, ARMATO S G III, MEYER C R, et al. The lung image database consortium (LIDC) data collection process for nodule detection and annotation[J]. Academic Radiology, 2007, 14(12): 1464-1474.
[25] ISENSEE F, JAEGER P F, KOHL S A A, et al. nnU-Net: A self-configuring method for deep learning-based biomedical image segmentation[J]. Nature Methods, 2021, 18(2): 203-211.
[26] CIPOLLA R, GAL Y, KENDALL A. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]// 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Shenzhen, China: IEEE, 2018: 7482-7491.
[27] KENDALL A, GAL Y, CIPOLLA R, et al. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics[C]// Proceedings of the IEEE conference on computer vision and pattern recognition. Salt Lake City, UT, USA: IEEE, 2018: 7482-7491.
Options
文章导航

/

〈 〉