J Shanghai Jiaotong Univ Sci ›› 2024, Vol. 29 ›› Issue (6): 945-957.doi: 10.1007/s12204-024-2743-y

• Medicine-Engineering Interdisciplinary •     Next Articles

TshFNA-Examiner: A Nuclei Segmentation and Cancer Assessment Framework for Thyroid Cytology Image

TshFNA-Examiner:甲状腺细胞学图像的核分割和癌症评估框架

KE Jing1*(柯晶), ZHU Junchao2 (朱俊超), YANG Xin1(杨鑫), ZHANG Haolin3 (张浩林), SUN Yuxiang1(孙宇翔), WANG Jiayi1(王嘉怡), LU Yizhou4(鲁亦舟), SHEN Yiqing5(沈逸卿), LIU Sheng6*(刘晟), JIANG Fusong7(蒋伏松), HUANG Qin8*(黄琴)   

  1. (1. School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai 200240, China; 2.School of Life Sciences and Biotechnology, Shanghai Jiao Tong University, Shanghai 200240, China; 3. School of Engineering, Shanghai Ocean University, Shanghai 201306, China; 4. Shanghai Advanced Research Institute, Chinese Academy of Sciences, Shanghai 201210, China; 5. Department of Computer Science, Johns Hopkins University, Baltimore 21218, USA; 6. Department of Thyroid Breast and Vascular Surgery, Shanghai Fourth People’s Hospital, Shanghai 200434,China; 7. Department of Endocrinology and Metabolism, Shanghai Sixth People’s Hospital, Shanghai 200233, China; 8.Department of Pathology, Shanghai Sixth People’s Hospital, Shanghai 200233, China)
  2. (1. 上海交通大学 电子信息与电子工程学院,上海200240;2. 上海交通大学 生命科学技术学院,上海200240;3. 上海海洋大学 工程学院,上海201306;4. 中国科学院上海高等研究院,上海 201210;5. 约翰霍普金斯大学 计算机科学系,美国巴尔的摩 21218;6. 上海市第四人民医院,上海 200434;7. 上海市第六人民医院内分泌代谢科,上海200233;8. 上海市第六人民医院病理科,上海200233)
  • Accepted:2023-10-23 Online:2024-11-28 Published:2024-11-28

Abstract: Examining thyroid fine-needle aspiration (FNA) can grade cancer risks, derive prognostic information, and guide follow-up care or surgery. The digitization of biopsy and deep learning techniques has recently enabled computational pathology. However, there is still lack of systematic diagnostic system for the complicated gigapixel cytopathology images, which can match physician-level basic perception. In this study, we design a deep learning framework, thyroid segmentation and hierarchy fine-needle aspiration (TshFNA)-Examiner to quantitatively profile the cancer risk of a thyroid FNA image. In the TshFNA-Examiner, cellular-intensive areas strongly correlated with diagnostic medical information are detected by a nuclei segmentation neural network; cell-level image patches are catalogued following The Bethesda System for Reporting Thyroid Cytopathology (TBSRTC) system, by a classification neural network which is further enhanced by leveraging unlabeled data. A cohort of 333 thyroid FNA cases collected from 2019 to 2022 from I to VI is studied, with pixel-wise and image-wise image patches annotated. Empirically, TshFNA-Examiner is evaluated with comprehensive metrics and multiple tasks to demonstrate its superiority to state-of-the-art deep learning approaches. The average performance of cellular area segmentation achieves a Dice of 0.931 and Jaccard index of 0.871. The cancer risk classifier achieves a macro-F1-score of 0.959, macro-AUC of 0.998, and accuracy of 0.959 following TBSRTC. The corresponding metrics can be enhanced to a macro-F1-score of 0.970, macro-AUC of 0.999, and accuracy of 0.970 by leveraging informative unlabeled data. In clinical practice, TshFNA-Examiner can help cytologists to visualize the output of deep learning networks in a convenient way to facilitate making the final decision.

Key words: thyroid fine-needle aspiration (FNA) cytology, The Bethesda System for Reporting Thyroid Cytopathology (TBSRTC), diagnostic system, nuclei segmentation, cancer risk classification

摘要: 通过甲状腺细针穿刺(FNA)可以评估癌症风险,获得预后信息,并指导后续护理或手术。生物检验数字化和深度学习技术推动了计算病理学的发展。然而,目前仍然缺乏,可以与医生基本水平相匹配的,针对复杂细胞病理学图像的系统性诊断系统。研究中,我们设计了一个深度学习框架,用于定量评估甲状腺细针穿刺图像的癌症风险,该框架名为TshFNA-Examiner。在TshFNA-Examiner中,通过细胞核分割神经网络检测与诊断医学信息强相关的细胞密集区域;通过分类神经网络按照报告甲状腺细胞病理学(TBSRTC)系统对细胞级图像子块进行分类,同时使用半监督网络基于未标记数据对分类网络进行增强。研究了从2019年到2022年收集的333例甲状腺细针穿刺样本,分为I到VI级,并完成了像素级和图像级的图像子块标注。通过综合指标和多个任务评估了TshFNA-Examiner,以证明其优于最先进的深度学习方法。细胞区域分割的平均性能达到了0.931 Dice系数和0.871 Jaccard指数。癌症风险分类器按照TBSRTC标准达到了0.959 Macro-F1-score、0.998 Macro-AUC和0.959准确率。通过利用大量未标记数据进行半监督学习,相应的指标可以提高至0.970 Macro-F1-score、0.999 Macro-AUC和0.970 准确率。在临床实践中,TshFNA-Examiner可以帮助细胞学家以便捷方式可视化深度学习网络的输出,以促进最终决策的制定。

关键词: 甲状腺细针穿刺细胞病理学,TBSRTC,诊断系统,细胞核分割,癌症风险分类

CLC Number: