基于点云语义分割算法的下颌角截骨面设计

doi:10.16183/j.cnki.jsjtu.2021.103

上海交通大学学报 ›› 2022, Vol. 56 ›› Issue (11): 1509-1517.doi: 10.16183/j.cnki.jsjtu.2021.103

基于点云语义分割算法的下颌角截骨面设计

吕超凡¹, 言颖杰², 林力²^,³, 柴岗², 鲍劲松¹()

1.东华大学机械工程学院,上海 201620
2.上海交通大学医学院附属第九人民医院整复外科, 上海 200011
3.上海交通大学塑性成形技术与装备研究院,上海 200030

收稿日期:2021-04-05 出版日期:2022-11-28 发布日期:2022-12-02
通讯作者: 鲍劲松 E-mail:bao@dhu.edu.cn
作者简介:吕超凡(1997-),男,安徽省黄山市人,硕士生,从事三维深度学习研究.
基金资助:
上海申康医院发展中心临床三年行动计划(SHDC2020CR3070B);上海市科学技术委员会项目(18DZ2201900);上海市科学技术委员会项目(19441912300)

Design of Mandibular Angle Osteotomy Plane Based on Point Cloud Semantic Segmentation Algorithm

LÜ Chaofan¹, YAN Yingjie², LIN Li²^,³, CHAI Gang², BAO Jinsong¹()

1. College of Mechanical Engineering, Donghua University, Shanghai 201620, China
2. Department of Plastic and Reconstructive Surgery, Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai 200011, China
3. Institute of Forming Technology and Equipment, Shanghai Jiao Tong University, Shanghai 200030, China

Received:2021-04-05 Online:2022-11-28 Published:2022-12-02
Contact: BAO Jinsong E-mail:bao@dhu.edu.cn

摘要/Abstract

摘要：

下颌角截骨手术是近年来较为热门的颅面整形手术.现阶段,下颌角截骨的术前方案设计通常由具有一定年资的医生完成,过程繁琐且耗时较长.为了提高截骨手术术前规划效率,提出一种基于点云语义分割网络的下颌角截骨面设计方法.对颅骨电子计算机断层扫描(CT)数据进行三维重建和表面点采样,将下颌骨三维模型转换为点云数据,然后通过基于Transformer的点云语义分割网络预测下颌骨点云中的截骨区域,最后根据点云分割结果计算出下颌角截骨平面.所提网络主要包括两个部分:一是基于注意力机制的本地特征提取层,用于提取细粒度局部结构信息;二是基于Transformer的非本地特征提取层,用于提取点云的全局上下文信息.在构建的下颌骨语义分割数据集上,将所提算法与其他点云语义分割算法进行比较.结果表明:所提算法能实现最佳的下颌角截骨区域预测,优于目前常见的点云语义分割算法.

关键词: 下颌角截骨术, 点云分割, 自注意力, 深度学习

Abstract:

Mandibular angle osteotomy is a popular craniofacial plastic surgery in recent years. Usually, preoperative planning of mandibular angle osteotomy is completed by an experienced doctor, which is cumbersome and time-consuming. In order to improve the efficiency of osteotomy planning, a design method of mandibular angle osteotomy plane based on point cloud semantic segmentation network is proposed. After three-dimensional reconstruction of the skull computer tomography (CT) scan data, the three-dimensional model of the mandible is converted into point cloud data through uniform sampling. The resection area of the mandible is predicted by the proposed algorithm, which is used to calculate the mandibular angle osteotomy plane. The proposed semantic segmentation network mainly includes 2 parts: a local feature extraction layer based on the attention mechanism, which is used to extract fine-grained local structure information, and a non-local feature extraction layer based on Transformer, which is used to extract the global context information of the point cloud. On the constructed mandible semantic segmentation data set, the proposed algorithm is compared with other point cloud semantic segmentation algorithms. The results show that the proposed algorithm can achieve the best prediction of the mandibular angle resection area, which is better than current common point cloud semantic segmentation algorithms.

Key words: mandibular angle osteotomy, point cloud segmentation, self-attention, deep learning

中图分类号:

TP183

吕超凡, 言颖杰, 林力, 柴岗, 鲍劲松. 基于点云语义分割算法的下颌角截骨面设计[J]. 上海交通大学学报, 2022, 56(11): 1509-1517.

LÜ Chaofan, YAN Yingjie, LIN Li, CHAI Gang, BAO Jinsong. Design of Mandibular Angle Osteotomy Plane Based on Point Cloud Semantic Segmentation Algorithm[J]. Journal of Shanghai Jiao Tong University, 2022, 56(11): 1509-1517.

图/表 13

图1

图2

图3

图4

图5

图6

图7

表1

表2

图8

表3

表4

表5

参考文献 19

[1]	ZHANG C, MA M W, XU J J, et al. Application of the 3D digital ostectomy template (DOT) in mandibular angle ostectomy (MAO)[J]. Journal of Cranio-Maxillofacial Surgery, 2018, 46(10): 1821-1827. doi: 10.1016/j.jcms.2018.07.026 URL
[2]	MENG H Y, GAO L, LAI Y K, et al. VV-net: Voxel VAE net with group convolutions for point cloud segmentation[C]∥2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea: IEEE, 2019: 8499-8507.
[3]	WANG Z J, LU F. VoxSegNet: Volumetric CNNs for semantic part segmentation of 3D shapes[J]. IEEE Transactions on Visualization and Computer Graphics, 2020, 26(9): 2919-2930. doi: 10.1109/TVCG.2019.2896310 URL
[4]	LE T, BUI G, DUAN Y. A multi-view recurrent neural network for 3D mesh segmentation[J]. Computers & Graphics, 2017, 66: 103-112. doi: 10.1016/j.cag.2017.05.011 URL
[5]	KUNDU A, YIN X Q, FATHI A, et al. Virtual multi-view fusion for 3D semantic segmentation[C]∥Computer Vision-ECCV 2020. Glasgow, UK: Springer, 2020: 518-535.
[6]	CHARLES R Q, HAO S, MO K C, et al. PointNet: Deep learning on point sets for 3D classification and segmentation[C]∥2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA: IEEE, 2017: 77-85.
[7]	CHARLES R Q, LI Y, HAO S, et al. PointNet++: Deep hierarchical feature learning on point sets in a metric space[C]∥Proceedings of the 31st International Conference on Neural Information Processing Systems. Long Beach, California, USA: NIPS, 2017: 5099-5108.
[8]	WANG Y, SUN Y B, LIU Z W, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 1-12.
[9]	LIU Y C, FAN B, XIANG S M, et al. Relation-shape convolutional neural network for point cloud analysis[C]∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019: 8887-8896.
[10]	WANG L, HUANG Y C, HOU Y L, et al. Graph attention convolution for point cloud semantic segmentation[C]∥2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA: IEEE, 2019: 10288-10297.
[11]	HU Q Y, YANG B, XIE L H, et al. RandLA-net: Efficient semantic segmentation of large-scale point clouds[C]∥2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, WA, USA: IEEE, 2020: 11105-11114.
[12]	GUO M H, CAI J X, LIU Z N, et al. PCT: Point cloud transformer[J]. Computational Visual Media, 2021, 7(2): 187-199. doi: 10.1007/s41095-021-0229-5 URL
[13]	赵沁园, 刘磊, 章一新, 等. 手术导板应用于下颌骨精确截骨的前瞻性随机对照研究[J]. 中国美容整形外科杂志, 2018, 29(9): 524-526.
	ZHAO Qinyuan, LIU Lei, ZHANG Yixin, et al. The accuracy of a surgical template for mandibular angle osteotomy: A prospective randomized controlled trial[J]. Chinese Journal of Aesthetic and Plastic Surgery, 2018, 29(9): 524-526.
[14]	LI J X, CHEN B M, LEE G H. SO-net: Self-organizing network for point cloud analysis[C]∥2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA: IEEE, 2018: 9397-9406.
[15]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30: 5998-6008.
[16]	LEE J, YOON W, KIM S, et al. BioBERT: A pre-trained biomedical language representation model for biomedical text mining[J]. Bioinformatics, 2019, 36(4): 1234-1240.
[17]	DONG L H, XU S, XU B. Speech-transformer: A no-recurrence sequence-to-sequence model for speech recognition[C]∥2018 IEEE International Conference on Acoustics, Speech and Signal Processing. Calgary, AB, Canada: IEEE, 2018: 5884-5888.
[18]	SHAW P, USZKOREIT J, VASWANI A. Self-attention with relative position representations[C]∥Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg, PA, USA: Association for Computational Linguistics, 2018: 464-468.
[19]	MAO X Y, FU X, NIU F, et al. Three-dimensional analysis of mandibular angle classification and aesthetic evaluation of the lower face in Chinese female adults[J]. Annals of Plastic Surgery, 2018, 81(1): 12-17. doi: 10.1097/SAP.0000000000001463 pmid: 29762450

模型	准确率	精确率	召回率	F₁值
PointNet++	97.17	80.16	85.09	82.31
DGCNN	98.79	93.17	91.36	92.03
PCT	99.08	94.29	94.01	94.11
本文	99.80	98.78	98.72	98.75

评价指标	最小值	最大值	平均值	达标率/%
Δd_CG/mm	0.14	3.16	1.39	97.5
d_s/mm	3.37	8.86	5.79	97.5
θ/(°)	112.23	126.55	120.08	95.0

k	精确率	准确率	召回率	F₁值
4	99.13	94.42	94.15	94.18
16	99.80	98.78	98.72	98.75
64	97.24	84.62	79.95	81.93

特征聚合函数	精确率	准确率	召回率	F₁值
平均池化	99.35	95.12	97.03	96.01
最大池化	99.62	97.31	97.66	97.41
注意力池化	99.80	98.78	98.72	98.75

位置编码	精确率	准确率	召回率	F₁值
没有位置编码	99.37	95.89	96.01	95.90
绝对位置编码	99.56	96.33	98.16	97.23
相对位置编码	99.80	98.78	98.72	98.75

基于点云语义分割算法的下颌角截骨面设计

Design of Mandibular Angle Osteotomy Plane Based on Point Cloud Semantic Segmentation Algorithm

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 13

参考文献 19

相关文章 15

编辑推荐

Metrics

本文评价

[1]	曾国治, 魏子清, 岳宝, 丁云霄, 郑春元, 翟晓强. 基于CNN-RNN组合模型的办公建筑能耗预测[J]. 上海交通大学学报, 2022, 56(9): 1256-1261.
[2]	吴庶宸, 戚宗锋, 李建勋. 基于深度学习的智能全局灵敏度分析[J]. 上海交通大学学报, 2022, 56(7): 840-849.
[3]	李高阳, 贾爱亭, 洪波, 李湘文, 高佳篷. 大型箱型梁高速GMAW三维位姿快速估计方法[J]. 上海交通大学学报, 2022, 56(6): 794-800.
[4]	陶海红, 闫莹菲. 一种基于GA-CNN的网络化雷达节点遴选算法[J]. 空天防御, 2022, 5(1): 1-5.
[5]	金丽洁, 武亚涛. 基于双CNN的雷达信号调制类型识别方法[J]. 空天防御, 2022, 5(1): 66-70.
[6]	王兴志, 翟海保, 严亚勤, 吴庆曦. 基于数字孪生和深度学习的新一代调控系统预调度方法[J]. 上海交通大学学报, 2021, 55(S2): 37-41.
[7]	王岩, 陈耀然, 韩兆龙, 周岱, 包艳. 基于互信息理论与递归神经网络的短期风速预测模型[J]. 上海交通大学学报, 2021, 55(9): 1080-1086.
[8]	王宇, 余岳峰, 朱小磊, 张忠孝. 基于光流法和深度学习的燃气火焰稳定性[J]. 上海交通大学学报, 2021, 55(4): 462-470.
[9]	蔡云泽, 张彦军. 基于双通道特征增强集成注意力网络的红外弱小目标检测方法[J]. 空天防御, 2021, 4(4): 14-22.
[10]	曹靖豪, 张俊举, 黄维, 姚若彤, 张平. 基于多尺度特征融合的无人机识别与检测[J]. 空天防御, 2021, 4(1): 60-64.
[11]	刘文海，胡洁，王伟明. 基于完备抓取构型和多阶段网络的软体手抓取[J]. 上海交通大学学报, 2020, 54(5): 507-514.
[12]	陈寂驰, 魏国华, 郭聪隆, 张立和. 一种基于红外图像序列的深度学习三维重建仿真方法初探[J]. 空天防御, 2020, 3(4): 21-29.
[13]	章云港,杨剑锋,易本顺. 低剂量CT图像去噪的改进型残差编解码网络[J]. 上海交通大学学报, 2019, 53(8): 983-989.
[14]	袁利毫, 昝英飞, 钟声华, 祝海涛. 基于YOLOv3的水下小目标自主识别[J]. 海洋工程装备与技术, 2018, 5(增刊): 118-123.
[15]	易平，王科迪，黄程，顾双驰，邹福泰，李建华. 人工智能对抗攻击研究综述[J]. 上海交通大学学报（自然版）, 2018, 52(10): 1298-1306.