With advancements in medical imaging and robotic technology, the idea of fully autonomous diagnosis and treatment has become appealing, from ethereal to tangible. Owing to its characteristics of non-invasiveness, non-radiation, and fast imaging speed, ultrasonography has been increasingly used in clinical practice, such as in obstetrics, gynecology, and surgical puncture. In this paper, we propose a real-time image-based visual servo control scheme using a hybrid slice-to-volume registration method. In this manner, the robot can autonomously locate the ultrasound probe to the desired posture according to preoperational planning, even in the presence of disturbances. The experiments are designed and conducted using a thyroid biopsy phantom model. The results show that the proposed scheme can achieve a refresh rate of up to 30 Hz and a tracking accuracy of (0.52±0.65) mm.
LI Jian1 (李坚)，WANG Xingchao1 (王星超)，ZHONG Min2 (钟敏)，ZHENG Jian2(郑剑)，SUN Zhenglong1*(孙正隆)
. Real-Time Slice-to-Volume Registration-Based Autonomous Navigation for Robot-Assisted Thyroid Biopsy[J]. Journal of Shanghai Jiaotong University(Science), 2023
, 28(3)
: 330
-338
.
DOI: 10.1007/s12204-023-2606-y
[1]CHAN V, PERLAS A. Basics of ultrasound imaging [M]//Atlas of ultrasound-guided procedures in interventional pain management. New York: Springer, 2011: 13-19.
[2]HARRISON G, HARRIS A. Work-related musculoskeletal disorders in ultrasound: Can You reduce risk? [J]. Ultrasound, 2015, 23(4): 224-230.
[3]MUIR M, HRYNKOW P, CHASE R, et al. The nature, cause, and extent of occupational musculoskeletal injuries among sonographers [J]. Journal of Diagnostic Medical Sonography, 2004, 20(5): 317-325.
[4]DE OLIVEIRA FILHO G R. The construction of learning curves for basic skills in anesthetic procedures: An application for the cumulative sum method [J]. Anesthesia and Analgesia, 2002, 95(2): 411-416.
[5]NING C P, JI X L, WANG H Q, et al. Association between the sonographer’s experience and diagnostic performance of IOTA simple rules [J]. World Journal of Surgical Oncology, 2018, 16(1): 1-7.
[6]WANG H X, LI R, CHEN X, et al. Remote intelligent assisted diagnosis system for hepatic echinococcosis [M]//Medical ultrasound, and preterm, perinatal and paediatric image analysis. Cham: Springer International Publishing, 2020: 3-12.
[7]HUANG Q H, LAN J L. Remote control of a robotic prosthesis arm with six-degree-of-freedom for ultrasonic scanning and three-dimensional imaging [J]. Biomedical Signal Processing and Control, 2019, 54: 101606.
[8]SARTORI E, TADIELLO C, SECCHI C, et al. Teleechography using a two-layer teleoperation algorithm with energy scaling [C]//2019 International Conference on Robotics and Automation. Montreal: IEEE, 2019: 1569-1575.
[9]MARCHAL M, TROCCAZ J. A one-DOF freehand haptic device for robotic tele-echography [J]. Studies in Health Technology and Informatics, 2004, 98: 231-233.
[10]KAMINSKI J T, RAFATZAND K, ZHANG H K. Feasibility of robot-assisted ultrasound imaging with force feedback for assessment of thyroid diseases [J]. Proceedings of SPIE, 2020, 11315: 113151D.
[11]NADEAU C, KRUPA A. Intensity-based ultrasound visual servoing: Modeling and validation with 2-D and 3-D probes [J]. IEEE Transactions on Robotics, 2013, 29(4): 1003-1015.
[12]NAKADATE R, SOLIS J, TAKANISHI A, et al. Out-of-plane visual servoing method for tracking the carotid artery with a robot-assisted ultrasound diagnostic system [C]//2011 IEEE International Conference on Robotics and Automation. Shanghai: IEEE, 2011: 5267-5272.
[13]HUANG Y W, XIAO W, WANG C Y, et al. Towards fully autonomous ultrasound scanning robot with imitation learning based on clinical protocols [J]. IEEE Robotics and Automation Letters, 2021, 6(2): 3671-3678.
[14]KRUPA A, FOLIO D, NOVALES C, et al. Robotized tele-echography: An assisting visibility tool to support expert diagnostic [J]. IEEE Systems Journal, 2014, 10(3): 974-983.
[15]JIANG Z L, DANIS N, BI Y, et al. Precise repositioning of robotic ultrasound: Improving registration-based motion compensation using ultrasound confidence optimization [J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-11.
[16]ZHANG S H, JIANG S, YANG Z Y, et al. An ultrasound image navigation robotic prostate brachytherapy system based on US to MRI deformable image registration method [J]. Hellenic Journal of Nuclear Medicine, 2016, 19(3): 223-230.
[17]ZHANG S H, JIANG S, YANG Z Y, et al. 2D ultrasound and 3D MR image registration of the prostate for brachytherapy surgical navigation [J]. Medicine, 2015, 94(40): e1643.
[18]ESTEBAN J, SIMSON W, WITZIG S R, et al. Robotic ultrasound-guided facet joint insertion [J]. International Journal of Computer Assisted Radiology and Surgery, 2018, 13(6): 895-904.
[19]ZETTINIG O, FUERST B, KOJCEV R, et al. Toward real-time 3D ultrasound registration-based visual servoing for interventional navigation [C]//2016 IEEE International Conference on Robotics and Automation. Stockholm: IEEE, 2016: 945-950.
[20]LASSO A, HEFFTER T, RANKIN A, et al. PLUS: Open-source toolkit for ultrasound-guided intervention systems [J]. IEEE Transactions on Biomedical Engineering, 2014, 61(10): 2527-2537.
[21]RONNEBERGER O, FISCHER P, BROX T. U-Net: Convolutional networks for biomedical image segmentation [M]//Medical image computing and computer-assisted intervention – MICCAI 2015. Cham: Springer, 2015: 234-241.
[22]WANG Q. GMM-based hidden markov random field for color image and 3D volume segmentation [DB/OL]. (2012-12-18). https://arxiv.org/abs/1212.4527
[23]FERRANTE E, PARAGIOS N. Slice-to-volume medical image registration: A survey [J]. Medical Image Analysis, 2017, 39: 101-123.
[24]MARAMI B, SIROUSPOUR S, CAPSON D W. Model-based 3D/2D deformable registration of MR images [C]//2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society. Boston: IEEE, 2011: 4880-4883.
[25]MOCKUS J. The Bayesian approach to global optimization [M]//System modeling and optimization. Berlin/Heidelberg: Springer-Verlag, 2005: 473-481.
[26]SRINIVAS N, KRAUSE A, KAKADE S M, et al. Gaussian process optimization in the bandit setting: No regret and experimental design [DB/OL]. (2009-12-21). https://arxiv.org/abs/0912.3995
[27]NELDER J A, MEAD R. A simplex method for function minimization [J]. The Computer Journal, 1965, 7(4): 308-313.
[28]CHAUMETTE F, HUTCHINSON S. Visual servo control. I. basic approaches [J]. IEEE Robotics & Automation Magazine, 2006, 13(4): 82-90.
[29]MALIS E, CHAUMETTE F, BOUDET S. 2 1/2 D visual servoing [J]. IEEE Transactions on Robotics & Automation, 1999, 15(2): 238-250.
[30]ALBU-SCHAFFER A, OTT C, FRESE U, et al. Cartesian impedance control of redundant robots: Recent results with the DLR-light-weight-arms [C]//2003 IEEE International Conference on Robotics and Automation. Taipei: IEEE, 2003: 3704-3709.
[31]PRAGER R W, ROHLING R N, GEE A H, et al. Rapid calibration for 3-D freehand ultrasound [J]. Ultrasound in Medicine & Biology, 1998, 24(6): 855-869.
