Visual Positioning of Nasal Swab Robot Based on Hierarchical Decision

doi:10.1007/s12204-023-2581-3

Abstract

Abstract: This study focuses on a robot vision localization method for coping with the operational task of automatic nasal swab sampling. The application is important in the detection and epidemic prevention of Corona Virus Disease 2019 (COVID-19) to alleviate the large-scale negative impact of individuals suffering from pneumonia owing to COVID-19. In this method, the idea of a hierarchical decision network is used to consider the strong infectious characteristics of the COVID-19, which is followed by processing the robot behavior constraint condition. The visual navigation and positioning method using a single-arm robot for sampling is also planned, which considers the operation characteristics of medical staff. In the decision network, the risk factor for potential contact infection caused by swab sampling operations is established to avoid the spread among personnel. A robot visual servo control with artificial intelligence characteristics is developed to achieve a stable and safe nasal swab sampling operation. Experiments demonstrate that the proposed method can achieve good vision positioning for the robots and provide technical support for managing new major public health situations.

Key words: surgical robot, nasal swab sampling, vision servo, hierarchical decision

摘要： 本文主要研究一种用于鼻拭子机器人自动采样操作的视觉定位方法。使用机器人完成鼻拭子采样任务可以减少医务人员对新型冠状病毒病（COVID-19）患者的直接接触，从而减小COVID-19带来的负面影响，对COVID-19的检测和防疫具有重要意义。该方法根据COVID-19的传播特点使用层次决策网络来处理机器人的行为约束条件，并结合医务人员的采样动作特点设计了使用单臂机器人进行鼻拭子采样操作的视觉导航定位方法。该方法所使用的决策网络综合考虑了人工采样操作中引起潜在接触感染风险的影响因素，以尽可能降低病毒在人员之间的传播概率。进一步形成具有人工智能特征的视觉伺服控制策略，并完成稳定、安全的鼻拭子机器人采样操作。实验证明，该方法能够实现良好的机器人视觉系统的定位，可以为公共卫生防控提供必要的技术支持。

关键词: 医疗机器人，鼻拭子采样，视觉伺服，层次决策

CLC Number:

TP 242
R 314

LI Guozhia a(李国志)，ZOU Shuizhong b*(邹水中),DING Shuacue a(丁数学). Visual Positioning of Nasal Swab Robot Based on Hierarchical Decision[J]. J Shanghai Jiaotong Univ Sci, 2023, 28(3): 323-329.

References 27

[1]	CHEN A T, RYSKINA K L, JUNG H Y. Long-term care, residential facilities, and COVID-19: An overview of federal and state policy responses [J]. Journal of the American Medical Directors Association, 2020, 21(9): 1186-1190.
[2]	FISHER D, HEYMANN D. Q&A: The novel coronavirus outbreak causing COVID-19 [J]. BMC Medicine, 2020, 18(1): 57.
[3]	TSIKALA VAFEA M, ATALLA E, GEORGAKAS J, et al. Emerging technologies for use in the study, diagnosis, and treatment of patients with COVID-19 [J]. Cellular and Molecular Bioengineering, 2020, 13(4): 249-257.
[4]	DING W P, NAYAK J, SWAPNAREKHA H, et al. Fusion of intelligent learning for COVID-19: A state-of-the-art review and analysis on real medical data [J]. Neurocomputing, 2021, 457: 40-66.
[5]	NAREN N, CHAMOLA V, BAITRAGUNTA S, et al. IoMT and DNN-enabled drone-assisted COVID-19 screening and detection framework for rural areas [J]. IEEE Internet of Things Magazine, 2021, 4(2): 4-9.
[6]	WANG X V, WANG L H. A literature survey of the robotic technologies during the COVID-19 pandemic [J]. Journal of Manufacturing Systems, 2021, 60: 823-836.
[7]	WU S Z, WU D D, YE R Z, et al. Pilot study of robot-assisted teleultrasound based on 5G network: A new feasible strategy for early imaging assessment during COVID-19 pandemic [J]. IEEE Transactions on Ultrasonics, Ferroelectrics, and Frequency Control, 2020, 67(11): 2241-2248.
[8]	XIE Z X, CHEN B H, LIU J Q, et al. A tapered soft robotic oropharyngeal swab for throat testing: A new way to collect sputa samples [J]. IEEE Robotics & Automation Magazine, 2021, 28(1): 90-100.
[9]	PETRUZZI G, DE VIRGILIO A, PICHI B, et al. COVID-19: Nasal and oropharyngeal swab [J]. Head & Neck, 2020, 42(6): 1303-1304.
[10]	MCDERMOTT A. Inner Workings: Researchers race to develop in-home testing for COVID-19, a potential game changer [J]. Proceedings of the National Academy of Sciences of the United States of America, 2020, 117(42): 25956-25959.
[11]	LI Z, FEILING J, REN H L, et al. A novel teleoperated flexible robot targeted for minimally invasive robotic surgery [J]. Engineering, 2015, 1(1): 73-78.
[12]	SEO J, SHIM S, PARK H, et al. Development of robot-assisted untact swab sampling system for upper respiratory disease [J]. Applied Sciences, 2020, 10(21): 7707.
[13]	SHI F, WANG J, SHI J, et al. Review of artificial intelligence techniques in imaging data acquisition, segmentation, and diagnosis for COVID-19 [J]. IEEE Reviews in Biomedical Engineering, 2021, 14: 4-15.
[14]	HUSSAIN K, WANG X S, OMAR Z, et al. Robotics and artificial intelligence applications in manage and control of COVID-19 pandemic [C]//2021 International Conference on Computer, Control and Robotics. Shanghai: IEEE, 2021: 66-69.
[15]	MBUNGE E, MILLHAM R C, SIBIYA M N, et al. Framework for ethical and acceptable use of social distancing tools and smart devices during COVID-19 pandemic in Zimbabwe [J]. Sustainable Operations and Computers, 2021, 2: 190-199.
[16]	MEI X Y, LEE H C, DIAO K Y, et al. Artificial intelligence-enabled rapid diagnosis of patients with COVID-19 [J]. Nature Medicine, 2020, 26(8): 1224-1228.
[17]	XU X B, YANG Y M, ZHOU Y Y, et al. Image segmentation of throat swab sampling based on mask R-CNN [C]//2020 Chinese Automation Congress. Shanghai: IEEE, 2020: 7451-7455.
[18]	ZHAO Z Y, WANG T, WANG D Q. Inverse kinematic analysis of the general 6R serial manipulators based on unit dual quaternion and Dixon resultant [C]//2017 Chinese Automation Congress. Jinan: IEEE, 2017: 2646-2650.
[19]	HE R B, ZHAO Y J, YANG S N, et al. Kinematic-parameter identification for serial-robot calibration based on POE formula [J]. IEEE Transactions on Robotics, 2010, 26(3): 411-423.
[20]	STONE H, SANDERSON A. A prototype arm signature identification system [C]//1987 IEEE International Conference on Robotics and Automation. Raleigh: IEEE, 1987: 175-182.
[21]	WU L, YANG X D, CHEN K, et al. A minimal POE-based model for robotic kinematic calibration with only position measurements [J]. IEEE Transactions on Automation Science and Engineering, 2015, 12(2): 758-763.
[22]	ASPRAGATHOS N A, DIMITROS J K. A comparative study of three methods for robot kinematics [J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 1998, 28(2): 135-145.
[23]	WANG W, WANG G, YUN C. A calibration method of kinematic parameters for serial industrial robots [J]. Industrial Robot: An International Journal, 2014, 41(2): 157-165.
[24]	HAGN U, KONIETSCHKE R, TOBERGTE A, et al DLR MiroSurge: A versatile system for research in endoscopic telesurgery [J]. International Journal of Computer Assisted Radiology and Surgery, 2010, 5(2): 183-193.
[25]	CHEN S F, KAO I. Conservative congruence transformation for joint and Cartesian stiffness matrices of robotic hands and fingers [J]. The International Journal of Robotics Research, 2000, 19(9): 835-847.
[26]	WANG Y N, XU Z C, ZHAO H C, et al. M-region segmentation of pharyngeal swab image based on improved U-net model [C]//2021 IEEE International Conference on Intelligence and Safety for Robotics. Tokoname: IEEE, 2021: 186-190.
[27]	PARK F C, OKAMURA K. Kinematic calibration and the product of exponential formula [M]// Advance in robot kinematics and computational geometry. Dordrecht: Springer, 1994: 119-128.