Safety Protection Method of Rehabilitation Robot Based on fNIRS and RGB-D Information Fusion

doi:10.1007/s12204-021-2365-6

Abstract

Abstract: In order to improve the safety protection performance of the rehabilitation robot, an active safety protection method is proposed in the rehabilitation scene. The oxyhemoglobin concentration information and RGB-D information are combined in this method, which aims to realize the comprehensive monitoring of the invasion target, the patient’s brain function movement state, and the joint angle in the rehabilitation scene. The main focus is to study the fusion method of the oxyhemoglobin concentration information and RGB-D information in the rehabilitation scene. Frequency analysis of brain functional connectivity coefficient was used to distinguish the basic motion states. The human skeleton recognition algorithm was used to realize the angle monitoring of the upper limb joint combined with the depth information. Compared with speed and separation monitoring, the protection method of multi-information fusion is safer and more comprehensive for stroke patients. By building the active safety protection platform of the upper limb rehabilitation robot, the performance of the system in different safety states is tested, and the safety protection performance of the method in the upper limb rehabilitation scene is verified.

Key words: upper limb rehabilitation robot, safety protection, functional near-infrared spectroscopy (fNIRS), multi-information fusion

CLC Number:

TP 242.3

LI Dong (李栋), FAN Yulin (樊钰琳), L v Na (吕娜), CHEN Guodong∗ (陈国栋), WANG Zheng (王正), CHI Wenzheng (迟文政). Safety Protection Method of Rehabilitation Robot Based on fNIRS and RGB-D Information Fusion[J]. J Shanghai Jiaotong Univ Sci, 2022, 27(1): 45-54.

References 31

[1]	WANG L D. Brief report on stroke prevention andtreatment in China, 2019: Report on stroke preventionand treatment in China Writing Group [J]. ChineseJournal of Cerebrovascular Diseases, 2020, 17(5): 272-281 (in Chinese).
[2]	FU L, WU R, ZHAO J. The evolution and enlightenmentof safety specification of cooperative robots:ISO/TS 15066 [J]. Robot, 2017, 39(4): 532-540 (inChinese).
[3]	DE SANTIS A, SICILIANO B, DE LUCA A, et al. Anatlas of physical human-robot interaction [J]. Mechanismand Machine Theory, 2008, 43(3): 253-270.
[4]	KIKUUWE R, YASUKOUCHI S, FUJIMOTO H, etal. Proxy-based sliding mode control: A safer extensionof PID position control [J]. IEEE Transactions onRobotics, 2010, 26(4): 670-683.
[5]	YU W, ROSEN J. A novel linear PID controller foran upper limb exoskeleton [C]//49th IEEE Conferenceon Decision and Control (CDC). Atlanta, GA: IEEE,2010: 3548-3553.
[6]	SUGAR T G, HE J P, KOENEMAN E J, et al. Designand control of RUPERT: A device for robotic upperextremity repetitive therapy [J]. IEEE Transactions onNeural Systems and Rehabilitation Engineering, 2007,15(3): 336-346.
[7]	ZHANG Y B, WANG Z X, JI L H, et al. The clinicalapplication of the upper extremity compound movementsrehabilitation training robot [C]//9th InternationalConference on Rehabilitation Robotics. Chicago,IL: IEEE, 2005: 91-94.
[8]	DUVINAGE M, CASTERMANS T, PETIEAU M, etal. A subjective assessment of a P300 BCI system forlower-limb rehabilitation purposes [C]//2012 AnnualInternational Conference of the IEEE Engineering inMedicine and Biology Society. San Diego, CA: IEEE,2012: 3845-3849.
[9]	TONG L N, HOU Z G, PENG L, et al. MultichannelsEMG time series analysis based human motionrecognition method [J]. Acta Automatica Sinica,2014, 40(5): 810-821 (in Chinese).
[10]	KHALID A, KIRISCI P, GHRAIRI Z, et al. Safetyrequirements in collaborative human–robot cyberphysicalsystem [M]//Dynamics in logistics. Cham:Springer International Publishing, 2016: 41-51.
[11]	WANG Z P, ZHOU Y J, CHEN L, et al. BCI monitorenhances electroencephalographic and cerebral hemodynamicactivations during motor training [J]. IEEETransactions on Neural Systems and RehabilitationEngineering, 2019, 27(4): 780-787.
[12]	LIQB, FENG J, GUOJ, et al. Effects of themultisensoryrehabilitation product for home-based hand trainingafter stroke on cortical activation by using NIRSmethods [J]. Neuroscience Letters, 2020, 717: 134682.
[13]	WANGMJ, HU Z S, LIU L, et al. Disrupted functionalbrain connectivity networks in children with attentiondeficit/hyperactivity disorder: Evidence from restingstatefunctional near-infrared spectroscopy [J]. Neurophotonics,2020, 7(1): 015012.
[14]	BU L G, WU Y, YAN Q, et al. Effects of physicaltraining on brain functional connectivity of methamphetaminedependencies as assessed using functionalnear-infrared spectroscopy [J]. Neuroscience Letters,2020, 715: 134605.
[15]	HUO C C, XU G C, LI Z Y, et al. Limb linkage rehabilitationtraining-related changes in cortical activationand effective connectivity after stroke: A functionalnear-infrared spectroscopy study [J]. ScientificReports, 2019, 9: 6226.
[16]	ZHANG S, ZHENG Y C, WANG D F, et al. Applicationof a common spatial pattern-based algorithm foran fNIRS-based motor imagery brain-computer interface[J]. Neuroscience Letters, 2017, 655: 35-40.
[17]	LU K, XU G C, LIWH, et al. Frequency-specific functionalconnectivity related to the rehabilitation task ofstroke patients [J]. Medical Physics, 2019, 46(4): 1545-1560.
[18]	LIU Z, ZHANG M, XU G, et al. Effective connectivityanalysis of the brain network in drivers during actualdriving using near-infrared spectroscopy [J]. Frontiersin Behavioral Neuroscience, 2017, 11: 211.
[19]	XU G C, ZHANG M, WANG Y, et al. Functionalconnectivity analysis of distracted drivers based onthe wavelet phase coherence of functional near-infraredspectroscopy signals [J]. PLoS One, 2017, 12(11):e0188329.
[20]	XU L W, WANG B T, XU G C, et al. Functionalconnectivity analysis using fNIRS in healthy subjectsduring prolonged simulated driving [J]. NeuroscienceLetters, 2017, 640: 21-28.
[21]	HU X, ZHUANG C, WANG F, et al. fNIRS evidencefor recognizably different positive emotions [J]. Frontiersin Human Neuroscience, 2019, 13: 120.
[22]	LIU Q Y,WANG B T, LIU Y, et al. Frequency-specificeffective connectivity in subjects with cerebral infarctionas revealed by NIRS method [J]. Neuroscience,2018, 373: 169-181.
[23]	NIKOLAKIS N, MARATOS V, MAKRIS S. A cyberphysical system (CPS) approach for safe human-robotcollaboration in a shared workplace [J]. Robotics andComputer-Integrated Manufacturing, 2019, 56: 233-243.
[24]	NI Z S, GU Y, LIU Q L, et al. Flexible calibrationmethod for binocular stereo vision in large field of view[J]. Optics and Precision Engineering, 2017, 25(7):1882-1889 (in Chinese).
[25]	CAO Z, HIDALGO G, SIMON T, et al. OpenPose:Realtime multi-person 2D pose estimation using partaffinity fields [J]. IEEE Transactions on Pattern Analysisand Machine Intelligence, 2021, 43(1): 172-186.
[26]	SNAVELY K N. Scene reconstruction and visualizationfrom Internet photo collections [D]. Seattle, WA:University of Washington, 2008.
[27]	STAUFFER C, GRIMSON W E L. Adaptivebackground mixture models for real-time tracking[C]//Proceedings of 1999 IEEE Computer SocietyConference on Computer Vision and Pattern Recognition(Cat. No PR00149). Fort Collins, CO: IEEE,1999: 246-252.
[28]	REDMON J, FARHADI A. YOLOv3: An incrementalimprovement [EB/OL]. [2021-01-30].https://arxiv.org/pdf/1804.02767.pdf.
[29]	SU H L, HUO C C, WANG B T, et al. Alterationsin the coupling functions between cerebraloxyhaemoglobin and arterial blood pressure signalsin post-stroke subjects [J]. PLoS One, 2018, 13(4):e0195936.
[30]	BU L G, QI L P, YAN W, et al. Acute kick-boxingexercise alters effective connectivity in the brain of femaleswith methamphetamine dependencies [J]. NeuroscienceLetters, 2020, 720: 134780.
[31]	JIAN C Y, DENG L C, LIANG L K, et al. Neuromuscularcontrol of the agonist-antagonist muscle coordinationaffected by visual dimension: An EMG-fNIRSstudy [J]. IEEE Access, 2020, 8: 100768-100777.