Real-Time Prediction of Elbow Motion Through sEMG-Based Hybrid BP-LSTM Network

doi:10.1007/s12204-024-2581-y

Abstract

Abstract: In the face of the large number of people with motor function disabilities, rehabilitation robots have attracted more and more attention. In order to promote the active participation of the user’s motion intention in the assisted rehabilitation process of the robots, it is crucial to establish the human motion prediction model. In this paper, a hybrid prediction model built on long short-term memory (LSTM) neural network using surface electromyography (sEMG) is applied to predict the elbow motion of the users in advance. This model includes two sub-models: a back-propagation neural network and an LSTM network. The former extracts a preliminary prediction of the elbow motion, and the latter corrects this prediction to increase accuracy. The proposed model takes time series data as input, which includes the sEMG signals measured by electrodes and the continuous angles from inertial measurement units. The offline and online tests were carried out to verify the established hybrid model. Finally, average root mean square errors of 3.52 ◦ and 4.18 ◦ were reached respectively for offline and online tests, and the correlation coefficients for both were above 0.98.

Key words: motion prediction, surface electromyography (sEMG), long short-term memory (LSTM), backpropagation neural network

摘要： 面对数量庞大的运动功能障碍群体，康复机器人越来越受到关注。为了促进用户意图在机器人辅助康复过程中的主动参与，建立人体运动预测模型至关重要。本文建立了一个基于长短期记忆网络（LSTM）的混合预测模型。该模型使用表面肌电信号（sEMG）作为输入，并成功应用于肘关节运动的提前预测。该模型包含两个子模型：反向传播神经网络和长短期记忆网络。首先，反向传播神经网络基于肌电信号对肘关节运动进行初步预测，然后，长短期记忆网络对这一预测进行修正以提高模型准确性。该模型使用时间序列数据作为输入，包括通过电极测量的表面肌电信号和来自惯性测量单元的连续的角度信号。使用离线和在线实验对所建立的混合模型进行了训练和验证。在离线和在线实验中，预测角度和实际角度之间的平均均方根误差分别为3.52°和4.18°，相关系数均在0.98以上。

关键词: 运动预测，表面肌电信号，长短期记忆网络，反向传播神经网络

CLC Number:

TP181
R496

Ma Yiyuan, Chen Huaiyuan, Chen Weidong. Real-Time Prediction of Elbow Motion Through sEMG-Based Hybrid BP-LSTM Network[J]. J Shanghai Jiaotong Univ Sci, 2025, 30(3): 455-462.

References

[1] KRATZER P, BIHLMAIER S, MIDLAGAJNI N B, et al. MoGaze: A dataset of full-body motions that includes workspace geometry and eye-gaze [J]. IEEE Robotics and Automation Letters, 2021, 6(2): 367-373.

[2] PENG L, HOU Z G, WANG C, et al. Physical interaction methods for rehabilitation and assistive robots [J]. Acta Automatica Sinica, 2018, 44(11): 2000-2010 (in Chinese).

[3] ERSEN M, OZTOP E, SARIEL S. Cognition-enabled robot manipulation in human environments: Requirements, recent work, and open problems [J]. IEEE Robotics & Automation Magazine, 2017, 24(3): 108-122.

[4] HUANG B Z, PAN L, YANG Y, et al. Neural MoCon: Neural motion control for physically plausible human motion capture [C]//2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 6407-6416.

[5] JARRASSÉ N, SANGUINETI V, BURDET E. Slaves no longer: Review on role assignment for human-robot joint motor action [J]. Adaptive Behavior - Animals, Animats, Software Agents, Robots, Adaptive Systems, 2014, 22(1): 70-82.

[6] TRIGILI E, CREA S, MOISÈ M, et al. Design and experimental characterization of a shoulder-elbow exoskeleton with compliant joints for post-stroke rehabilitation [J]. IEEE/ASME Transactions on Mechatronics, 2019, 24(4): 1485-1496.

[7] ZHANG R, WANG C S, HE S H, et al. An adaptive brain-computer interface to enhance motor recovery after stroke [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2023, 31: 2268-2278.

[8] LOBO-PRAT J, KOOREN P N, STIENEN A H A, et al. Non-invasive control interfaces for intention detection in active movement-assistive devices [J]. Journal of Neuroengineering and Rehabilitation, 2014, 11: 168.

[9] CASTELLINI C. Peripheral nervous system interfaces: Invasive or non-invasive? [J]. Frontiers in Neurorobotics, 2022, 16: 846866.

[10] LIU L, YANG P, LIU Z J, et al. Prosthetic leg locomotion-mode identification based on high-order zero-crossing analysis surface electromyography [J]. Journal of Shanghai Jiao Tong University (Science), 2021, 26(1): 84-92.

[11] YUAN Z W, PENG Y, WANG L S, et al. Effect of BCI-controlled pedaling training system with multiple modalities of feedback on motor and cognitive function rehabilitation of early subacute stroke patients [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2021, 29: 2569-2577.

[12] GUO N, WANG X J, DUANMU D H, et al. SSVEP-based brain computer interface controlled soft robotic glove for post-stroke hand function rehabilitation [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2022, 30: 1737-1744.

[13] HAN J D, DING Q C, XIONG A B, et al. A state-space EMG model for the estimation of continuous joint movements [J]. IEEE Transactions on Industrial Electronics, 2015, 62(7): 4267-4275.

[14] SITOLE S P, SUP F C. Continuous prediction of human joint mechanics using EMG signals: A review of model-based and model-free approaches [J]. IEEE Transactions on Medical Robotics and Bionics, 2023, 5(3): 528-546.

[15] SAPUTRA A A, HONG C W, MATSUDA T, et al. A real-time control system of upper-limb human musculoskeletal model with environmental integration [J]. IEEE Access, 2023, 11: 74337-74363.

[16] XI X G, JIANG W J, HUA X, et al. Simultaneous and continuous estimation of joint angles based on surface electromyography state-space model [J]. IEEE Sensors Journal, 2021, 21(6): 8089-8099.

[17] KASHI S, POLAK R F, LERNER B, et al. A machine-learning model for automatic detection of movement compensations in stroke patients [J]. IEEE Transactions on Emerging Topics in Computing, 2021, 9(3): 1234-1247.

[18] GUO S X, YANG Z Y, LIU Y. EMG-based continuous prediction of the upper limb elbow joint angle using GRNN [C]//2019 IEEE International Conference on Mechatronics and Automation. Tianjin: IEEE, 2019: 2168-2173.

[19] LIU J C, LIANG Y Y, CHEN Z, et al. A double-layer vehicle speed prediction based on BPNN-LSTM for off-road vehicles [J]. Sensors, 2023, 23(14): 6385.

[20] ZHANG L. An upper limb movement estimation from electromyography by using BP neural network [J]. Biomedical Signal Processing and Control, 2019, 49: 434-439.

[21] KIM D, KOH K, OPPIZZI G, et al. Simultaneous estimations of joint angle and torque in interactions with environments using EMG [C]//2020 IEEE International Conference on Robotics and Automation. Paris: IEEE, 2020: 3818-3824.

[22] MA C F, LIN C, SAMUEL O W, et al. Continuous estimation of upper limb joint angle from sEMG signals based on SCA-LSTM deep learning approach [J]. Biomedical Signal Processing and Control, 2020, 61: 102024.

[23] MA C F, LIN C, SAMUEL O W, et al. A Bi-directional LSTM network for estimating continuous upper limb movement from surface electromyography [J]. IEEE Robotics and Automation Letters, 2021, 6(4): 7217-7224.

[24] SONG H L, LI Y T, ZOU X F, et al. Elite male table tennis matches diagnosis using SHAP and a hybrid LSTM-BPNN algorithm [J]. Scientific Reports, 2023, 13(1): 11533.

[25] JIA H M, ZHOU X F. Water quality prediction method based on LSTM-BP [C]//2020 12th International Conference on Intelligent Human-Machine Systems and Cybernetics. Hangzhou: IEEE, 2020: 27-30.

[26] XIE Y W, HU P F, ZHU N, et al. A hybrid short-term load forecasting model and its application in ground source heat pump with cooling storage system [J]. Renewable Energy, 2020, 161: 1244-1259.

[27] STEGEMAN D, HERMENS H. Standards for surface electromyography: The European project Surface EMG for non-invasive assessment of muscles (SENIAM) [J]. Enschede: Roessingh Research and Development, 2007, 10: 8-12.

[28] CHU J U, MOON I, LEE Y J, et al. A supervised feature-projection-based real-time EMG pattern recognition for multifunction myoelectric hand control [J]. IEEE/ASME Transactions on Mechatronics, 2007, 12(3): 282-290.

[29] HUANG Y, SONG R, ARGHA A, et al. Continuous description of human 3D motion intent through switching mechanism [J]. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 2020, 28(1): 277-286.

[30] XU M J, CHEN X, SUN A T, et al. A novel event-driven spiking convolutional neural network for electromyography pattern recognition [J]. IEEE Transactions on Biomedical Engineering, 2023, 70(9): 2604-2615.

[31] LIU J, KANG S H, XU D L, et al. EMG-based continuous and simultaneous estimation of arm kinematics in able-bodied individuals and stroke survivors [J]. Frontiers in Neuroscience, 2017, 11: 480.

[32] HOCHREITER S, SCHMIDHUBER J. Long short-term memory [J]. Neural Computation, 1997, 9: 1735-1780.

[33] ABADI M, BARHAM P, CHEN J M, et al. TensorFlow: A system for large-scale machine learning [C]//Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation. Savannah: USENIX, 2016: 265-283.

[34] REAZ M B I, HUSSAIN M S, MOHD-YASIN F. Techniques of EMG signal analysis: Detection, processing, classification and applications (Correction) [J]. Biological Procedures Online, 2006, 8: 163.

[35] FUKANO K, IIAZAWA K, SOEDA T, et al. Deep learning for gesture recognition based on surface EMG data [C]//2021 International Conference on Advanced Mechatronic Systems. Tokyo: IEEE, 2021: 41-45.

[36] SIMÃO M, NETO P, GIBARU O. EMG-based online classification of gestures with recurrent neural networks [J]. Pattern Recognition Letters, 2019, 128: 45-51.