Abstract: With the continuous advancement of sensors and algorithms, an increasing number of deep learning methods have been applied to fine-grained upper limb motion intention recognition using multimodal physiological signals. However, effectively and quantifiably integrating correlations between electroencephalogram (EEG) and electromyogram (EMG) signal channels as well as within EEG signal channels as a clue to improve performance remained challenging. In this paper, we proposed a novel framework that achieved accurate prediction of upper limb motion intentions via fusing EEG and EMG signals. Firstly, the raw input signals were fed into the feature extraction module, respectively, enabling feature decomposition in the channel dimension. Secondly, the graph convolution module with learnable edge weights was proposed to adaptively learn correlations between different modalities. Thirdly, we designed a self-attention graph pooling module that employed the self-attention mechanism to compute the attention score for each node as the basis for pooling. Compared with calculation methods using the mean or maximum value, this approach was more likely to retain nodes with stronger correlations to motor intentions. Finally, the prediction results were obtained through a classifier. We validated the effectiveness of our method on a publicly available multimodal upper limb dataset, achieving an accuracy of 93.17%.

Key words: motion intention recognition, graph convolution network (GCN), electroencephalogram (EEG), electromyogram (EMG), multi-modality

摘要： 随着传感器和算法的不断进步，越来越多的深度学习方法被应用于利用多模态生理信号进行精细化的上肢运动意图识别。然而，如何有效且可量化地整合脑电（EEG）与肌电（EMG）信号通道之间，以及脑电信号通道内部的相关性，仍然是提升识别性能的一大挑战。本文提出了一种神经网络结构，通过融合脑电和肌电信号，实现对上肢运动意图的准确预测。首先，将原始输入信号输入至特征提取模块，实现通道维度上的特征分解。其次，提出了一种边权可学习的图卷积模块，用于自适应学习不同模态之间的相关性。第三，设计了一种自注意力图池化模块，通过自注意力机制计算每个节点的注意力分数，作为池化的依据；相比采用均值或最大值的计算方法，该方法能更有效地保留与运动意图相关性更强的节点。最后，通过分类器得到最终预测结果。我们在一个公开的多模态上肢数据集上验证了该方法的有效性，准确率达到 93.17%。

关键词: 运动意图识别，图卷积网络，脑电图，肌电图，多模态