Multi-source domain generalization and adaptation toward cross-subject myoelectric pattern recognition

Enhancing and Optimizing User-Machine Closed-Loop Co-Adaptation in Dynamic Myoelectric Interface

Co-adaptation interfaces, developed through user-machine collaboration, have the capacity to transform surface electromyography (sEMG) into control signals, thereby enabling external devices to facilitate or augment the sensory-motor capabilities of individuals with physical disabilities. However, the efficacy and reliability of myoelectric interfaces in untrained environments over extensive spatial range have not been thoroughly explored. We propose a user-machine closed-loop co-adaptation strategy, which consists of a multimodal progressive domain adversarial neural network (MPDANN), an augmented reality (AR) system and a scenario-based dynamic asymmetric training scheme. MPDANN employs both sEMG and Inertial Measurement Unit (IMU) data using dual-domain adversarial training, with the aim of facilitating knowledge transfer and enabling multi-source domain adaptation. The AR system allows users to perform 10 holographic object repositioning tasks in a stereoscopic mixed reality environment using a virtual prosthesis represented as an extension of the residual limb. The scenario-based dynamic asymmetric training scheme, which employs incremental learning in MPDANN and incremental training in the AR system, enables the continuous updating and optimization of the system parameters. A group of non-disable participants and two amputees performed a five-day offline data collection in multiple limb position conditions and a five-day real-time holographic object manipulation task. The average completion rate for subjects utilizing MPDANN reached ${83}.{37}\% \pm {2}.{50}\%$ on the final day, marking a significant improvement compared to the other groups. These findings provide a novel approach to designing myoelectric interfaces with cross-scene recognition through user-machine closed-loop co-adaptation.

Toward Hand Gesture Recognition Using a Channel-Wise Cumulative Spike Train Image-Driven Model

Recognizing hand gestures from neural control signals is essential for natural human-machine interaction, which is extensively applied to prosthesis control and rehabilitation. However, establishing associations between the neural control signals of motor units and gestures remains an open question. Here, we propose a channel-wise cumulative spike train (cw-CST) image-driven model (cwCST-CNN) for hand gesture recognition, leveraging the spatial activation patterns of motor unit firings to distinguish motor intentions. Specifically, the cw-CSTs of motor units were decomposed from high-density surface electromyography using a spatial spike detection algorithm and were further reconstructed into images according to their spatial recording positions. Then, the resultant cwCST-images were fed into a customized convolutional neural network to recognize gestures. Additionally, we conducted an experiment involving 10 gestures and 10 subjects and compared the proposed method with 2 root-mean-square (RMS)-based approaches and a cw-CST-based approach, namely, RMS-image-driven convolutional neural network classification model, RMS feature with linear discrimination analysis classifier, and cw-CST discharge rate feature with linear discrimination analysis classifier. The results demonstrated that cwCST-CNN outperformed the other 3 methods with a higher classification accuracy of 96.92% ± 1.77%. Moreover, analysis of cw-CST and RMS features showed that the former had better separability across gestures and consistency considering training and testing datasets. This study provides a new solution and enhances the accuracy of gesture recognition using neural drive signals in human-machine interaction.

Privacy-preserving multi-source semi-supervised domain adaptation for seizure prediction

Domain adaptation (DA) has been frequently used to solve the inter-patient variability problem in EEG-based seizure prediction. However, existing DA methods require access to the existing patients' data when adapting the model, which leads to privacy concerns. Besides, most of them treat the whole existing patients' data as one single source and attempt to minimize the discrepancy with the target patient. This manner ignores the inter-patient variability among source patients, making the adaptation more difficult. Considering theses issues simultaneously, we present a novel multi-source-free semi-supervised domain adaptive seizure prediction model (MSF-SSDA-SPM). MSF-SSDA-SPM considers each source patient as one single source and generates a pretrained model from each source. Without requiring access to the source data, MSF-SSDA-SPM performs adaptation just using these pretrained source models and limited labeled target data. Specifically, we freeze the classifiers of all the source models and optimize the source feature extractors in a joint manner. Then we design a knowledge distillation strategy to integrate the knowledge of these well-adapted source models into one single target model. On the CHB-MIT dataset, MSF-SSDA-SPM attains a sensitivity of 88.6%, a FPR of 0.182/h and an AUC of 0.856; on the Kaggle dataset, it achieves 78.6%, 0.178/h and 0.784, respectively. Experimental results demonstrate that MSF-SSDA-SPM achieves both high privacy-protection and promising prediction performance.

A Novel Active Learning Framework for Cross-Subject Human Activity Recognition from Surface Electromyography

Wearable sensor-based human activity recognition (HAR) methods hold considerable promise for upper-level control in exoskeleton systems. However, such methods tend to overlook the critical role of data quality and still encounter challenges in cross-subject adaptation. To address this, we propose an active learning framework that integrates the relation network architecture with data sampling techniques. Initially, target data are used to fine tune two auxiliary classifiers of the pre-trained model, thereby establishing subject-specific classification boundaries. Subsequently, we assess the significance of the target data based on classifier discrepancy and partition the data into sample and template sets. Finally, the sampled data and a category clustering algorithm are employed to tune model parameters and optimize template data distribution, respectively. This approach facilitates the adaptation of the model to the target subject, enhancing both accuracy and generalizability. To evaluate the effectiveness of the proposed adaptation framework, we conducted evaluation experiments on a public dataset and a self-constructed electromyography (EMG) dataset. Experimental results demonstrate that our method outperforms the compared methods across all three statistical metrics. Furthermore, ablation experiments highlight the necessity of data screening. Our work underscores the practical feasibility of implementing user-independent HAR methods in exoskeleton control systems.

A Robust and Real-Time Framework of Cross-Subject Myoelectric Control Model Calibration via Multi-Source Domain Adaptation

Surface electromyogram (sEMG) has been widely used in hand gesture recognition. However, most previous studies focused on user-personalized models, which require a great amount of data from each new target user to learn the user-specific EMG patterns. In this work, we present a novel real-time gesture recognition framework based on multi-source domain adaptation, which learns extra knowledge from the data of other users, thereby reducing the data collection burdens on the target user. Additionally, compared with conventional domain adaptation methods which treat data from all users in the source domain as a whole, the proposed multi-source method treat data from different users as multiple separate source domains. Therefore, more detailed statistical information on the data distribution from each user can be learned effectively. High-density sEMG (256 channels) from 20 subjects was used to validate the proposed method. Importantly, we evaluated our method with a simulated real-time processing pipeline on continuous sEMG data stream, rather than well-segmented data. The false alarm rate during rest periods in an EMG data stream, which is typically neglected by previous studies performing offline analyses, was also considered. Our results showed that, with only 1 s sEMG data per gesture from the new user, the 10-gesture classification accuracy reached 87.66 % but the false alarm rate was reduced to 1.95 %. Our method can reduce the frustratingly heavy data collection burdens on each new user.

One-shot random forest model calibration for hand gesture decoding

Objective.Most existing machine learning models for myoelectric control require a large amount of data to learn user-specific characteristics of the electromyographic (EMG) signals, which is burdensome. Our objective is to develop an approach to enable the calibration of a pre-trained model with minimal data from a new myoelectric user.Approach.We trained a random forest (RF) model with EMG data from 20 people collected during the performance of multiple hand grips. To adapt the decision rules for a new user, first, the branches of the pre-trained decision trees were pruned using the validation data from the new user. Then new decision trees trained merely with data from the new user were appended to the pruned pre-trained model.Results.Real-time myoelectric experiments with 18 participants over two days demonstrated the improved accuracy of the proposed approach when compared to benchmark user-specific RF and the linear discriminant analysis models. Furthermore, the RF model that was calibrated on day one for a new participant yielded significantly higher accuracy on day two, when compared to the benchmark approaches, which reflects the robustness of the proposed approach.Significance.The proposed model calibration procedure is completely source-free, that is, once the base model is pre-trained, no access to the source data from the original 20 people is required. Our work promotes the use of efficient, explainable, and simple models for myoelectric control.

Cross-User Electromyography Pattern Recognition Based on a Novel Spatial-Temporal Graph Convolutional Network

With the goal of promoting the development of myoelectric control technology, this paper focuses on exploring graph neural network (GNN) based robust electromyography (EMG) pattern recognition solutions. Given that high-density surface EMG (HD-sEMG) signal contains rich temporal and spatial information, the multi-view spatial-temporal graph convolutional network (MSTGCN)is adopted as the basic classifier, and a feature extraction convolutional neural network (CNN) module is designed and integrated into MSTGCN to generate a new model called CNN-MSTGCN. The EMG pattern recognition experiments are conducted on HD-sEMG data of 17 gestures from 11 subjects. The ablation experiments show that each functional module of the proposed CNN-MSTGCN network has played a more or less positive role in improving the performance of EMG pattern recognition. The user-independent recognition experiments and the transfer learning-based cross-user recognition experiments verify the advantages of the proposed CNN-MSTGCN network in improving recognition rate and reducing user training burden. In the user-independent recognition experiments, CNN-MSTGCN achieves the recognition rate of 68%, which is significantly better than those obtained by residual network-50 (ResNet50, 47.5%, p < 0.001) and long-short-term-memory (LSTM, 57.1%, p=0.045). In the transfer learning-based cross-user recognition experiments, TL-CMSTGCN achieves an impressive recognition rate of 92.3%, which is significantly superior to both TL-ResNet50 (84.6%, p = 0.003) and TL-LSTM (85.3%, p = 0.008). The research results of this paper indicate that GNN has certain advantages in overcoming the impact of individual differences, and can be used to provide possible solutions for achieving robust EMG pattern recognition technology.

SFDA: Domain Adaptation With Source Subject Fusion Based on Multi-Source and Single-Target Fall Risk Assessment

In cross-subject fall risk classification based on plantar pressure, a challenge is that data from different subjects have significant individual information. Thus, the models with insufficient generalization ability can't perform well on new subjects, which limits their application in daily life. To solve this problem, domain adaptation methods are applied to reduce the gap between source and target domain. However, these methods focus on the distribution of the source and the target domain, but ignore the potential correlation among multiple source subjects, which deteriorates domain adaptation performance. In this paper, we proposed a novel method named domain adaptation with subject fusion (SFDA) for fall risk assessment, greatly improving the cross-subject assessment ability. Specifically, SFDA synchronously carries out source target adaptation and multiple source subject fusion by domain adversarial module to reduce source-target gap and distribution distance within source subjects of same class. Consequently, target samples can learn more task-specific features from source subjects to improve the generalization ability. Experiment results show that SFDA achieved mean accuracy of 79.17 % and 73.66 % based on two backbones in a cross-subject classification manner, outperforming the state-of-the-art methods on continuous plantar pressure dataset. This study proves the effectiveness of SFDA and provides a novel tool for implementing cross-subject and few-gait fall risk assessment.