Continuous reach-to-grasp motion recognition based on an extreme learning machine algorithm using sEMG signals

Recognizing user intention in reach-to-grasp motions is a critical challenge in rehabilitation engineering. To address this, a Machine Learning (ML) algorithm based on the Extreme Learning Machine (ELM) was developed for identifying motor actions using surface Electromyography (sEMG) during continuous reach-to-grasp movements, involving multiple Degrees of Freedom (DoFs). This study explores feature extraction methods based on time domain and autoregressive models to evaluate ELM performance under different conditions. The experimental setup encompassed variations in neuron size, time windows, validation with each muscle, increase in the number of features, comparison with five conventional ML-based classifiers, inter-subjects variability, and temporal dynamic response. To evaluate the efficacy of the proposed ELM-based method, an openly available sEMG dataset containing data from 12 participants was used. Results highlight the method's performance, achieving Accuracy above 85%, F-score above 90%, Recall above 85%, Area Under the Curve of approximately 84% and compilation times (computational cost) of less than 1 ms. These metrics significantly outperform standard methods (p < 0.05). Additionally, specific trends were found in increasing and decreasing performance in identifying specific tasks, as well as variations in the continuous transitions in the temporal dynamics response. Thus, the ELM-based method effectively identifies continuous reach-to-grasp motions through myoelectric data. These findings hold promise for practical applications. The method's success prompts future research into implementing it for more reliable and effective Human-Machine Interface (HMI) control. This can revolutionize real-time upper limb rehabilitation, enabling natural and complex Activities of Daily Living (ADLs) like object manipulation. The robust results encourages further research and innovative solutions to improve people's quality of life through more effective interventions.

Phasor-Based Myoelectric Synergy Features: A Fast Hand-Crafted Feature Extraction Scheme for Boosting Performance in Gait Phase Recognition

Gait phase recognition systems based on surface electromyographic signals (EMGs) are crucial for developing advanced myoelectric control schemes that enhance the interaction between humans and lower limb assistive devices. However, machine learning models used in this context, such as Linear Discriminant Analysis (LDA) and Support Vector Machine (SVM), typically experience performance degradation when modeling the gait cycle with more than just stance and swing phases. This study introduces a generalized phasor-based feature extraction approach (PHASOR) that captures spatial myoelectric features to improve the performance of LDA and SVM in gait phase recognition. A publicly available dataset of 40 subjects was used to evaluate PHASOR against state-of-the-art feature sets in a five-phase gait recognition problem. Additionally, fully data-driven deep learning architectures, such as Rocket and Mini-Rocket, were included for comparison. The separability index (SI) and mean semi-principal axis (MSA) analyses showed mean SI and MSA metrics of 7.7 and 0.5, respectively, indicating the proposed approach's ability to effectively decode gait phases through EMG activity. The SVM classifier demonstrated the highest accuracy of 82% using a five-fold leave-one-trial-out testing approach, outperforming Rocket and Mini-Rocket. This study confirms that in gait phase recognition based on EMG signals, novel and efficient muscle synergy information feature extraction schemes, such as PHASOR, can compete with deep learning approaches that require greater processing time for feature extraction and classification.

Leveraging Motor Unit Spatial Activation Patterns for Channel Selection in Finger Force Regression

Individual finger forces can be predicted by regression of high-density surface electromyography (sEMG) signals. This is promising for applications in human-machine interfaces, specifically prosthesis control, although the large number of electrodes imposes high computational requirements. In this study, we present strategies for a-priori channel selection guided by motor unit spatial activation patterns to reduce computational costs without compromising decoding accuracy. In contrast to subject-specific data-driven selection, we test the hypothesis that pre-selecting sEMG channels for finger-specific force estimation can still generalize across subjects. We show that a subset of 32 channels, out of a total of 256, achieves an RMSE of 6.32 ± 2.34 % of the Maximum Voluntary Contraction (MVC) on the HYSER RANDOM dataset, competitive with the state-of-the-art baseline model, using all channels, which attains an RMSE of 5.57 ± 1.94 % MVC. These results highlight the potential of simple, a-priori channel selection strategies in decoding finger forces from sEMG, which would be particularly suited for applications with limited computational resources.

Intelligent Human-Computer Interaction: Combined Wrist and Forearm Myoelectric Signals for Handwriting Recognition

Recent studies have highlighted the possibility of using surface electromyographic (EMG) signals to develop human-computer interfaces that are also able to recognize complex motor tasks involving the hand as the handwriting of digits. However, the automatic recognition of words from EMG information has not yet been studied. The aim of this study is to investigate the feasibility of using combined forearm and wrist EMG probes for solving the handwriting recognition problem of 30 words with consolidated machine-learning techniques and aggregating state-of-the-art features extracted in the time and frequency domains. Six healthy subjects, three females and three males aged between 25 and 40 years, were recruited for the study. Two tests in pattern recognition were conducted to assess the possibility of classifying fine hand movements through EMG signals. The first test was designed to assess the feasibility of using consolidated myoelectric control technology with shallow machine-learning methods in the field of handwriting detection. The second test was implemented to assess if specific feature extraction schemes can guarantee high performances with limited complexity of the processing pipeline. Among support vector machine, linear discriminant analysis, and K-nearest neighbours (KNN), the last one showed the best classification performances in the 30-word classification problem, with a mean accuracy of 95% and 85% when using all the features and a specific feature set known as TDAR, respectively. The obtained results confirmed the validity of using combined wrist and forearm EMG data for intelligent handwriting recognition through pattern recognition approaches in real scenarios.

Study on Gesture Recognition Method with Two-Stream Residual Network Fusing sEMG Signals and Acceleration Signals

Currently, surface EMG signals have a wide range of applications in human-computer interaction systems. However, selecting features for gesture recognition models based on traditional machine learning can be challenging and may not yield satisfactory results. Considering the strong nonlinear generalization ability of neural networks, this paper proposes a two-stream residual network model with an attention mechanism for gesture recognition. One branch processes surface EMG signals, while the other processes hand acceleration signals. Segmented networks are utilized to fully extract the physiological and kinematic features of the hand. To enhance the model's capacity to learn crucial information, we introduce an attention mechanism after global average pooling. This mechanism strengthens relevant features and weakens irrelevant ones. Finally, the deep features obtained from the two branches of learning are fused to further improve the accuracy of multi-gesture recognition. The experiments conducted on the NinaPro DB2 public dataset resulted in a recognition accuracy of 88.25% for 49 gestures. This demonstrates that our network model can effectively capture gesture features, enhancing accuracy and robustness across various gestures. This approach to multi-source information fusion is expected to provide more accurate and real-time commands for exoskeleton robots and myoelectric prosthetic control systems, thereby enhancing the user experience and the naturalness of robot operation.

Transferable non-invasive modal fusion-transformer (NIMFT) for end-to-end hand gesture recognition

Objective.Recent studies have shown that integrating inertial measurement unit (IMU) signals with surface electromyographic (sEMG) can greatly improve hand gesture recognition (HGR) performance in applications such as prosthetic control and rehabilitation training. However, current deep learning models for multimodal HGR encounter difficulties in invasive modal fusion, complex feature extraction from heterogeneous signals, and limited inter-subject model generalization. To address these challenges, this study aims to develop an end-to-end and inter-subject transferable model that utilizes non-invasively fused sEMG and acceleration (ACC) data.Approach.The proposed non-invasive modal fusion-transformer (NIMFT) model utilizes 1D-convolutional neural networks-based patch embedding for local information extraction and employs a multi-head cross-attention (MCA) mechanism to non-invasively integrate sEMG and ACC signals, stabilizing the variability induced by sEMG. The proposed architecture undergoes detailed ablation studies after hyperparameter tuning. Transfer learning is employed by fine-tuning a pre-trained model on new subject and a comparative analysis is performed between the fine-tuning and subject-specific model. Additionally, the performance of NIMFT is compared to state-of-the-art fusion models.Main results.The NIMFT model achieved recognition accuracies of 93.91%, 91.02%, and 95.56% on the three action sets in the Ninapro DB2 dataset. The proposed embedding method and MCA outperformed the traditional invasive modal fusion transformer by 2.01% (embedding) and 1.23% (fusion), respectively. In comparison to subject-specific models, the fine-tuning model exhibited the highest average accuracy improvement of 2.26%, achieving a final accuracy of 96.13%. Moreover, the NIMFT model demonstrated superiority in terms of accuracy, recall, precision, and F1-score compared to the latest modal fusion models with similar model scale.Significance.The NIMFT is a novel end-to-end HGR model, utilizes a non-invasive MCA mechanism to integrate long-range intermodal information effectively. Compared to recent modal fusion models, it demonstrates superior performance in inter-subject experiments and offers higher training efficiency and accuracy levels through transfer learning than subject-specific approaches.

Alignment-Enhanced Interactive Fusion Model for Complete and Incomplete Multimodal Hand Gesture Recognition

Hand gesture recognition (HGR) based on surface electromyogram (sEMG) and Accelerometer (ACC) signals is increasingly attractive where fusion strategies are crucial for performance and remain challenging. Currently, neural network-based fusion methods have gained superior performance. Nevertheless, these methods typically fuse sEMG and ACC either in the early or late stages, overlooking the integration of entire cross-modal hierarchical information within each individual hidden layer, thus inducing inefficient inter-modal fusion. To this end, we propose a novel Alignment-Enhanced Interactive Fusion (AiFusion) model, which achieves effective fusion via a progressive hierarchical fusion strategy. Notably, AiFusion can flexibly perform both complete and incomplete multimodal HGR. Specifically, AiFusion contains two unimodal branches and a cascaded transformer-based multimodal fusion branch. The fusion branch is first designed to adequately characterize modality-interactive knowledge by adaptively capturing inter-modal similarity and fusing hierarchical features from all branches layer by layer. Then, the modality-interactive knowledge is aligned with that of unimodality using cross-modal supervised contrastive learning and online distillation from embedding and probability spaces respectively. These alignments further promote fusion quality and refine modality-specific representations. Finally, the recognition outcomes are set to be determined by available modalities, thus contributing to handling the incomplete multimodal HGR problem, which is frequently encountered in real-world scenarios. Experimental results on five public datasets demonstrate that AiFusion outperforms most state-of-the-art benchmarks in complete multimodal HGR. Impressively, it also surpasses the unimodal baselines in the challenging incomplete multimodal HGR. The proposed AiFusion provides a promising solution to realize effective and robust multimodal HGR-based interfaces.

Transformer-based hand gesture recognition from instantaneous to fused neural decomposition of high-density EMG signals

Designing efficient and labor-saving prosthetic hands requires powerful hand gesture recognition algorithms that can achieve high accuracy with limited complexity and latency. In this context, the paper proposes a Compact Transformer-based Hand Gesture Recognition framework referred to as [Formula: see text], which employs a vision transformer network to conduct hand gesture recognition using high-density surface EMG (HD-sEMG) signals. Taking advantage of the attention mechanism, which is incorporated into the transformer architectures, our proposed [Formula: see text] framework overcomes major constraints associated with most of the existing deep learning models such as model complexity; requiring feature engineering; inability to consider both temporal and spatial information of HD-sEMG signals, and requiring a large number of training samples. The attention mechanism in the proposed model identifies similarities among different data segments with a greater capacity for parallel computations and addresses the memory limitation problems while dealing with inputs of large sequence lengths. [Formula: see text] can be trained from scratch without any need for transfer learning and can simultaneously extract both temporal and spatial features of HD-sEMG data. Additionally, the [Formula: see text] framework can perform instantaneous recognition using sEMG image spatially composed from HD-sEMG signals. A variant of the [Formula: see text] is also designed to incorporate microscopic neural drive information in the form of Motor Unit Spike Trains (MUSTs) extracted from HD-sEMG signals using Blind Source Separation (BSS). This variant is combined with its baseline version via a hybrid architecture to evaluate potentials of fusing macroscopic and microscopic neural drive information. The utilized HD-sEMG dataset involves 128 electrodes that collect the signals related to 65 isometric hand gestures of 20 subjects. The proposed [Formula: see text] framework is applied to 31.25, 62.5, 125, 250 ms window sizes of the above-mentioned dataset utilizing 32, 64, 128 electrode channels. Our results are obtained via 5-fold cross-validation by first applying the proposed framework on the dataset of each subject separately and then, averaging the accuracies among all the subjects. The average accuracy over all the participants using 32 electrodes and a window size of 31.25 ms is 86.23%, which gradually increases till reaching 91.98% for 128 electrodes and a window size of 250 ms. The [Formula: see text] achieves accuracy of 89.13% for instantaneous recognition based on a single frame of HD-sEMG image. The proposed model is statistically compared with a 3D Convolutional Neural Network (CNN) and two different variants of Support Vector Machine (SVM) and Linear Discriminant Analysis (LDA) models. The accuracy results for each of the above-mentioned models are paired with their precision, recall, F1 score, required memory, and train/test times. The results corroborate effectiveness of the proposed [Formula: see text] framework compared to its counterparts.

Research on Lower Limb Step Speed Recognition Method Based on Electromyography

Wearable exoskeletons play an important role in people's lives, such as helping stroke and amputation patients to carry out rehabilitation training and so on. How to make the exoskeleton accurately judge the human action intention is the basic requirement to ensure that it can complete the corresponding task. Traditional exoskeleton control signals include pressure values, joint angles and acceleration values, which can only reflect the current motion information of the human lower limbs and cannot be used to predict motion. The electromyography (EMG) signal always occurs before a certain movement; it can be used to predict the target's gait speed and movement as the input signal. In this study, the generalization ability of a BP neural network and the timing property of a hidden Markov chain are used to properly fuse the two, and are finally used in the research of this paper. Experiments show that, using the same training samples, the recognition accuracy of the three-layer BP neural network is only 91%, while the recognition accuracy of the fusion discriminant model proposed in this paper can reach 95.1%. The results show that the fusion of BP neural network and hidden Markov chain has a strong solving ability for the task of wearable exoskeleton recognition of target step speed.

Feature–Classifier Pairing Compatibility for sEMG Signals in Hand Gesture Recognition under Joint Effects of Processing Procedures

Gesture recognition using surface electromyography (sEMG) serves many applications, from human–machine interfaces to prosthesis control. Many features have been adopted to enhance recognition accuracy. However, studies mostly compare features under a prechosen feature window size or a classifier, biased to a specific application. The bias is evident in the reported accuracy drop, around 10%, from offline gesture recognition in experiment settings to real-time clinical environment studies. This paper explores the feature–classifier pairing compatibility for sEMG. We demonstrate that it is the primary determinant of gesture recognition accuracy under various window sizes and normalization ranges, thus removing application bias. The proposed pairing ranking provides a guideline for choosing the proper feature or classifier in future research. For instance, random forest (RF) performed best, with a mean accuracy of around 74.0%; however, it was optimal with the mean absolute value feature (MAV), giving 86.8% accuracy. Additionally, our ranking showed that the proper pairing enables low-computational models to surpass complex ones. The Histogram feature with linear discriminant analysis classifier (HIST-LDA) was the top pair with 88.6% accuracy. We also concluded that a 1250 ms window and a (−1, 1) signal normalization were the optimal procedures for gesture recognition on the used dataset.

Internet of Things for beyond-the-laboratory prosthetics research

Research on upper-limb prostheses is typically laboratory-based. Evidence indicates that research has not yet led to prostheses that meet user needs. Inefficient communication loops between users, clinicians and manufacturers limit the amount of quantitative and qualitative data that researchers can use in refining their innovations. This paper offers a first demonstration of an alternative paradigm by which remote, beyond-the-laboratory prosthesis research according to user needs is feasible. Specifically, the proposed Internet of Things setting allows remote data collection, real-time visualization and prosthesis reprogramming through Wi-Fi and a commercial cloud portal. Via a dashboard, the user can adjust the configuration of the device and append contextual information to the prosthetic data. We evaluated this demonstrator in real-time experiments with three able-bodied participants. Results promise the potential of contextual data collection and system update through the internet, which may provide real-life data for algorithm training and reduce the complexity of send-home trials. This article is part of the theme issue 'Advanced neurotechnologies: translating innovation for health and well-being'.

ViT-HGR: Vision Transformer-based Hand Gesture Recognition from High Density Surface EMG Signals

Recently, there has been a surge of significant interest on application of Deep Learning (DL) models to autonomously perform hand gesture recognition using surface Electromyogram (sEMG) signals. Many of the existing DL models are, however, designed to be applied on sparse sEMG signals. Furthermore, due to the complex structure of these models, typically, we are faced with memory constraint issues, require large training times and a large number of training samples, and; there is the need to resort to data augmentation and/or transfer learning. In this paper, for the first time (to the best of our knowledge), we investigate and design a Vision Transformer (ViT) based architecture to perform hand gesture recognition from High Density (HD-sEMG) signals. Intuitively speaking, we capitalize on the recent breakthrough role of the transformer architecture in tackling different com-plex problems together with its potential for employing more input parallelization via its attention mechanism. The proposed Vision Transformer-based Hand Gesture Recognition (ViT-HGR) framework can overcome the aforementioned training time problems and can accurately classify a large number of hand gestures from scratch without any need for data augmentation and/or transfer learning. The efficiency of the proposed ViT-HGR framework is evaluated using a recently-released HD-sEMG dataset consisting of 65 isometric hand gestures. Our experiments with 64-sample (31.25 ms) window size yield average test accuracy of 84.62 ± 3.07%, where only 78,210 learnable parameters are utilized in the model. The compact structure of the proposed ViT-based ViT-HGR framework (i.e., having significantly reduced number of trainable parameters) shows great potentials for its practical application for prosthetic control.

The Effectiveness of Narrowing the Window size for LD & HD EMG Channels based on Novel Deep Learning Wavelet Scattering Transform Feature Extraction Approach

The use of the Electromyogram (EMG) signals as a source of control to command externally powered prostheses is often challenged by the signal complexity and non-stationary behavior. Mainly, two factors affect classification accuracy: selecting the optimum feature extraction methods and overlapping segmentation/window size. Nowadays, studies attempt to use deep learning (DL) methods to improve classification accuracy. However, DL models are frequently hampered by their requirements of a vast quantity of training data to attain decent performance and the high computing costs. Therefore, researchers tried to replace the deep learning models with other low computational cost methods like deep wavelet scattering transform (DWST) as a feature extraction technique. In terms of windows size, selecting a larger window size increases the classification accuracy, but at the same time, it increases the processing time, which makes the system unsuitable for real-time applications. Accordingly, researchers attempted to minimise the size of the overlapping windows as much as possible without impacting classification performance. This work suggests to utilise DWST transform to achieve two goals (a) extracting the features from EMG signal with low computational cost. Even though many studies have used DWST approaches to extract features from other biological signals, but not been examined before for EMG signals. (b) study the effect of extracting the features from high-density EMG datasets (HD EMG) and low-density EMG datasets (LD EMG), reducing the analysis window size by up to 32msec with minimal impact on classification performance. The outcomes of the proposed method are compared with other well-known feature extraction algorithms to validate these achievements. The proposed strategy exceeds other methods by more than 25% in accuracy.

CSAC-Net: Fast Adaptive sEMG Recognition through Attention Convolution Network and Model-Agnostic Meta-Learning

Gesture recognition through surface electromyography (sEMG) provides a new method for the control algorithm of bionic limbs, which is a promising technology in the field of human–computer interaction. However, subject specificity of sEMG along with the offset of the electrode makes it challenging to develop a model that can quickly adapt to new subjects. In view of this, we introduce a new deep neural network called CSAC-Net. Firstly, we extract the time-frequency feature from the raw signal, which contains rich information. Secondly, we design a convolutional neural network supplemented by an attention mechanism for further feature extraction. Additionally, we propose to utilize model-agnostic meta-learning to adapt to new subjects and this learning strategy achieves better results than the state-of-the-art methods. By the basic experiment on CapgMyo and three ablation studies, we demonstrate the advancement of CSAC-Net.