Locomo-Net: A Low -Complex Deep Learning Framework for sEMG-Based Hand Movement Recognition for Prosthetic Control

Hand gestures classification of sEMG signals based on BiLSTM-metaheuristic optimization and hybrid U-Net-MobileNetV2 encoder architecture

Surface electromyography (sEMG) data has been extensively utilized in deep learning algorithms for hand movement classification. This paper aims to introduce a novel method for hand gesture classification using sEMG data, addressing accuracy challenges seen in previous studies. We propose a U-Net architecture incorporating a MobileNetV2 encoder, enhanced by a novel Bidirectional Long Short-Term Memory (BiLSTM) and metaheuristic optimization for spatial feature extraction in hand gesture and motion recognition. Bayesian optimization is employed as the metaheuristic approach to optimize the BiLSTM model's architecture. To address the non-stationarity of sEMG signals, we employ a windowing strategy for signal augmentation within deep learning architectures. The MobileNetV2 encoder and U-Net architecture extract relevant features from sEMG spectrogram images. Edge computing integration is leveraged to further enhance innovation by enabling real-time processing and decision-making closer to the data source. Six standard databases were utilized, achieving an average accuracy of 90.23% with our proposed model, showcasing a 3-4% average accuracy improvement and a 10% variance reduction. Notably, Mendeley Data, BioPatRec DB3, and BioPatRec DB1 surpassed advanced models in their respective domains with classification accuracies of 88.71%, 90.2%, and 88.6%, respectively. Experimental results underscore the significant enhancement in generalizability and gesture recognition robustness. This approach offers a fresh perspective on prosthetic management and human-machine interaction, emphasizing its efficacy in improving accuracy and reducing variance for enhanced prosthetic control and interaction with machines through edge computing integration.

Improved Surface Electromyogram-Based Hand-Wrist Force Estimation Using Deep Neural Networks and Cross-Joint Transfer Learning

Deep neural networks (DNNs) and transfer learning (TL) have been used to improve surface electromyogram (sEMG)-based force estimation. However, prior studies focused mostly on applying TL within one joint, which limits dataset size and diversity. Herein, we investigated cross-joint TL between two upper-limb joints with four DNN architectures using sliding windows. We used two feedforward and two recurrent DNN models with feature engineering and feature learning, respectively. We found that the dependencies between sEMG and force are short-term (<400 ms) and that sliding windows are sufficient to capture them, suggesting that more complicated recurrent structures may not be necessary. Also, using DNN architectures reduced the required sliding window length. A model pre-trained on elbow data was fine-tuned on hand-wrist data, improving force estimation accuracy and reducing the required training data amount. A convolutional neural network with a 391 ms sliding window fine-tuned using 20 s of training data had an error of 6.03 ± 0.49% maximum voluntary torque, which is statistically lower than both our multilayer perceptron model with TL and a linear regression model using 40 s of training data. The success of TL between two distinct joints could help enrich the data available for future deep learning-related studies.

A multimodal framework based on deep belief network for human locomotion intent prediction

Accurate prediction of human locomotion intent benefits the seamless switching of lower limb exoskeleton controllers in different terrains to assist humans in walking safely. In this paper, a deep belief network (DBN) was developed to construct a multimodal framework for recognizing various locomotion modes and predicting transition tasks. Three fusion strategies (data level, feature level, and decision level) were explored, and optimal network performance was obtained. This method could be tested on public datasets. For the continuous performance of steady state, the best prediction accuracy achieved was 97.64% in user-dependent testing and 96.80% in user-independent testing. During the transition state, the system accurately predicted all transitions (user-dependent: 96.37%, user-independent: 95.01%). The multimodal framework based on DBN can accurately predict the human locomotion intent. The experimental results demonstrate the potential of the proposed model in the volition control of the lower limb exoskeleton.

Recognition of Human Lower Limb Motion and Muscle Fatigue Status Using a Wearable FES-sEMG System

Functional electrical stimulation (FES) devices are widely employed for clinical treatment, rehabilitation, and sports training. However, existing FES devices are inadequate in terms of wearability and cannot recognize a user's intention to move or muscle fatigue. These issues impede the user's ability to incorporate FES devices into their daily life. In response to these issues, this paper introduces a novel wearable FES system based on customized textile electrodes. The system is driven by surface electromyography (sEMG) movement intention. A parallel structured deep learning model based on a wearable FES device is used, which enables the identification of both the type of motion and muscle fatigue status without being affected by electrical stimulation. Five subjects took part in an experiment to test the proposed system, and the results showed that our method achieved a high level of accuracy for lower limb motion recognition and muscle fatigue status detection. The preliminary results presented here prove the effectiveness of the novel wearable FES system in terms of recognizing lower limb motions and muscle fatigue status.

Prediction of Gait Kinematics and Kinetics: A Systematic Review of EMG and EEG Signal Use and Their Contribution to Prediction Accuracy

Human-machine interfaces hold promise in enhancing rehabilitation by predicting and responding to subjects' movement intent. In gait rehabilitation, neural network architectures utilize lower-limb muscle and brain activity to predict continuous kinematics and kinetics during stepping and walking. This systematic review, spanning five databases, assessed 16 papers meeting inclusion criteria. Studies predicted lower-limb kinematics and kinetics using electroencephalograms (EEGs), electromyograms (EMGs), or a combination with kinematic data and anthropological parameters. Long short-term memory (LSTM) and convolutional neural network (CNN) tools demonstrated highest accuracies. EEG focused on joint angles, while EMG predicted moments and torque joints. Useful EEG electrode locations included C3, C4, Cz, P3, F4, and F8. Vastus Lateralis, Rectus Femoris, and Gastrocnemius were the most commonly accessed muscles for kinematic and kinetic prediction using EMGs. No studies combining EEGs and EMGs to predict lower-limb kinematics and kinetics during stepping or walking were found, suggesting a potential avenue for future development in this technology.

Continuous Motion Estimation of Knee Joint Based on a Parameter Self-Updating Mechanism Model

Estimation of continuous motion of human joints using surface electromyography (sEMG) signals has a critical part to play in intelligent rehabilitation. Traditional methods always use sEMG signals as inputs to build regression or biomechanical models to estimate continuous joint motion variables. However, it is challenging to accurately estimate continuous joint motion in new subjects due to the non-stationarity and individual differences in sEMG signals, which greatly limits the generalisability of the method. In this paper, a continuous motion estimation model for the human knee joint with a parameter self-updating mechanism based on the fusion of particle swarm optimization (PSO) and deep belief network (DBN) is proposed. According to the original sEMG signals of different subjects, the method adaptively optimized the parameters of the DBN model and completed the optimal reconstruction of signal feature structure in high-dimensional space to achieve the optimal estimation of continuous joint motion. Extensive experiments were conducted on knee joint motions. The results suggested that the average root mean square errors (RMSEs) of the proposed method were 9.42° and 7.36°, respectively, which was better than the results obtained by common neural networks. This finding lays a foundation for the human-robot interaction (HRI) of the exoskeleton robots based on the sEMG signals.

Progressive Rehabilitation Based on EMG Gesture Classification and an MPC-Driven Exoskeleton

Stroke is a leading cause of disability and death worldwide, with a prevalence of 200 millions of cases worldwide. Motor disability is presented in 80% of patients. In this context, physical rehabilitation plays a fundamental role for gradually recovery of mobility. In this work, we designed a robotic hand exoskeleton to support rehabilitation of patients after a stroke episode. The system acquires electromyographic (EMG) signals in the forearm, and automatically estimates the movement intention for five gestures. Subsequently, we developed a predictive adaptive control of the exoskeleton to compensate for three different levels of muscle fatigue during the rehabilitation therapy exercises. The proposed system could be used to assist the rehabilitation therapy of the patients by providing a repetitive, intense, and adaptive assistance.

High-Performance Surface Electromyography Armband Design for Gesture Recognition

Wearable surface electromyography (sEMG) signal-acquisition devices have considerable potential for medical applications. Signals obtained from sEMG armbands can be used to identify a person's intentions using machine learning. However, the performance and recognition capabilities of commercially available sEMG armbands are generally limited. This paper presents the design of a wireless high-performance sEMG armband (hereinafter referred to as the α Armband), which has 16 channels and a 16-bit analog-to-digital converter and can reach 2000 samples per second per channel (adjustable) with a bandwidth of 0.1-20 kHz (adjustable). The α Armband can configure parameters and interact with sEMG data through low-power Bluetooth. We collected sEMG data from the forearms of 30 subjects using the α Armband and extracted three different image samples from the time-frequency domain for training and testing convolutional neural networks. The average recognition accuracy for 10 hand gestures was as high as 98.6%, indicating that the α Armband is highly practical and robust, with excellent development potential.

Estimation of Knee Joint Angle from Surface EMG Using Multiple Kernels Relevance Vector Regression

In wearable robots, the application of surface electromyography (sEMG) signals in motion intention recognition is a hot research issue. To improve the viability of human-robot interactive perception and to reduce the complexity of the knee joint angle estimation model, this paper proposed an estimation model for knee joint angle based on the novel method of multiple kernel relevance vector regression (MKRVR) through offline learning. The root mean square error, mean absolute error, and R2_score are used as performance indicators. By comparing the estimation model of MKRVR and least squares support vector regression (LSSVR), the MKRVR performs better on the estimation of the knee joint angle. The results showed that the MKRVR can estimate the knee joint angle with a continuous global MAE of 3.27° ± 1.2°, RMSE of 4.81° ± 1.37°, and R² of 0.8946 ± 0.07. Therefore, we concluded that the MKRVR for the estimation of the knee joint angle from sEMG is viable and could be used for motion analysis and the application of recognition of the wearer's motion intentions in human-robot collaboration control.

Low-Density sEMG-Based Pattern Recognition of Unrelated Movements Rejection for Wrist Joint Rehabilitation

sEMG-based pattern recognition commonly assumes a limited number of target categories, and the classifiers often predict each target category depending on probability. In wrist rehabilitation training, the patients may make movements that do not belong to the target category unconsciously. However, most pattern recognition methods can only identify limited patterns and are prone to be disturbed by abnormal movement, especially for wrist joint movements. To address the above the problem, a sEMG-based rejection method for unrelated movements is proposed to identify wrist joint unrelated movements using center loss. In this paper, the sEMG signal collected by the Myo armband is used as the input of the sEMG control method. First, the sEMG signal is processed by sliding signal window and image coding. Then, the CNN with center loss and softmax loss is used to describe the spatial information from the sEMG image to extract discriminative features and target movement recognition. Finally, the deep spatial information is used to train the AE to reject unrelated movements based on the reconstruction loss. The results show that the proposed method can realize the target movements recognition and reject unrelated movements with an F-score of 93.4% and a rejection accuracy of 95% when the recall is 0.9, which reveals the effectiveness of the proposed method.

Time–frequency feature transform suite for deep learning-based gesture recognition using sEMG signals

Recently, deep learning methods have achieved considerable performance in gesture recognition using surface electromyography signals. However, improving the recognition accuracy in multi-subject gesture recognition remains a challenging problem. In this study, we aimed to improve recognition performance by adding subject-specific prior knowledge to provide guidance for multi-subject gesture recognition. We proposed a time–frequency feature transform suite (TFFT) that takes the maps generated by continuous wavelet transform (CWT) as input. The TFFT can be connected to a neural network to obtain an end-to-end architecture. Thus, we integrated the suite into traditional neural networks, such as convolutional neural networks and long short-term memory, to adjust the intermediate features. The results of comparative experiments showed that the deep learning models with the TFFT suite based on CWT improved the recognition performance of the original architectures without the TFFT suite in gesture recognition tasks. Our proposed TFFT suite has promising applications in multi-subject gesture recognition and prosthetic control.

ViT-HGR: Vision Transformer-based Hand Gesture Recognition from High Density Surface EMG Signals

Recently, there has been a surge of significant interest on application of Deep Learning (DL) models to autonomously perform hand gesture recognition using surface Electromyogram (sEMG) signals. Many of the existing DL models are, however, designed to be applied on sparse sEMG signals. Furthermore, due to the complex structure of these models, typically, we are faced with memory constraint issues, require large training times and a large number of training samples, and; there is the need to resort to data augmentation and/or transfer learning. In this paper, for the first time (to the best of our knowledge), we investigate and design a Vision Transformer (ViT) based architecture to perform hand gesture recognition from High Density (HD-sEMG) signals. Intuitively speaking, we capitalize on the recent breakthrough role of the transformer architecture in tackling different com-plex problems together with its potential for employing more input parallelization via its attention mechanism. The proposed Vision Transformer-based Hand Gesture Recognition (ViT-HGR) framework can overcome the aforementioned training time problems and can accurately classify a large number of hand gestures from scratch without any need for data augmentation and/or transfer learning. The efficiency of the proposed ViT-HGR framework is evaluated using a recently-released HD-sEMG dataset consisting of 65 isometric hand gestures. Our experiments with 64-sample (31.25 ms) window size yield average test accuracy of 84.62 ± 3.07%, where only 78,210 learnable parameters are utilized in the model. The compact structure of the proposed ViT-based ViT-HGR framework (i.e., having significantly reduced number of trainable parameters) shows great potentials for its practical application for prosthetic control.

Feature Extraction Method of Piano Performance Technique Based on Recurrent Neural Network

In order to solve the problem of low efficiency in traditional feature extraction methods of piano performance techniques, a feature extraction method of piano performance techniques based on recurrent neural network is proposed. Analyze the types of piano playing techniques, and establish the hand model. On this basis, the hand action signals of piano performance are collected from the two aspects of finger key strength and hand action video image. Finally, the feature extraction of piano performance techniques is realized from the time domain and frequency domain. Through the comparison with the traditional extraction method, it is concluded that the extraction efficiency of the optimized design of piano performance technique feature extraction method has been significantly improved, and it has obvious application advantages in the identification of piano performance techniques.

Deep learning for biosignal control: insights from basic to real-time methods with recommendations

Biosignal control is an interaction modality that allows users to interact with electronic devices by decoding the biological signals emanating from the movements or thoughts of the user. This manner of interaction with devices can enhance the sense of agency for users and enable persons suffering from a paralyzing condition to interact with everyday devices that would otherwise be challenging for them to use. It can also improve control of prosthetic devices and exoskeletons by making the interaction feel more natural and intuitive. However, with the current state of the art, several issues still need to be addressed to reliably decode user intent from biosignals and provide an improved user experience over other interaction modalities. One solution is to leverage advances in Deep Learning (DL) methods to provide more reliable decoding at the expense of added computational complexity. This scoping review introduces the basic concepts of DL and assists readers in deploying DL methods to a real-time control system that should operate under real-world conditions. The scope of this review covers any electronic device, but with an emphasis on robotic devices, as this is the most active area of research in biosignal control. We review the literature pertaining to the implementation and evaluation of control systems that incorporate DL to identify the main gaps and issues in the field, and formulate suggestions on how to mitigate them. Additionally, we formulate guidelines on the best approach to designing, implementing and evaluating research prototypes that use DL in their biosignal control systems.

Prediction of muscular paralysis disease based on hybrid feature extraction with machine learning technique for COVID-19 and post-COVID-19 patients

Many Coronavirus disease 2019 (COVID-19) and post-COVID-19 patients experience muscle fatigues. Early detection of muscle fatigue and muscular paralysis helps in the diagnosis, prediction, and prevention of COVID-19 and post-COVID-19 patients. Nowadays, the biomedical and clinical domains widely used the electromyography (EMG) signal due to its ability to differentiate various neuromuscular diseases. In general, nerves or muscles and the spinal cord influence numerous neuromuscular disorders. The clinical examination plays a major role in early finding and diagnosis of these diseases; this research study focused on the prediction of muscular paralysis using EMG signals. Machine learning-based diagnosis of the diseases has been widely used due to its efficiency and the hybrid feature extraction (FE) methods with deep learning classifier are used for the muscular paralysis disease prediction. The discrete wavelet transform (DWT) method is applied to decompose the EMG signal and reduce feature degradation. The proposed hybrid FE method consists of Yule-Walker, Burg's method, Renyi entropy, mean absolute value, min-max voltage FE, and other 17 conventional features for prediction of muscular paralysis disease. The hybrid FE method has the advantage of extract the relevant features from the signals and the Relief-F feature selection (FS) method is applied to select the optimal relevant feature for the deep learning classifier. The University of California, Irvine (UCI), EMG-Lower Limb Dataset is used to determine the performance of the proposed classifier. The evaluation shows that the proposed hybrid FE method achieved 88% of precision, while the existing neural network (NN) achieved 65% of precision and the support vector machine (SVM) achieved 35% of precision on whole EMG signal.

Design of Mirror Therapy System Base on Multi-Channel Surface-Electromyography Signal Pattern Recognition and Mobile Augmented Reality

Numerous studies have proven that the mirror therapy can make rehabilitation more effective on hemiparesis following a stroke. Using surface electromyography (SEMG) to predict gesture presents one of the important subjects in related research areas, including rehabilitation medicine, sports medicine, prosthetic control, and so on. However, current signal analysis methods still fail to achieve accurate recognition of multimode motion in a very reliable way due to the weak physiological signal and low noise-ratio. In this paper, a mirror therapy system based on multi-channel SEMG signal pattern recognition and mobile augmented reality is studied. Besides, wavelet transform method is designed to mitigate the noise. The spectrogram obtained by analyzing electromyography signals is proposed to be used as an image. Two approaches, including Convolutional Neural Network (CNN) and grid-optimized Support Vector Machine (SVM), are designed to classify the SEMG of different gestures. The mobile augmented reality provides a virtual hand movement in the real environment to perform mirror therapy process. The experimental results show that the overall accuracy of SVM is 93.07%, and that of CNN is up to 97.8%.

sEMG-Based Neural Network Prediction Model Selection of Gesture Fatigue and Dataset Optimization

The fatigue energy consumption of independent gestures can be obtained by calculating the power spectrum of surface electromyography (sEMG) signals. The existing research studies focus on the fatigue of independent gestures, while the research studies on integrated gestures are few. However, the actual gesture operation mode is usually integrated by multiple independent gestures, so the fatigue degree of integrated gestures can be predicted by training neural network of independent gestures. Three natural gestures including browsing information, playing games, and typing are divided into nine independent gestures in this paper, and the predicted model is established and trained by calculating the energy consumption of independent gestures. The artificial neural networks (ANNs) including backpropagation (BP) neural network, recurrent neural network (RNN), and long short-term memory (LSTM) are used to predict the fatigue of gesture. The support vector machine (SVM) is used to assist verification. Mean square error (MSE), root mean square error (RMSE), and mean absolute error (MAE) are utilized to evaluate the optimal prediction model. Furthermore, the different datasets of the processed sEMG signal and its decomposed wavelet coefficients are trained, respectively, and the changes of error functions of them are compared. The experimental results show that LSTM model is more suitable for gesture fatigue prediction. The processed sEMG signals are appropriate for using as the training set the fatigue degree of one-handed gesture. It is better to use wavelet decomposition coefficients as datasets to predict the high-dimensional sEMG signals of two-handed gestures. The experimental results can be applied to predict the fatigue degree of complex human-machine interactive gestures, help to avoid unreasonable gestures, and improve the user's interactive experience.