Multi-feature gait recognition with DNN based on sEMG signals

A Novel TCN-LSTM Hybrid Model for sEMG-Based Continuous Estimation of Wrist Joint Angles

Surface electromyography (sEMG) offers a novel method in human-machine interactions (HMIs) since it is a distinct physiological electrical signal that conceals human movement intention and muscle information. Unfortunately, the nonlinear and non-smooth features of sEMG signals often make joint angle estimation difficult. This paper proposes a joint angle prediction model for the continuous estimation of wrist motion angle changes based on sEMG signals. The proposed model combines a temporal convolutional network (TCN) with a long short-term memory (LSTM) network, where the TCN can sense local information and mine the deeper information of the sEMG signals, while LSTM, with its excellent temporal memory capability, can make up for the lack of the ability of the TCN to capture the long-term dependence of the sEMG signals, resulting in a better prediction. We validated the proposed method in the publicly available Ninapro DB1 dataset by selecting the first eight subjects and picking three types of wrist-dependent movements: wrist flexion (WF), wrist ulnar deviation (WUD), and wrist extension and closed hand (WECH). Finally, the proposed TCN-LSTM model was compared with the TCN and LSTM models. The proposed TCN-LSTM outperformed the TCN and LSTM models in terms of the root mean square error (RMSE) and average coefficient of determination (R2). The TCN-LSTM model achieved an average RMSE of 0.064, representing a 41% reduction compared to the TCN model and a 52% reduction compared to the LSTM model. The TCN-LSTM also achieved an average R2 of 0.93, indicating an 11% improvement over the TCN model and an 18% improvement over the LSTM model.

Revolutionizing motor dysfunction treatment: A novel closed-loop electrical stimulator guided by multiple motor tasks with predictive control

Functional electrical stimulation (FES) has been demonstrated as a viable method for addressing motor dysfunction in individuals affected by stroke, spinal cord injury, and other etiologies. By eliciting muscle contractions to facilitate joint movements, FES plays a crucial role in fostering the restoration of motor function compromised nervous system. In response to the challenge of muscle fatigue associated with conventional FES protocols, a novel biofeedback electrical stimulator incorporating multi-motor tasks and predictive control algorithms has been developed to enable adaptive modulation of stimulation parameters. The study initially establishes a Hammerstein model for the stimulated muscle group, representing a time-varying relationship between the stimulation pulse width and the root mean square (RMS) of the surface electromyography (sEMG). An online parameter identification algorithm utilizing recursive least squares is employed to estimate the time-varying parameters of the Hammerstein model. Predictive control is then implemented through feedback corrections based on the comparison between predicted and actual outputs, guided by an optimization objective function. The integration of predictive control and roll optimization enables closed-loop control of muscle stimulation. The motor training tasks of elbow flexion and extension, wrist flexion and extension, and five-finger grasping were selected for experimental validation. The results indicate that the model parameters were accurately identified, with a RMS error of 3.83 % between actual and predicted values. Furthermore, the predictive control algorithm, based on the motor tasks, effectively adjusted the stimulus parameters to ensure that the stimulated muscle groups can achieve the desired sEMG characteristic trajectory. The biofeedback electrical stimulator that was developed has the potential to assist patients experiencing motor dysfunction in achieving the appropriate joint movements. This research provides a foundation for a novel intelligent electrical stimulation model.

Ensemble Learning Method for the Continuous Decoding of Hand Joint Angles

Human-machine interface technology is fundamentally constrained by the dexterity of motion decoding. Simultaneous and proportional control can greatly improve the flexibility and dexterity of smart prostheses. In this research, a new model using ensemble learning to solve the angle decoding problem is proposed. Ultimately, seven models for angle decoding from surface electromyography (sEMG) signals are designed. The kinematics of five angles of the metacarpophalangeal (MCP) joints are estimated using the sEMG recorded during functional tasks. The estimation performance was evaluated through the Pearson correlation coefficient (CC). In this research, the comprehensive model, which combines CatBoost and LightGBM, is the best model for this task, whose average CC value and RMSE are 0.897 and 7.09. The mean of the CC and the mean of the RMSE for all the test scenarios of the subjects' dataset outperform the results of the Gaussian process model, with significant differences. Moreover, the research proposed a whole pipeline that uses ensemble learning to build a high-performance angle decoding system for the hand motion recognition task. Researchers or engineers in this field can quickly find the most suitable ensemble learning model for angle decoding through this process, with fewer parameters and fewer training data requirements than traditional deep learning models. In conclusion, the proposed ensemble learning approach has the potential for simultaneous and proportional control (SPC) of future hand prostheses.

Exploring the EMG transient: the muscular activation sequences used as novel time-domain features for hand gestures classification

Introduction

Muscular activation sequences have been shown to be suitable time-domain features for classification of motion gestures. However, their clinical application in myoelectric prosthesis control was never investigated so far. The aim of the paper is to evaluate the robustness of these features extracted from the EMG signal in transient state, on the forearm, for classifying common hand tasks.

Methods

The signal associated to four hand gestures and the rest condition were acquired from ten healthy people and two persons with trans-radial amputation. A feature extraction algorithm allowed for encoding the EMG signals into muscular activation sequences, which were used to train four commonly used classifiers, namely Linear Discriminant Analysis (LDA), Support Vector Machine (SVM), Non-linear Logistic Regression (NLR) and Artificial Neural Network (ANN). The offline performances were assessed with the entire sample of recruited people. The online performances were assessed with the amputee subjects. Moreover, a comparison of the proposed method with approaches based on the signal envelope in the transient state and in the steady state was conducted.

Results

The highest performance were obtained with the NLR classifier. Using the sequences, the offline classification accuracy was higher than 93% for healthy and amputee subjects and always higher than the approach with the signal envelope in transient state. As regards the comparison with the steady state, the performances obtained with the proposed method are slightly lower (<4%), but the classification occurred at least 200 ms earlier. In the online application, the motion completion rate reached up to 85% of the total classification attempts, with a motion selection time that never exceeded 218 ms.

Discussion

Muscular activation sequences are suitable alternatives to the time-domain features commonly used in classification problems belonging to the sole EMG transient state and could be potentially exploited in control strategies of myoelectric prosthesis hands.

A gait stability evaluation method based on wearable acceleration sensors

In this study, an accurate tool is provided for the evaluation of the effect of joint motion effect on gait stability. This quantitative gait evaluation method relies exclusively on the analysis of data acquired using acceleration sensors. First, the acceleration signal of lower limb motion is collected dynamically in real-time through the acceleration sensor. Second, an algorithm based on improved dynamic time warping (DTW) is proposed and used to calculate the gait stability index of the lower limbs. Finally, the effects of different joint braces on gait stability are analyzed. The experimental results show that the joint brace at the ankle and the knee reduces the range of motions of both ankle and knee joints, and a certain impact is exerted on the gait stability. In comparison to the ankle joint brace, the knee joint brace inflicts increased disturbance on the gait stability. Compared to the joint motion of the braced side, which showed a large deviation, the joint motion of the unbraced side was more similar to that of the normal walking process. In this paper, the quantitative evaluation algorithm based on DTW makes the results more intuitive and has potential application value in the evaluation of lower limb dysfunction, clinical training and rehabilitation.

Mechanomyography Signal Pattern Recognition of Knee and Ankle Movements Using Swarm Intelligence Algorithm-Based Feature Selection Methods

Pattern recognition of lower-limb movements based on mechanomyography (MMG) signals has a certain application value in the study of wearable rehabilitation-training devices. In this paper, MMG feature selection methods based on a chameleon swarm algorithm (CSA) and a grasshopper optimization algorithm (GOA) are proposed for the pattern recognition of knee and ankle movements in the sitting and standing positions. Wireless multichannel MMG acquisition systems were designed and used to collect MMG movements from four sites on the subjects thighs. The relationship between the threshold values and classification accuracy was analyzed, and comparatively high recognition rates were obtained after redundant information was eliminated. When the threshold value rose, the recognition rates from the CSA fluctuated within a small range: up to 88.17% (sitting position) and 90.07% (standing position). However, the recognition rates from the GOA drop dramatically when increasing the threshold value. The comparison results demonstrated that using a GOA consumes less time and selects fewer features, while a CSA gives higher recognition rates of knee and ankle movements.

Online detection of compensatory strategies in human movement with supervised classification: a pilot study

Introduction

Stroke survivors often compensate for the loss of motor function in their distal joints by altered use of more proximal joints and body segments. Since this can be detrimental to the rehabilitation process in the long-term, it is imperative that such movements are indicated to the patients and their caregiver. This is a difficult task since compensation strategies are varied and multi-faceted. Recent works that have focused on supervised machine learning methods for compensation detection often require a large training dataset of motions with compensation location annotations for each time-step of the recorded motion. In contrast, this study proposed a novel approach that learned a linear classifier from energy-based features to discriminate between healthy and compensatory movements and identify the compensating joints without the need for dense and explicit annotations.

Methods

Six healthy physiotherapists performed five different tasks using healthy movements and acted compensations. The resulting motion capture data was transformed into joint kinematic and dynamic trajectories. Inspired by works in bio-mechanics, energy-based features were extracted from this dataset. Support vector machine (SVM) and logistic regression (LR) algorithms were then applied for detection of compensatory movements. For compensating joint identification, an additional condition enforcing the independence of the feature calculation for each observable degree of freedom was imposed.

Results

Using leave-one-out cross validation, low values of mean brier score (<0.15), mis-classification rate (<0.2) and false discovery rate (<0.2) were obtained for both SVM and LR classifiers. These methods were found to outperform deep learning classifiers that did not use energy-based features. Additionally, online classification performance by our methods were also shown to outperform deep learning baselines. Furthermore, qualitative results obtained from the compensation joint identification experiment indicated that the method could successfully identify compensating joints.

Discussion

Results from this study indicated that including prior bio-mechanical information in the form of energy based features can improve classification performance even when linear classifiers are used, both for offline and online classification. Furthermore, evaluation compensation joint identification algorithm indicated that it could potentially provide a straightforward and interpretable way of identifying compensating joints, as well as the degree of compensation being performed.

Recent advancements in digital health management using multi-modal signal monitoring

Healthcare is the method of keeping or enhancing physical and mental well-being with its aid of illness and injury prevention, diagnosis, and treatment. The majority of conventional healthcare practices involve manual management and upkeep of client demographic information, case histories, diagnoses, medications, invoicing, and drug stock upkeep, which can result in human errors that have an impact on clients. By linking all the essential parameter monitoring equipment through a network with a decision-support system, digital health management based on Internet of Things (IoT) eliminates human errors and aids the doctor in making more accurate and timely diagnoses. The term "Internet of Medical Things" (IoMT) refers to medical devices that have the ability to communicate data over a network without requiring human-to-human or human-to-computer interaction. Meanwhile, more effective monitoring gadgets have been made due to the technology advancements, and these devices can typically record a few physiological signals simultaneously, including the electrocardiogram (ECG) signal, the electroglottography (EGG) signal, the electroencephalogram (EEG) signal, and the electrooculogram (EOG) signal. Yet, there has not been much research on the connection between digital health management and multi-modal signal monitoring. To bridge the gap, this article reviews the latest advancements in digital health management using multi-modal signal monitoring. Specifically, three digital health processes, namely, lower-limb data collection, statistical analysis of lower-limb data, and lower-limb rehabilitation via digital health management, are covered in this article, with the aim to fully review the current application of digital health technology in lower-limb symptom recovery.

A gait phase prediction model trained on benchmark datasets for evaluating a controller for prosthetic legs

Powered lower-limb assistive devices, such as prostheses and exoskeletons, are a promising option for helping mobility-impaired individuals regain functional gait. Gait phase prediction plays an important role in controlling these devices and evaluating whether the device generates a gait similar to that of individuals with intact limbs. This study proposes a gait phase prediction method based on a deep neural network (DNN). The long short-term memory (LSTM)-based model predicts a continuous gait phase from the 250 ms history of the vertical load, thigh angle, knee angle, and ankle angle, commonly available on powered lower-limb assistive devices. One unified model was trained using publicly available benchmark datasets containing intact limb gaits for level-ground walking (LGW) and ascending stairs (SA). A phase prediction error of 1.28% for all benchmark datasets was obtained. The model was subsequently applied to a state machine-controlled powered prosthetic leg dataset collected from four individuals with unilateral transfemoral amputation. The gait phase prediction results (a phase prediction error of 5.70%) indicate that the model trained on benchmark data can be used for a system not included in the training dataset with no post-processing, such as model adaptation. Furthermore, it provided information regarding evaluation of the controller: whether the prosthetic leg generated normal gait. In conclusion, the proposed gait phase prediction model will facilitate efficient gait prediction and evaluation of controllers for powered lower-limb assistive devices.

A sequential learning model with GNN for EEG-EMG-based stroke rehabilitation BCI

Introduction

Brain-computer interfaces (BCIs) have the potential in providing neurofeedback for stroke patients to improve motor rehabilitation. However, current BCIs often only detect general motor intentions and lack the precise information needed for complex movement execution, mainly due to insufficient movement execution features in EEG signals.

Methods

This paper presents a sequential learning model incorporating a Graph Isomorphic Network (GIN) that processes a sequence of graph-structured data derived from EEG and EMG signals. Movement data are divided into sub-actions and predicted separately by the model, generating a sequential motor encoding that reflects the sequential features of the movements. Through time-based ensemble learning, the proposed method achieves more accurate prediction results and execution quality scores for each movement.

Results

A classification accuracy of 88.89% is achieved on an EEG-EMG synchronized dataset for push and pull movements, significantly outperforming the benchmark method's performance of 73.23%.

Discussion

This approach can be used to develop a hybrid EEG-EMG brain-computer interface to provide patients with more accurate neural feedback to aid their recovery.

Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing

Robust classification of natural hand grasp type based on electromyography (EMG) still has some shortcomings in the practical prosthetic hand control, owing to the influence of dynamic arm position changing during hand actions. This study provided a framework for robust hand grasp type classification during dynamic arm position changes, improving both the “hardware” and “algorithm” components. In the hardware aspect, co-located synchronous EMG and force myography (FMG) signals are adopted as the multi-modal strategy. In the algorithm aspect, a sequential decision algorithm is proposed by combining the RNN-based deep learning model with a knowledge-based post-processing model. Experimental results showed that the classification accuracy of multi-modal EMG-FMG signals was increased by more than 10% compared with the EMG-only signal. Moreover, the classification accuracy of the proposed sequential decision algorithm improved the accuracy by more than 4% compared with other baseline models when using both EMG and FMG signals.

A Multi-Information Fusion Method for Gait Phase Classification in Lower Limb Rehabilitation Exoskeleton

Gait phase classification is important for rehabilitation training in patients with lower extremity motor dysfunction. Classification accuracy of the gait phase also directly affects the effect and rehabilitation training cycle. In this article, a multiple information (multi-information) fusion method for gait phase classification in lower limb rehabilitation exoskeleton is proposed to improve the classification accuracy. The advantage of this method is that a multi-information acquisition system is constructed, and a variety of information directly related to gait movement is synchronously collected. Multi-information includes the surface electromyography (sEMG) signals of the human lower limb during the gait movement, the angle information of the knee joints, and the plantar pressure information. The acquired multi-information is processed and input into a modified convolutional neural network (CNN) model to classify the gait phase. The experiment of gait phase classification with multi-information is carried out under different speed conditions, and the experiment is analyzed to obtain higher accuracy. At the same time, the gait phase classification results of multi-information and single information are compared. The experimental results verify the effectiveness of the multi-information fusion method. In addition, the delay time of each sensor and model classification time is measured, which shows that the system has tremendous real-time performance.

Human Gait Recognition: A Single Stream Optimal Deep Learning Features Fusion

Human Gait Recognition (HGR) is a biometric technique that has been utilized for security purposes for the last decade. The performance of gait recognition can be influenced by various factors such as wearing clothes, carrying a bag, and the walking surfaces. Furthermore, identification from differing views is a significant difficulty in HGR. Many techniques have been introduced in the literature for HGR using conventional and deep learning techniques. However, the traditional methods are not suitable for large datasets. Therefore, a new framework is proposed for human gait recognition using deep learning and best feature selection. The proposed framework includes data augmentation, feature extraction, feature selection, feature fusion, and classification. In the augmentation step, three flip operations were used. In the feature extraction step, two pre-trained models were employed, Inception-ResNet-V2 and NASNet Mobile. Both models were fine-tuned and trained using transfer learning on the CASIA B gait dataset. The features of the selected deep models were optimized using a modified three-step whale optimization algorithm and the best features were chosen. The selected best features were fused using the modified mean absolute deviation extended serial fusion (MDeSF) approach. Then, the final classification was performed using several classification algorithms. The experimental process was conducted on the entire CASIA B dataset and achieved an average accuracy of 89.0. Comparison with existing techniques showed an improvement in accuracy, recall rate, and computational time.