Wavelet Packet Feature Assessment for High-Density Myoelectric Pattern Recognition and Channel Selection toward Stroke Rehabilitation

Empirical Myoelectric Feature Extraction and Pattern Recognition in Hemiplegic Distal Movement Decoding

In myoelectrical pattern recognition (PR), the feature extraction methods for stroke-oriented applications are challenging and remain discordant due to a lack of hemiplegic data and limited knowledge of skeletomuscular function. Additionally, technical and clinical barriers create the need for robust, subject-independent feature generation while using supervised learning (SL). To the best of our knowledge, we are the first study to investigate the brute-force analysis of individual and combinational feature vectors for acute stroke gesture recognition using surface electromyography (EMG) of 19 patients. Moreover, post-brute-force singular vectors were concatenated via a Fibonacci-like spiral net ranking as a novel, broadly applicable concept for feature selection. This semi-brute-force navigated amalgamation in linkage (SNAiL) of EMG features revealed an explicit classification rate performance advantage of 10-17% compared to canonical feature sets, which can drastically extend PR capabilities in biosignal processing.

Supervised Myoelectrical Hand Gesture Recognition in Post-Acute Stroke Patients with Upper Limb Paresis on Affected and Non-Affected Sides

In clinical practice, acute post-stroke paresis of the extremities fundamentally complicates timely rehabilitation of motor functions; however, recently, residual and distorted musculoskeletal signals have been used to initiate feedback-driven solutions for establishing motor rehabilitation. Here, we investigate the possibilities of basic hand gesture recognition in acute stroke patients with hand paresis using a novel, acute stroke, four-component multidomain feature set (ASF-4) with feature vector weight additions (ASF-14NP, ASF-24P) and supervised learning algorithms trained only by surface electromyography (sEMG). A total of 19 (65.9 ± 12.4 years old; 12 men, seven women) acute stroke survivors (12.4 ± 6.3 days since onset) with hand paresis (Brunnstrom stage 4 ± 1/4 ± 1, SIAS 3 ± 1/3 ± 2, FMA-UE 40 ± 20) performed 10 repetitive hand movements reflecting basic activities of daily living (ADLs): rest, fist, pinch, wrist flexion, wrist extension, finger spread, and thumb up. Signals were recorded using an eight-channel, portable sEMG device with electrode placement on the forearms and thenar areas of both limbs (four sensors on each extremity). Using data preprocessing, semi-automatic segmentation, and a set of extracted feature vectors, support vector machine (SVM), linear discriminant analysis (LDA), and k-nearest neighbors (k-NN) classifiers for statistical comparison and validity (paired t-tests, p-value < 0.05), we were able to discriminate myoelectrical patterns for each gesture on both paretic and non-paretic sides. Despite any post-stroke conditions, the evaluated total accuracy rate by the 10-fold cross-validation using SVM among four-, five-, six-, and seven-gesture models were 96.62%, 94.20%, 94.45%, and 95.57% for non-paretic and 90.37%, 88.48%, 88.60%, and 89.75% for paretic limbs, respectively. LDA had competitive results using PCA whereas k-NN was a less efficient classifier in gesture prediction. Thus, we demonstrate partial efficacy of the combination of sEMG and supervised learning for upper-limb rehabilitation procedures for early acute stroke motor recovery and various treatment applications.

Domain Adaptation with Self-Guided Adaptive Sampling Strategy: Feature Alignment for Cross-User Myoelectric Pattern Recognition

Gestural interfaces based on surface electromyo-graphic (sEMG) signal have been widely explored. Nevertheless, due to the individual differences in the sEMG signals, it is very challenging for a myoelectric pattern recognition control system to adapt cross-user variability. Unsupervised domain adaptation (UDA) has achieved unprecedented success in improving the cross-domain robustness, and it is a promising approach to solve the cross-user challenge. Existing UDA methods largely ignore the instantaneous data distribution during model updating, thus deteriorating the feature representation given a large domain shift. To address this issue, a novel method is proposed based on a UDA model incorporated with a self-guided adaptive sampling (SGAS) strategy. This strategy is designed to utilize the domain distance in a kernel space as an indicator to screen out reliable instantaneous samples for updating the classifier. Thus, it enables improved alignment of feature representations of myoelectric patterns across users. To evaluate the performance of the proposed method, sEMG data were recorded from forearm muscles of nine subjects performing six finger and wrist gestures. Experiment results show that the UDA method with the SGAS strategy achieved a mean accuracy of 90.41% ± 14.44% in a cross-user classification manner, outperformed the state-of-the-art methods with statistical significance. This study demonstrates the effectiveness of the proposed UDA framework and offers a novel tool for implementing cross-user myoelectric pattern recognition towards a multi-user and user-independent control.

Rejecting Novel Motions in High-Density Myoelectric Pattern Recognition Using Hybrid Neural Networks

The objective of this study is to develop a method for alleviating a novel pattern interference toward achieving a robust myoelectric pattern-recognition control system. To this end, a framework was presented for surface electromyogram (sEMG) pattern classification and novelty detection using hybrid neural networks, i.e., a convolutional neural network (CNN) and autoencoder networks. In the framework, the CNN was first used to extract spatio-temporal information conveyed in the sEMG data recorded via high-density (HD) 2-dimensional electrode arrays. Given the target motion patterns well-characterized by the CNN, autoencoder networks were applied to learn variable correlation in the spatio-temporal information, where samples from any novel pattern appeared to be significantly different from those from target patterns. Therefore, it was straightforward to discriminate and then reject the novel motion interferences identified as untargeted and unlearned patterns. The performance of the proposed method was evaluated with HD-sEMG data recorded by two 8 × 6 electrode arrays placed over the forearm extensors and flexors of 9 subjects performing seven target motion tasks and six novel motion tasks. The proposed method achieved high accuracies over 95% for identifying and rejecting novel motion tasks, and it outperformed conventional methods with statistical significance (p < 0.05). The proposed method is demonstrated to be a promising solution for rejecting novel motion interferences, which are ubiquitous in myoelectric control. This study will enhance the robustness of the myoelectric control system against novelty interference.

Towards optimizing electrode configurations for silent speech recognition based on high-density surface electromyography

Objective. Silent speech recognition (SSR) based on surface electromyography (sEMG) is an attractive non-acoustic modality of human-machine interfaces that convert the neuromuscular electrophysiological signals into computer-readable textual messages. The speaking process involves complex neuromuscular activities spanning a large area over the facial and neck muscles, thus the locations of the sEMG electrodes considerably affected the performance of the SSR system. However, most of the previous studies used only a quite limited number of electrodes that were placed empirically without prior quantitative analysis, resulting in uncertainty and unreliability of the SSR outcomes. Approach. In this study, the technique of high-density sEMG was proposed to provide a full representation of the articulatory muscle activities so that the optimal electrode configuration for SSR could be systemically explored. A total of 120 closely spaced electrodes were placed on the facial and neck muscles to collect the high-density sEMG signals for classifying ten digits (0–9) silently spoken in both English and Chinese. The sequential forward selection algorithm was adopted to explore the optimal electrodes configurations. Main Results. The results showed that the classification accuracy increased rapidly and became saturated quickly when the number of selected electrodes increased from 1 to 120. Using only ten optimal electrodes could achieve a classification accuracy of 86% for English and 94% for Chinese, whereas as many as 40 non-optimized electrodes were required to obtain comparable accuracies. Also, the optimally selected electrodes seemed to be mostly distributed on the neck instead of the facial region, and more electrodes were required for English recognition to achieve the same accuracy. Significance. The findings of this study can provide useful guidelines about electrode placement for developing a clinically feasible SSR system and implementing a promising approach of human-machine interface, especially for patients with speaking difficulties.

Improved High-Density Myoelectric Pattern Recognition Control Against Electrode Shift Using Data Augmentation and Dilated Convolutional Neural Network

OBJECTIVE

the objective of this work is to develop a robust method for myoelectric control towards alleviating the interference of electrode shift. M ethods: In the proposed method, a preprocessing approach was first performed to convert high-density surface electromyogram (HD-sEMG) signals into a series of images, and the electrode shift appeared as pixel shift in these images. Next, a data augmentation approach was applied to the training data from just one position (no shift), so as to simulate HD-sEMG images derived from fictitious shift positions. The dilated convolutional neural network (DCNN) was subsequently adopted for classification. Compared to common convolutional neural network, DCNN always contained a larger receptive field that was supposed to be adept at mining wider spatial contextual information in images. This property was further confirmed to facilitate the classification of myoelectric patterns using HD-sEMG. The performance of the proposed method was evaluated with HD-sEMG data recorded by a 10×10 electrode array placed over forearm extensors of ten subjects during their performance of six wrist and finger extension tasks.

RESULTS

Under a variety of actual electrode shift conditions, the proposed method achieved a mean classification accuracy of 95.34%, and it outperformed other common methods.

CONCLUSION

This work demonstrated feasibility and usability of combining data augmentation and DCNN in predicting myoelectric patterns in the context of electrode shifts.

SIGNIFICANCE

The proposed method is a practical solution for robust myoelectric control against electrode array shifts.

Decoding Attempted Hand Movements in Stroke Patients Using Surface Electromyography

Brain- and muscle-triggered exoskeletons have been proposed as a means for motor training after a stroke. With the possibility of performing different movement types with an exoskeleton, it is possible to introduce task variability in training. It is difficult to decode different movement types simultaneously from brain activity, but it may be possible from residual muscle activity that many patients have or quickly regain. This study investigates whether nine different motion classes of the hand and forearm could be decoded from forearm EMG in 15 stroke patients. This study also evaluates the test-retest reliability of a classical, but simple, classifier (linear discriminant analysis) and advanced, but more computationally intensive, classifiers (autoencoders and convolutional neural networks). Moreover, the association between the level of motor impairment and classification accuracy was tested. Three channels of surface EMG were recorded during the following motion classes: Hand Close, Hand Open, Wrist Extension, Wrist Flexion, Supination, Pronation, Lateral Grasp, Pinch Grasp, and Rest. Six repetitions of each motion class were performed on two different days. Hudgins time-domain features were extracted and classified using linear discriminant analysis and autoencoders, and raw EMG was classified with convolutional neural networks. On average, 79 ± 12% and 80 ± 12% (autoencoders) of the movements were correctly classified for days 1 and 2, respectively, with an intraclass correlation coefficient of 0.88. No association was found between the level of motor impairment and classification accuracy (Spearman correlation: 0.24). It was shown that nine motion classes could be decoded from residual EMG, with autoencoders being the best classification approach, and that the results were reliable across days; this may have implications for the development of EMG-controlled exoskeletons for training in the patient’s home.

Quantitative Assessment of Traumatic Upper-Limb Peripheral Nerve Injuries Using Surface Electromyography

Background

There is a great demand for convenient and quantitative assessment of upper-limb traumatic peripheral nerve injuries (PNIs) beyond their clinical routine. This would contribute to improved PNI management and rehabilitation.

Objective

The aim of this study was to develop a novel surface EMG examination method for quantitatively evaluating traumatic upper-limb PNIs.

Methods

Experiments were conducted to collect surface EMG data from forearm muscles on both sides of seven male subjects during their performance of eight designated hand and wrist motion tasks. All participants were clinically diagnosed as unilateral traumatic upper-limb PNIs on the ulnar nerve, median nerve, or radial nerve. Ten healthy control participants were also enrolled in the study. A novel framework consisting of two modules was also proposed for data analysis. One module was first used to identify whether a PNI occurs on a tested forearm using a machine learning algorithm by extracting and classifying features from surface EMG data. The second module was then used to quantitatively evaluate the degree of injury on three individual nerves on the examined arm.

Results

The evaluation scores yielded by the proposed method were highly consistent with the clinical assessment decisions for three nerves of all 34 examined arms (7 × 2 + 10 × 2), with a sensitivity of 81.82%, specificity of 98.90%, and significate linear correlation (p < 0.05) in quantitative decision points between the proposed method and the routine clinical approach.

Conclusion

This study offers a useful tool for PNI assessment and helps to promote extensive clinical applications of surface EMG.

Visualized Evidences for Detecting Novelty in Myoelectric Pattern Recognition using 3D Convolutional Neural Networks

Although myoelectric pattern recognition (MPR) has been considered as a milestone technique to enable dexterous control of multiple degrees of freedom, outlier data interference (i.e., novelty) is a big issue affecting stability of conventional MPR control and its wide applications. Inspired by video classification techniques, we propose a novel method using 3-dimensional (3D) convolutional neural network (CNN) for extracting sufficient spatial-temporal features from high-density surface electromyogram (EMG) recordings processed as a video stream where time-varying information of muscular activity was taken into account. Given the targeted task patterns accurately characterized by the proposed method, it is straightforward to discriminate and then reject outlier data interferences regarded as untargeted and unlearnt patterns. Meanwhile, the strength of the 3D CNN in discriminating targeted task patterns through spatial-temporal features is further visualized and confirmed by t-Distributed Stochastic Neighbor Embedding (t-SNE). The performance of the proposed method was evaluated with surface EMG recorded by two 6 × 8 electrode arrays placed over forearm flexors and extensors of 3 subjects performing 7 targeted and 4 outlier motions. The visualized results by t-SNE showed the targeted task patterns were separated with each concentrated into a small region, while the outlier patterns were scattered around. On this basis, a traditional Mahalanobis Distance (MD) based method was applied for novelty detection and targeted task classification. Finally, the proposed method yielded averaged error rate of 10.98% for the targeted task patterns and <; 5% for all outlier data, respectively, which outperformed a common baseline method. All these findings demonstrate the advantage of characterizing myoelectric patterns using the proposed method and the potential of applying it to reject outlier data interference.

A Novel Time-Domain based Feature for EMG-PR Prosthetic and Rehabilitation Application

EMG signal is widely accepted in human-machine interaction applications, such as prosthesis control and rehabilitation devices. The existing feature extraction methods struggle to separate a variety of EMG based activities. In the proposed work, a novel feature defined as PAP (peak average power) has been proposed. This feature has been validated for NinaPro database which includes isometric, isotonic, grasp and finger force based upper limb motions. Further, the comparison of classification accuracy has been performed with well-known time domain based features. Significant classification performance enhancement has been observed in terms of accuracy with LDA and QDA techniques. In this experiment, three datasets have been created and analysis was performed. Consequently, the results show an average enhancement of 17.60%, 7.52% and 15.37% using the proposed approach for LDA in dataset-1, dataset-2, and dataset-3 respectively. Similarly for the same datasets, when QDA is used the proposed approach overrules the existing techniques with the average enhanced performance of 13.52%, 12.72%, and 15.40%. All the analysis has been done using MATLAB 2015a in the i7 core.

Adaptive Calibration of Electrode Array Shifts Enables Robust Myoelectric Control

OBJECTIVE

The objective of this work is to develop a novel method for adaptive calibration of the electrode array shifts toward achieving robust myoelectric pattern-recognition control.

METHODS

This work is inspired by the idea of image object detection when high-density surface electromyogram signals recorded from a two-dimensional electrode array carry rich spatial information and can serve as a muscular activation image. A convolutional neural network involving transfer learning (from a network for image recognition) is used to learn muscular activity patterns at an original/baseline position of an electrode array. The shift of an electrode array can be estimated by identifying and matching partially overlapped regions between the training images of muscular activation and the testing images at any other position, in an unsupervised manner. Given the calibration of an electrode array shift, the identification of muscular activity patterns is implemented accordingly. The performance of the proposed method was evaluated with data recorded by a 10 × 10 electrode array placed over the forearm extensors of 10 subjects performing 6 wrist and finger extension tasks.

RESULTS

The proposed method achieved high task classification accuracies around 95% and outperformed five traditional methods under conditions with multiple designated shifts in offline experiments and a random shift in online testing.

CONCLUSION

The proposed method is demonstrated to be a promising solution for the automatic and adaptive calibration of electrode array shifts.

SIGNIFICANCE

This work will enhance the robustness of myoelectric control systems.

Upper Arm Motion High-Density sEMG Recognition Optimization Based on Spatial and Time-Frequency Domain Features

Background

Spatial characteristics of sEMG signals are obtained by high-density matrix sEMG electrodes for further complex upper arm movement classification. Multiple electrode channels of the high-density sEMG acquisition device aggravate the burden of the microprocessor and deteriorate control system's real-time performance at the same time. A shoulder motion recognition optimization method based on the maximizing mutual information from multiclass CSP selected spatial feature channels and wavelet packet features extraction is proposed in this study.

Results

The relationship between the number of channels and recognition rate is obtained by the recognition optimization method. The original 64 electrodes channels are reduced to only 4-5 active signal channels with the accuracy over 92%.

Conclusion

The shoulder motion recognition optimization method is combined with the spatial-domain and time-frequency-domain features. In addition, the spatial feature channel selection is independent of feature extraction and classification algorithm. Therefore, it is more convenient to use less channels to achieve the desired classification accuracy.

An IoT-Enabled Stroke Rehabilitation System Based on Smart Wearable Armband and Machine Learning

Surface electromyography signal plays an important role in hand function recovery training. In this paper, an IoT-enabled stroke rehabilitation system was introduced which was based on a smart wearable armband (SWA), machine learning (ML) algorithms, and a 3-D printed dexterous robot hand. User comfort is one of the key issues which should be addressed for wearable devices. The SWA was developed by integrating a low-power and tiny-sized IoT sensing device with textile electrodes, which can measure, pre-process, and wirelessly transmit bio-potential signals. By evenly distributing surface electrodes over user’s forearm, drawbacks of classification accuracy poor performance can be mitigated. A new method was put forward to find the optimal feature set. ML algorithms were leveraged to analyze and discriminate features of different hand movements, and their performances were appraised by classification complexity estimating algorithms and principal components analysis. According to the verification results, all nine gestures can be successfully identified with an average accuracy up to 96.20%. In addition, a 3-D printed five-finger robot hand was implemented for hand rehabilitation training purpose. Correspondingly, user’s hand movement intentions were extracted and converted into a series of commands which were used to drive motors assembled inside the dexterous robot hand. As a result, the dexterous robot hand can mimic the user’s gesture in a real-time manner, which shows the proposed system can be used as a training tool to facilitate rehabilitation process for the patients after stroke.