Towards reducing the impacts of unwanted movements on identification of motion intentions

EMG-FRNet: A feature reconstruction network for EMG irrelevant gesture recognition

With the development of deep learning technology, gesture recognition based on surface electromyography (EMG) signals has shown broad application prospects in various human-computer interaction fields. Most current gesture recognition technologies can achieve high recognition accuracy on a wide range of gesture actions. However, in practical applications, gesture recognition based on surface EMG signals is susceptible to interference from irrelevant gesture movements, which affects the accuracy and security of the system. Therefore, it is crucial to design an irrelevant gesture recognition method. This paper introduces the GANomaly network from the field of image anomaly detection into surface EMG-based irrelevant gesture recognition. The network has a small feature reconstruction error for target samples and a large feature reconstruction error for irrelevant samples. By comparing the relationship between the feature reconstruction error and the predefined threshold, we can determine whether the input samples are from the target category or the irrelevant category. In order to improve the performance of EMG irrelevant gesture recognition, this paper proposes a feature reconstruction network named EMG-FRNet for EMG irrelevant gesture recognition. This network is based on GANomaly and incorporates structures such as channel cropping (CC), cross-layer encoding-decoding feature fusion (CLEDFF), and SE channel attention (SE). In this paper, Ninapro DB1, Ninapro DB5 and self-collected datasets were used to verify the performance of the proposed model. The Area Under the receiver operating characteristic Curve (AUC) values of EMG-FRNet on the above three datasets were 0.940, 0.926 and 0.962, respectively. Experimental results demonstrate that the proposed model achieves the highest accuracy among related research.

Toward Improving the Reliability of Discrete Movement Recognition of sEMG Signals

Currently, the classification accuracy of surface electromyography (sEMG) signals is high in literature, but the conventional recognition system may classify untrained movements or the trained movements of low reliability to one of its target classes by mistake. If such a system is used for prosthetic control, sometimes it may cause a disaster. A two-layer classifier that fuses the Gaussian mixture model (GMM) and k-nearest neighbor (kNN) in a sequential structure is proposed in this study. The proposed algorithm can reject the trained movements with low reliability and is efficient in rejecting the untrained movements, thus enhancing the reliability of the myoelectric control system. The results show that the proposed algorithm can produce 95.7% active accuracy in recognizing 12 trained movements and a 30.3% error rate for rejecting 12 untrained movements. When the movement number is six, the active accuracy for trained movements can reach 99.2%, and the error rate of untrained movement is only 17.4%, which is much better than previous studies. Therefore, the proposed classifier can accurately recognize the trained movements and reject untrained movement patterns effectively.

Identification of Upper-Limb Movements Based on Muscle Shape Change Signals for Human-Robot Interaction

Towards providing efficient human-robot interaction, surface electromyogram (EMG) signals have been widely adopted for the identification of different limb movement intentions. Since the available EMG signal sensors are highly susceptible to external interferences such as electromagnetic artifacts and muscle fatigues, the quality of EMG recordings would be mostly corrupted, which may decay the performance of EMG-based control systems. Given the fact that the muscle shape changes (MSC) would be different when doing various limb movements, the MSC signal would be nonsensitive to electromagnetic artifacts and muscle fatigues and maybe promising for movement intention recognition. In this study, a novel nanogold flexible and stretchable sensor was developed for the acquisition of MSC signals utilized for decoding multiple classes of limb movement intents. More precisely, four sensors were used to measure the MSC signals from the right forearm of each subject when they performed seven classes of movements. Also, six different features were extracted from the measured MSC signals, and a linear discriminant analysis- (LDA-) based classifier was built for movement classification tasks. The experimental results showed that using MSC signals could achieve an average recognition rate of about 96.06 ± 1.84% by properly placing the four flexible and stretchable sensors on the forearm. Additionally, when the MSC sampling rate was greater than 100 Hz and the analysis window length was greater than 20 ms, the movement recognition accuracy would be only slightly increased. These pilot results suggest that the MSC-based method should be feasible in movement identifications for human-robot interaction, and at the same time, they provide a systematic reference for the use of the flexible and stretchable sensors in human-robot interaction systems.

A Classification and Calibration Procedure for Gesture Specific Home-Based Therapy Exercise in Young People With Cerebral Palsy

Movement-based video games can provide engaging practice for repetitive therapeutic gestures towards improving manual ability in youth with cerebral palsy (CP). However, home-based gesture calibration and classification is needed to personalize therapy and ensure an optimal challenge point. Nineteen youth with CP controlled a video game during a 4-week home-based intervention using therapeutic hand gestures detected via electromyography and inertial sensors. The in-game calibration and classification procedure selects the most discriminating, person-specific features using random forest classification. Then, a support vector machine is trained with this feature subset for in-game interaction. The procedure uses features intended to be sensitive to signs of CP and leverages directional statistics to characterize muscle activity around the forearm. Home-based calibration showed good agreement with video verified ground truths (0.86 ± 0.11, 95%CI = 0.93-0.97). Across participants, classifier performance (F1-score) for the primary therapeutic gesture was 0.90 ± 0.05 (95%CI = 0.87-0.92) and, for the secondary gesture, 0.82 ± 0.09 (95%CI = 0.77-0.86). Features sensitive to signs of CP were significant contributors to classification and correlated to wrist extension improvement and increased practice time. This study contributes insights for classifying gestures in people with CP and demonstrates a new gesture controller to facilitate home-based therapy gaming.

Rejection of Systemic and Operator Errors in a Real-Time Myoelectric Control Task

In myoelectric pattern-recognition control, the rejection of movement decisions based on confidence - the likelihood of a correct classification - has been shown to improve system usability, however it is not known to what extent this is due directly to error mitigation, and to what extent this is due to users having opportunities to change the way they contract. To understand this, 24 subjects participated in a real-time pattern recognition control task with rejection at seven different confidence thresholds, and without rejection. Errors were classified into systemic errors (i.e., those produced by the classifier) and operator errors (i.e., those produced by user behavior). It was found that the error permitted by the rejection controller was reduced by about half at high rejection thresholds, with both systemic and operator errors significantly affected, while the errors produced by the user remained essentially constant throughout. Conversely, correct decisions were filtered out by the rejection controller at significantly greater rates at high rejection thresholds, which may be excessive enough to ultimately impair usability. While some subjects reported being experienced in myoelectric control, no significant differences were observed due to experience level.

Adaptive Hybrid Classifier for Myoelectric Pattern Recognition Against the Interferences of Outlier Motion, Muscle Fatigue, and Electrode Doffing

Traditional myoelectric prostheses that employ a static pattern recognition model to identify human movement intention from surface electromyography (sEMG) signals hardly adapt to the changes in the sEMG characteristics caused by interferences from daily activities, which hinders the clinical applications of such prostheses. In this paper, we focus on methods to reduce or eliminate the impacts of three types of daily interferences on myoelectric pattern recognition (MPR), i.e., outlier motion, muscle fatigue, and electrode doffing/donning. We constructed an adaptive incremental hybrid classifier (AIHC) by combining one-class support vector data description and multi-class linear discriminant analysis in conjunction with two specific update schemes. We developed an AIHC-based MPR strategy to improve the robustness of MPR against the three interferences. Extensive experiments on hand-motion recognition were conducted to demonstrate the performance of the proposed method. Experimental results show that the AIHC has significant advantages over non-adaptive classifiers under various interferences, with improvements in the classification accuracy ranging from 7.1% to 39% ( ). The additional evaluations on data deviations demonstrate that the AIHC can accommodate large-scale changes in the sEMG characteristics, revealing the potential of the AIHC-based MPR strategy in the development of clinical myoelectric prostheses.

Upper Arm Motion High-Density sEMG Recognition Optimization Based on Spatial and Time-Frequency Domain Features

Background

Spatial characteristics of sEMG signals are obtained by high-density matrix sEMG electrodes for further complex upper arm movement classification. Multiple electrode channels of the high-density sEMG acquisition device aggravate the burden of the microprocessor and deteriorate control system's real-time performance at the same time. A shoulder motion recognition optimization method based on the maximizing mutual information from multiclass CSP selected spatial feature channels and wavelet packet features extraction is proposed in this study.

Results

The relationship between the number of channels and recognition rate is obtained by the recognition optimization method. The original 64 electrodes channels are reduced to only 4-5 active signal channels with the accuracy over 92%.

Conclusion

The shoulder motion recognition optimization method is combined with the spatial-domain and time-frequency-domain features. In addition, the spatial feature channel selection is independent of feature extraction and classification algorithm. Therefore, it is more convenient to use less channels to achieve the desired classification accuracy.

A novel application of a surface ElectroMyoGraphy-based control strategy for a hand exoskeleton system: A single-case study

Robotics is increasingly involving many aspects of daily life and robotic-based assistance to physically impaired people is considered one of the most promising application of this largely investigated technology. However, the World Health Organization reports that, so far, only 10% of people in need can get access to the so-called assistive technology also due to its high costs. This work aims to tackle the aforementioned point presenting an innovative control strategy for a low-cost hand exoskeleton system based on surface electromyography signals. Most of the activities of daily living are, in fact, carried out thanks to the hands while the exploitation of surface electromyography signals represents a non-invasive technique in straightforwardly controlling wearable devices. Although such approach results deeply studied in literature, it has not been deeply tested on real patients yet. The main contribution of this article is hence not only to describe a novel control strategy but also to provide a detailed explanation of its implementation into a real device, ready to be used. Finally, the authors have evaluated and preliminary tested the proposed technique enrolling a patient in a single-case study.

Effects of Confidence-Based Rejection on Usability and Error in Pattern Recognition-Based Myoelectric Control

Rejection of movements based on the confidence in the classification decision has previously been demonstrated to improve the usability of pattern recognition based myoelectric control. To this point, however, the optimal rejection threshold has been determined heuristically, and it is not known how different thresholds affect the tradeoff between error mitigation and false rejections in real-time closed-loop control. To answer this question, 24 able-bodied subjects completed a real-time Fitts' law-style virtual cursor control task using a support vector machine classifier. It was found that rejection improved information throughput at all thresholds, with the best performance coming at thresholds between 0.60 and 0.75. Two fundamental types of error were defined and identified: operator error (identifiable, repeatable behaviors, directly attributable to the user), and systemic error (other errors attributable to misclassification or noise). The incidence of both operator and systemic errors were found to decrease as rejection threshold increased. Moreover, while the incidence of all error types correlated strongly with path efficiency, only systemic errors correlated strongly with throughput and trial completion rate. Interestingly, more experienced users were found to commit as many errors as novice users, despite performing better in the Fitts' task, suggesting that there is more to usability than error prevention alone. Nevertheless, these results demonstrate the usability gains possible with rejection across a range of thresholds for both novice and experienced users alike.

Motor imagery classification of upper limb movements based on spectral domain features of EEG patterns

Surface electromyography pattern recognition methods have been widely applied to decode limb movement intentions for prosthesis control. These methods generally require amputees to provide sufficient myoelectric signals from their residual limb muscles. Previous studies have shown that amputees with high level amputation or neuromuscular disorder usually do not have sufficient residual limb muscles to provide enough myoelectric signals for accurate identification of limb movements. Electroencephalography (EEG), another bioelectric signal associated with limb movements has also been proposed and used for decoding the limb motion intents of humans. With an attempt to improve the performance of EEG-based method in identifying multiple classes of upper limb movement intents, four spectral domain features of EEG were proposed in this study. Motor imagery patterns associated with five different classes of imagined upper limb movements were distinctively decoded based on the four features extracted from 64-channel EEG recordings in four transhumeral amputees. Experimental results show that an average accuracy of 97.81% was achieved across all the subjects and limb movement classes. By applying a sequential channel selection method, an accuracy of around 95.00% was realized with about 20-channels of EEG. Thus, the proposed method might be potential for providing accurate control input for neuroprosthesis.

Increasing the robustness against force variation in EMG motion classification by common spatial patterns

In the practical use of an electromyography (EMG) pattern-recognition based myoelectric prosthesis, the variation of force levels to do a motion would be inevitable, which will cause a change of EMG patterns. Therefore, the force variation will decay the performance of a trained classifier. In this study, the common spatial pattern (CSP) method was proposed with an attempt to improve the robustness of EMG-PR based classifier against force variation. The EMG signals were acquired from three able-bodied subjects when they were performing the motions at low, medium, and high force levels, respectively. And in the pattern recognition, CSP features were extracted from the EMG signals for motion classification. By comparing the classification accuracies between the CSP and the commonly used time-domain (TD) features, the CSP features showed a better robustness against force variation with an increment of 5.3% of the average classification accuracy. Especially, the classification accuracy of a classifier was 84.2% when tested at low force level by using CSP features, which was 18.5% higher than that of the TD features. These preliminary results suggest that using CSP features may increase the robustness of EMG-based myoelectric control.

Towards Efficient Decoding of Multiple Classes of Motor Imagery Limb Movements Based on EEG Spectral and Time Domain Descriptors

To control multiple degrees of freedom (MDoF) upper limb prostheses, pattern recognition (PR) of electromyogram (EMG) signals has been successfully applied. This technique requires amputees to provide sufficient EMG signals to decode their limb movement intentions (LMIs). However, amputees with neuromuscular disorder/high level amputation often cannot provide sufficient EMG control signals, and thus the applicability of the EMG-PR technique is limited especially to this category of amputees. As an alternative approach, electroencephalograph (EEG) signals recorded non-invasively from the brain have been utilized to decode the LMIs of humans. However, most of the existing EEG based limb movement decoding methods primarily focus on identifying limited classes of upper limb movements. In addition, investigation on EEG feature extraction methods for the decoding of multiple classes of LMIs has rarely been considered. Therefore, 32 EEG feature extraction methods (including 12 spectral domain descriptors (SDDs) and 20 time domain descriptors (TDDs)) were used to decode multiple classes of motor imagery patterns associated with different upper limb movements based on 64-channel EEG recordings. From the obtained experimental results, the best individual TDD achieved an accuracy of 67.05 ± 3.12% as against 87.03 ± 2.26% for the best SDD. By applying a linear feature combination technique, an optimal set of combined TDDs recorded an average accuracy of 90.68% while that of the SDDs achieved an accuracy of 99.55% which were significantly higher than those of the individual TDD and SDD at p < 0.05. Our findings suggest that optimal feature set combination would yield a relatively high decoding accuracy that may improve the clinical robustness of MDoF neuroprosthesis.

TRIAL REGISTRATION

The study was approved by the ethics committee of Institutional Review Board of Shenzhen Institutes of Advanced Technology, and the reference number is SIAT-IRB-150515-H0077.

Improving the Robustness of Electromyogram-Pattern Recognition for Prosthetic Control by a Postprocessing Strategy

Electromyogram (EMG) contains rich information for motion decoding. As one of its major applications, EMG-pattern recognition (PR)-based control of prostheses has been proposed and investigated in the field of rehabilitation robotics for decades. These prostheses can offer a higher level of dexterity compared to the commercially available ones. However, limited progress has been made toward clinical application of EMG-PR-based prostheses, due to their unsatisfactory robustness against various interferences during daily use. These interferences may lead to misclassifications of motion intentions, which damage the control performance of EMG-PR-based prostheses. A number of studies have applied methods that undergo a postprocessing stage to determine the current motion outputs, based on previous outputs or other information, which have proved effective in reducing erroneous outputs. In this study, we proposed a postprocessing strategy that locks the outputs during the constant contraction to block out occasional misclassifications, upon detecting the motion onset using a threshold. The strategy was investigated using three different motion onset detectors, namely mean absolute value, Teager–Kaiser energy operator, or mechanomyogram (MMG). Our results indicate that the proposed strategy could suppress erroneous outputs, during rest and constant contractions in particular. In addition, with MMG as the motion onset detector, the strategy was found to produce the most significant improvement in the performance, reducing the total errors up to around 50% (from 22.9 to 11.5%) in comparison to the original classification output in the online test, and it is the most robust against threshold value changes. We speculate that motion onset detectors that are both smooth and responsive would further enhance the efficacy of the proposed postprocessing strategy, which would facilitate the clinical application of EMG-PR-based prosthetic control.

Using spatial features for classification of combined motions based on common spatial pattern

Motion recognition is an important application of electromyography (EMG) analysis. While discrete motions such as hand open, hand close and wrist pronation have been extensively investigated, studies on combined motions involving two or more degrees of freedom (DOFs) are relatively few and the classification accuracy of the combined motions reported in previous studies is barely satisfactory. To improve the accuracy of the combined motion recognition, common spatial pattern (CSP) was employed in this study to extract spatial features. 18 forearm motion classes, consisted of 8 discrete motions and 10 combined motions, were classified by the proposed method. Our results showed that the accuracy rate of CSP features was 96.3%, which outperformed the commonly used time-domain (TD) features by 2.4% and TD combined with auto-regression coefficients (TDAR) by 0.6%. Moreover, CSP features cost noticeable much less time than TDAR and quite less time than TD in testing. These results suggest that CSP features could be a better feature set for multi-DOF myoelectric control than conventional features.

Improving the Robustness of Real-Time Myoelectric Pattern Recognition against Arm Position Changes in Transradial Amputees

Previous studies have showed that arm position variations would significantly degrade the classification performance of myoelectric pattern-recognition-based prosthetic control, and the cascade classifier (CC) and multiposition classifier (MPC) have been proposed to minimize such degradation in offline scenarios. However, it remains unknown whether these proposed approaches could also perform well in the clinical use of a multifunctional prosthesis control. In this study, the online effect of arm position variation on motion identification was evaluated by using a motion-test environment (MTE) developed to mimic the real-time control of myoelectric prostheses. The performance of different classifier configurations in reducing the impact of arm position variation was investigated using four real-time metrics based on dataset obtained from transradial amputees. The results of this study showed that, compared to the commonly used motion classification method, the CC and MPC configurations improved the real-time performance across seven classes of movements in five different arm positions (8.7% and 12.7% increments of motion completion rate, resp.). The results also indicated that high offline classification accuracy might not ensure good real-time performance under variable arm positions, which necessitated the investigation of the real-time control performance to gain proper insight on the clinical implementation of EMG-pattern-recognition-based controllers for limb amputees.

A motion-classification strategy based on sEMG-EEG signal combination for upper-limb amputees

Background

Most of the modern motorized prostheses are controlled with the surface electromyography (sEMG) recorded on the residual muscles of amputated limbs. However, the residual muscles are usually limited, especially after above-elbow amputations, which would not provide enough sEMG for the control of prostheses with multiple degrees of freedom. Signal fusion is a possible approach to solve the problem of insufficient control commands, where some non-EMG signals are combined with sEMG signals to provide sufficient information for motion intension decoding. In this study, a motion-classification method that combines sEMG and electroencephalography (EEG) signals were proposed and investigated, in order to improve the control performance of upper-limb prostheses.

Methods

Four transhumeral amputees without any form of neurological disease were recruited in the experiments. Five motion classes including hand-open, hand-close, wrist-pronation, wrist-supination, and no-movement were specified. During the motion performances, sEMG and EEG signals were simultaneously acquired from the skin surface and scalp of the amputees, respectively. The two types of signals were independently preprocessed and then combined as a parallel control input. Four time-domain features were extracted and fed into a classifier trained by the Linear Discriminant Analysis (LDA) algorithm for motion recognition. In addition, channel selections were performed by using the Sequential Forward Selection (SFS) algorithm to optimize the performance of the proposed method.

Results

The classification performance achieved by the fusion of sEMG and EEG signals was significantly better than that obtained by single signal source of either sEMG or EEG. An increment of more than 14% in classification accuracy was achieved when using a combination of 32-channel sEMG and 64-channel EEG. Furthermore, based on the SFS algorithm, two optimized electrode arrangements (10-channel sEMG + 10-channel EEG, 10-channel sEMG + 20-channel EEG) were obtained with classification accuracies of 84.2 and 87.0%, respectively, which were about 7.2 and 10% higher than the accuracy by using only 32-channel sEMG input.

Conclusions

This study demonstrated the feasibility of fusing sEMG and EEG signals towards improving motion classification accuracy for above-elbow amputees, which might enhance the control performances of multifunctional myoelectric prostheses in clinical application.

Trial registration

The study was approved by the ethics committee of Institutional Review Board of Shenzhen Institutes of Advanced Technology, and the reference number is SIAT-IRB-150515-H0077.