A real-time pattern recognition based myoelectric control usability study implemented in a virtual environment

A Review on the Usability, Flexibility, Affinity, and Affordability of Virtual Technology for Rehabilitation Training of Upper Limb Amputees

(1) Background: Prosthetic rehabilitation is essential for upper limb amputees to regain their ability to work. However, the abandonment rate of prosthetics is higher than 50% due to the high cost of rehabilitation. Virtual technology shows potential for improving the availability and cost-effectiveness of prosthetic rehabilitation. This article systematically reviews the application of virtual technology for the prosthetic rehabilitation of upper limb amputees. (2) Methods: We followed PRISMA review guidance, STROBE, and CASP to evaluate the included articles. Finally, 17 articles were screened from 22,609 articles. (3) Results: This study reviews the possible benefits of using virtual technology from four aspects: usability, flexibility, psychological affinity, and long-term affordability. Three significant challenges are also discussed: realism, closed-loop control, and multi-modality integration. (4) Conclusions: Virtual technology allows for flexible and configurable control rehabilitation, both during hospital admissions and after discharge, at a relatively low cost. The technology shows promise in addressing the critical barrier of current prosthetic training issues, potentially improving the practical availability of prosthesis techniques for upper limb amputees.

Correcting Temporal Inaccuracies in Labeled Training Data for Electromyographic Control Algorithms

Electromyographic (EMG) control relies on supervised-learning algorithms that correlate EMG to motor intent. The quality of the training dataset is critical to the runtime performance of the algorithm, but labeling motor intent is imprecise and imperfect. Traditional EMG training data is collected while participants mimic predetermined movements of a virtual hand with their own hand. This assumes participants are perfectly synchronized with the predetermined movements, which is unlikely due to reaction time and signal-processing delays. Prior work has used cross-correlation to globally shift and re-align kinematic data and EMG. Here, we quantify the impact of this global re-alignment on both classification algorithms and regression algorithms with and without a human in the loop. We also introduce a novel trial-by-trial re-alignment method to re-align EMG with kinematics on a per-movement basis. We show that EMG and kinematic data are inherently misaligned, and that reaction time is inconsistent throughout data collection. Both global and trial-by-trial re-alignment significantly improved offline performance for classification and regression. Our trial-by-trial re-alignment further improved offline classification performance relative to global realignment. However, online performance, with a human actively in the loop, was no different with or without re-alignment. This work highlights inaccuracies in labeled EMG data and has broad implications for EMG-control applications.

The Influence of Training with Visual Biofeedback on the Predictability of Myoelectric Control Usability

Studies have shown that closed-loop myoelectric control schemes can lead to changes in user performance and behavior compared to open-loop systems. When users are placed within the control loop, such as during real-time use, they must correct for errors made by the controller and learn what behavior is necessary to produce desired outcomes. Augmented feedback, consequently, has been used to incorporate the user throughout the training process and to facilitate learning. This work explores the effect of visual feedback presented during user training on both the performance and predictability of a myoelectric classification-based control system. Our results suggest that properly designed feedback mechanisms and training tasks can influence the quality of the training data and the ability to predict usability using linear combinations of metrics derived from feature space. Furthermore, our results confirm that the most common in-lab training protocol, screen guided training, may yield training data that are less representative of online use than training protocols that incorporate the user in the loop. These results suggest that training protocols should be designed that better parallel the testing environment to more effectively prepare both the algorithms and users for real-time control.

Serious Games Are Not Serious Enough for Myoelectric Prosthetics

Serious games show a lot of potential for use in movement rehabilitation (eg, after a stroke, injury to the spinal cord, or limb loss). However, the nature of this research leads to diversity both in the background of the researchers and in the approaches of their investigation. Our close examination and categorization of virtual training software for upper limb prosthetic rehabilitation found that researchers typically followed one of two broad approaches: (1) focusing on the game design aspects to increase engagement and muscle training and (2) concentrating on an accurate representation of prosthetic training tasks, to induce task-specific skill transfer. Previous studies indicate muscle training alone does not lead to improved prosthetic control without a transfer-enabling task structure. However, the literature shows a recent surge in the number of game-based prosthetic training tools, which focus on engagement without heeding the importance of skill transfer. This influx appears to have been strongly influenced by the availability of both software and hardware, specifically the launch of a commercially available acquisition device and freely available high-profile game development engines. In this Viewpoint, we share our perspective on the current trends and progress of serious games for prosthetic training.

An Adaptive Multi-Modal Control Strategy to Attenuate the Limb Position Effect in Myoelectric Pattern Recognition

Over the last few decades, pattern recognition algorithms have shown promising results in the field of upper limb prostheses myoelectric control and are now gradually being incorporated in commercial devices. A widely used approach is based on a classifier which assigns a specific input value to a selected hand motion. While this method guarantees good performance and robustness within each class, it still shows limitations in adapting to different conditions encountered in real-world applications, such as changes in limb position or external loads. This paper proposes an adaptive method based on a pattern recognition classifier that takes advantage of an augmented dataset-i.e., representing variations in limb position or external loads-to selectively adapt to underrepresented variations. The proposed method was evaluated using a series of target achievement control tests with ten able-bodied volunteers. Results indicated a higher median completion rate >3.33% for the adapted algorithm compared to a classical pattern recognition classifier used as a baseline model. Subject-specific performance showed the potential for improved control after adaptation and a ≤13% completion rate; and in many instances, the adapted points were able to provide new information within classes. These preliminary results show the potential of the proposed method and encourage further development.

Upper-Limb Electromyogram Classification of Reaching-to-Grasping Tasks Based on Convolutional Neural Networks for Control of a Prosthetic Hand

In recent years, myoelectric interfaces using surface electromyogram (EMG) signals have been developed for assisting people with physical disabilities. Especially, in the myoelectric interfaces for robotic hands or arms, decoding the user’s upper-limb movement intentions is cardinal to properly control the prosthesis. However, because previous experiments were implemented with only healthy subjects, the possibility of classifying reaching-to-grasping based on the EMG signals from the residual limb without the below-elbow muscles was not investigated yet. Therefore, we aimed to investigate the possibility of classifying reaching-to-grasping tasks using the EMG from the upper arm and upper body without considering wrist muscles for prosthetic users. In our study, seven healthy subjects, one trans-radial amputee, and one wrist amputee were participated and performed 10 repeatable 12 reaching-to-grasping tasks based on the Southampton Hand Assessment Procedure (SHAP) with 12 different weighted (light and heavy) objects. The acquired EMG was processed using the principal component analysis (PCA) and convolutional neural network (CNN) to decode the tasks. The PCA–CNN method showed that the average accuracies of the healthy subjects were 69.4 ± 11.4%, using only the EMG signals by the upper arm and upper body. The result with the PCA–CNN method showed 8% significantly higher accuracies than the result with the widely used time domain and auto-regressive-support vector machine (TDAR–SVM) method as 61.6 ± 13.7%. However, in the cases of the amputees, the PCA–CNN showed slightly lower performance. In addition, in the aspects of assistant daily living, because grip force is also important when grasping an object after reaching, the possibility of classifying the two light and heavy objects in each reaching-to-grasping task was also investigated. Consequently, the PCA–CNN method showed higher accuracy at 70.1 ± 9.8%. Based on our results, the PCA–CNN method can help to improve the performance of classifying reaching-to-grasping tasks without wrist EMG signals. Our findings and decoding method can be implemented to further develop a practical human–machine interface using EMG signals.

How Many Muscles? Optimal Muscles Set Search for Optimizing Myocontrol Performance

In myo-control, for computational and setup constraints, the measurement of a high number of muscles is not always possible: the choice of the muscle set to use in a myo-control strategy depends on the desired application scope and a search for a reduced muscle set, tailored to the application, has never been performed. The identification of such set would involve finding the minimum set of muscles whose difference in terms of intention detection performance is not statistically significant when compared to the original set. Also, given the intrinsic sensitivity of muscle synergies to variations of EMG signals matrix, the reduced set should not alter synergies that come from the initial input, since they provide physiological information on motor coordination. The advantages of such reduced set, in a rehabilitation context, would be the reduction of the inputs processing time, the reduction of the setup bulk and a higher sensitivity to synergy changes after training, which can eventually lead to modifications of the ongoing therapy. In this work, the existence of a minimum muscle set, called optimal set, for an upper-limb myoelectric application, that preserves performance of motor activity prediction and the physiological meaning of synergies, has been investigated. Analyzing isometric contractions during planar reaching tasks, two types of optimal muscle sets were examined: a subject-specific one and a global one. The former relies on the subject-specific movement strategy, the latter is composed by the most recurrent muscles among subjects specific optimal sets and shared by all the subjects. Results confirmed that the muscle set can be reduced to achieve comparable hand force estimation performances. Moreover, two types of muscle synergies namely "Pose-Shared" (extracted from a single multi-arm-poses dataset) and "Pose-Related" (clustering pose-specific synergies), extracted from the global optimal muscle set, have shown a significant similarity with full-set related ones meaning a high consistency of the motor primitives. Pearson correlation coefficients assessed the similarity of each synergy. The discovering of dominant muscles by means of the optimization of both muscle set size and force estimation error may reveal a clue on the link between synergistic patterns and the force task.

A Multi-Variate Approach to Predicting Myoelectric Control Usability

Pattern recognition techniques leveraging the use of electromyography signals have become a popular approach to provide intuitive control of myoelectric devices. Performance of these control interfaces is commonly quantified using offline classification accuracy, despite studies having shown that this metric is a poor indicator of usability. Researchers have identified alternative offline metrics that better correlate with online performance; however, the relationship has yet to be fully defined in the literature. This has necessitated the continued trial-and-error-style online testing of algorithms developed using offline approaches. To bridge this information divide, we conducted an exploratory study where thirty-two different metrics from the offline training data were extracted. A correlation analysis and an ordinary least squares regression were implemented to investigate the relationship between the offline metrics and six aspects online use. The results indicate that the current offline standard, classification accuracy, is a poor indicator of usability and that other metrics may hold predictive power. The metrics identified in this work also may constitute more representative evaluation criteria when designing and reporting new control schemes. Furthermore, linear combinations of offline training metrics generate substantially more accurate predictions than using individual metrics. We found that the offline metric feature efficiency generated the best predictions for the usability metric throughput. A combination of two offline metrics (mean semi-principal axes and mean absolute value) significantly outperformed feature efficiency alone, with a 166% increase in the predicted R² value (i.e., VEcv). These findings suggest that combinations of metrics could provide a more robust framework for predicting usability.

Evaluation of windowing techniques for intramuscular EMG-based diagnostic, rehabilitative and assistive devices

Objective.Intramuscular electromyography (iEMG) signals, invasively recorded, directly from the muscles are used to diagnose various neuromuscular disorders/diseases and to control rehabilitative and assistive robotic devices. iEMG signals are potentially being used in neurology, kinesiology, rehabilitation and ergonomics, to detect/diagnose various diseases/disorders. Electromyography-based classification and analysis systems are being designed and tested for the classification of various neuromuscular disorders and to control rehabilitative and assistive robotic devices. Many studies have explored parameters such as the pre-processing, feature extraction and selection of classifiers that can affect the performance and efficacy of iEMG-based classification systems. The pre-processing stage includes the removal of any unwanted noise from the original signal and windowing of the signal.Approach.This study investigated and presented the optimum windowing configurations for robust control and better performance results of an iEMG-based analysis system based on the stationarity rate (SR) and classification accuracy. Both disjoint and overlap, windowing techniques with varying window and overlap sizes have been investigated using a machine learning-based classification algorithm called linear discriminant analysis.Main results.The optimum window size ranges are from 200-300 ms for the disjoint and 225-300 ms for the overlap windowing technique, respectively. The inferred results show that for the overlap windowing technique the optimum range of overlap size is from 10%-30% of the length of window size. The mean classification accuracy (MCA) and mean stationarity rate (MSR) were found to be lower in the disjoint windowing technique compared to overlap windowing at all investigated overlap sizes. Statistical analysis (two-way analysis of variance test) showed that the MSR and MCA of the overlap windowing technique was significantly different at overlap sizes of 10%-30% (p-values < 0.05).Significance.The presented results can be used to achieve the best possible classification results and SR for any iEMG-based real-time diagnosis, detection and control system, which can enhance the performance of the system significantly.

Understanding Limb Position and External Load Effects on Real-Time Pattern Recognition Control in Amputees

Limb position is a factor that negatively affects myoelectric pattern recognition classification accuracy. However, prior studies evaluating impact on real-time control for upper-limb amputees have done so without a physical prosthesis on the residual limb. It remains unclear how limb position affects real-time pattern recognition control in amputees when their residual limb is supporting various weights. We used a virtual reality target achievement control test to evaluate the effects of limb position and external load on real-time pattern recognition control in fourteen intact limb subjects and six major upper limb amputee subjects. We also investigated how these effects changed based on different control system training methods. In a static training method, subjects kept their unloaded arm by their side with the elbow bent whereas in the dynamic training method, subjects moved their arm throughout a workspace while supporting a load. When static training was used, limb position significantly affected real-time control in all subjects. However, amputee subjects were still able to adequately complete tasks in all conditions, even in untrained limb positions. Moreover, increasing external loads decreased controller performance, albeit to a lesser extent in amputee subjects. The effects of limb position did not change as load increased, and vice versa. In intact limb subjects, dynamic training significantly reduced the limb position effect but did not completely remove them. In contrast, in amputee subjects, dynamic training eliminated the limb position effect in three out of four outcome measures. However, it did not reduce the effects of load for either subject population. These findings suggest that results obtained from intact limb subjects may not generalize to amputee subjects and that advanced training methods can substantially improve controller robustness to different limb positions regardless of limb loading.

Real-Time Hand Gesture Recognition Using Surface Electromyography and Machine Learning: A Systematic Literature Review

Today, daily life is composed of many computing systems, therefore interacting with them in a natural way makes the communication process more comfortable. Human-Computer Interaction (HCI) has been developed to overcome the communication barriers between humans and computers. One form of HCI is Hand Gesture Recognition (HGR), which predicts the class and the instant of execution of a given movement of the hand. One possible input for these models is surface electromyography (EMG), which records the electrical activity of skeletal muscles. EMG signals contain information about the intention of movement generated by the human brain. This systematic literature review analyses the state-of-the-art of real-time hand gesture recognition models using EMG data and machine learning. We selected and assessed 65 primary studies following the Kitchenham methodology. Based on a common structure of machine learning-based systems, we analyzed the structure of the proposed models and standardized concepts in regard to the types of models, data acquisition, segmentation, preprocessing, feature extraction, classification, postprocessing, real-time processing, types of gestures, and evaluation metrics. Finally, we also identified trends and gaps that could open new directions of work for future research in the area of gesture recognition using EMG.

A Subject-Transfer Framework Based on Single-Trial EMG Analysis Using Convolutional Neural Networks

In recent years, electromyography (EMG)-based practical myoelectric interfaces have been developed to improve the quality of daily life for people with physical disabilities. With these interfaces, it is very important to decode a user's movement intention, to properly control the external devices. However, improving the performance of these interfaces is difficult due to the high variations in the EMG signal patterns caused by intra-user variability. Therefore, this paper proposes a novel subject-transfer framework for decoding hand movements, which is robust in terms of intra-user variability. In the proposed framework, supportive convolutional neural network (CNN) classifiers, which are pre-trained using the EMG data of several subjects, are selected and fine-tuned for the target subject via single-trial analysis. Then, the target subject's hand movements are classified by voting the outputs of the supportive CNN classifiers. The feasibility of the proposed framework is validated with NinaPro databases 2 and 3, which comprise 49 hand movements of 40 healthy and 11 amputee subjects, respectively. The experimental results indicate that, when compared to the self-decoding framework, which uses only the target subject's data, the proposed framework can successfully decode hand movements with improved performance in both healthy and amputee subjects. From the experimental results, the proposed subject-transfer framework can be seen to represent a useful tool for EMG-based practical myoelectric interfaces controlling external devices.

PaWFE: Fast Signal Feature Extraction Using Parallel Time Windows

Motivation: Hand amputations can dramatically affect the quality of life of a person. Researchers are developing surface electromyography and machine learning solutions to control dexterous and robotic prosthetic hands, however long computational times can slow down this process.

Objective: This paper aims at creating a fast signal feature extraction algorithm that can extract widely used features and allow researchers to easily add new ones.

Methods: PaWFE (Parallel Window Feature Extractor) extracts the signal features from several time windows in parallel. The MATLAB code is publicly available and supports several time domain and frequency features. The code was tested and benchmarked using 1,2,4,8,16,32, and 48 threads on a server with four Xeon E7- 4820 and 128 GB RAM using the first 5 datasets of the Ninapro database, that are recorded with different acquisition setups.

Results: The parallel time window analysis approach allows to reduce the computational time up to 20 times when using 32 cores, showing a very good scalability. Signal features can be extracted in few seconds from an entire data acquisition and in <100 ms from a single time window, easily reducing of up to over 15 times the feature extraction procedure in comparison to traditional approaches. The code allows users to easily add new signal feature extraction scripts, that can be added to the code and on the Ninapro website upon request.

Significance: The code allows researchers in machine learning and biosignals data analysis to easily and quickly test modern machine learning approaches on big datasets and it can be used as a resource for real time data analysis too.

A Physics-based Virtual Reality Environment to Quantify Functional Performance of Upper-limb Prostheses

Usability of upper-limb prostheses remains to be a challenge due to the complexity of hand-object interactions in activities of daily living. Functional evaluation is critical for the optimization of prosthesis performance during device design and parameter tuning phase. Therefore, we implemented a low-cost physics-based virtual reality environment (VRE) capable of simulating wide range of object grasping and manipulation tasks to enable human-in-the-loop optimization. Importantly, our novel VRE can assess user performance quantitatively using movement kinematics and interaction forces. We present a preliminary experiment to validate our VRE. Four able-bodied subjects performed object transfer tasks with a simulated myoelectric one DoF soft-synergy prosthetic hand, while wearing braces to restrain different levels of wrist motion. We found that the task completion time was similar across conditions, however limited wrist pronation led to more shoulder compensatory motion whereas challenging object orientation caused more torso compensatory motion.

Adapting myoelectric control in real-time using a virtual environment

BACKGROUND

Pattern recognition technology allows for more intuitive control of myoelectric prostheses. However, the need to collect electromyographic data to initially train the pattern recognition system, and to re-train it during prosthesis use, adds complexity that can make using such a system difficult. Although experienced clinicians may be able to guide users to ensure successful data collection methods, they may not always be available when a user needs to (re)train their device.

METHODS

Here we present an engaging and interactive virtual reality environment for optimal training of a myoelectric controller. Using this tool, we evaluated the importance of training a classifier actively (i.e., moving the residual limb during data collection) compared to passively (i.e., maintaining the limb in a single, neutral orientation), and whether computational adaptation through serious gaming can improve performance.

RESULTS

We found that actively trained classifiers performed significantly better than passively trained classifiers for non-amputees (P < 0.05). Furthermore, collecting data passively with minimal instruction, paired with computational adaptation in a virtual environment, significantly improved real-time performance of myoelectric controllers.

CONCLUSION

These results further support previous work which suggested active movements during data collection can improve pattern recognition systems. Furthermore, adaptation within a virtual guided serious game environment can improve real-time performance of myoelectric controllers.

EMG Pattern Recognition Control of the DEKA Arm: Impact on User Ratings of Satisfaction and Usability

The DEKA Arm has multiple degrees of freedom which historically have been operated primarily by inertial measurement units (IMUs). However, the IMUs are not appropriate for all potential users; new control methods are needed. The purposes of this study were: 1) to describe usability and satisfaction of two controls methods—IMU and myoelectric pattern recognition (EMG-PR) controls—and 2) to compare ratings by control and amputation level. A total of 36 subjects with transradial (TR) or transhumeral (TH) amputation participated in the study. The subjects included 11 EMG-PR users (82% TR) and 25 IMU users (68% TR). The study consisted of in-laboratory training (Part A) and home use (Part B). The subjects were administered the Trinity Amputation and Prosthesis Experience satisfaction scale and other usability and satisfaction measures. Wilcoxon rank-sum tests compared the differences by control type. The differences were compared for those who did and did not want a DEKA Arm. The preferences for features of the DEKA Arm were compared by control type. The comparisons revealed poorer ratings of skill, comfort, and weight among EMG-PR users. The TR amputees using IMUs rated usability more favorably. TH amputees rated usability similarly. The TR amputees using EMG-PR were less satisfied with weight, pinch grip, and wrist display, whereas the TH amputees were less satisfied with the full system, wires/cables, and battery. Usability and satisfaction declined after Part B for EMG-PR users. Overall, we found that the IMU users rated the DEKA Arm and the controls more favorably than the EMG-PR users. The findings indicate that the EMG-PR system we tested was less well accepted than the IMUs for control of the DEKA Arm.

Improving the functionality, robustness, and adaptability of myoelectric control for dexterous motion restoration

The development of advanced and effective human-machine interfaces, especially for amputees to control their prostheses, is very high priority and a very active area of research. An intuitive control method should retain an adequate level of functionality for dexterous operation, provide robustness against confounding factors, and supply adaptability for diverse long-term usage, all of which are current problems being tackled by researchers. This paper reviews the state-of-the-art, as well as, the limitations of current myoelectric signal control (MSC) methods. To address the research topic on functionality, we review different approaches to prosthetic hand control (DOF configuration, discrete or simultaneous, etc.), and how well the control is performed (accuracy, response, intuitiveness, etc.). To address the research on robustness, we review the confounding factors (limb positions, electrode shift, force variance, and inadvertent activity) that affect the stability of the control performance. Lastly, to address adaptability, we review the strategies that can automatically adjust the classifier for different individuals and for long-term usage. This review provides a thorough overview of the current MSC methods and helps highlight the current areas of research focus and resulting clinic usability for the MSC methods for upper-limb prostheses.

Effects of Confidence-Based Rejection on Usability and Error in Pattern Recognition-Based Myoelectric Control

Rejection of movements based on the confidence in the classification decision has previously been demonstrated to improve the usability of pattern recognition based myoelectric control. To this point, however, the optimal rejection threshold has been determined heuristically, and it is not known how different thresholds affect the tradeoff between error mitigation and false rejections in real-time closed-loop control. To answer this question, 24 able-bodied subjects completed a real-time Fitts' law-style virtual cursor control task using a support vector machine classifier. It was found that rejection improved information throughput at all thresholds, with the best performance coming at thresholds between 0.60 and 0.75. Two fundamental types of error were defined and identified: operator error (identifiable, repeatable behaviors, directly attributable to the user), and systemic error (other errors attributable to misclassification or noise). The incidence of both operator and systemic errors were found to decrease as rejection threshold increased. Moreover, while the incidence of all error types correlated strongly with path efficiency, only systemic errors correlated strongly with throughput and trial completion rate. Interestingly, more experienced users were found to commit as many errors as novice users, despite performing better in the Fitts' task, suggesting that there is more to usability than error prevention alone. Nevertheless, these results demonstrate the usability gains possible with rejection across a range of thresholds for both novice and experienced users alike.

Virtual Integration Environment as an Advanced Prosthetic Limb Training Platform

Background: Despite advances in prosthetic development and neurorehabilitation, individuals with upper extremity (UE) loss continue to face functional and psychosocial challenges following amputation. Recent advanced myoelectric prostheses offer intuitive control over multiple, simultaneous degrees of motion and promise sensory feedback integration, but require complex training to effectively manipulate. We explored whether a virtual reality simulator could be used to teach dexterous prosthetic control paradigms to individuals with UE loss. Methods: Thirteen active-duty military personnel with UE loss (14 limbs) completed twenty, 30-min passive motor training sessions over 1-2 months. Participants were asked to follow the motions of a virtual avatar using residual and phantom limbs, and electrical activity from the residual limb was recorded using surface electromyography. Eight participants (nine limbs), also completed twenty, 30-min active motor training sessions. Participants controlled a virtual avatar through three motion sets of increasing complexity (Basic, Advanced, and Digit) and were scored on how accurately they performed requested motions. Score trajectory was assessed as a function of time using longitudinal mixed effects linear regression. Results: Mean classification accuracy for passive motor training was 43.8 ± 10.7% (14 limbs, 277 passive sessions). In active motor sessions, >95% classification accuracy (which we used as the threshold for prosthetic acceptance) was achieved by all participants for Basic sets and by 50% of participants in Advanced and Digit sets. Significant improvement in active motor scores over time was observed in Basic and Advanced sets (per additional session: β-coefficient 0.125, p = 0.022; β-coefficient 0.45, p = 0.001, respectively), and trended toward significance for Digit sets (β-coefficient 0.594, p = 0.077). Conclusions: These results offer robust evidence that a virtual reality training platform can be used to quickly and efficiently train individuals with UE loss to operate advanced prosthetic control paradigms. Participants can be trained to generate muscle contraction patterns in residual limbs that are interpreted with high accuracy by computer software as distinct active motion commands. These results support the potential viability of advanced myoelectric prostheses relying on pattern recognition feedback or similar controls systems.

Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses

Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.

User experience of controlling the DEKA Arm with EMG pattern recognition

Introduction

A commercially available EMG Pattern Recognition (EMG-PR) control system was adapted to interface with the multi-degree of freedom (DOF) DEKA Arm.

Purpose

To describe users’ experience of controlling the DEKA Arm using EMG-PR.

Methods

Sample: Twelve persons with upper limb amputation participated, 10 with transradial (TR), 2 with transhumeral (TH) level amputation. Ten were male, and 11 were users of a prosthesis at baselines. Design: This was a two-part study consisting of in-laboratory prosthetic training (Part A) and up to 12 weeks of home use of the prosthesis (Part B). Data collection: Qualitative data were collected through open-ended survey questions and semi-structured interviews. Data Analysis: The study used a qualitative case series design with a constant comparative approach to identify common categories of experience. Coding categories were iteratively refined until saturation of categories was achieved. The data were organized in a primary category, major categories of experience, factors impacting experience, and broader contextual factors.

Results

Users had mixed perspectives on the desirability of the EMG-PR control system in combination with the DEKA Arm. Major aspects of user experience related to the system complexity, process of calibrating, and functional benefits. Factors influencing user experience included training and acclimation, fatigue, prosthesis design, technical issues and control changes. Broader contextual factors, both personal and environmental, also impacted users’ experience.

Discussion/Conclusion

This study provided an in-depth description of user experience operating the DEKA Arm using EMG-PR control. The majority of participants expressed a preference for the controls of their personal prosthesis and controls rather than the iteration of EMG-PR controlled DEKA Arm used in this study. Most were positive about the future potential of EMG-PR as a control method. An understanding of patient experience will assist clinicians and patients choosing prosthetic options.

Control within a virtual environment is correlated to functional outcomes when using a physical prosthesis

Background

Advances such as targeted muscle reinnervation and pattern recognition control may provide improved control of upper limb myoelectric prostheses, but evaluating user function remains challenging. Virtual environments are cost-effective and immersive tools that are increasingly used to provide practice and evaluate prosthesis control, but the relationship between virtual and physical outcomes—i.e., whether practice in a virtual environment translates to improved physical performance—is not understood.

Methods

Nine people with transhumeral amputations who previously had targeted muscle reinnervation surgery were fitted with a myoelectric prosthesis comprising a commercially available elbow, wrist, terminal device, and pattern recognition control system. Virtual and physical outcome measures were obtained before and after a 6-week home trial of the prosthesis.

Results

After the home trial, subjects showed statistically significant improvements (p < 0.05) in offline classification error, the virtual Target Achievement Control test, and the physical Southampton Hand Assessment Procedure and Box and Blocks Test. A trend toward improvement was also observed in the physical Clothespin Relocation task and Jebsen-Taylor test; however, these changes were not statistically significant. The median completion time in the virtual test correlated strongly and significantly with the Southampton Hand Assessment Procedure (p = 0.05, R = − 0.86), Box and Blocks Test (p = 0.007, R = − 0.82), Jebsen-Taylor Test (p = 0.003, R = 0.87), and the Assessment of Capacity for Myoelectric Control (p = 0.005,R = − 0.85). The classification error performance only had a significant correlation with the Clothespin Relocation Test (p = 0.018, R = .76).

Conclusions

In-home practice with a pattern recognition-controlled prosthesis improves functional control, as measured by both virtual and physical outcome measures. However, virtual measures need to be validated and standardized to ensure reliability in a clinical or research setting.

Trial registration

This is a registered clinical trial: NCT03097978.

High-density force myography: A possible alternative for upper-limb prosthetic control

Several multiple degree-of-freedom upper-limb prostheses that have the promise of highly dexterous control have recently been developed. Inadequate controllability, however, has limited adoption of these devices. Introducing more robust control methods will likely result in higher acceptance rates. This work investigates the suitability of using high-density force myography (HD-FMG) for prosthetic control. HD-FMG uses a high-density array of pressure sensors to detect changes in the pressure patterns between the residual limb and socket caused by the contraction of the forearm muscles. In this work, HD-FMG outperforms the standard electromyography (EMG)-based system in detecting different wrist and hand gestures. With the arm in a fixed, static position, eight hand and wrist motions were classified with 0.33% error using the HD-FMG technique. Comparatively, classification errors in the range of 2.2%-11.3% have been reported in the literature for multichannel EMG-based approaches. As with EMG, position variation in HD-FMG can introduce classification error, but incorporating position variation into the training protocol reduces this effect. Channel reduction was also applied to the HD-FMG technique to decrease the dimensionality of the problem as well as the size of the sensorized area. We found that with informed, symmetric channel reduction, classification error could be decreased to 0.02%.

Characterisation of the Clothespin Relocation Test as a functional assessment tool

Method

The Clothespin Relocation Test has been adapted from an arm training tool to create an instrument to measure hand function. It is based on the time to move three clothespins from a horizontal to a vertical bar, and back. To be generally useful, the measures need to have their psychometric properties investigated. This paper measures the characteristics of an able-bodied population to gain an understanding of the underlying statistical properties of the test, in order that it can then be used to compare with different subject groups. Fifty adults (29 males, 21 females, mean age 31) were tested with five runs of three clothespins moved up and then down. Ten subjects returned twice more to observe repeatability.

Results

There was a non-Gaussian range of times, from 2.5 to 7.37 s. Mean time for Up was 4.1 s, and was 4.0 s for Down, with a skew towards the faster times of 0.57 for Up and 0.97 for Down. Over the three sessions there was a small (not significant) increase in speed 4.1 ± 0.5 s first run Down to 3.5 ± 0.4 s for third.

Conclusion

These initial tests confirm that it has potential to be used as a measurement of the performance of arm movement.

Evaluation of Computer-Based Target Achievement Tests for Myoelectric Control

Real-time evaluation of novel prosthetic control schemes is critical for translational research on artificial limbs. Recently, two computer-based, real-time evaluation tools, the target achievement control (TAC) test and the Fitts' law test (FLT), have been proposed to assess real-time controllability. Whereas TAC tests provides an anthropomorphic visual representation of the limb at the cost of confusing visual feedback, FLT clarifies the current and target locations by simplified non-anthropomorphic representations. Here, we investigated these two approaches and quantified differences in common performance metrics that can result from the chosen method of visual feedback. Ten able-bodied and one amputee subject performed target achievement tasks corresponding to the FLT and TAC test with equivalent indices of difficulty. Able-bodied subjects exhibited significantly (p <0.05) better completion rate, path efficiency, and overshoot when performing the FLT, although no significant difference was seen in throughput performance. The amputee subject showed significantly better performance in overshoot at the FLT, but showed no significant difference in completion rate, path efficiency, and throughput. Results from the FLT showed a strong linear relationship between the movement time and the index of difficulty (R² = 0.96), whereas TAC test results showed no apparent linear relationship (R² = 0.19). These results suggest that in relatively similar conditions, the confusing location of virtual limb representation used in the TAC test contributed to poorer performance. Establishing an understanding of the biases of various evaluation protocols is critical to the translation of research into clinical practice.

Comparison of six electromyography acquisition setups on hand movement classification tasks

Hand prostheses controlled by surface electromyography are promising due to the non-invasive approach and the control capabilities offered by machine learning. Nevertheless, dexterous prostheses are still scarcely spread due to control difficulties, low robustness and often prohibitive costs. Several sEMG acquisition setups are now available, ranging in terms of costs between a few hundred and several thousand dollars. The objective of this paper is the relative comparison of six acquisition setups on an identical hand movement classification task, in order to help the researchers to choose the proper acquisition setup for their requirements. The acquisition setups are based on four different sEMG electrodes (including Otto Bock, Delsys Trigno, Cometa Wave + Dormo ECG and two Thalmic Myo armbands) and they were used to record more than 50 hand movements from intact subjects with a standardized acquisition protocol. The relative performance of the six sEMG acquisition setups is compared on 41 identical hand movements with a standardized feature extraction and data analysis pipeline aimed at performing hand movement classification. Comparable classification results are obtained with three acquisition setups including the Delsys Trigno, the Cometa Wave and the affordable setup composed of two Myo armbands. The results suggest that practical sEMG tests can be performed even when costs are relevant (e.g. in small laboratories, developing countries or use by children). All the presented datasets can be used for offline tests and their quality can easily be compared as the data sets are publicly available.

Gaussian Process Autoregression for Simultaneous Proportional Multi-Modal Prosthetic Control With Natural Hand Kinematics

Matching the dexterity, versatility, and robustness of the human hand is still an unachieved goal in bionics, robotics, and neural engineering. A major limitation for hand prosthetics lies in the challenges of reliably decoding user intention from muscle signals when controlling complex robotic hands. Most of the commercially available prosthetic hands use muscle-related signals to decode a finite number of predefined motions and some offer proportional control of open/close movements of the whole hand. Here, in contrast, we aim to offer users flexible control of individual joints of their artificial hand. We propose a novel framework for decoding neural information that enables a user to independently control 11 joints of the hand in a continuous manner-much like we control our natural hands. Toward this end, we instructed six able-bodied subjects to perform everyday object manipulation tasks combining both dynamic, free movements (e.g., grasping) and isometric force tasks (e.g., squeezing). We recorded the electromyographic and mechanomyographic activities of five extrinsic muscles of the hand in the forearm, while simultaneously monitoring 11 joints of hand and fingers using a sensorized data glove that tracked the joints of the hand. Instead of learning just a direct mapping from current muscle activity to intended hand movement, we formulated a novel autoregressive approach that combines the context of previous hand movements with instantaneous muscle activity to predict future hand movements. Specifically, we evaluated a linear vector autoregressive moving average model with exogenous inputs and a novel Gaussian process ( ) autoregressive framework to learn the continuous mapping from hand joint dynamics and muscle activity to decode intended hand movement. Our approach achieves high levels of performance (RMSE of 8°/s and ). Crucially, we use a small set of sensors that allows us to control a larger set of independently actuated degrees of freedom of a hand. This novel undersensored control is enabled through the combination of nonlinear autoregressive continuous mapping between muscle activity and joint angles. The system evaluates the muscle signals in the context of previous natural hand movements. This enables us to resolve ambiguities in situations, where muscle signals alone cannot determine the correct action as we evaluate the muscle signals in their context of natural hand movements. autoregression is a particularly powerful approach which makes not only a prediction based on the context but also represents the associated uncertainty of its predictions, thus enabling the novel notion of risk-based control in neuroprosthetics. Our results suggest that autoregressive approaches with exogenous inputs lend themselves for natural, intuitive, and continuous control in neurotechnology, with the particular focus on prosthetic restoration of natural limb function, where high dexterity is required for complex movements.

NLR, MLP, SVM, and LDA: a comparative analysis on EMG data from people with trans-radial amputation

Background

Currently, the typically adopted hand prosthesis surface electromyography (sEMG) control strategies do not provide the users with a natural control feeling and do not exploit all the potential of commercially available multi-fingered hand prostheses. Pattern recognition and machine learning techniques applied to sEMG can be effective for a natural control based on the residual muscles contraction of amputated people corresponding to phantom limb movements. As the researches has reached an advanced grade accuracy, these algorithms have been proved and the embedding is necessary for the realization of prosthetic devices. The aim of this work is to provide engineering tools and indications on how to choose the most suitable classifier, and its specific internal settings for an embedded control of multigrip hand prostheses.

Methods

By means of an innovative statistical analysis, we compare 4 different classifiers: Nonlinear Logistic Regression, Multi-Layer Perceptron, Support Vector Machine and Linear Discriminant Analysis, which was considered as ground truth. Experimental tests have been performed on sEMG data collected from 30 people with trans-radial amputation, in which the algorithms were evaluated for both performance and computational burden, then the statistical analysis has been based on the Wilcoxon Signed-Rank test and statistical significance was considered at p < 0.05.

Results

The comparative analysis among NLR, MLP and SVM shows that, for either classification performance and for the number of classification parameters, SVM attains the highest values followed by MLP, and then by NLR. However, using as unique constraint to evaluate the maximum acceptable complexity of each classifier one of the typically available memory of a high performance microcontroller, the comparison pointed out that for people with trans-radial amputation the algorithm that produces the best compromise is NLR closely followed by MLP. This result was also confirmed by the comparison with LDA with time domain features, which provided not significant differences of performance and computational burden between NLR and LDA.

Conclusions

The proposed analysis would provide innovative engineering tools and indications on how to choose the most suitable classifier based on the application and the desired results for prostheses control.

Study on Interaction Between Temporal and Spatial Information in Classification of EMG Signals for Myoelectric Prostheses

Advanced forearm prosthetic devices employ classifiers to recognize different electromyography (EMG) signal patterns, in order to identify the user's intended motion gesture. The classification accuracy is one of the main determinants of real-time controllability of a prosthetic limb and hence the necessity to achieve as high an accuracy as possible. In this paper, we study the effects of the temporal and spatial information provided to the classifier on its off-line performance and analyze their inter-dependencies. EMG data associated with seven practical hand gestures were recorded from partial-hand and trans-radial amputee volunteers as well as able-bodied volunteers. An extensive investigation was conducted to study the effect of analysis window length, window overlap, and the number of electrode channels on the classification accuracy as well as their interactions. Our main discoveries are that the effect of analysis window length on classification accuracy is practically independent of the number of electrodes for all participant groups; window overlap has no direct influence on classifier performance, irrespective of the window length, number of channels, or limb condition; the type of limb deficiency and the existing channel count influence the reduction in classification error achieved by adding more number of channels; partial-hand amputees outperform trans-radial amputees, with classification accuracies of only 11.3% below values achieved by able-bodied volunteers.

Comparison of Constant-Posture Force-Varying EMG-Force Dynamic Models About the Elbow

Numerous techniques have been used to minimize error in relating the surface electromyogram (EMG) to elbow joint torque. We compare the use of three techniques to further reduce error. First, most EMG-torque models only use estimates of EMG standard deviation as inputs. We studied the additional features of average waveform length, slope sign change rate and zero crossing rate. Second, multiple channels of EMG from the biceps, and separately from the triceps, have been combined to produce two low-variance model inputs. We contrasted this channel combination with using each EMG separately. Third, we previously modeled nonlinearity in the EMG-torque relationship via a polynomial. We contrasted our model versus that of the classic exponential power law of Vredenbregt and Rau (1973). Results from 65 subjects performing constant-posture, force-varying contraction gave a "baseline" comparison error (i.e., error with none of the new techniques) of 5.5 ± 2.3% maximum flexion voluntary contraction (%MVC_F). Combining the techniques of multiple features with individual channels reduced error to 4.8 ± 2.2 %MVC_F, while combining individual channels with the power-law model reduced error to 4.7 ± 2.0 %MVC_F. The new techniques further reduced error from that of the baseline by ≈ 15 %.

Processing of surface EMG through pattern recognition techniques aimed at classifying shoulder joint movements

Artificial arms for shoulder disarticulation need a high number of degrees of freedom to be controlled. In order to control a prosthetic shoulder joint, an intention detection system based on surface electromyography (sEMG) pattern recognition methods was proposed and experimentally investigated. Signals from eight trunk muscles that are generally preserved after shoulder disarticulation were recorded from a group of eight normal subjects in nine shoulder positions. After data segmentation, four different features were extracted (sample entropy, cepstral coefficients of the 4th order, root mean square and waveform length) and classified by means of linear discriminant analysis. The classification accuracy was 92.1% and this performance reached 97.9% after reducing the positions considered to five classes. To reduce the computational cost, the two channels with the least discriminating information were neglected yielding to a classification accuracy diminished by just 4.08%.

Deep Learning with Convolutional Neural Networks Applied to Electromyography Data: A Resource for the Classification of Movements for Prosthetic Hands

Natural control methods based on surface electromyography (sEMG) and pattern recognition are promising for hand prosthetics. However, the control robustness offered by scientific research is still not sufficient for many real life applications, and commercial prostheses are capable of offering natural control for only a few movements. In recent years deep learning revolutionized several fields of machine learning, including computer vision and speech recognition. Our objective is to test its methods for natural control of robotic hands via sEMG using a large number of intact subjects and amputees. We tested convolutional networks for the classification of an average of 50 hand movements in 67 intact subjects and 11 transradial amputees. The simple architecture of the neural network allowed to make several tests in order to evaluate the effect of pre-processing, layer architecture, data augmentation and optimization. The classification results are compared with a set of classical classification methods applied on the same datasets. The classification accuracy obtained with convolutional neural networks using the proposed architecture is higher than the average results obtained with the classical classification methods, but lower than the results obtained with the best reference methods in our tests. The results show that convolutional neural networks with a very simple architecture can produce accurate results comparable to the average classical classification methods. They show that several factors (including pre-processing, the architecture of the net and the optimization parameters) can be fundamental for the analysis of sEMG data. Larger networks can achieve higher accuracy on computer vision and object recognition tasks. This fact suggests that it may be interesting to evaluate if larger networks can increase sEMG classification accuracy too.

A mechatronics platform to study prosthetic hand control using EMG signals

In this paper, a low-cost mechatronics platform for the design and development of robotic hands as well as a surface electromyogram (EMG) pattern recognition system is proposed. This paper also explores various EMG classification techniques using a low-cost electronics system in prosthetic hand applications. The proposed platform involves the development of a four channel EMG signal acquisition system; pattern recognition of acquired EMG signals; and development of a digital controller for a robotic hand. Four-channel surface EMG signals, acquired from ten healthy subjects for six different movements of the hand, were used to analyse pattern recognition in prosthetic hand control. Various time domain features were extracted and grouped into five ensembles to compare the influence of features in feature-selective classifiers (SLR) with widely considered non-feature-selective classifiers, such as neural networks (NN), linear discriminant analysis (LDA) and support vector machines (SVM) applied with different kernels. The results divulged that the average classification accuracy of the SVM, with a linear kernel function, outperforms other classifiers with feature ensembles, Hudgin's feature set and auto regression (AR) coefficients. However, the slight improvement in classification accuracy of SVM incurs more processing time and memory space in the low-level controller. The Kruskal-Wallis (KW) test also shows that there is no significant difference in the classification performance of SLR with Hudgin's feature set to that of SVM with Hudgin's features along with AR coefficients. In addition, the KW test shows that SLR was found to be better in respect to computation time and memory space, which is vital in a low-level controller. Similar to SVM, with a linear kernel function, other non-feature selective LDA and NN classifiers also show a slight improvement in performance using twice the features but with the drawback of increased memory space requirement and time. This prototype facilitated the study of various issues of pattern recognition and identified an efficient classifier, along with a feature ensemble, in the implementation of EMG controlled prosthetic hands in a laboratory setting at low-cost. This platform may help to motivate and facilitate prosthetic hand research in developing countries.

Cardinality as a highly descriptive feature in myoelectric pattern recognition for decoding motor volition

Accurate descriptors of muscular activity play an important role in clinical practice and rehabilitation research. Such descriptors are features of myoelectric signals extracted from sliding time windows. A wide variety of myoelectric features have been used as inputs to pattern recognition algorithms that aim to decode motor volition. The output of these algorithms can then be used to control limb prostheses, exoskeletons, and rehabilitation therapies. In the present study, cardinality is introduced and compared with traditional time-domain (Hudgins' set) and other recently proposed myoelectric features (for example, rough entropy). Cardinality was found to consistently outperform other features, including those that are more sophisticated and computationally expensive, despite variations in sampling frequency, time window length, contraction dynamics, type, and number of movements (single or simultaneous), and classification algorithms. Provided that the signal resolution is kept between 12 and 14 bits, cardinality improves myoelectric pattern recognition for the prediction of motion volition. This technology is instrumental for the rehabilitation of amputees and patients with motor impairments where myoelectric signals are viable. All code and data used in this work is available online within BioPatRec.

Real-Time Task Discrimination for Myoelectric Control Employing Task-Specific Muscle Synergies

We present a novel formulation that employs task-specific muscle synergies and state-space representation of neural signals to tackle the challenging myoelectric control problem for lower arm prostheses. The proposed framework incorporates information about muscle configurations, e.g., muscles acting synergistically or in agonist/antagonist pairs, using the hypothesis of muscle synergies. The synergy activation coefficients are modeled as the latent system state and are estimated using a constrained Kalman filter. These task-dependent synergy activation coefficients are estimated in real-time from the electromyogram (EMG) data and are used to discriminate between various tasks. The task discrimination is helped by a post-processing algorithm that uses posterior probabilities. The proposed algorithm is robust as well as computationally efficient, yielding a decision with > 90% discrimination accuracy in approximately 3 ms . The real-time performance and controllability of the algorithm were evaluated using the targeted achievement control (TAC) test. The proposed algorithm outperformed common machine learning algorithms for single- as well as multi-degree-of-freedom (DOF) tasks in both off-line discrimination accuracy and real-time controllability (p < 0.01).

Spatial correlation of high density EMG signals provides features robust to electrode number and shift in pattern recognition for myocontrol

Research on pattern recognition for myoelectric control has usually focused on a small number of electromyography (EMG) channels because of better clinical acceptability and low computational load with respect to multi-channel EMG. However, recently, high density (HD) EMG technology has substantially improved, also in practical usability, and can thus be applied in myocontrol. HD EMG provides several closely spaced recordings in multiple locations over the skin surface. This study considered the use of HD EMG for controlling upper limb prostheses, based on pattern recognition. In general, robustness and reliability of classical pattern recognition systems are influenced by electrode shift in dons and doff, and by the presence of malfunctioning channels. The aim of this study is to propose a new approach to attenuate these issues. The HD EMG grid of electrodes is an ensemble of sensors that records data spatially correlated. The experimental variogram, which is a measure of the degree of spatial correlation, was used as feature for classification, contrary to previous approaches that are based on temporal or frequency features. The classification based on the variogram was tested on seven able-bodied subjects and one subject with amputation, for the classification of nine and seven classes, respectively. The performance of the proposed approach was comparable with the classic methods based on time-domain and autoregressive features (average classification accuracy over all methods ∼ 95% for nine classes). However, the new spatial features demonstrated lower sensitivity to electrode shift ( ± 1 cm) with respect to the classic features . When even just one channel was noisy, the classification accuracy dropped by ∼ 10% for all methods. However, the new method could be applied without any retraining to a subset of high-quality channels whereas the classic methods require retraining when some channels are omitted. In conclusion, the new spatial feature space proposed in this study improved the robustness to electrode number and shift in myocontrol with respect to previous approaches.

Boosting-based EMG patterns classification scheme for robustness enhancement

The high conventional accuracy of pattern recognition-based surface myoelectric classification in laboratory experiments does not necessarily result in high accessibility to practical protheses. An obvious reason is the effect of signals of untrained classes caused by the relatively small training dataset. In order to make the classifier robust to untrained classes, a classification scheme is developed based on boosting and random forest classifiers in this paper. Meanwhile, a threshold, the post probability of the prediction, is introduced as a balance (i.e., adjust) between the accurate classification and the rejection of the samples belonging to some untrained classes. The experiments are conducted to compare with other two schemes using linear discriminant analysis and support vector machines. Surface electromyogram signals, labeled with seven isometric movements, are collected from six healthy subjects' forearm. It is shown that the proposed scheme can reach up to about 92% accuracy in recognizing trained classes and 20% for untrained classes. Through adjusting the threshold, the accuracy of rejecting untrained classes reaches up to around 80%, with small decrease in recognizing trained classes (down to 80%). In the analysis of experiments' results, we also find that the proposed scheme has better error distribution among the classes.

Extrinsic finger and thumb muscles command a virtual hand to allow individual finger and grasp control

Fine-wire intramuscular electrodes were used to obtain electromyogram (EMG) signals from six extrinsic hand muscles associated with the thumb, index, and middle fingers. Subjects' EMG activity was used to control a virtual three-degree-of-freedom (DOF) hand as they conformed the hand to a sequence of hand postures testing two controllers: direct EMG control and pattern recognition control. Subjects tested two conditions using each controller: starting the hand from a predefined neutral posture before each new posture and starting the hand from the previous posture in the sequence. Subjects demonstrated their abilities to simultaneously, yet individually, move all three DOFs during the direct EMG control trials; however, results showed subjects did not often utilize this feature. Performance metrics such as failure rate and completion time showed no significant difference between the two controllers.

On the usability of intramuscular EMG for prosthetic control: a Fitts' Law approach

Previous studies on intramuscular EMG based control used offline data analysis. The current study investigates the usability of intramuscular EMG in two degree-of-freedom using a Fitts' Law approach by combining classification and proportional control to perform a task, with real time feedback of user performance. Nine able-bodied subjects participated in the study. Intramuscular and surface EMG signals were recorded concurrently from the right forearm. Five performance metrics (Throughput,Path efficiency, Average Speed, Overshoot and Completion Rate) were used for quantification of usability. Intramuscular EMG based control performed significantly better than surface EMG for Path Efficiency (80.5±2.4% vs. 71.5±3.8%, P=0.004) and Overshoot (22.0±3.0% vs. 45.1±6.6%, P=0.01). No difference was found between Throughput and Completion Rate. However the Average Speed was significantly higher for surface (51.8±5.5%) than for intramuscular EMG (35.7±2.7%). The results obtained in this study imply that intramuscular EMG has great potential as control source for advanced myoelectric prosthetic devices.

Evaluation of classifier topologies for the real-time classification of simultaneous limb motions

The prediction of motion intent through the decoding of myoelectric signals has the potential to improve the functionally of limb prostheses. Considerable research on individual motion classifiers has been done to exploit this idea. A drawback with the individual prediction approach, however, is its limitation to serial control, which is slow, cumbersome, and unnatural. In this work, different classifier topologies suitable for the decoding of mixed classes, and thus capable of predicting simultaneous motions, were investigated in real-time. These topologies resulted in higher offline accuracies than previously achieved, but more importantly, positive indications of their suitability for real-time systems were found. Furthermore, in order to facilitate further development, benchmarking, and cooperation, the algorithms and data generated in this study are freely available as part of BioPatRec, an open source framework for the development of advanced prosthetic control strategies.

System training and assessment in simultaneous proportional myoelectric prosthesis control

Background

Pattern recognition control of prosthetic hands take inputs from one or more myoelectric sensors and controls one or more degrees of freedom. However, most systems created allow only sequential control of one motion class at a time. Additionally, only recently have researchers demonstrated proportional myoelectric control in such systems, an option that is believed to make fine control easier for the user. Recent developments suggest improved reliability if the user follows a so-called prosthesis guided training (PGT) scheme.

Methods

In this study, a system for simultaneous proportional myoelectric control has been developed for a hand prosthesis with two motor functions (hand open/close, and wrist pro-/supination). The prosthesis has been used with a prosthesis socket equivalent designed for normally-limbed subjects. An extended version of PGT was developed for use with proportional control. The control system’s performance was tested for two subjects in the Clothespin Relocation Task and the Southampton Hand Assessment Procedure (SHAP). Simultaneous proportional control was compared with three other control strategies implemented on the same prosthesis: mutex proportional control (the same system but with simultaneous control disabled), mutex on-off control, and a more traditional, sequential proportional control system with co-contractions for state switching.

Results

The practical tests indicate that the simultaneous proportional control strategy and the two mutex-based pattern recognition strategies performed equally well, and superiorly to the more traditional sequential strategy according to the chosen outcome measures.

Conclusions

This is the first simultaneous proportional myoelectric control system demonstrated on a prosthesis affixed to the forearm of a subject. The study illustrates that PGT is a promising system training method for proportional control. Due to the limited number of subjects in this study, no definite conclusions can be drawn.

Movement error rate for evaluation of machine learning methods for sEMG-based hand movement classification

There has been increasing interest in applying learning algorithms to improve the dexterity of myoelectric prostheses. In this work, we present a large-scale benchmark evaluation on the second iteration of the publicly released NinaPro database, which contains surface electromyography data for 6 DOF force activations as well as for 40 discrete hand movements. The evaluation involves a modern kernel method and compares performance of three feature representations and three kernel functions. Both the force regression and movement classification problems can be learned successfully when using a nonlinear kernel function, while the exp- χ(2) kernel outperforms the more popular radial basis function kernel in all cases. Furthermore, combining surface electromyography and accelerometry in a multimodal classifier results in significant increases in accuracy as compared to when either modality is used individually. Since window-based classification accuracy should not be considered in isolation to estimate prosthetic controllability, we also provide results in terms of classification mistakes and prediction delay. To this extent, we propose the movement error rate as an alternative to the standard window-based accuracy. This error rate is insensitive to prediction delays and it allows us therefore to quantify mistakes and delays as independent performance characteristics. This type of analysis confirms that the inclusion of accelerometry is superior, as it results in fewer mistakes while at the same time reducing prediction delay.

Improving transient state myoelectric signal recognition in hand movement classification using gyroscopes

Pattern recognition of myoelectric signals in upper-limb prosthesis control has been subject to intense research for several years. However, few systems have yet been successfully clinically implemented. One possible explanation for this discrepancy is that published reports mostly focus on classification accuracy of myoelectric signals recorded under laboratory conditions as the metric for the system's performance. These data are usually acquired only during the static state of the contraction in a fixed seated position. This supports the test subject in performing repeatable contractions throughout the experiment and generally results in an unrealistically high classification accuracy. In clinical testing however, subjects have to perform various activities of daily living, causing the limb to move in different positions. These variations in limb positions can significantly decrease robustness and usability of myoelectric control systems. Recent reports have shown that the so-called limb position effect can be resolved for the static state of the signal by adding accelerometer data to the feature vector. Including data from the transient state of the signals for classifier training generally significantly increases the classification error so it is mostly not considered in published reports. In this paper, we investigate the classification accuracy of transient EMG data, taking into account the limb position effect. We demonstrate that a classifier trained with features from EMG, accelerometer and gyroscope outperforms classifiers using only EMG or EMG and accelerometer data when classifying transient EMG data.

Real-time control of an interactive impulsive virtual prosthesis

An interactive virtual dynamic environment for testing control strategies for neural machine interfacing with artificial limbs offers several advantages. The virtual environment is low-cost, easily configured, and offers a wealth of data for post-hoc analysis compared with real physical prostheses and robots. For use with prosthetics and research involving amputee subjects it allows the valuable time with the subject to be spent in experiments rather than fixing hardware issues. The usefulness of the virtual environment increases as the realism of the environment increases. Most tasks performed with limbs require interactions with objects in the environment. To simulate these tasks the dynamics of frictional contact, in addition to inertial limb dynamics must be modeled. Here, subjects demonstrate real-time control of an eight degree-of-freedom virtual prosthesis while performing an interactive box-and-blocks task. With practice, four nonamputee subjects and one shoulder disarticulation subject were able to successfully transfer blocks in the virtual environment at an average rate of just under two blocks per minute. The virtual environment is configurable in terms of the virtual arm design, control strategy, and task.

Assessment of a multigrasp myoelectric control approach for use by transhumeral amputees

The authors have previously developed a multigrasp myoelectric controller, and assessed the ability of healthy subjects to control the configuration of a multigrasp hand prosthesis using musculature on the anterior and posterior aspects of the forearm, as would be representative of controller use by a transradial amputee population. In this paper, the authors conduct a similar study, this time to assess the capability of a transhumeral amputee to control a multigrasp hand from residual musculature on the upper arm. Specifically, experiments are conducted on five healthy subjects, comparing their ability to obtain one of seven hand postures in a virtual prosthesis from EMG measurement of the respective biceps and triceps musculature. The ability to control the virtual hand prosthesis is compared with their ability to do so with their intact hand, as measured by a dataglove. Results indicate an average transition time using the EMG controller on the biceps and triceps of 1.86 seconds, relative to 0.82 seconds with the dataglove.

Preliminary functional assessment of a multigrasp myoelectric prosthesis

The authors have previously described a multigrasp hand prosthesis prototype, and a two-site surface EMG based multigrasp control interface for its control. In this paper, the authors present a preliminary assessment of the efficacy of the prosthesis and multigrasp controller in performing tasks requiring interaction and manipulation. The authors use as a performance measure the Southampton Hand Assessment Procedure (SHAP), which entails manipulation of various objects designed to emulate activities of daily living, and provides a set of scores that indicate level of functionality in various types of hand function. In this preliminary assessment, a single non-amputee subject performed the SHAP while wearing the multigrasp prosthesis via an able-bodied adaptor. The results from this testing are presented, and compared to recently published SHAP results obtained with commercially available single-grasp and multigrasp prosthetic hands.