Context-informed incremental learning improves both the performance and resilience of myoelectric control

Despite its rich history of success in controlling powered prostheses and emerging commercial interests in ubiquitous computing, myoelectric control continues to suffer from a lack of robustness. In particular, EMG-based systems often degrade over prolonged use resulting in tedious recalibration sessions, user frustration, and device abandonment. Unsupervised adaptation is one proposed solution that updates a model's parameters over time based on its own predictions during real-time use to maintain robustness without requiring additional user input or dedicated recalibration. However, these strategies can actually accelerate performance deterioration when they begin to classify (and thus adapt) incorrectly, defeating their own purpose. To overcome these limitations, we propose a novel adaptive learning strategy, Context-Informed Incremental Learning (CIIL), that leverages in situ context to better inform the prediction of pseudo-labels. In this work, we evaluate these CIIL strategies in an online target acquisition task for two use cases: (1) when there is a lack of training data and (2) when a drastic and enduring alteration in the input space has occurred. A total of 32 participants were evaluated across the two experiments. The results show that the CIIL strategies significantly outperform the current state-of-the-art unsupervised high-confidence adaptation and outperform models trained with the conventional screen-guided training approach, even after a 45-degree electrode shift (p < 0.05). Consequently, CIIL has substantial implications for the future of myoelectric control, potentially reducing the training burden while bolstering model robustness, and leading to improved real-time control.

Recent trends and challenges of surface electromyography in prosthetic applications

Surface electromyography (sEMG) meets extensive applications in the field of prosthesis in the current period. The effectiveness of sEMG in prosthesis applications has been verified by numerous revolutionary developments and extensive research attempts. A large volume of research and literature works have explored and validated the vast use of these signals in prostheses as an assistive technology. The objective of this paper is to conduct a systematic review and offer a detailed overview of the work record in the prosthesis and myoelectric interfaces framework. This review utilized a systematic search strategy to identify published articles discussing the state-of-the-art applications of sEMG in prostheses (including upper limb prosthesis and lower limb prostheses). Relevant studies were identified using electronic databases such as PubMed, IEEE Explore, SCOPUS, ScienceDirect, Google Scholar and Web of Science. Out of 3791 studies retrieved from the databases, 188 articles were found to be potentially relevant (after screening of abstracts and application of inclusion-exclusion criteria) and included in this review. This review presents an investigative analysis of sEMG-based prosthetic applications to assist the readers in making further advancements in this field. It also discusses the fundamental advantages and disadvantages of using sEMG in prosthetic applications. It also includes some important guidelines to follow in order to improve the performance of sEMG-based prosthesis. The findings of this study support the widespread use of sEMG in prosthetics. It is concluded that sEMG-based prosthesis technology, still in its sprouting phase, requires significant explorations for further development. Supplementary investigations are necessary in the direction of making a seamless mechanism of biomechatronics for sEMG-based prosthesis by cohesive efforts of robotic researchers and biomedical engineers.

Soft Robotics Enables Neuroprosthetic Hand Design

Development and implementation of neuroprosthetic hands is a multidisciplinary field at the interface between humans and artificial robotic systems, which aims at replacing the sensorimotor function of the upper-limb amputees as their own. Although prosthetic hand devices with myoelectric control can be dated back to more than 70 years ago, their applications with anthropomorphic robotic mechanisms and sensory feedback functions are still at a relatively preliminary and laboratory stage. Nevertheless, a recent series of proof-of-concept studies suggest that soft robotics technology may be promising and useful in alleviating the design complexity of the dexterous mechanism and integration difficulty of multifunctional artificial skins, in particular, in the context of personalized applications. Here, we review the evolution of neuroprosthetic hands with the emerging and cutting-edge soft robotics, covering the soft and anthropomorphic prosthetic hand design and relating bidirectional neural interactions with myoelectric control and sensory feedback. We further discuss future opportunities on revolutionized mechanisms, high-performance soft sensors, and compliant neural-interaction interfaces for the next generation of neuroprosthetic hands.

Virtual regression-based myoelectric hand-wrist prosthesis control and electrode site selection using no force feedback

Most transradial prosthesis users with conventional "Sequential" myoelectric control have two electrode sites which control one degree of freedom (DoF) at a time. Rapid EMG co-activation toggles control between DoFs (e.g., hand and wrist), providing limited function. We implemented a regression-based EMG control method which achieved simultaneous and proportional control of two DoFs in a virtual task. We automated electrode site selection using a short-duration (90 s) calibration period, without force feedback. Backward stepwise selection located the best electrodes for either six or 12 electrodes (selected from a pool of 16). We additionally studied two, 2-DoF controllers: "Intuitive" control (hand open-close and wrist pronation-supination controlled virtual target size and rotation, respectively) and "Mapping" control (wrist flexion-extension and ulnar-radial deviation controlled virtual target left-right and up-down movement, respectively). In practice, a Mapping controller would be mapped to control prosthesis hand open-close and wrist pronation-supination. Eleven able-bodied subjects and 4 limb-absent subjects completed virtual target matching tasks (fixed target moves to a new location after being "matched," and subject immediately pursues) and fixed (static) target tasks. For all subjects, both 2-DoF controllers with 6 optimally-sited electrodes had statistically better target matching performance than Sequential control in number of matches (average of 4-7 vs. 2 matches, p< 0.001) and throughput (average of 0.75-1.25 vs. 0.4 bits/s, p< 0.001), but not overshoot rate and path efficiency. There were no statistical differences between 6 and 12 optimally-sited electrodes for both 2-DoF controllers. These results support the feasibility of 2-DoF simultaneous, proportional myoelectric control.

Limb loading enhances skill transfer between augmented and physical reality tasks during limb loss rehabilitation

BACKGROUND

Virtual and augmented reality (AR) have become popular modalities for training myoelectric prosthesis control with upper-limb amputees. While some systems have shown moderate success, it is unclear how well the complex motor skills learned in an AR simulation transfer to completing the same tasks in physical reality. Limb loading is a possible dimension of motor skill execution that is absent in current AR solutions that may help to increase skill transfer between the virtual and physical domains.

METHODS

We implemented an immersive AR environment where individuals could operate a myoelectric virtual prosthesis to accomplish a variety of object relocation manipulations. Intact limb participants were separated into three groups, the load control (CG_LD; [Formula: see text]), the AR control (CG_AR; [Formula: see text]), and the experimental group (EG; [Formula: see text]). Both the CG_AR and EG completed a 5-session prosthesis training protocol in AR while the CG_LD performed simple muscle training. The EG attempted manipulations in AR while undergoing limb loading. The CG_AR attempted the same manipulations without loading. All participants performed the same manipulations in physical reality while operating a real prosthesis pre- and post-training. The main outcome measure was the change in the number of manipulations completed during the physical reality assessments (i.e. completion rate). Secondary outcomes included movement kinematics and visuomotor behavior.

RESULTS

The EG experienced a greater increase in completion rate post-training than both the CG_AR and CG_LD. This performance increase was accompanied by a shorter motor learning phase, the EG's performance saturating in less sessions of AR training than the CG_AR.

CONCLUSION

The results demonstrated that limb loading plays an important role in transferring complex motor skills learned in virtual spaces to their physical reality analogs. While participants who did not receive limb loading were able to receive some functional benefit from AR training, participants who received the loading experienced a greater positive change in motor performance with their performance saturating in fewer training sessions.

Combined spatial and frequency encoding for electrotactile feedback of myoelectric signals

Electrotactile stimulation has been commonly used in human–machine interfaces to provide feedback to the user, thereby closing the control loop and improving performance. The encoding approach, which defines the mapping of the feedback information into stimulation profiles, is a critical component of an electrotactile interface. Ideally, the encoding will provide a high-fidelity representation of the feedback variable while being easy to perceive and interpret by the subject. In the present study, we performed a closed-loop experiment wherein discrete and continuous coding schemes are combined to exploit the benefits of both techniques. Subjects performed a muscle activation-matching task relying solely on electrotactile feedback representing the generated myoelectric signal (EMG). In particular, we investigated the performance of two different coding schemes (spatial and spatial combined with frequency) at two feedback resolutions (low: 3 and high: 5 intervals). In both schemes, the stimulation electrodes were placed circumferentially around the upper arm. The magnitude of the normalized EMG was divided into intervals, and each electrode was associated with one interval. When the generated EMG entered one of the intervals, the associated electrode started stimulating. In the combined encoding, the additional frequency modulation of the active electrode also indicated the momentary magnitude of the signal within the interval. The results showed that combined coding decreased the undershooting rate, variability and absolute deviation when the resolution was low but not when the resolution was high, where it actually worsened the performance. This demonstrates that combined coding can improve the effectiveness of EMG feedback, but that this effect is limited by the intrinsic variability of myoelectric control. Our findings, therefore, provide important insights as well as elucidate limitations of the information encoding methods when using electrotactile stimulation to convey a feedback signal characterized by high variability (EMG biofeedback).

Starfish Procedure

For those patients with partial hand level amputation who would benefit from myoelectric prosthetic digits for enhanced prehensile function, the Starfish Procedure provides muscle transfers, which allow for the generation of intuitively controlled electromyographic signals for individual digital control with minimal myoelectric cross-talk. Thoughtful preoperative planning allows for creation of multiple sources of high-quality myoelectric signal in a single operation, which does not require microsurgery, providing for wide applicability to hand surgeons of all backgrounds.

Transfer of mode switching performance: from training to upper-limb prosthesis use

BACKGROUND

Current myoelectric prostheses are multi-articulated and offer multiple modes. Switching between modes is often done through pre-defined myosignals, so-called triggers, of which the training hardly is studied. We evaluated if switching skills trained without using a prosthesis transfer to actual prosthesis use and whether the available feedback during training influences this transfer. Furthermore we examined which clinically relevant performance measures and which myosignal features were adapted during training.

METHODS

Two experimental groups and one control group participated in a five day pre-test-post-test design study. Both experimental groups used their myosignals to perform a task. One group performed a serious game without seeing their myosignals, the second group was presented their myosignal on a screen. The control group played the serious game using the touchpad of the laptop. Each training session lasted 15 min. The pre- and post-test were identical for all groups and consisted of performing a task with an actual prosthesis, where switches had to be produced to change grip mode to relocate clothespins. Both clinically relevant performance measures and myosignal features were analysed.

RESULTS

10 participants trained using the serious game, 10 participants trained with the visual myosignal and 8 the control task. All participants were unimpaired. Both experimental groups showed significant transfer of skill from training to prosthesis use, the control group did not. The degree of transfer did not differ between the two training groups. Clinically relevant measure 'accuracy' and feature of the myosignals 'variation in phasing' changed during training.

CONCLUSIONS

Training switching skills appeared to be successful. The skills trained in the game transferred to performance in a functional task. Learning switching skills is independent of the type of feedback used during training. Outcome measures hardly changed during training and further research is needed to explain this. It should be noted that five training sessions did not result in a level of performance needed for actual prosthesis use. Trial registration The study was approved by the local ethics committee (ECB 2014.02.28_1) and was included in the Dutch trial registry (NTR5876).

Immersive augmented reality system for the training of pattern classification control with a myoelectric prosthesis

BACKGROUND

Hand amputation can have a truly debilitating impact on the life of the affected person. A multifunctional myoelectric prosthesis controlled using pattern classification can be used to restore some of the lost motor abilities. However, learning to control an advanced prosthesis can be a challenging task, but virtual and augmented reality (AR) provide means to create an engaging and motivating training.

METHODS

In this study, we present a novel training framework that integrates virtual elements within a real scene (AR) while allowing the view from the first-person perspective. The framework was evaluated in 13 able-bodied subjects and a limb-deficient person divided into intervention (IG) and control (CG) groups. The IG received training by performing simulated clothespin task and both groups conducted a pre- and posttest with a real prosthesis. When training with the AR, the subjects received visual feedback on the generated grasping force. The main outcome measure was the number of pins that were successfully transferred within 20 min (task duration), while the number of dropped and broken pins were also registered. The participants were asked to score the difficulty of the real task (posttest), fun-factor and motivation, as well as the utility of the feedback.

RESULTS

The performance (median/interquartile range) consistently increased during the training sessions (4/3 to 22/4). While the results were similar for the two groups in the pretest, the performance improved in the posttest only in IG. In addition, the subjects in IG transferred significantly more pins (28/10.5 versus 14.5/11), and dropped (1/2.5 versus 3.5/2) and broke (5/3.8 versus 14.5/9) significantly fewer pins in the posttest compared to CG. The participants in IG assigned (mean ± std) significantly lower scores to the difficulty compared to CG (5.2 ± 1.9 versus 7.1 ± 0.9), and they highly rated the fun factor (8.7 ± 1.3) and usefulness of feedback (8.5 ± 1.7).

CONCLUSION

The results demonstrated that the proposed AR system allows for the transfer of skills from the simulated to the real task while providing a positive user experience. The present study demonstrates the effectiveness and flexibility of the proposed AR framework. Importantly, the developed system is open source and available for download and further development.

Exploring augmented grasping capabilities in a multi-synergistic soft bionic hand

BACKGROUND

State-of-the-art bionic hands incorporate hi-tech devices which try to overcome limitations of conventional single grip systems. Unfortunately, their complexity often limits mechanical robustness and intuitive prosthesis control. Recently, the translation of neuroscientific theories (i.e. postural synergies) in software and hardware architecture of artificial devices is opening new approaches for the design and control of upper-limb prostheses.

METHODS

Following these emerging principles, previous research on the SoftHand Pro, which embeds one physical synergy, showed promising results in terms of intuitiveness, robustness, and grasping performance. To explore these principles also in hands with augmented capabilities, this paper describes the SoftHand 2 Pro, a second generation of the device with 19 degrees-of-freedom and a second synergistic layer. After a description of the proposed device, the work explores a continuous switching control method based on a myoelectric pattern recognition classifier.

RESULTS

The combined system was validated using standardized assessments with able-bodied and, for the first time, amputee subjects. Results show an average improvement of more than 30% of fine grasp capabilities and about 10% of hand function compared with the first generation SoftHand Pro.

CONCLUSIONS

Encouraging results suggest how this approach could be a viable way towards the design of more natural, reliable, and intuitive dexterous hands.

Automatic discovery of resource-restricted Convolutional Neural Network topologies for myoelectric pattern recognition

Convolutional Neural Networks (CNNs) have been subject to extensive attention in the pattern recognition literature due to unprecedented performance in tasks of information extraction from unstructured data. Whereas available methods for supervised training of a CNN with a given network topology are well-defined with rigorous theoretical justification, procedures for the initial selection of topology are currently not. Work incorporating selection of the CNN topology has instead substantially been guided by the domain-specific expertise of the creator(s), followed by iterative improvement via empirical evaluation. This limitation of methodology is restricting in the pursuit of naturally controlled muscle-computer interfaces, where CNNs have been identified as a promising research avenue but effective topology selection heuristics are lacking. With the goal of mitigating ambiguities in topology selection, this paper presents a systematic approach wherein we apply a novel evolutionary algorithm to search a space of candidate topologies. Furthermore, we constrain the search-space by excluding topologies with excessive inference-time computational complexity, making the obtained results implementable in embedded systems. In contrast to manual topology design, our algorithm requires the user to only specify a relatively small set of intuitive hyperparameters. To validate our approach, we use it in order to create topologies for myoelectric pattern recognition via movement decoding of surface electromyography signals. By collating offline classification accuracies obtained from experiments on a collection of publicly available databases, we demonstrate that our method generates computationally lightweight topologies with performance comparable to those of available alternatives.

Performance among different types of myocontrolled tasks is not related

Studies on myocontrolled assistive technology (AT), such as myoelectric prostheses, as well as rehabilitation practice using myoelectric controlled interfaces, commonly assume the existence of a general myocontrol skill. This is the skill to control myosignals in such a way that they are employable in multiple tasks. If this skill exists, training any myocontrolled task using a certain set of muscles would improve the use of myocontrolled AT when the AT is controlled using these muscles. We examined whether a general myocontrol skill exists in myocontrolled tasks with and without a prosthesis. Unimpaired, right-handed adults used the sEMG of wrist flexors and extensors to perform several tasks in two experiments. In Experiment 1, twelve participants trained a myoelectric prosthesis-simulator task and a myocontrolled serious game for five consecutive days. Performance was compared between tasks and over the course of the training period. In Experiment 2, thirty-one participants performed five myocontrolled tasks consisting of two serious games, two prosthesis-simulator tasks and one digital signal matching task. All tasks were based on tasks currently used in clinical practice or research settings. Kendall rank correlation coefficients were computed to analyze correlations between the performance on different tasks. In Experiment 1 performance on the tasks showed no correlation for multiple outcome measures. Rankings within tasks did not change over the training period. In Experiment 2 performance did not correlate between any of the tasks. Since performance between different tasks did not correlate, results suggest that a general myocontrol skill does not exist and that each myocontrolled task requires a specific skill. Generalization of those findings to amputees using AT should be done with caution since in both experiments unimpaired participants were included. Moreover, training duration in Experiment 2 was short. Our findings indicate that training and assessment methods for myocontrolled AT use should focus on tasks frequently performed in daily life by the individual using the AT instead of merely focusing on training myosignals.

Stable, three degree-of-freedom myoelectric prosthetic control via chronic bipolar intramuscular electrodes: a case study

Background

Modern prosthetic hands are typically controlled using skin surface electromyographic signals (EMG) from remaining muscles in the residual limb. However, surface electrode performance is limited by changes in skin impedance over time, day-to-day variations in electrode placement, and relative motion between the electrodes and underlying muscles during movement: these limitations require frequent retraining of controllers. In the presented study, we used chronically implanted intramuscular electrodes to minimize these effects and thus create a more robust prosthetic controller.

Methods

A study participant with a transradial amputation was chronically implanted with 8 intramuscular EMG electrodes. A K Nearest Neighbor (KNN) regression velocity controller was trained to predict intended joint movement direction using EMG data collected during a single training session. The resulting KNN was evaluated over 12 weeks and in multiple arm posture configurations, with the participant controlling a 3 Degree-of-Freedom (DOF) virtual reality (VR) hand to match target VR hand postures. The performance of this EMG-based controller was compared to a position-based controller that used movement measured from the participant’s opposite (intact) hand. Surface EMG was also collected for signal quality comparisons.

Results

Signals from the implanted intramuscular electrodes exhibited less crosstalk between the various channels and had a higher Signal-to-Noise Ratio than surface electrode signals. The performance of the intramuscular EMG-based KNN controller in the VR control task showed no degradation over time, and was stable over the 6 different arm postures. Both the EMG-based KNN controller and the intact hand-based controller had 100% hand posture matching success rates, but the intact hand-based controller was slightly superior in regards to speed (trial time used) and directness of the VR hand control (path efficiency).

Conclusions

Chronically implanted intramuscular electrodes provide negligible crosstalk, high SNR, and substantial VR control performance, including the ability to use a fixed controller over 12 weeks and under different arm positions. This approach can thus be a highly effective platform for advanced, multi-DOF prosthetic control.

Myoelectric signal from below the level of spinal cord injury as a command source for an implanted upper extremity neuroprosthesis - a case report

Implanted motor neuroprostheses offer significant restoration of function for individuals with spinal cord injury. Providing adequate user control for these devices is a challenge but is crucial for successful performance. Electromyographic (EMG) signals can serve as effective control sources, but the number of above-injury muscles suitable to provide EMG-based control signals is very limited. Previous work has shown the presence of below-injury volitional myoelectric signals even in subjects diagnosed with motor complete spinal cord injury. In this case report, we present a demonstration of a hand grasp neuroprosthesis being controlled by a user with a C6 level, motor complete injury through EMG signals from their toe flexor. These signals were successfully translated into a functional grasp output, which performed similarly to the participant’s usual shoulder position control in a grasp-release functional test. This proof-of-concept demonstrates the potential for below-injury myoelectric activity to serve as a novel form of neuroprosthesis control.

Mechanics and energetics of post-stroke walking aided by a powered ankle exoskeleton with speed-adaptive myoelectric control

BACKGROUND

Ankle exoskeletons offer a promising opportunity to offset mechanical deficits after stroke by applying the needed torque at the paretic ankle. Because joint torque is related to gait speed, it is important to consider the user's gait speed when determining the magnitude of assistive joint torque. We developed and tested a novel exoskeleton controller for delivering propulsive assistance which modulates exoskeleton torque magnitude based on both soleus muscle activity and walking speed. The purpose of this research is to assess the impact of the resulting exoskeleton assistance on post-stroke walking performance across a range of walking speeds.

METHODS

Six participants with stroke walked with and without assistance applied to a powered ankle exoskeleton on the paretic limb. Walking speed started at 60% of their comfortable overground speed and was increased each minute (n00, n01, n02, etc.). We measured lower limb joint and limb powers, metabolic cost of transport, paretic and non-paretic limb propulsion, and trailing limb angle.

RESULTS

Exoskeleton assistance increased with walking speed, verifying the speed-adaptive nature of the controller. Both paretic ankle joint power and total limb power increased significantly with exoskeleton assistance at six walking speeds (n00, n01, n02, n03, n04, n05). Despite these joint- and limb-level benefits associated with exoskeleton assistance, no subject averaged metabolic benefits were evident when compared to the unassisted condition. Both paretic trailing limb angle and integrated anterior paretic ground reaction forces were reduced with assistance applied as compared to no assistance at four speeds (n00, n01, n02, n03).

CONCLUSIONS

Our results suggest that despite appropriate scaling of ankle assistance by the exoskeleton controller, suboptimal limb posture limited the conversion of exoskeleton assistance into forward propulsion. Future studies could include biofeedback or verbal cues to guide users into limb configurations that encourage the conversion of mechanical power at the ankle to forward propulsion.

TRIAL REGISTRATION

N/A.

Adapting myoelectric control in real-time using a virtual environment

Background

Pattern recognition technology allows for more intuitive control of myoelectric prostheses. However, the need to collect electromyographic data to initially train the pattern recognition system, and to re-train it during prosthesis use, adds complexity that can make using such a system difficult. Although experienced clinicians may be able to guide users to ensure successful data collection methods, they may not always be available when a user needs to (re)train their device.

Methods

Here we present an engaging and interactive virtual reality environment for optimal training of a myoelectric controller. Using this tool, we evaluated the importance of training a classifier actively (i.e., moving the residual limb during data collection) compared to passively (i.e., maintaining the limb in a single, neutral orientation), and whether computational adaptation through serious gaming can improve performance.

Results

We found that actively trained classifiers performed significantly better than passively trained classifiers for non-amputees (P < 0.05). Furthermore, collecting data passively with minimal instruction, paired with computational adaptation in a virtual environment, significantly improved real-time performance of myoelectric controllers.

Conclusion

These results further support previous work which suggested active movements during data collection can improve pattern recognition systems. Furthermore, adaptation within a virtual guided serious game environment can improve real-time performance of myoelectric controllers.

Electronic supplementary material

The online version of this article (10.1186/s12984-019-0480-5) contains supplementary material, which is available to authorized users.

Myoelectric control algorithm for robot-assisted therapy: a hardware-in-the-loop simulation study

BACKGROUND

A direct blow to the knee is one way to injure the anterior cruciate ligament (ACL), e.g., during a football or traffic accident. Robot-assisted therapy (RAT) rehabilitation, simulating regular walking, improves walking and balance abilities, and extensor strength after ACL reconstruction. However, there is a need to perform RAT during other phases of ACL injury rehabilitation before attempting an advanced exercise such as walking. This paper aims to propose a myoelectric control (MEC) algorithm for a robot-assisted rehabilitation system, "Nukawa", to assist knee movement during these types of exercises, i.e., such as in active-assisted extension exercises.

METHODS

Surface electromyography (sEMG) signal processing algorithm was developed to detect the motion intention of the knee joint. The sEMG signal processing algorithm and the movement control algorithm, reported by the authors in a previous publication, were joined together as a hardware-in-the-loop simulation to create and test the MEC algorithm, instead of using the actual robot.

EXPERIMENTS AND RESULTS

An experimental protocol was conducted with 17 healthy subjects to acquire sEMG signals and their lower limb kinematics during 12 ACL rehabilitation exercises. The proposed motion intention algorithm detected the orientation of the intention 100% of the times for the extension and flexion exercises. Also, it detected in 94% and 59% of the cases the intensity of the movement intention in a comparable way to the maximum voluntary contraction (MVC) during extension exercises and flexion exercises, respectively. The maximum position mean absolute error was [Formula: see text], [Formula: see text], and [Formula: see text] for the hip, knee, and ankle joints, respectively.

CONCLUSIONS

The MEC algorithm detected the intensity of the movement intention, approximately, in a comparable way to the MVC and the orientation. Moreover, it requires no prior training or additional torque sensors. Also, it controls the speed of the knee joint of Nukawa to assist the knee movement, i.e., such as in active-assisted extension exercises.

Control within a virtual environment is correlated to functional outcomes when using a physical prosthesis

Background

Advances such as targeted muscle reinnervation and pattern recognition control may provide improved control of upper limb myoelectric prostheses, but evaluating user function remains challenging. Virtual environments are cost-effective and immersive tools that are increasingly used to provide practice and evaluate prosthesis control, but the relationship between virtual and physical outcomes—i.e., whether practice in a virtual environment translates to improved physical performance—is not understood.

Methods

Nine people with transhumeral amputations who previously had targeted muscle reinnervation surgery were fitted with a myoelectric prosthesis comprising a commercially available elbow, wrist, terminal device, and pattern recognition control system. Virtual and physical outcome measures were obtained before and after a 6-week home trial of the prosthesis.

Results

After the home trial, subjects showed statistically significant improvements (p < 0.05) in offline classification error, the virtual Target Achievement Control test, and the physical Southampton Hand Assessment Procedure and Box and Blocks Test. A trend toward improvement was also observed in the physical Clothespin Relocation task and Jebsen-Taylor test; however, these changes were not statistically significant. The median completion time in the virtual test correlated strongly and significantly with the Southampton Hand Assessment Procedure (p = 0.05, R = − 0.86), Box and Blocks Test (p = 0.007, R = − 0.82), Jebsen-Taylor Test (p = 0.003, R = 0.87), and the Assessment of Capacity for Myoelectric Control (p = 0.005,R = − 0.85). The classification error performance only had a significant correlation with the Clothespin Relocation Test (p = 0.018, R = .76).

Conclusions

In-home practice with a pattern recognition-controlled prosthesis improves functional control, as measured by both virtual and physical outcome measures. However, virtual measures need to be validated and standardized to ensure reliability in a clinical or research setting.

Trial registration

This is a registered clinical trial: NCT03097978.

Decoding the grasping intention from electromyography during reaching motions

BACKGROUND

Active upper-limb prostheses are used to restore important hand functionalities, such as grasping. In conventional approaches, a pattern recognition system is trained over a number of static grasping gestures. However, training a classifier in a static position results in lower classification accuracy when performing dynamic motions, such as reach-to-grasp. We propose an electromyography-based learning approach that decodes the grasping intention during the reaching motion, leading to a faster and more natural response of the prosthesis.

METHODS AND RESULTS

Eight able-bodied subjects and four individuals with transradial amputation gave informed consent and participated in our study. All the subjects performed reach-to-grasp motions for five grasp types, while the elecromyographic (EMG) activity and the extension of the arm were recorded. We separated the reach-to-grasp motion into three phases, with respect to the extension of the arm. A multivariate analysis of variance (MANOVA) on the muscular activity revealed significant differences among the motion phases. Additionally, we examined the classification performance on these phases. We compared the performance of three different pattern recognition methods; Linear Discriminant Analysis (LDA), Support Vector Machines (SVM) with linear and non-linear kernels, and an Echo State Network (ESN) approach. Our off-line analysis shows that it is possible to have high classification performance above 80% before the end of the motion when with three-grasp types. An on-line evaluation with an upper-limb prosthesis shows that the inclusion of the reaching motion in the training of the classifier importantly improves classification accuracy and enables the detection of grasp intention early in the reaching motion.

CONCLUSIONS

This method offers a more natural and intuitive control of prosthetic devices, as it will enable controlling grasp closure in synergy with the reaching motion. This work contributes to the decrease of delays between the user's intention and the device response and improves the coordination of the device with the motion of the arm.

Two ways to improve myoelectric control for a transhumeral amputee after targeted muscle reinnervation: a case study

BACKGROUND

Myoelectric control of multifunctional prostheses is challenging for individuals with high-level amputations due to insufficient surface electromyography (sEMG) signals. A surgical technique called targeted muscle reinnervation (TMR) has achieved impressive improvements in myoelectric control by providing more sEMG control signals. In this case, some channels of sEMG signals are coupled after TMR, which limits the performance of conventional amplitude-based control for upper-limb prostheses.

METHODS

In this paper, two different ways (training and algorithms) were attempted to solve the problem in a transhumeral amputee after TMR. Firstly, effect of rehabilitation training on generating independent sEMG signals was investigated. The results indicated that some sEMG signals recorded were still coupled over the targeted muscles after rehabilitation training for about two months. Secondly, pattern recognition (PR) algorithm was then applied to classify the sEMG signals. In the second way, to further improve the real-time performance of prosthetic control, a post-processing method named as mean absolute value-based (MAV-based) threshold switches was utilized.

RESULTS

Using the improved algorithms, substantial improvement was shown in a subset of the modified Action Research Arm Test (ARAT). Compared with common PR control without post-processing method, the total scores increased more than 18% with majority vote and more than 58% with MAV-based threshold switches. The amputee was able to finish all the tasks within the allotted time with the standard MAV-based threshold switches. Subjectively the amputee preferred the PR control with MAV-based threshold switches and reported it to be more accurate and much smoother both in experiment and practical use.

CONCLUSIONS

Although the sEMG signals were still coupled after rehabilitation training on the TMR patient, the online performance of the prosthetic operation was improved through application of PR control with combination of the MAV-based threshold switches.

TRIAL REGISTRATION

Retrospectively registered http://www.chictr.org.cn/showproj.aspx?proj=22058 .

The effect of time on EMG classification of hand motions in able-bodied and transradial amputees

While several studies have demonstrated the short-term performance of pattern recognition systems, long-term investigations are very limited. In this study, we investigated changes in classification performance over time. Ten able-bodied individuals and six amputees took part in this study. EMG signals were recorded concurrently from surface and intramuscular electrodes, with intramuscular electrodes kept in the muscles for seven days. Seven hand motions were evaluated daily using linear discriminant analysis and the classification error quantified within (WCE) and between (BCE) days. BCE was computed for all possible combinations between the days. For all subjects, surface sEMG (7.2 ± 7.6%), iEMG (11.9 ± 9.1%) and cEMG (4.6 ± 4.8%) were significantly different (P < 0.001) from each other. A regression between WCE and days (1-7) was on average not significant implying that performance may be considered similar within each day. Regression between BCE and time difference (Df) in days was significant. The slope between BCE and Df (0-6) was significantly different from zero for sEMG (R² = 89%) and iEMG (R² = 95%) in amputees. Results indicate that performance continuously degrades as the time difference between training and testing day increases. Furthermore, for iEMG, performance in amputees was directly proportional to the size of the residual limb.

Evaluation of EMG pattern recognition for upper limb prosthesis control: a case study in comparison with direct myoelectric control

Background

Although electromyogram (EMG) pattern recognition (PR) for multifunctional upper limb prosthesis control has been reported for decades, the clinical benefits have rarely been examined. The study purposes were to: 1) compare self-report and performance outcomes of a transradial amputee immediately after training and one week after training of direct myoelectric control and EMG pattern recognition (PR) for a two-degree-of-freedom (DOF) prosthesis, and 2) examine the change in outcomes one week after pattern recognition training and the rate of skill acquisition in two subjects with transradial amputations.

Methods

In this cross-over study, participants were randomized to receive either PR control or direct control (DC) training of a 2 DOF myoelectric prosthesis first. Participants were 2 persons with traumatic transradial (TR) amputations who were 1 DOF myoelectric users. Outcomes, including measures of dexterity with and without cognitive load, activity performance, self-reported function, and prosthetic satisfaction were administered immediately and 1 week after training. Speed of skill acquisition was assessed hourly. One subject completed training under both PR control and DC conditions. Both subjects completed PR training and testing. Outcomes of test metrics were analyzed descriptively.

Results

Comparison of the two control strategies in one subject who completed training in both conditions showed better scores in 2 (18%) dexterity measures, 1 (50%) dexterity measure with cognitive load, and 1 (50%) self-report functional measure using DC, as compared to PR. Scores of all other metrics were comparable. Both subjects showed decline in dexterity after training. Findings related to rate of skill acquisition varied considerably by subject.

Conclusions

Outcomes of PR and DC for operating a 2-DOF prosthesis in a single subject cross-over study were similar for 74% of metrics, and favored DC in 26% of metrics. The two subjects who completed PR training showed decline in dexterity one week after training ended. Findings related to rate of skill acquisition varied considerably by subject. This study, despite its small sample size, highlights a need for additional research quantifying the functional and clinical benefits of PR control for upper limb prostheses.

Electronic supplementary material

The online version of this article (10.1186/s12984-018-0361-3) contains supplementary material, which is available to authorized users.

Improved prosthetic hand control with concurrent use of myoelectric and inertial measurements

Background

Myoelectric pattern recognition systems can decode movement intention to drive upper-limb prostheses. Despite recent advances in academic research, the commercial adoption of such systems remains low. This limitation is mainly due to the lack of classification robustness and a simultaneous requirement for a large number of electromyogram (EMG) electrodes. We propose to address these two issues by using a multi-modal approach which combines surface electromyography (sEMG) with inertial measurements (IMs) and an appropriate training data collection paradigm. We demonstrate that this can significantly improve classification performance as compared to conventional techniques exclusively based on sEMG signals.

Methods

We collected and analyzed a large dataset comprising recordings with 20 able-bodied and two amputee participants executing 40 movements. Additionally, we conducted a novel real-time prosthetic hand control experiment with 11 able-bodied subjects and an amputee by using a state-of-the-art commercial prosthetic hand. A systematic performance comparison was carried out to investigate the potential benefit of incorporating IMs in prosthetic hand control.

Results

The inclusion of IM data improved performance significantly, by increasing classification accuracy (CA) in the offline analysis and improving completion rates (CRs) in the real-time experiment. Our findings were consistent across able-bodied and amputee subjects. Integrating the sEMG electrodes and IM sensors within a single sensor package enabled us to achieve high-level performance by using on average 4-6 sensors.

Conclusions

The results from our experiments suggest that IMs can form an excellent complimentary source signal for upper-limb myoelectric prostheses. We trust that multi-modal control solutions have the potential of improving the usability of upper-extremity prostheses in real-life applications.

Electronic supplementary material

The online version of this article (doi:10.1186/s12984-017-0284-4) contains supplementary material, which is available to authorized users.

Tactile feedback is an effective instrument for the training of grasping with a prosthesis at low- and medium-force levels

Grasping is a complex task routinely performed in an anticipatory (feedforward) manner, where sensory feedback is responsible for learning and updating the internal model of grasp dynamics. This study aims at evaluating whether providing a proportional tactile force feedback during the myoelectric control of a prosthesis facilitates learning a stable internal model of the prosthesis force control. Ten able-bodied subjects controlled a sensorized myoelectric prosthesis performing four blocks of consecutive grasps at three levels of target force (30, 50, and 70%), repeatedly closing the fully opened hand. In the first and third block, the subjects received tactile and visual feedback, respectively, while during the second and fourth block, the feedback was removed. The subjects also performed an additional block with no feedback 1 day after the training (Retest). The median and interquartile range of the generated forces was computed to assess the accuracy and precision of force control. The results demonstrated that the feedback was indeed an effective instrument for the training of prosthesis control. After the training, the subjects were still able to accurately generate the desired force for the low and medium target (30 and 50% of maximum force available in a prosthesis), despite the feedback being removed within the session and during the retest (low target force). However, the training was substantially less successful for high forces (70% of prosthesis maximum force), where subjects exhibited a substantial loss of accuracy as soon as the feedback was removed. The precision of control decreased with higher forces and it was consistent across conditions, determined by an intrinsic variability of repeated myoelectric grasping. This study demonstrated that the subject could rely on the tactile feedback to adjust the motor command to the prosthesis across trials. The subjects adjusted the mean level of muscle activation (accuracy), whereas the precision could not be modulated as it depends on the intrinsic myoelectric variability. They were also able to maintain the feedforward command even after the feedback was removed, demonstrating thereby a stable learning, but the retention depended on the level of the target force. This is an important insight into the role of feedback as an instrument for learning of anticipatory prosthesis force control.

Two degrees of freedom quasi-static EMG-force at the wrist using a minimum number of electrodes

Surface electromyogram-controlled powered hand/wrist prostheses return partial upper-limb function to limb-absent persons. Typically, one degree of freedom (DoF) is controlled at a time, with mode switching between DoFs. Recent research has explored using large-channel EMG systems to provide simultaneous, independent and proportional (SIP) control of two joints-but such systems are not practical in current commercial prostheses. Thus, we investigated site selection of a minimum number of conventional EMG electrodes in an EMG-force task, targeting four sites for a two DoF controller. In a laboratory experiment with 10 able-bodied subjects and three limb-absent subjects, 16 electrodes were placed about the proximal forearm. Subjects produced 1-DoF and 2-DoF slowly force-varying contractions up to 30% maximum voluntary contraction (MVC). EMG standard deviation was related to forces via regularized regression. Backward stepwise selection was used to retain those progressively fewer electrodes that exhibited minimum error. For 1-DoF models using two retained electrodes (which mimics the current state of the art), subjects had average RMS errors of (depending on the DoF): 7.1-9.5% MVC for able-bodied and 13.7-17.1% MVC for limb-absent subjects. For 2-DoF models, subjects using four electrodes had errors on 1-DoF trials of 6.7-8.5% MVC for able-bodied and 11.9-14.0% MVC for limb-absent; and errors on 2-DoF trials of 9.9-11.2% MVC for able-bodied and 15.8-16.7% MVC for limb-absent subjects. For each model, retaining more electrodes did not statistically improve performance. The able-bodied results suggest that backward selection is a viable method for minimum error selection of as few as four electrode sites for these EMG-force tasks. Performance evaluation in a prosthesis control task is a necessary and logical next step for this site selection method.

Role of Muscle Synergies in Real-Time Classification of Upper Limb Motions using Extreme Learning Machines

Background

Myoelectric signals offer significant insights in interpreting the motion intention and extent of effort involved in performing a movement, with application in prostheses, orthosis and exoskeletons. Feature extraction plays a vital role, and follows two approaches: EMG and synergy features. More recently, muscle synergy based features are being increasingly explored, since it simplifies dimensionality of control, and are considered to be more robust to signal variations. Another important aspect in a myoelectrically controlled devices is the learning capability and speed of performance for online decoding. Extreme learning machine (ELM) is a relatively new neural-network based learning algorithm: its performance hasn’t been explored in the context of online control, which is a more reliable measure compared to offline analysis. To this purpose we aim at focusing our investigation on a myoelectric-based interface which is able to identify and online classify, upper limb motions involving shoulder and elbow. The main objective is to compare the performance of the decoder trained using ELM, for two different features: EMG and synergy features.

Methods

The experiments are broadly divided in two phases training/calibration and testing respectively. ELM is used to train the decoder using data acquired during the calibration phase. The performance of the decoder is then tested in online motion control by using a simulated graphical user interface replicating the human limb: subjects are requested to control a virtual arm by using their muscular activity. The decoder performance is quantified using ad-hoc metrics based on the following indicators: motion selection time, motion completion time, and classification accuracy.

Results

Performance has been evaluated for both offline and online contexts. The offline classification results indicated better performance in the case of EMG features, whereas a better classification accuracy for synergy feature was observed for online decoding. Also the other indicators as motion selection time and motion completion time, showed better trend in the case of synergy than time-domain features.

Conclusion

This work demonstrates better robustness of online decoding of upper-limb motions and motor intentions when using synergy feature. Furthermore, we have quantified the performance of the decoder trained using ELM for online control, providing a potential and viable option for real-time myoelectric control in assistive technology.

Electronic supplementary material

The online version of this article (doi:10.1186/s12984-016-0183-0) contains supplementary material, which is available to authorized users.

Spatial distribution of HD-EMG improves identification of task and force in patients with incomplete spinal cord injury

Background

Recent studies show that spatial distribution of High Density surface EMG maps (HD-EMG) improves the identification of tasks and their corresponding contraction levels. However, in patients with incomplete spinal cord injury (iSCI), some nerves that control muscles are damaged, leaving some muscle parts without an innervation. Therefore, HD-EMG maps in patients with iSCI are affected by the injury and they can be different for every patient. The objective of this study is to investigate the spatial distribution of intensity in HD-EMG recordings to distinguish co-activation patterns for different tasks and effort levels in patients with iSCI. These patterns are evaluated to be used for extraction of motion intention.

Method

HD-EMG was recorded in patients during four isometric tasks of the forearm at three different effort levels. A linear discriminant classifier based on intensity and spatial features of HD-EMG maps of five upper-limb muscles was used to identify the attempted tasks. Task and force identification were evaluated for each patient individually, and the reliability of the identification was tested with respect to muscle fatigue and time interval between training and identification.

Results

Three feature sets were analyzed in the identification: 1) intensity of the HD-EMG map, 2) intensity and center of gravity of HD-EMG maps and 3) intensity of a single differential EMG channel (gold standard). Results show that the combination of intensity and spatial features in classification identifies tasks and effort levels properly (Acc = 98.8 %; S = 92.5 %; P = 93.2 %; SP = 99.4 %) and outperforms significantly the other two feature sets (p < 0.05).

Conclusion

In spite of the limited motor functionality, a specific co-activation pattern for each patient exists for both intensity, and spatial distribution of myoelectric activity. The spatial distribution is less sensitive than intensity to myoelectric changes that occur due to fatigue, and other time-dependent influences.

Wiener filtering of surface EMG with a priori SNR estimation toward myoelectric control for neurological injury patients

Voluntary surface electromyogram (EMG) signals from neurological injury patients are often corrupted by involuntary background interference or spikes, imposing difficulties for myoelectric control. We present a novel framework to suppress involuntary background spikes during voluntary surface EMG recordings. The framework applies a Wiener filter to restore voluntary surface EMG signals based on tracking a priori signal to noise ratio (SNR) by using the decision-directed method. Semi-synthetic surface EMG signals contaminated by different levels of involuntary background spikes were constructed from a database of surface EMG recordings in a group of spinal cord injury subjects. After the processing, the onset detection of voluntary muscle activity was significantly improved against involuntary background spikes. The magnitude of voluntary surface EMG signals can also be reliably estimated for myoelectric control purpose. Compared with the previous sample entropy analysis for suppressing involuntary background spikes, the proposed framework is characterized by quick and simple implementation, making it more suitable for application in a myoelectric control system toward neurological injury rehabilitation.

EMG feature assessment for myoelectric pattern recognition and channel selection: a study with incomplete spinal cord injury

Myoelectric pattern recognition with a large number of electromyogram (EMG) channels provides an approach to assessing motor control information available from the recorded muscles. In order to develop a practical myoelectric control system, a feature dependent channel reduction method was developed in this study to determine a small number of EMG channels for myoelectric pattern recognition analysis. The method selects appropriate raw EMG features for classification of different movements, using the minimum Redundancy Maximum Relevance (mRMR) and the Markov random field (MRF) methods to rank a large number of EMG features, respectively. A k-nearest neighbor (KNN) classifier was used to evaluate the performance of the selected features in terms of classification accuracy. The method was tested using 57 channels' surface EMG signals recorded from forearm and hand muscles of individuals with incomplete spinal cord injury (SCI). Our results demonstrate that appropriate selection of a small number of raw EMG features from different recording channels resulted in similar high classification accuracies as achieved by using all the EMG channels or features. Compared with the conventional sequential forward selection (SFS) method, the feature dependent method does not require repeated classifier implementation. It can effectively reduce redundant information not only cross different channels, but also cross different features in the same channel. Such hybrid feature-channel selection from a large number of EMG recording channels can reduce computational cost for implementation of a myoelectric pattern recognition based control system.

Several practical issues toward implementing myoelectric pattern recognition for stroke rehabilitation

High density surface electromyogram (sEMG) recording and pattern recognition techniques have demonstrated that substantial motor control information can be extracted from neurologically impaired muscles. In this study, a series of pattern recognition parameters were investigated in classification of 20 different movements involving the affected limb of 12 chronic stroke subjects. The experimental results showed that classification performance could be improved with spatial filtering and be maintained with a limited number of electrodes. It was also found that appropriate adjustment of analysis window length, sampling rate, and high-pass cut-off frequency in sEMG conditioning and processing would be potentially useful in reducing computational cost and meanwhile ensuring classification performance. The quantitative analyses are useful for practical myoelectric control toward improved stroke rehabilitation.