Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses

ChatEMG: Synthetic Data Generation to Control a Robotic Hand Orthosis for Stroke

Intent inferral on a hand orthosis for stroke patients is challenging due to the difficulty of data collection. Additionally, EMG signals exhibit significant variations across different conditions, sessions, and subjects, making it hard for classifiers to generalize. Traditional approaches require a large labeled dataset from the new condition, session, or subject to train intent classifiers; however, this data collection process is burdensome and time-consuming. In this paper, we propose ChatEMG, an autoregressive generative model that can generate synthetic EMG signals conditioned on prompts (i.e., a given sequence of EMG signals). ChatEMG enables us to collect only a small dataset from the new condition, session, or subject and expand it with synthetic samples conditioned on prompts from this new context. ChatEMG leverages a vast repository of previous data via generative training while still remaining context-specific via prompting. Our experiments show that these synthetic samples are classifier-agnostic and can improve intent inferral accuracy for different types of classifiers. We demonstrate that our complete approach can be integrated into a single patient session, including the use of the classifier for functional orthosis-assisted tasks. To the best of our knowledge, this is the first time an intent classifier trained partially on synthetic data has been deployed for functional control of an orthosis by a stroke survivor.

EMG Dataset for Gesture Recognition with Arm Translation

Myoelectric control has emerged as a promising approach for a wide range of applications, including controlling limb prosthetics, teleoperating robots and enabling immersive interactions in the Metaverse. However, the accuracy and robustness of myoelectric control systems are often affected by various factors, including muscle fatigue, perspiration, drifts in electrode positions and changes in arm position. The latter has received less attention despite its significant impact on signal quality and decoding accuracy. To address this gap, we present a novel dataset of surface electromyographic (EMG) signals captured from multiple arm positions. This dataset, comprising EMG and hand kinematics data from 8 participants performing 6 different hand gestures, provides a comprehensive resource for investigating position-invariant myoelectric control decoding algorithms. We envision this dataset to serve as a valuable resource for both training and benchmark arm position-invariant myoelectric control algorithms. Additionally, to expand the publicly available data capturing the variability of EMG signals across diverse arm positions, we propose a novel data acquisition protocol that can be utilized for future data collection.

Enhancing neuroprosthesis calibration: the advantage of integrating prior training over exclusive use of new data

Objective.Neuroprostheses typically operate under supervised learning, in which a machine-learning algorithm is trained to correlate neural or myoelectric activity with an individual's motor intent. Due to the stochastic nature of neuromyoelectric signals, algorithm performance decays over time. This decay is accelerated when attempting to regress proportional control of multiple joints in parallel, compared with the more typical classification-based pattern recognition control. To overcome this degradation, neuroprostheses and commercial myoelectric prostheses are often recalibrated and retrained frequently so that only the most recent, up-to-date data influences the algorithm performance. Here, we introduce and validate an alternative training paradigm in which training data from past calibrations is aggregated and reused in future calibrations for regression control.Approach.Using a cohort of four transradial amputees implanted with intramuscular electromyographic recording leads, we demonstrate that aggregating prior datasets improves prosthetic regression-based control in offline analyses and an online human-in-the-loop task. In offline analyses, we compared the performance of a convolutional neural network (CNN) and a modified Kalman filter (MKF) to simultaneously regress the kinematics of an eight-degree-of-freedom prosthesis. Both algorithms were trained under the traditional paradigm using a single dataset, as well as under the new paradigm using aggregated datasets from the past five or ten trainings.Main results.Dataset aggregation reduced the root-mean-squared error (RMSE) of algorithm estimates for both the CNN and MKF, although the CNN saw a greater reduction in error. Further offline analyses revealed that dataset aggregation improved CNN robustness when reusing the same algorithm on subsequent test days, as indicated by a smaller increase in RMSE per day. Finally, data from an online virtual-target-touching task with one amputee showed significantly better real-time prosthetic control when using aggregated training data from just two prior datasets.Significance.Altogether, these results demonstrate that training data from past calibrations should not be discarded but, rather, should be reused in an aggregated training dataset such that the increased amount and diversity of data improve algorithm performance. More broadly, this work supports a paradigm shift for the field of neuroprostheses away from daily data recalibration for linear classification models and towards daily data aggregation for non-linear regression models.

From Simulation to Reality: Predicting Torque With Fatigue Onset via Transfer Learning

Muscle fatigue impacts upper extremity function but is often overlooked in biomechanical models. The present work leveraged a transfer learning approach to improve torque predictions during fatiguing upper extremity movements. We developed two artificial neural networks to model sustained elbow flexion: one trained solely on recorded data (i.e., direct learning) and one pre-trained on simulated data and fine-tuned on recorded data (i.e., transfer learning). We simulated muscle activations and joint torques using a musculoskeletal model and a muscle fatigue model (n = 1,701 simulations). We also recorded static subject-specific features (e.g., anthropometric measurements) and dynamic muscle activations and torques during sustained elbow flexion in healthy young adults (n = 25 subjects). Using the simulated dataset, we pre-trained a long short-term memory neural network (LSTM) to regress fatiguing elbow flexion torque from muscle activations. We concatenated this pre-trained LSTM with a feedforward architecture, and fine-tuned the model on recorded muscle activations and static features to predict elbow flexion torques. We trained a similar architecture solely on the recorded data and compared each neural network's predictions on 5 leave-out subjects' data. The transfer learning model outperformed the direct learning model, as indicated by a decrease of 24.9% in their root-mean-square-errors (6.22 Nm and 8.28 Nm, respectively). The transfer learning model and direct learning model outperformed analogous musculoskeletal simulations, which consistently underpredicted elbow flexion torque. Our results suggest that transfer learning from simulated to recorded datasets can decrease reliance on assumptions inherent to biomechanical models and yield predictions robust to real-world conditions.

Simultaneous Estimation of Digit Tip Forces and Hand Postures in a Simulated Real-Life Condition With High-Density Electromyography and Deep Learning

In myoelectric control, continuous estimation of multiple degrees of freedom has an important role. Most studies have focused on estimating discrete postures or forces of the human hand but for a practical prosthetic system, both should be considered. In daily life activities, hand postures vary for grasping different objects and the amount of force exerted on each fingertip depends on the shape and weight of the object. This study aims to investigate the feasibility of continuous estimation of multiple degrees of freedom. We proposed a reach and grasp framework to study both absolute fingertip forces and hand movement types using deep learning techniques applied to high-density surface electromyography (HD-sEMG). Four daily life grasp types were examined and absolute fingertip forces were simultaneously estimated while grasping various objects, along with the grasp types. We showed that combining a 3-dimensional Convolutional Neural Network (3DCNN) with a Long Short-term Memory (LSTM) can reliably and continuously estimate the digit tip forces and classify different hand postures in human individuals. The mean absolute error (MAE) and Pearson correlation coefficient (PCC) results of the force estimation problem across all fingers and subjects were 0.46 ± 0.23 and 0.90 ± 0.03% respectively and for the classification problem, they were 0.04 ± 0.01 and 0.97 ± 0.02%. The results demonstrated that both absolute digit tip forces and hand postures can be successfully estimated through deep learning and HD-sEMG.

Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction

Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.

Sonomyography for Control of Upper-Limb Prostheses: Current State and Future Directions

Problem Statement

Despite the recent advancements in technology, many individuals with upper-limb loss struggle to achieve stable control over multiple degrees of freedom in a prosthesis. There is an ongoing need to develop noninvasive prosthesis control modalities that could improve functional patient outcomes.

Proposed Solution

Ultrasound-based sensing of muscle deformation, known as sonomyography, is an emerging sensing modality for upper-limb prosthesis control with the potential to significantly improve functionality. Sonomyography enables spatiotemporal characterization of both superficial and deep muscle activity, making it possible to distinguish the contributions of individual muscles during functional movements and derive a large set of independent prosthesis control signals. Using sonomyography to control a prosthesis has shown great promise in the research literature but has not yet been fully adapted for clinical use. This article describes the implementation of sonomyography for upper-limb prosthesis control, ongoing technological development, considerations for deploying this technology in clinical settings, and recommendations for future study.

Clinical Relevance

Sonomyography may soon become a clinically viable modality for upper-limb prosthesis control that could offer prosthetists an additional solution when selecting optimal treatment plans for their patients.

Characterizing sEMG changes with muscle fatigue for lower-extremity prostheses and exoskeletons control

In powered lower limb prostheses and exoskeleton control, sEMG based methods have been widely explored and utilized due to non-invasive nature and ability to directly reflect user intentions. However, in practical use, neuromuscular fatigue (NMF) caused by long-term use can cause significant changes in sEMG signals. These issues are constraining sEMG's active prosthetics from the laboratory to real life. In this study, the changes in sEMG signals with muscle fatigue was investigated. sEMG data from one healthy subject and a below-knee amputee were logged and analyzed when they walked on a treadmill for a long time. They will experience from no NMF to NMF and then adapt to no NMF. A variational mode decomposition-based soft interval threshold (VMD-SIT) method was used to denoise the sEMG signals. Feature extraction methods were used to extract the features of the sEMG signal, and the distribution changes of these features during walking were analyzed. The results indicated that the NMF of the amputee was quite different from and more variable than that of the healthy subject, and the ratios of features' mean value of amputee and healthy are much higher in residual-limb side (12.35-26.18) than normal side (3.52-5.91).

Alternative muscle synergy patterns of upper limb amputees

Myoelectric hand prostheses are effective tools for upper limb amputees to regain hand functions. Much progress has been made with pattern recognition algorithms to recognize surface electromyography (sEMG) patterns, but few attentions was placed on the amputees' motor learning process. Many potential myoelectric prostheses users could not fully master the control or had declined performance over time. It is possible that learning to produce distinct and consistent muscle activation patterns with the residual limb could help amputees better control the myoelectric prosthesis. In this study, we observed longitudinal effect of motor skill learning with 2 amputees who have developed alternative muscle activation patterns in response to the same set of target prosthetic actions. During a 10-week program, amputee participants were trained to produce distinct and constant muscle activations with visual feedback of live sEMG and without interaction with prosthesis. At the end, their sEMG patterns were different from each other and from non-amputee control groups. For certain intended hand motion, gradually reducing root mean square (RMS) variance was observed. The learning effect was also assessed with a CNN-LSTM mixture classifier designed for mobile sEMG pattern recognition. The classification accuracy had a rising trend over time, implicating potential performance improvement of myoelectric prosthesis control. A follow-up session took place 6 months after the program and showed lasting effect of the motor skill learning in terms of sEMG pattern classification accuracy. The results indicated that with proper feedback training, amputees could learn unique muscle activation patterns that allow them to trigger intended prosthesis functions, and the original motor control scheme is updated. The effect of such motor skill learning could help to improve myoelectric prosthetic control performance.

Online prediction of sustained muscle force from individual motor unit activities using adaptive surface EMG decomposition

Decoding movement intentions from motor unit (MU) activities to represent neural drive information plays a central role in establishing neural interfaces, but there remains a great challenge for obtaining precise MU activities during sustained muscle contractions. In this paper, we presented an online muscle force prediction method driven by individual MU activities that were decomposed from prolonged surface electromyogram (SEMG) signals in real time. In the training stage of the proposed method, a set of separation vectors was initialized for decomposing MU activities. After transferring each decomposed MU activity into a twitch force train according to its action potential waveform, a neural network was designed and trained for predicting muscle force. In the subsequent online stage, a practical double-thread-parallel algorithm was developed. One frontend thread predicted the muscle force in real time utilizing the trained network and the other backend thread simultaneously updated the separation vectors. To assess the performance of the proposed method, SEMG signals were recorded from the abductor pollicis brevis muscles of eight subjects and the contraction force was simultaneously collected. With the update procedure in the backend thread, the force prediction performance of the proposed method was significantly improved in terms of lower root mean square deviation (RMSD) of around 10% and higher fitness (R2) of around 0.90, outperforming two conventional methods. This study provides a promising technique for real-time myoelectric applications in movement control and health.

Multimodal fusion of EMG and vision for human grasp intent inference in prosthetic hand control

Objective: For transradial amputees, robotic prosthetic hands promise to regain the capability to perform daily living activities. Current control methods based on physiological signals such as electromyography (EMG) are prone to yielding poor inference outcomes due to motion artifacts, muscle fatigue, and many more. Vision sensors are a major source of information about the environment state and can play a vital role in inferring feasible and intended gestures. However, visual evidence is also susceptible to its own artifacts, most often due to object occlusion, lighting changes, etc. Multimodal evidence fusion using physiological and vision sensor measurements is a natural approach due to the complementary strengths of these modalities. Methods: In this paper, we present a Bayesian evidence fusion framework for grasp intent inference using eye-view video, eye-gaze, and EMG from the forearm processed by neural network models. We analyze individual and fused performance as a function of time as the hand approaches the object to grasp it. For this purpose, we have also developed novel data processing and augmentation techniques to train neural network components. Results: Our results indicate that, on average, fusion improves the instantaneous upcoming grasp type classification accuracy while in the reaching phase by 13.66% and 14.8%, relative to EMG (81.64% non-fused) and visual evidence (80.5% non-fused) individually, resulting in an overall fusion accuracy of 95.3%. Conclusion: Our experimental data analyses demonstrate that EMG and visual evidence show complementary strengths, and as a consequence, fusion of multimodal evidence can outperform each individual evidence modality at any given time.

Novel near E-Field Topography Sensor for Human-Machine Interfacing in Robotic Applications

This work investigates a new sensing technology for use in robotic human-machine interface (HMI) applications. The proposed method uses near E-field sensing to measure small changes in the limb surface topography due to muscle actuation over time. The sensors introduced in this work provide a non-contact, low-computational-cost, and low-noise method for sensing muscle activity. By evaluating the key sensor characteristics, such as accuracy, hysteresis, and resolution, the performance of this sensor is validated. Then, to understand the potential performance in intention detection, the unmodified digital output of the sensor is analysed against movements of the hand and fingers. This is done to demonstrate the worst-case scenario and to show that the sensor provides highly targeted and relevant data on muscle activation before any further processing. Finally, a convolutional neural network is used to perform joint angle prediction over nine degrees of freedom, achieving high-level regression performance with an RMSE value of less than six degrees for thumb and wrist movements and 11 degrees for finger movements. This work demonstrates the promising performance of this novel approach to sensing for use in human-machine interfaces.

Effects of Training and Calibration Data on Surface Electromyogram-Based Recognition for Upper Limb Amputees

Surface electromyogram (sEMG)-based gesture recognition has emerged as a promising avenue for developing intelligent prostheses for upper limb amputees. However, the temporal variations in sEMG have rendered recognition models less efficient than anticipated. By using cross-session calibration and increasing the amount of training data, it is possible to reduce these variations. The impact of varying the amount of calibration and training data on gesture recognition performance for amputees is still unknown. To assess these effects, we present four datasets for the evaluation of calibration data and examine the impact of the amount of training data on benchmark performance. Two amputees who had undergone amputations years prior were recruited, and seven sessions of data were collected for analysis from each of them. Ninapro DB6, a publicly available database containing data from ten healthy subjects across ten sessions, was also included in this study. The experimental results show that the calibration data improved the average accuracy by 3.03%, 6.16%, and 9.73% for the two subjects and Ninapro DB6, respectively, compared to the baseline results. Moreover, it was discovered that increasing the number of training sessions was more effective in improving accuracy than increasing the number of trials. Three potential strategies are proposed in light of these findings to enhance cross-session models further. We consider these findings to be of the utmost importance for the commercialization of intelligent prostheses, as they demonstrate the criticality of gathering calibration and cross-session training data, while also offering effective strategies to maximize the utilization of the entire dataset.

Reducing Motor Variability Enhances Myoelectric Control Robustness Across Untrained Limb Positions

The limb position effect is a multi-faceted problem, associated with decreased upper-limb prosthesis control acuity following a change in arm position. Factors contributing to this problem can arise from distinct environmental or physiological sources. Despite their differences in origin, the effect of each factor manifests similarly as increased input data variability. This variability can cause incorrect decoding of user intent. Previous research has attempted to address this by better capturing input data variability with data abundance. In this paper, we take an alternative approach and investigate the effect of reducing trial-to-trial variability by improving the consistency of muscle activity through user training. Ten participants underwent 4 days of myoelectric training with either concurrent or delayed feedback in a single arm position. At the end of training participants experienced a zero-feedback retention test in multiple limb positions. In doing so, we tested how well the skill learned in a single limb position generalized to untrained positions. We found that delayed feedback training led to more consistent muscle activity across both the trained and untrained limb positions. Analysis of patterns of activations in the delayed feedback group suggest a structured change in muscle activity occurs across arm positions. Our results demonstrate that myoelectric user-training can lead to the retention of motor skills that bring about more robust decoding across untrained limb positions. This work highlights the importance of reducing motor variability with practice, prior to examining the underlying structure of muscle changes associated with limb position.

Concurrent and continuous estimation of multi-finger forces by synergy mapping and reconstruction: a pilot study

Objective.The absence of intuitive control in present myoelectric interfaces makes it a challenge for users to communicate with assistive devices efficiently in real-world conditions. This study aims to tackle this difficulty by incorporating neurophysiological entities, namely muscle and force synergies, onto multi-finger force estimation to allow intuitive myoelectric control.Approach. Eleven healthy subjects performed six isometric grasping tasks at three muscle contraction levels. The exerted fingertip forces were collected concurrently with the surface electromyographic (sEMG) signals from six extrinsic and intrinsic muscles of hand. Muscle synergies were then extracted from recorded sEMG signals, while force synergies were identified from measured force data. Afterwards, a linear regressor was trained to associate the two types of synergies. This would allow us to predict multi-finger forces simply by multiplying the activation signals derived from muscle synergies with the weighting matrix of initially identified force synergies. To mitigate the false activation of unintended fingers, the force predictions were finally corrected by a finger state recognition procedure.Main results. We found that five muscle synergies and four force synergies are able to make a tradeoff between the computation load and the prediction accuracy for the proposed model; When trained and tested on all six grasping tasks, our method (SYN-II) achieved better performance (R2= 0.80 ± 0.04, NRMSE = 0.19 ± 0.01) than conventional sEMG amplitude-based method; Interestingly, SYN-II performed better than all other methods when tested on two unknown tasks outside the four training tasks (R2= 0.74 ± 0.03, NRMSE = 0.22 ± 0.02), which indicated better generalization ability.Significance. This study shows the first attempt to link between muscle and force synergies to allow concurrent and continuous estimation of multi-finger forces from sEMG. The proposed approach may lay the foundation for high-performance myoelectric interfaces that allow users to control robotic hands in a more natural and intuitive manner.

Progressive unsupervised control of myoelectric upper limbs

Objective.Unsupervised myocontrol methods aim to create control models for myoelectric prostheses while avoiding the complications of acquiring reliable, regular, and sufficient labeled training data. A limitation of current unsupervised methods is that they fix the number of controlled prosthetic functions a priori, thus requiring an initial assessment of the user's motor skills and neglecting the development of novel motor skills over time.Approach.We developed a progressive unsupervised myocontrol (PUM) paradigm in which the user and the control model coadaptively identify distinct muscle synergies, which are then used to control arbitrarily associated myocontrol functions, each corresponding to a hand or wrist movement. The interaction starts with learning a single function and the user may request additional functions after mastering the available ones, which aligns the evolution of their motor skills with an increment in system complexity. We conducted a multi-session user study to evaluate PUM and compare it against a state-of-the-art non-progressive unsupervised alternative. Two participants with congenital upper-limb differences tested PUM, while ten non-disabled control participants tested either PUM or the non-progressive baseline. All participants engaged in myoelectric control of a virtual hand and wrist.Main results.PUM enabled autonomous learning of three myocontrol functions for participants with limb differences, and of all four available functions for non-disabled subjects, using both existing or newly identified muscle synergies. Participants with limb differences achieved similar success rates to non-disabled ones on myocontrol tests, but faced greater difficulties in internalizing new motor skills and exhibited slightly inferior movement quality. The performance was comparable with either PUM or the non-progressive baseline for the group of non-disabled participants.Significance.The PUM paradigm enables users to autonomously learn to operate the myocontrol system, adapts to the users' varied preexisting motor skills, and supports the further development of those skills throughout practice.

Consistent control information driven musculoskeletal model for multiday myoelectric control

Objective.Musculoskeletal model (MM)-based myoelectric interface has aroused great interest in human-machine interaction. However, the performance of electromyography (EMG)-driven MM in long-term use would be degraded owing to the inherent non-stationary characteristics of EMG signals. Here, to improve the estimation performance without retraining, we proposed a consistent muscle excitation extraction approach based on an improved non-negative matrix factorization (NMF) algorithm for MM when applied to simultaneous hand and wrist movement prediction.Approach.We added constraints andL2-norm regularization terms to the objective function of classic NMF regarding muscle weighting matrix and time-varying profiles, through which stable muscle synergies across days were identified. The resultant profiles of these synergies were then used to drive the MM. Both offline and online experiments were conducted to evaluate the performance of the proposed method in inter-day scenarios.Main results.The results demonstrated significantly better and more robust performance over several competitive methods in inter-day experiments, including machine learning methods, EMG envelope-driven MM, and classic NMF-based MM. Furthermore, the analysis of control information on different days revealed the effectiveness of the proposed method in obtaining consistent muscle excitations.Significance.The outcomes potentially provide a novel and promising pathway for the robust and zero-retraining control of myoelectric interfaces.

IMU Sensors Measurements Towards the Development of Novel Prosthetic Arm Control Strategies

The complexity of the human upper limb makes replicating it in a prosthetic device a significant challenge. With advancements in mechatronic developments involving the addition of a large number of degrees of freedom, novel control strategies are required. To accommodate this need, this study aims at developing an IMU-based control for the HannesARM upper-limb prosthetic device, as a proof-of-concept for new control strategies integrating data-fusion approaches. The natural human control of the upper-limb is based on different inputs that allow adaptive control. To mimic this in prostheses, the implementation of IMUs provides kinematic information of both the stump and the prosthesis to enrich the EMG control. The principle of operation is to decode upper limb movements by using a custom-made system and to replicate them in prosthetic arms improving the control algorithms. To evaluate the system's effectiveness, the custom algorithm's motion extraction was compared to a motion capture system using fifteen able-bodied subjects. The results showed that this system scored 0.16 ± 0.04 and 0.81 ± 0.12 in Root Mean Squared Error and Cross-Correlation compared to the motion capture system. Experimental results demonstrate how this work can extract valuable kinematic information necessary for new and improved control strategies, such as intention detection or pattern recognition, to allow users to perform a broader range of tasks and enhancing in turn their quality of life.

Asymmetric Effects of Different Training-Testing Mismatch Types on Myoelectric Regression via Deep Learning

This paper investigates how predictions of a convolutional neural network (CNN) suited for myoelectric simultaneous and proportional control (SPC) are affected when training and testing conditions differ. We used a dataset composed of electromyogram (EMG) signals and joint angular accelerations measured from volunteers drawing a star. This task was repeated multiple times using different combinations of motion amplitude and frequency. CNNs were trained with data from a given combination and tested under different combinations. Predictions were compared between situations in which training and testing conditions matched versus when there was a training-testing mismatch. Changes in predictions were assessed through three metrics: normalized root mean squared error (NRMSE), correlation, and slope of the linear regression between targets and predictions. We found that predictive performance declined differently depending on whether the confounding factors (amplitude and frequency) increased or decreased between training and testing. Correlations dropped as the factors decreased, whereas slopes deteriorated when factors increased. NRMSEs worsened when factors increased or decreased, with more accentuated deterioration for decreasing factors. We argue that worse correlations could be related to differences in EMG signal-to-ratio (SNR) between training and testing, which affected the noise robustness of the CNNs' learned internal features. Slope deterioration could be a result of the networks' inability to predict accelerations outside the range seen during training. These two mechanisms may also asymmetrically increase NRMSE. Finally, our findings open further possibilities to develop strategies to mitigate the negative impact of confounding factor variability on myoelectric SPC devices.

Stiffness Optimal Modulation of a Variable Stiffness Energy Storage Hip Exoskeleton and Experiments on Its Assistance Effect

Lower limb energy storage assisted exoskeletons realize walking assistance by using the energy stored by elastic elements during walking. Such exoskeletons are characterized by a small volume, light weight and low price. However, energy storage assisted exoskeletons adopt fixed stiffness joints typically, which cannot adapt to changes of the wearer's height, weight, or walking speed. In this study, based on the analysis of the energy flow characteristics and stiffness change characteristics of lower limb joints during a human walking on flat ground, a novel variable stiffness energy storage assisted hip exoskeleton is designed, and a stiffness optimization modulation method is proposed to store most of the negative work done by the human hip joint when walking. Through the analysis of the surface electromyography signals of the rectus femoris and long head of the biceps femoris, it is found that the muscle fatigue of the rectus femoris is reduced by 8.5% under the optimal stiffness assistance condition, and the exoskeleton provides better assistance under the optimal stiffness assistance condition.

Recalibration of myoelectric control with active learning

Introduction

Improving the robustness of myoelectric control to work over many months without the need for recalibration could reduce prosthesis abandonment. Current approaches rely on post-hoc error detection to verify the certainty of a decoder's prediction using predefined threshold value. Since the decoder is fixed, the performance decline over time is inevitable. Other approaches such as supervised recalibration and unsupervised self-recalibration entail limitations in scaling up and computational resources. The objective of this paper is to study active learning as a scalable, human-in-the-loop framework, to improve the robustness of myoelectric control.

Method

Active learning and linear discriminate analysis methods were used to create an iterative learning process, to modify decision boundaries based on changes in the data. We simulated a real-time scenario. We exploited least confidence, smallest margin and entropy reduction sampling strategies in single and batch-mode sample selection. Optimal batch-mode sampling was considered using ranked batch-mode active learning.

Results

With only 3.2 min of data carefully selected by the active learner, the decoder outperforms random sampling by 4-5 and ~2% for able-bodied and people with limb difference, respectively. We observed active learning strategies to systematically and significantly enhance the decoders adaptation while optimizing the amount of training data on a class-specific basis. Smallest margin and least confidence uncertainty were shown to be the most supreme.

Discussion

We introduce for the first time active learning framework for long term adaptation in myoelectric control. This study simulates closed-loop environment in an offline manner and proposes a pipeline for future real-time deployment.

Adaptive Lower Limb Pattern Recognition for Multi-Day Control

Pattern recognition in EMG-based control systems suffer from increase in error rate over time, which could lead to unwanted behavior. This so-called concept drift in myoelectric control systems could be caused by fatigue, sensor replacement and varying skin conditions. To circumvent concept drift, adaptation strategies could be used to retrain a pattern recognition system, which could lead to comparable error rates over multiple days. In this study, we investigated the error rate development over one week and compared three adaptation strategies to reduce the error rate increase. The three adaptation strategies were based on entropy, on backward prediction and a combination of backward prediction and entropy. Ten able-bodied subjects were measured on four measurement days while performing gait-related activities. During the measurement electromyography and kinematics were recorded. The three adaptation strategies were implemented and compared against the baseline error rate and against adaptation using the ground truth labels. It can be concluded that without adaptation the baseline error rate increases significantly from day 1 to 2, but plateaus on day 2, 3 and 7. Of the three tested adaptation strategies, entropy based adaptation showed the smallest increase in error rate over time. It can be concluded that entropy based adaptation is simple to implement and can be considered a feasible adaptation strategy for lower limb pattern recognition.

Improving the Robustness of Human-Machine Interactive Control for Myoelectric Prosthetic Hand During Arm Position Changing

Robust classification of natural hand grasp type based on electromyography (EMG) still has some shortcomings in the practical prosthetic hand control, owing to the influence of dynamic arm position changing during hand actions. This study provided a framework for robust hand grasp type classification during dynamic arm position changes, improving both the “hardware” and “algorithm” components. In the hardware aspect, co-located synchronous EMG and force myography (FMG) signals are adopted as the multi-modal strategy. In the algorithm aspect, a sequential decision algorithm is proposed by combining the RNN-based deep learning model with a knowledge-based post-processing model. Experimental results showed that the classification accuracy of multi-modal EMG-FMG signals was increased by more than 10% compared with the EMG-only signal. Moreover, the classification accuracy of the proposed sequential decision algorithm improved the accuracy by more than 4% compared with other baseline models when using both EMG and FMG signals.

SeNic: An Open Source Dataset for sEMG-Based Gesture Recognition in Non-ideal Conditions

In order to reduce the gap between the laboratory environment and actual use in daily life of human-machine interaction based on surface electromyogram (sEMG) intent recognition, this paper presents a benchmark dataset of sEMG in non-ideal conditions (SeNic). The dataset mainly consists of 8-channel sEMG signals, and electrode shifts from an 3D-printed annular ruler. A total of 36 subjects participate in our data acquisition experiments of 7 gestures in non-ideal conditions, where non-ideal factors of 1) electrode shifts, 2) individual difference, 3) muscle fatigue, 4) inter-day difference, and 5) arm postures are elaborately involved. Signals of sEMG are validated first in temporal and frequency domains. Results of recognizing gestures in ideal conditions indicate the high quality of the dataset. Adverse impacts in non-ideal conditions are further revealed in the amplitudes of these data and recognition accuracies. To be concluded, SeNic is a benchmark dataset that introduces several non-ideal factors which often degrade the robustness of sEMG-based systems. It could be used as a freely available dataset and a common platform for researchers in the sEMG-based recognition community. The benchmark dataset SeNic are available online via the website3.

Classification Performance and Feature Space Characteristics in Individuals With Upper Limb Loss Using Sonomyography

Objective: Sonomyography, or ultrasound-based sensing of muscle deformation, is an emerging modality for upper limb prosthesis control. Although prior studies have shown that individuals with upper limb loss can achieve successful motion classification with sonomyography, it is important to better understand the time-course over which proficiency develops. In this study, we characterized user performance during their initial and subsequent exposures to sonomyography. Method: Ultrasound images corresponding to a series of hand gestures were collected from individuals with transradial limb loss under three scenarios: during their initial exposure to sonomyography (Experiment 1), during a subsequent exposure to sonomyography where they were provided biofeedback as part of a training protocol (Experiment 2), and during testing sessions held on different days (Experiment 3). User performance was characterized by offline classification accuracy, as well as metrics describing the consistency and separability of the sonomyography signal patterns in feature space. Results: Classification accuracy was high during initial exposure to sonomyography (96.2 ± 5.9%) and did not systematically change with the provision of biofeedback or on different days. Despite this stable classification performance, some of the feature space metrics changed. Conclusions: User performance was strong upon their initial exposure to sonomyography and did not improve with subsequent exposure. Clinical Impact: Prosthetists may be able to quickly assess if a patient will be successful with sonomyography without submitting them to an extensive training protocol, leading to earlier socket fabrication and delivery.

First Demonstration of Functional Task Performance Using a Sonomyographic Prosthesis: A Case Study

Ultrasound-based sensing of muscle deformation, known as sonomyography, has shown promise for accurately classifying the intended hand grasps of individuals with upper limb loss in offline settings. Building upon this previous work, we present the first demonstration of real-time prosthetic hand control using sonomyography to perform functional tasks. An individual with congenital bilateral limb absence was fitted with sockets containing a low-profile ultrasound transducer placed over forearm muscle tissue in the residual limbs. A classifier was trained using linear discriminant analysis to recognize ultrasound images of muscle contractions for three discrete hand configurations (rest, tripod grasp, index finger point) under a variety of arm positions designed to cover the reachable workspace. A prosthetic hand mounted to the socket was then controlled using this classifier. Using this real-time sonomyographic control, the participant was able to complete three functional tasks that required selecting different hand grasps in order to grasp and move one-inch wooden blocks over a broad range of arm positions. Additionally, these tests were successfully repeated without retraining the classifier across 3 hours of prosthesis use and following simulated donning and doffing of the socket. This study supports the feasibility of using sonomyography to control upper limb prostheses in real-world applications.

Discrete action control for prosthetic digits

We aim to develop a paradigm for simultaneous and independent control of multiple degrees of freedom (DOFs) for upper-limb prostheses. To that end, we introduce action control, a novel method to operate prosthetic digits with surface electromyography (EMG) based on multi-output, multi-class classification. At each time step, the decoder classifies movement intent for each controllable DOF into one of three categories: open, close, or stall (i.e., no movement). We implemented a real-time myoelectric control system using this method and evaluated it by running experiments with one unilateral and two bilateral amputees. Participants controlled a six-DOF bar interface on a computer display, with each DOF corresponding to a motor function available in multi-articulated prostheses. We show that action control can significantly and systematically outperform the state-of-the-art method of position control via multi-output regression in both task- and non-task-related measures. Using the action control paradigm, improvements in median task performance over regression-based control ranged from 20.14% to 62.32% for individual participants. Analysis of a post-experimental survey revealed that all participants rated action higher than position control in a series of qualitative questions and expressed an overall preference for the former. Action control has the potential to improve the dexterity of upper-limb prostheses. In comparison with regression-based systems, it only requires discrete instead of real-valued ground truth labels, typically collected with motion tracking systems. This feature makes the system both practical in a clinical setting and also suitable for bilateral amputation. This work is the first demonstration of myoelectric digit control in bilateral upper-limb amputees. Further investigation and preclinical evaluation are required to assess the translational potential of the method.

A generic neural network model to estimate populational neural activity for robust neural decoding

BACKGROUND

Robust and continuous neural decoding is crucial for reliable and intuitive neural-machine interactions. This study developed a novel generic neural network model that can continuously predict finger forces based on decoded populational motoneuron firing activities.

METHOD

We implemented convolutional neural networks (CNNs) to learn the mapping from high-density electromyogram (HD-EMG) signals of forearm muscles to populational motoneuron firing frequency. We first extracted the spatiotemporal features of EMG energy and frequency maps to improve learning efficiency, given that EMG signals are intrinsically stochastic. We then established a generic neural network model by training on the populational neuron firing activities of multiple participants. Using a regression model, we continuously predicted individual finger forces in real-time. We compared the force prediction performance with two state-of-the-art approaches: a neuron-decomposition method and a classic EMG-amplitude method.

RESULTS

Our results showed that the generic CNN model outperformed the subject-specific neuron-decomposition method and the EMG-amplitude method, as demonstrated by a higher correlation coefficient between the measured and predicted forces, and a lower force prediction error. In addition, the CNN model revealed more stable force prediction performance over time.

CONCLUSIONS

Overall, our approach provides a generic and efficient continuous neural decoding approach for real-time and robust human-robot interactions.

A New Labeling Approach for Proportional Electromyographic Control

Different control strategies are available for human machine interfaces based on electromyography (EMG) to map voluntary muscle signals to control signals of a remote controlled device. Complex systems such as robots or multi-fingered hands require a natural commanding, which can be realized with proportional and simultaneous control schemes. Machine learning approaches and methods based on regression are often used to realize the desired functionality. Training procedures often include the tracking of visual stimuli on a screen or additional sensors, such as cameras or force sensors, to create labels for decoder calibration. In certain scenarios, where ground truth, such as additional sensor data, can not be measured, e.g., with people suffering from physical disabilities, these methods come with the challenge of generating appropriate labels. We introduce a new approach that uses the EMG-feature stream recorded during a simple training procedure to generate continuous labels. The method avoids synchronization mismatches in the labels and has no need for additional sensor data. Furthermore, we investigated the influence of the transient phase of the muscle contraction when using the new labeling approach. For this purpose, we performed a user study involving 10 subjects performing online 2D goal-reaching and tracking tasks on a screen. In total, five different labeling methods were tested, including three variations of the new approach as well as methods based on binary labels, which served as a baseline. Results of the evaluation showed that the introduced labeling approach in combination with the transient phase leads to a proportional command that is more accurate than using only binary labels. In summary, this work presents a new labeling approach for proportional EMG control without the need of a complex training procedure or additional sensors.

End-to-End Estimation of Hand- and Wrist Forces From Raw Intramuscular EMG Signals Using LSTM Networks

Processing myoelectrical activity in the forearm has for long been considered a promising framework to allow transradial amputees to control motorized prostheses. In spite of expectations, contemporary muscle-computer interfaces built for this purpose typically fail to satisfy one or more important desiderata, such as accuracy, robustness, and/or naturalness of control, in part due to difficulties in acquiring high-quality signals continuously outside laboratory conditions. In light of such problems, surgically implanted electrodes have been made a viable option that allows for long-term acquisition of intramuscular electromyography (iEMG) measurements of spatially precise origin. As it stands, the question of how information embedded in such signals is best extracted and combined across multiple channels remains open. This study presents and evaluates an approach to this end that uses deep neural networks based on the Long Short-Term Memory (LSTMs) architecture to regress forces exerted by multiple degrees of freedom (DoFs) from multichannel iEMG. Three deep learning models, representing three distinct regression strategies, were evaluated: (I) One-to-One, wherein each DoF is separately estimated by an LSTM model processing a single iEMG channel, (II) All-to-One, wherein each DoF is separately estimated by an LSTM model processing all iEMG channels, and (III) All-to-All, wherein a single LSTM model with access to all iEMG channels estimates all DoFs simultaneously. All models operate on raw iEMG, with no preliminary feature extraction required. When evaluated on a dataset comprising six iEMG channels with concurrent force measurements acquired from 14 subjects, all LSTM strategies were found to significantly outperform a baseline feature-based linear control regression method. This finding indicates that recurrent neural networks can learn to transform raw forearm iEMG signals directly into representations that correlate with forces exerted at the level of the hand to a greater degree than simple features do. Furthermore, the All-to-All and All-to-One strategies were found to exhibit better performance than the One-to-One strategy. This finding suggests that, in spite of the spatially local nature of signals, iEMG from muscles not directly actuating the relevant DoF can provide contextual information that aid in decoding motor intent.

Design and Validation of a Sensor Fault-Tolerant Module for Real-Time High-Density EMG Pattern Recognition

With the advancements in electronics technology, high-density (HD) EMG sensing systems have become available and have been investigated for their feasibility and performance in neural-machine interface (NMI) applications. Comparing to the traditional single channel-based targeted muscle sensing method, HD EMG sensing performs a sampling of the electrical activity over a larger surface area and has the promise of 1) providing richer neural information from one temporal and two spatial dimensions and 2) ease of wear in real life without the need of anatomically targeted electrode placement. To use HD EMG in real-time NMI applications, challenges including high computational burden and unreliability of EMG recordings over time need to be addressed. This paper presented an HD EMG PR based NMI which seamlessly integrates HD EMG PR with a Sensor Fault-Tolerant Module (SFTM) which aimed to provide robust PR in real time. Experimental results showed that the SFTM was able to recover the PR accuracies by 6%-22% from disturbances including contact artifacts and loose contacts. A Python-based implementation of the proposed HD EMG SFTM was developed and was demonstrated to be computationally efficient for real-time performance. These results have demonstrated the feasibility of a robust real-time HD EMG PR-based NMI.

In-silico development and assessment of a Kalman filter motor decoder for prosthetic hand control

Up to 50% of amputees abandon their prostheses, partly due to rapid degradation of the control systems, which require frequent recalibration. The goal of this study was to develop a Kalman filter-based approach to decoding motoneuron activity to identify movement kinematics and thereby provide stable, long-term, accurate, real-time decoding. The Kalman filter-based decoder was examined via biologically varied datasets generated from a high-fidelity computational model of the spinal motoneuron pool. The estimated movement kinematics controlled a simulated MuJoCo prosthetic hand. This clear-box approach showed successful estimation of hand movements under eight varied physiological conditions with no retraining. The mean correlation coefficient of 0.98 and mean normalized root mean square error of 0.06 over these eight datasets provide proof of concept that this decoder would improve long-term integrity of performance while performing new, untrained movements. Additionally, the decoder operated in real-time (~0.3 ms). Further results include robust performance of the Kalman filter when re-trained to more severe post-amputation limitations in the type and number of motoneurons remaining. An additional analysis shows that the decoder achieves better accuracy when using the firing of individual motoneurons as input, compared to using aggregate pool firing. Moreover, the decoder demonstrated robustness to noise affecting both the trained decoder parameters and the decoded motoneuron activity. These results demonstrate the utility of a proof of concept Kalman filter decoder that can support prosthetics' control systems to maintain accurate and stable real-time movement performance.

Gesture Recognition Using Surface Electromyography and Deep Learning for Prostheses Hand: State-of-the-Art, Challenges, and Future

Amputation of the upper limb brings heavy burden to amputees, reduces their quality of life, and limits their performance in activities of daily life. The realization of natural control for prosthetic hands is crucial to improving the quality of life of amputees. Surface electromyography (sEMG) signal is one of the most widely used biological signals for the prediction of upper limb motor intention, which is an essential element of the control systems of prosthetic hands. The conversion of sEMG signals into effective control signals often requires a lot of computational power and complex process. Existing commercial prosthetic hands can only provide natural control for very few active degrees of freedom. Deep learning (DL) has performed surprisingly well in the development of intelligent systems in recent years. The significant improvement of hardware equipment and the continuous emergence of large data sets of sEMG have also boosted the DL research in sEMG signal processing. DL can effectively improve the accuracy of sEMG pattern recognition and reduce the influence of interference factors. This paper analyzes the applicability and efficiency of DL in sEMG-based gesture recognition and reviews the key techniques of DL-based sEMG pattern recognition for the prosthetic hand, including signal acquisition, signal preprocessing, feature extraction, classification of patterns, post-processing, and performance evaluation. Finally, the current challenges and future prospects in clinical application of these techniques are outlined and discussed.

Learning regularized representations of categorically labelled surface EMG enables simultaneous and proportional myoelectric control

Background

Processing the surface electromyogram (sEMG) to decode movement intent is a promising approach for natural control of upper extremity prostheses. To this end, this paper introduces and evaluates a new framework which allows for simultaneous and proportional myoelectric control over multiple degrees of freedom (DoFs) in real-time. The framework uses multitask neural networks and domain-informed regularization in order to automatically find nonlinear mappings from the forearm sEMG envelope to multivariate and continuous encodings of concurrent hand- and wrist kinematics, despite only requiring categorical movement instruction stimuli signals for calibration.

Methods

Forearm sEMG with 8 channels was collected from healthy human subjects (N = 20) and used to calibrate two myoelectric control interfaces, each with two output DoFs. The interfaces were built from (I) the proposed framework, termed Myoelectric Representation Learning (MRL), and, to allow for comparisons, from (II) a standard pattern recognition framework based on Linear Discriminant Analysis (LDA). The online performances of both interfaces were assessed with a Fitts’s law type test generating 5 quantitative performance metrics. The temporal stabilities of the interfaces were evaluated by conducting identical tests without recalibration 7 days after the initial experiment session.

Results

Metric-wise two-way repeated measures ANOVA with factors method (MRL vs LDA) and session (day 1 vs day 7) revealed a significant (\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$p<0.05$$\end{document}p<0.05) advantage for MRL over LDA in 5 out of 5 performance metrics, with metric-wise effect sizes (Cohen’s \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$d$$\end{document}d) separating MRL from LDA ranging from \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left|d\right|=0.62$$\end{document}d=0.62 to \documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$\left|d\right|=1.13$$\end{document}d=1.13. No significant effect on any metric was detected for neither session nor interaction between method and session, indicating that none of the methods deteriorated significantly in control efficacy during one week of intermission.

Conclusions

The results suggest that MRL is able to successfully generate stable mappings from EMG to kinematics, thereby enabling myoelectric control with real-time performance superior to that of the current commercial standard for pattern recognition (as represented by LDA). It is thus postulated that the presented MRL approach can be of practical utility for muscle-computer interfaces.

Muscle Fatigue Analysis With Optimized Complementary Ensemble Empirical Mode Decomposition and Multi-Scale Envelope Spectral Entropy

The preprocessing of surface electromyography (sEMG) signals with complementary ensemble empirical mode decomposition (CEEMD) improves frequency identification precision and temporal resolution, and lays a good foundation for feature extraction. However, a mode-mixing problem often occurs when the CEEMD decomposes an sEMG signal that exhibits intermittency and contains components with a near-by spectrum into intrinsic mode functions (IMFs). This paper presents a method called optimized CEEMD (OCEEMD) to solve this problem. The method integrates the least-squares mutual information (LSMI) and the chaotic quantum particle swarm optimization (CQPSO) algorithm in signal decomposition. It uses the LSMI to calculate the correlation between IMFs so as to reduce mode mixing and uses the CQPSO to optimize the standard deviation of Gaussian white noise so as to improve iteration efficiency. Then, useful IMFs are selected and added to reconstruct a de-noised signal. Finally, considering that the IMFs contain abundant frequency and envelope information, this paper extracts the multi-scale envelope spectral entropy (MSESEn) from the reconstructed sEMG signal. Some original sEMG signals, which were collected from experiments, were used to validate the methods. Compared with the CEEMD and complete ensemble empirical mode decomposition with adaptive noise (CEEMDAN), the OCEEMD effectively suppresses mode mixing between IMFs with rapid iteration. Compared with approximate entropy (ApEn) and sample entropy (SampEn), the MSESEn clearly shows a declining tendency with time and is sensitive to muscle fatigue. This suggests a potential use of this approach for sEMG signal preprocessing and the analysis of muscle fatigue.

Real-time finger force prediction via parallel convolutional neural networks: a preliminary study

Continuous and accurate decoding of intended motions is critical for human-machine interactions. Here, we developed a novel approach for real-time continuous prediction of forces in individual fingers using parallel convolutional neural networks (CNNs). We extracted populational motor unit discharge frequency using CNNs in a parallel structure without spike sorting. The CNN parameters were trained based on two features from high-density electromyogram (HD-EMG), namely temporal energy heatmaps and frequency spectrum maps. The populational motor unit discharge frequency was then used to continuously predict finger forces based on a linear regression model. The force prediction performance was compared with a motor unit decomposition method and the conventional EMG amplitude-based method. Our results showed that the correlation coefficient between the predicted and the recorded forces of the CNN approach was on average 0.91, compared with the offline decomposition method of 0.89, the online decomposition method of 0.82, and the EMG amplitude method of 0.81. Additionally, the CNN based approach showed generalizable performance, with CNN trained on one finger applicable to a different finger. The outcomes suggest that our CNN based algorithm can offer an accurate and efficient force decoding method for human-machine interactions.

Should Hands Be Restricted When Measuring Able-Bodied Participants to Evaluate Machine Learning Controlled Prosthetic Hands?

OBJECTIVE

When evaluating methods for machine-learning controlled prosthetic hands, able-bodied participants are often recruited, for practical reasons, instead of participants with upper limb absence (ULA). However, able-bodied participants have been shown to often perform myoelectric control tasks better than participants with ULA. It has been suggested that this performance difference can be reduced by restricting the wrist and hand movements of able-bodied participants. However, the effect of such restrictions on the consistency and separability of the electromyogram's (EMG) features remains unknown. The present work investigates whether the EMG separability and consistency between unaffected and affected arms differ and whether they change after restricting the unaffected limb in persons with ULA.

METHODS

Both arms of participants with unilateral ULA were compared in two conditions: with the unaffected hand and wrist restricted or not. Furthermore, it was tested if the effect of arm and restriction is influenced by arm posture (arm down, arm in front, or arm up).

RESULTS

Fourteen participants (two women, age = 53.4±4.05) with acquired transradial limb loss were recruited. We found that the unaffected limb generated more separated EMG than the affected limb. Furthermore, restricting the unaffected hand and wrist lowered the separability of the EMG when the arm was held down.

CONCLUSION

Limb restriction is a viable method to make the EMG of able-bodied participants more similar to that of participants with ULA.

SIGNIFICANCE

Future research that evaluates methods for machine learning controlled hands in able-bodied participants should restrict the participants' hand and wrist.

Current Trends and Confounding Factors in Myoelectric Control: Limb Position and Contraction Intensity

This manuscript presents a hybrid study of a comprehensive review and a systematic(research) analysis. Myoelectric control is the cornerstone ofmany assistive technologies used in clinicalpractice, such as prosthetics and orthoses, and human-computer interaction, such as virtual reality control.Although the classification accuracy of such devices exceeds 90% in a controlled laboratory setting,myoelectric devices still face challenges in robustness to variability of daily living conditions.The intrinsic physiological mechanisms limiting practical implementations of myoelectric deviceswere explored: the limb position effect and the contraction intensity effect. The degradationof electromyography (EMG) pattern recognition in the presence of these factors was demonstratedon six datasets, where classification performance was 13% and 20% lower than the controlledsetting for the limb position and contraction intensity effect, respectively. The experimental designsof limb position and contraction intensity literature were surveyed. Current state-of-the-art trainingstrategies and robust algorithms for both effects were compiled and presented. Recommendationsfor future limb position effect studies include: the collection protocol providing exemplars of at least 6positions (four limb positions and three forearm orientations), three-dimensional space experimentaldesigns, transfer learning approaches, and multi-modal sensor configurations. Recommendationsfor future contraction intensity effect studies include: the collection of dynamic contractions, nonlinearcomplexity features, and proportional control.

Gaze, visual, myoelectric, and inertial data of grasps for intelligent prosthetics

A hand amputation is a highly disabling event, having severe physical and psychological repercussions on a person's life. Despite extensive efforts devoted to restoring the missing functionality via dexterous myoelectric hand prostheses, natural and robust control usable in everyday life is still challenging. Novel techniques have been proposed to overcome the current limitations, among them the fusion of surface electromyography with other sources of contextual information. We present a dataset to investigate the inclusion of eye tracking and first person video to provide more stable intent recognition for prosthetic control. This multimodal dataset contains surface electromyography and accelerometry of the forearm, and gaze, first person video, and inertial measurements of the head recorded from 15 transradial amputees and 30 able-bodied subjects performing grasping tasks. Besides the intended application for upper-limb prosthetics, we also foresee uses for this dataset to study eye-hand coordination in the context of psychophysics, neuroscience, and assistive robotics.

Comparing EMG-Based Human-Machine Interfaces for Estimating Continuous, Coordinated Movements

Electromyography (EMG)-based interfaces are trending toward continuous, simultaneous control with multiple degrees of freedom. Emerging methods range from data-driven approaches to biomechanical model-based methods. However, there has been no direct comparison between these two types of continuous EMG-based interfaces. The aim of this study was to compare a musculoskeletal model (MM) with two data-driven approaches, linear regression (LR) and artificial neural network (ANN), for predicting continuous wrist and hand motions for EMG-based interfaces. Six able-bodied subjects and one transradial amputee subject performed (missing) metacarpophalangeal (MCP) and wrist flexion/extension, simultaneously or independently, while four EMG signals were recorded from forearm muscles. To add variation to the EMG signals, the subjects repeated the MCP and wrist motions at various upper extremity postures. For each subject, the EMG signals collected from the neutral posture were used to build the EMG interfaces; the EMG signals collected from all postures were used to evaluate the interfaces. The performance of the interface was quantified by Pearson's correlation coefficient (r) and the normalized root mean square error (NRMSE) between measured and estimated joint angles. The results demonstrated that the MM predicted movements more accurately, with higher r values and lower NRMSE, than either LR or ANN. Similar results were observed in the transradial amputee. Additionally, the variation in r across postures, an indicator of reliability against posture changes, was significantly lower (better) for the MM than for either LR or ANN. Our findings suggest that incorporating musculoskeletal knowledge into EMG-based human-machine interfaces could improve the estimation of continuous, coordinated motion.