Continuous-Context, User-Independent, Real-Time Intent Recognition for Powered Lower-Limb Prostheses

Community ambulation is essential for maintaining a healthy lifestyle, but it poses significant challenges for individuals with limb loss due to complex task demands. In wearable robotics, particularly powered prostheses, there is a critical need to accurately estimate environmental context, such as walking speed and slope, to offer intuitive and seamless assistance during varied ambulation tasks. We developed a user-independent and multicontext, intent recognition system that was deployed in real-time on an Open Source Leg (OSL). We recruited 11 individuals with transfemoral amputation, with seven participants used for real-time validation. Our findings revealed two main conclusions: (1) the user-independent (IND) performance across speed and slope was not statistically different from user-dependent (DEP) models in real-time and did not degrade compared to its offline counterparts, and (2) IND walking speed estimates showed ∼0.09 m/s mean absolute error (MAE) and slope estimates showed ∼0.95 deg MAE across multicontext scenarios. Additionally, we provide an open-source dataset to facilitate further research in accurately estimating speed and slope using an IND approach in real-world walking tasks on a powered prosthesis.

Integrating Upper-Limb Prostheses with the Human Body: Technology Advances, Readiness, and Roles in Human-Prosthesis Interaction

Significant advances in bionic prosthetics have occurred in the past two decades. The field's rapid expansion has yielded many exciting technologies that can enhance the physical, functional, and cognitive integration of a prosthetic limb with a human. We review advances in the engineering of prosthetic devices and their interfaces with the human nervous system, as well as various surgical techniques for altering human neuromusculoskeletal systems for seamless human-prosthesis integration. We discuss significant advancements in research and clinical translation, focusing on upper limbprosthetics since they heavily rely on user intent for daily operation, although many discussed technologies have been extended to lower limb prostheses as well. In addition, our review emphasizes the roles of advanced prosthetics technologies in complex interactions with humans and the technology readiness levels (TRLs) of individual research advances. Finally, we discuss current gaps and controversies in the field and point out future research directions, guided by TRLs.

Movement recognition via channel-activation-wise sEMG attention

CONTEXT

Surface electromyography (sEMG) signals contain rich information recorded from muscle movements and therefore reflect the user's intention. sEMG has seen dominant applications in rehabilitation, clinical diagnosis as well as human engineering, etc. However, current feature extraction methods for sEMG signals have been seriously limited by their stochasticity, transiency, and non-stationarity.

OBJECTIVE

Our objective is to combat the difficulties induced by the aforementioned downsides of sEMG and thereby extract representative features for various downstream movement recognition.

METHOD

We propose a novel 3-axis view of sEMG features composed of temporal, spatial, and channel-wise summary. We leverage the state-of-the-art architecture Transformer to enforce efficient parallel search and to get rid of limitations imposed by previous work in gesture classification. The transformer model is designed on top of an attention-based module, which allows for the extraction of global contextual relevance among channels and the use of this relevance for sEMG recognition.

RESULTS

We compared the proposed method against existing methods on two Ninapro datasets consisting of data from both healthy people and amputees. Experimental results show the proposed method attains the state-of-the-art (SOTA) accuracy on both datasets. We further show that the proposed method enjoys strong generalization ability: a new SOTA is achieved by pretraining the model on a different dataset followed by fine-tuning it on the target dataset.

Multi-source domain generalization and adaptation toward cross-subject myoelectric pattern recognition

Objective.Myoelectric pattern recognition (MPR) has shown satisfactory performance under ideal laboratory conditions. Nevertheless, the individual variances lead to dramatic performance degradation in cross-user MPR applications. It is crucial to enable the myoelectric interface to adapt to multiple users' surface electromyography (sEMG) distributions in practical.Approach.Domain adaptation (DA) is a promising approach to tackle cross-user challenges due to its ability to diminish the divergence between individual users' EMG distributions and escalate model generalization performance. However, existing DA methods in sEMG control are based on single-source domain adaptation (SDA). SDA solely mixes multiple training users' data as a combined source domain and attempts to align with a novel user. This simple data mixing manner ignores the sEMG distribution variations between disparate training users, leading to an insufficient variance elimination and lower performance. To this end, this paper proposes a multi-source synchronize domain adaptation framework with both DA and domain generalization (DG) capability. This multi-source framework aligns each source user and the new user in individual feature spaces, which better transfers the knowledge of existing users to the new user. Moreover, we retain the source-combined data to preserve the effectiveness of SDA. The property was further confirmed by evaluating the performance of the proposed method on data from nine subjects performing six tasks.Main results.Experiment results prove that the proposed multi-source framework achieved both positive DG and DA performance in a cross-user classification manner.Significance.This work demonstrates the usability and feasibility of the proposed multi-source framework in cross-user myoelectric control.

Reduce the User Burden of Multiuser Myoelectric Interface via Few-Shot Domain Adaptation

Due to physiological and anatomical variations across users, myoelectric interfaces trained by multiple users cannot be adapted to the unique hand movement patterns of the new user. Most current work requires the new user to provide one or more trials per gesture (dozens to hundreds of samples), applying domain adaptation methods to calibrate the model and achieve promising movement recognition performance. However, the user burden associated with time-consuming electromyography signal acquisition and annotation is a key factor hindering the practical application of myoelectric control. As shown in this work, once the number of calibration samples is reduced, the performance of previous cross-user myoelectric interfaces will degrade due to the lack of enough statistics to characterize the distributions. In this paper, a few-shot supervised domain adaptation (FSSDA) framework is proposed to address this issue. It aligns the distributions of different domains by calculating the distribution distances of point-wise surrogates. Specifically, we introduce a positive-negative pair distance loss to find a shared embedding subspace where each scarce sample from the new user will be closer to the positive samples and away from the negative samples of multiple users. Thus, FSSDA allows every target domain sample to be paired with all source domain samples and optimizes the feature distance between each target domain sample and the source domain samples within the same batch, instead of direct estimation of the data distribution of the target domain. The proposed method is validated on two high-density EMG datasets, which achieves the averaged recognition accuracies of 97.59% and 82.78% with only 5 samples per gesture. In addition, FSSDA is also effective even when only one sample per gesture is provided. The experimental results show that FSSDA greatly reduces the user burden and further facilitates the development of myoelectric pattern recognition techniques.

Robust Long-Term Hand Grasp Recognition With Raw Electromyographic Signals Using Multidimensional Uncertainty-Aware Models

Hand grasp recognition with surface electromyography (sEMG) has been used as a possible natural strategy to control hand prosthetics. However, effectively performing activities of daily living for users relies significantly on the long-term robustness of such recognition, which is still a challenging task due to confused classes and several other variabilities. We hypothesise that this challenge can be addressed by introducing uncertainty-aware models because the rejection of uncertain movements has previously been demonstrated to improve the reliability of sEMG-based hand gesture recognition. With a particular focus on a very challenging benchmark dataset (NinaPro Database 6), we propose a novel end-to-end uncertainty-aware model, an evidential convolutional neural network (ECNN), which can generate multidimensional uncertainties, including vacuity and dissonance, for robust long-term hand grasp recognition. To avoid heuristically determining the optimal rejection threshold, we examine the performance of misclassification detection in the validation set. Extensive comparisons of accuracy under the non-rejection and rejection scheme are conducted when classifying 8 hand grasps (including rest) over 8 subjects across proposed models. The proposed ECNN is shown to improve recognition performance, achieving an accuracy of 51.44% without the rejection option and 83.51% under the rejection scheme with multidimensional uncertainties, significantly improving the current state-of-the-art (SoA) by 3.71% and 13.88%, respectively. Furthermore, its overall rejection-capable recognition accuracy remains stable with only a small accuracy degradation after the last data acquisition over 3 days. These results show the potential design of a reliable classifier that yields accurate and robust recognition performance.

Domain Adaptation with Self-Guided Adaptive Sampling Strategy: Feature Alignment for Cross-User Myoelectric Pattern Recognition

Gestural interfaces based on surface electromyo-graphic (sEMG) signal have been widely explored. Nevertheless, due to the individual differences in the sEMG signals, it is very challenging for a myoelectric pattern recognition control system to adapt cross-user variability. Unsupervised domain adaptation (UDA) has achieved unprecedented success in improving the cross-domain robustness, and it is a promising approach to solve the cross-user challenge. Existing UDA methods largely ignore the instantaneous data distribution during model updating, thus deteriorating the feature representation given a large domain shift. To address this issue, a novel method is proposed based on a UDA model incorporated with a self-guided adaptive sampling (SGAS) strategy. This strategy is designed to utilize the domain distance in a kernel space as an indicator to screen out reliable instantaneous samples for updating the classifier. Thus, it enables improved alignment of feature representations of myoelectric patterns across users. To evaluate the performance of the proposed method, sEMG data were recorded from forearm muscles of nine subjects performing six finger and wrist gestures. Experiment results show that the UDA method with the SGAS strategy achieved a mean accuracy of 90.41% ± 14.44% in a cross-user classification manner, outperformed the state-of-the-art methods with statistical significance. This study demonstrates the effectiveness of the proposed UDA framework and offers a novel tool for implementing cross-user myoelectric pattern recognition towards a multi-user and user-independent control.

User-Independent Hand Gesture Recognition Classification Models Using Sensor Fusion

Recently, it has been proven that targeting motor impairments as early as possible while using wearable mechatronic devices for assisted therapy can improve rehabilitation outcomes. However, despite the advanced progress on control methods for wearable mechatronic devices, the need for a more natural interface that allows for better control remains. To address this issue, electromyography (EMG)-based gesture recognition systems have been studied as a potential solution for human–machine interface applications. Recent studies have focused on developing user-independent gesture recognition interfaces to reduce calibration times for new users. Unfortunately, given the stochastic nature of EMG signals, the performance of these interfaces is negatively impacted. To address this issue, this work presents a user-independent gesture classification method based on a sensor fusion technique that combines EMG data and inertial measurement unit (IMU) data. The Myo Armband was used to measure muscle activity and motion data from healthy subjects. Participants were asked to perform seven types of gestures in four different arm positions while using the Myo on their dominant limb. Data obtained from 22 participants were used to classify the gestures using three different classification methods. Overall, average classification accuracies in the range of 67.5–84.6% were obtained, with the Adaptive Least-Squares Support Vector Machine model obtaining accuracies as high as 92.9%. These results suggest that by using the proposed sensor fusion approach, it is possible to achieve a more natural interface that allows better control of wearable mechatronic devices during robot assisted therapies.

Noninvasive Human-Prosthesis Interfaces for Locomotion Intent Recognition: A Review

The lower-limb robotic prostheses can provide assistance for amputees' daily activities by restoring the biomechanical functions of missing limb(s). To set proper control strategies and develop the corresponding controller for robotic prosthesis, a prosthesis user's intent must be acquired in time, which is still a major challenge and has attracted intensive attentions. This work focuses on the robotic prosthesis user's locomotion intent recognition based on the noninvasive sensing methods from the recognition task perspective (locomotion mode recognition, gait event detection, and continuous gait phase estimation) and reviews the state-of-the-art intent recognition techniques in a lower-limb prosthesis scope. The current research status, including recognition approach, progress, challenges, and future prospects in the human's intent recognition, has been reviewed. In particular for the recognition approach, the paper analyzes the recent studies and discusses the role of each element in locomotion intent recognition. This work summarizes the existing research results and problems and contributes a general framework for the intent recognition based on lower-limb prosthesis.

Multiuser gesture recognition using sEMG signals via canonical correlation analysis and optimal transport

Myoelectric interfaces have received much attention in the field of prosthesis control, neuro-rehabilitation systems and human-computer interaction. However, when different users perform the same gesture, the electromyography (EMG) signals can vary greatly. It is essential to design a multiuser myoelectric interface that can be simply used by novel users while maintaining good gesture classification performance. To cope with this problem, canonical correlation analysis (CCA) has been used to extract the inherent user-independent properties of EMG signals generated from the same gestures from multiple users and demonstrated superior performance. In this paper, we move forward to propose a novel framework based on CCA and optimal transport (OT), termed as CCA-OT. By optimal transport, the discrepancies in data distribution between the transformed feature matrix from the training and the testing sets can be further reduced. Experimental results on the defined 13 Chinese sign language gestures performed by 10 intact-limbed subjects demonstrated that the classification rate of our proposed CCA-OT framework is significantly higher than that of the CCA-only framework with an 8.49% promotion, which shows the necessity to reduce the drift in probability distribution functions (PDFs) of the different domains. The CCA-OT framework provides a promising method for the multiuser myoelectric interface which can be easily adapted to new users. This improvement will further facilitate the widespread implementation of myoelectric control systems using pattern recognition techniques.

Ultrasound Sensing Can Improve Continuous Classification of Discrete Ambulation Modes Compared to Surface Electromyography

Clinical translation of "intelligent" lower-limb assistive technologies relies on robust control interfaces capable of accurately detecting user intent. To date, mechanical sensors and surface electromyography (EMG) have been the primary sensing modalities used to classify ambulation. Ultrasound (US) imaging can be used to detect user-intent by characterizing structural changes of muscle. Our study evaluates wearable US imaging as a new sensing modality for continuous classification of five discrete ambulation modes: level, incline, decline, stair ascent, and stair descent ambulation, and benchmarks performance relative to EMG sensing. Ten able-bodied subjects were equipped with a wearable US scanner and eight unilateral EMG sensors. Time-intensity features were recorded from US images of three thigh muscles. Features from sliding windows of EMG signals were analyzed in two configurations: one including 5 EMG sensors on muscles around the thigh, and another with 3 additional sensors placed on the shank. Linear discriminate analysis was implemented to continuously classify these phase-dependent features of each sensing modality as one of five ambulation modes. US-based sensing statistically improved mean classification accuracy to 99.8% (99.5-100% CI) compared to 8-EMG sensors (85.8%; 84.0-87.6% CI) and 5-EMG sensors (75.3%; 74.5-76.1% CI). Further, separability analyses show the importance of superficial and deep US information for stair classification relative to other modes. These results are the first to demonstrate the ability of US-based sensing to classify discrete ambulation modes, highlighting the potential for improved assistive device control using less widespread, less superficial and higher resolution sensing of skeletal muscle.

Hand Gesture Recognition Using Compact CNN Via Surface Electromyography Signals

By training the deep neural network model, the hidden features in Surface Electromyography(sEMG) signals can be extracted. The motion intention of the human can be predicted by analysis of sEMG. However, the models recently proposed by researchers often have a large number of parameters. Therefore, we designed a compact Convolution Neural Network (CNN) model, which not only improves the classification accuracy but also reduces the number of parameters in the model. Our proposed model was validated on the Ninapro DB5 Dataset and the Myo Dataset. The classification accuracy of gesture recognition achieved good results.

A Subject-Transfer Framework Based on Single-Trial EMG Analysis Using Convolutional Neural Networks

In recent years, electromyography (EMG)-based practical myoelectric interfaces have been developed to improve the quality of daily life for people with physical disabilities. With these interfaces, it is very important to decode a user's movement intention, to properly control the external devices. However, improving the performance of these interfaces is difficult due to the high variations in the EMG signal patterns caused by intra-user variability. Therefore, this paper proposes a novel subject-transfer framework for decoding hand movements, which is robust in terms of intra-user variability. In the proposed framework, supportive convolutional neural network (CNN) classifiers, which are pre-trained using the EMG data of several subjects, are selected and fine-tuned for the target subject via single-trial analysis. Then, the target subject's hand movements are classified by voting the outputs of the supportive CNN classifiers. The feasibility of the proposed framework is validated with NinaPro databases 2 and 3, which comprise 49 hand movements of 40 healthy and 11 amputee subjects, respectively. The experimental results indicate that, when compared to the self-decoding framework, which uses only the target subject's data, the proposed framework can successfully decode hand movements with improved performance in both healthy and amputee subjects. From the experimental results, the proposed subject-transfer framework can be seen to represent a useful tool for EMG-based practical myoelectric interfaces controlling external devices.

Design of User-Independent Hand Gesture Recognition Using Multilayer Perceptron Networks and Sensor Fusion Techniques

According to the World Health Organization, stroke is the third leading cause of disability. A common consequence of stroke is hemiparesis, which leads to the impairment of one side of the body and affects the performance of activities of daily living. It has been proven that targeting the motor impairments as early as possible while using wearable mechatronic devices as a robot assisted therapy, and letting the patient be in control of the robotic system, can improve the rehabilitation outcomes. However, despite the increased progress on control methods for wearable mechatronic devices, a need for a more natural interface that allows for better control remains. In this work, a user-independent gesture classification method based on a sensor fusion technique using surface electromyography (EMG) and an inertial measurement unit (IMU) is presented. The Myo Armband was used to extract EMG and IMU data from healthy subjects. Participants were asked to perform 10 types of gestures in 4 different arm positions while using the Myo on their dominant limb. Data obtained from 14 participants were used to classify the gestures using a Multilayer Perceptron Network. Finally, the classification algorithm was tested on 5 novel users, obtaining an average accuracy of 78.94%. These results demonstrate that by using the proposed approach, it is possible to achieve a more natural human machine interface that allows better control of wearable mechatronic devices during robot assisted therapies.

Adaptive Hybrid Classifier for Myoelectric Pattern Recognition Against the Interferences of Outlier Motion, Muscle Fatigue, and Electrode Doffing

Traditional myoelectric prostheses that employ a static pattern recognition model to identify human movement intention from surface electromyography (sEMG) signals hardly adapt to the changes in the sEMG characteristics caused by interferences from daily activities, which hinders the clinical applications of such prostheses. In this paper, we focus on methods to reduce or eliminate the impacts of three types of daily interferences on myoelectric pattern recognition (MPR), i.e., outlier motion, muscle fatigue, and electrode doffing/donning. We constructed an adaptive incremental hybrid classifier (AIHC) by combining one-class support vector data description and multi-class linear discriminant analysis in conjunction with two specific update schemes. We developed an AIHC-based MPR strategy to improve the robustness of MPR against the three interferences. Extensive experiments on hand-motion recognition were conducted to demonstrate the performance of the proposed method. Experimental results show that the AIHC has significant advantages over non-adaptive classifiers under various interferences, with improvements in the classification accuracy ranging from 7.1% to 39% ( ). The additional evaluations on data deviations demonstrate that the AIHC can accommodate large-scale changes in the sEMG characteristics, revealing the potential of the AIHC-based MPR strategy in the development of clinical myoelectric prostheses.

Improving the functionality, robustness, and adaptability of myoelectric control for dexterous motion restoration

The development of advanced and effective human-machine interfaces, especially for amputees to control their prostheses, is very high priority and a very active area of research. An intuitive control method should retain an adequate level of functionality for dexterous operation, provide robustness against confounding factors, and supply adaptability for diverse long-term usage, all of which are current problems being tackled by researchers. This paper reviews the state-of-the-art, as well as, the limitations of current myoelectric signal control (MSC) methods. To address the research topic on functionality, we review different approaches to prosthetic hand control (DOF configuration, discrete or simultaneous, etc.), and how well the control is performed (accuracy, response, intuitiveness, etc.). To address the research on robustness, we review the confounding factors (limb positions, electrode shift, force variance, and inadvertent activity) that affect the stability of the control performance. Lastly, to address adaptability, we review the strategies that can automatically adjust the classifier for different individuals and for long-term usage. This review provides a thorough overview of the current MSC methods and helps highlight the current areas of research focus and resulting clinic usability for the MSC methods for upper-limb prostheses.

Effects of Confidence-Based Rejection on Usability and Error in Pattern Recognition-Based Myoelectric Control

Rejection of movements based on the confidence in the classification decision has previously been demonstrated to improve the usability of pattern recognition based myoelectric control. To this point, however, the optimal rejection threshold has been determined heuristically, and it is not known how different thresholds affect the tradeoff between error mitigation and false rejections in real-time closed-loop control. To answer this question, 24 able-bodied subjects completed a real-time Fitts' law-style virtual cursor control task using a support vector machine classifier. It was found that rejection improved information throughput at all thresholds, with the best performance coming at thresholds between 0.60 and 0.75. Two fundamental types of error were defined and identified: operator error (identifiable, repeatable behaviors, directly attributable to the user), and systemic error (other errors attributable to misclassification or noise). The incidence of both operator and systemic errors were found to decrease as rejection threshold increased. Moreover, while the incidence of all error types correlated strongly with path efficiency, only systemic errors correlated strongly with throughput and trial completion rate. Interestingly, more experienced users were found to commit as many errors as novice users, despite performing better in the Fitts' task, suggesting that there is more to usability than error prevention alone. Nevertheless, these results demonstrate the usability gains possible with rejection across a range of thresholds for both novice and experienced users alike.

Muscle Synergy Analysis of a Hand-Grasp Dataset: A Limited Subset of Motor Modules May Underlie a Large Variety of Grasps

Background: Kinematic and muscle patterns underlying hand grasps have been widely investigated in the literature. However, the identification of a reduced set of motor modules, generalizing across subjects and grasps, may be valuable for increasing the knowledge of hand motor control, and provide methods to be exploited in prosthesis control and hand rehabilitation.

Methods: Motor muscle synergies were extracted from a publicly available database including 28 subjects, executing 20 hand grasps selected for daily-life activities. The spatial synergies and temporal components were analyzed with a clustering algorithm to characterize the patterns underlying hand-grasps.

Results: Motor synergies were successfully extracted on all 28 subjects. Clustering orders ranging from 2 to 50 were tested. A subset of ten clusters, each one represented by a spatial motor module, approximates the original dataset with a mean maximum error of 5% on reconstructed modules; however, each spatial synergy might be employed with different timing and recruited at different grasp stages. Two temporal activation patterns are often recognized, corresponding to the grasp/hold phase, and to the pre-shaping and release phase.

Conclusions: This paper presents one of the biggest analysis of muscle synergies of hand grasps currently available. The results of 28 subjects performing 20 different grasps suggest that a limited number of time dependent motor modules (shared among subjects), correctly elicited by a control activation signal, may underlie the execution of a large variety of hand grasps. However, spatial synergies are not strongly related to specific motor functions but may be recruited at different stages, depending on subject and grasp. This result can lead to applications in rehabilitation and assistive robotics.

Causes of Performance Degradation in Non-invasive Electromyographic Pattern Recognition in Upper Limb Prostheses

Surface Electromyography (EMG)-based pattern recognition methods have been investigated over the past years as a means of controlling upper limb prostheses. Despite the very good reported performance of myoelectric controlled prosthetic hands in lab conditions, real-time performance in everyday life conditions is not as robust and reliable, explaining the limited clinical use of pattern recognition control. The main reason behind the instability of myoelectric pattern recognition control is that EMG signals are non-stationary in real-life environments and present a lot of variability over time and across subjects, hence affecting the system's performance. This can be the result of one or many combined changes, such as muscle fatigue, electrode displacement, difference in arm posture, user adaptation on the device over time and inter-subject singularity. In this paper an extensive literature review is performed to present the causes of the drift of EMG signals, ways of detecting them and possible techniques to counteract for their effects in the application of upper limb prostheses. The suggested techniques are organized in a table that can be used to recognize possible problems in the clinical application of EMG-based pattern recognition methods for upper limb prosthesis applications and state-of-the-art methods to deal with such problems.

Progress in Neuroengineering for brain repair: New challenges and open issues

Background

In recent years, biomedical devices have proven to be able to target also different neurological disorders. Given the rapid ageing of the population and the increase of invalidating diseases affecting the central nervous system, there is a growing demand for biomedical devices of immediate clinical use. However, to reach useful therapeutic results, these tools need a multidisciplinary approach and a continuous dialogue between neuroscience and engineering, a field that is named neuroengineering. This is because it is fundamental to understand how to read and perturb the neural code in order to produce a significant clinical outcome.

Results

In this review, we first highlight the importance of developing novel neurotechnological devices for brain repair and the major challenges expected in the next years. We describe the different types of brain repair strategies being developed in basic and clinical research and provide a brief overview of recent advances in artificial intelligence that have the potential to improve the devices themselves. We conclude by providing our perspective on their implementation to humans and the ethical issues that can arise.

Conclusions

Neuroengineering approaches promise to be at the core of future developments for clinical applications in brain repair, where the boundary between biology and artificial intelligence will become increasingly less pronounced.

Adaptive learning to speed-up control of prosthetic hands: A few things everybody should know

Domain adaptation methods have been proposed to reduce the training efforts needed to control an upper-limb prosthesis by adapting well performing models from previous subjects to the new subject. These studies generally reported impressive reductions in the required number of training samples to achieve a certain level of accuracy for intact subjects. We further investigate two popular methods in this field to verify whether this result also applies to amputees. Our findings show instead that this improvement can largely be attributed to a suboptimal hyperparameter configuration. When hyperparameters are appropriately tuned, the standard approach that does not exploit prior information performs on par with the more complicated transfer learning algorithms. Additionally, earlier studies erroneously assumed that the number of training samples relates proportionally to the efforts required from the subject. However, a repetition of a movement is the atomic unit for subjects and the total number of repetitions should therefore be used as reliable measure for training efforts. Also when correcting for this mistake, we do not find any performance increase due to the use of prior models.

A Novel Unsupervised Adaptive Learning Method for Long-Term Electromyography (EMG) Pattern Recognition

Performance degradation will be caused by a variety of interfering factors for pattern recognition-based myoelectric control methods in the long term. This paper proposes an adaptive learning method with low computational cost to mitigate the effect in unsupervised adaptive learning scenarios. We presents a particle adaptive classifier (PAC), by constructing a particle adaptive learning strategy and universal incremental least square support vector classifier (LS-SVC). We compared PAC performance with incremental support vector classifier (ISVC) and non-adapting SVC (NSVC) in a long-term pattern recognition task in both unsupervised and supervised adaptive learning scenarios. Retraining time cost and recognition accuracy were compared by validating the classification performance on both simulated and realistic long-term EMG data. The classification results of realistic long-term EMG data showed that the PAC significantly decreased the performance degradation in unsupervised adaptive learning scenarios compared with NSVC (9.03% ± 2.23%, p < 0.05) and ISVC (13.38% ± 2.62%, p = 0.001), and reduced the retraining time cost compared with ISVC (2 ms per updating cycle vs. 50 ms per updating cycle).

Exploiting Knowledge Composition to Improve Real-Life Hand Prosthetic Control

In myoelectric prosthesis control, one of the hottest topics nowadays is enforcing simultaneous and proportional (s/p) control over several degrees of freedom. This problem is particularly hard and the scientific community has so far failed to provide a stable and reliable s/p control, effective in daily-life activities. In order to improve the reliability of this form of control, in this paper we propose on-the-fly knowledge composition, thereby reducing the burden of matching several patterns at the same time, and simplifying the task of the system. In particular, we show that using our method it is possible to dynamically compose a model by juxtaposing subsets of previously gathered (sample, target) pairs in real-time, rather than composing a single model in the beginning and then hoping it can reliably distinguish all patterns. Fourteen intact subjects participated in an experiment, where repetitive daily-life tasks (e.g. ironing a cloth) were performed using a commercially available dexterous prosthetic hand mounted on a splint and wirelessly controlled using a machine learning method. During the experiment, the subjects performed these tasks using myocontrol with and without knowledge composition and the results demonstrate that employing knowledge composition allowed better performance, i.e. reducing the overall task completion time by 30%.

Application of real-time machine learning to myoelectric prosthesis control: A case series in adaptive switching

BACKGROUND

Myoelectric prostheses currently used by amputees can be difficult to control. Machine learning, and in particular learned predictions about user intent, could help to reduce the time and cognitive load required by amputees while operating their prosthetic device.

OBJECTIVES

The goal of this study was to compare two switching-based methods of controlling a myoelectric arm: non-adaptive (or conventional) control and adaptive control (involving real-time prediction learning).

STUDY DESIGN

Case series study.

METHODS

We compared non-adaptive and adaptive control in two different experiments. In the first, one amputee and one non-amputee subject controlled a robotic arm to perform a simple task; in the second, three able-bodied subjects controlled a robotic arm to perform a more complex task. For both tasks, we calculated the mean time and total number of switches between robotic arm functions over three trials.

RESULTS

Adaptive control significantly decreased the number of switches and total switching time for both tasks compared with the conventional control method.

CONCLUSION

Real-time prediction learning was successfully used to improve the control interface of a myoelectric robotic arm during uninterrupted use by an amputee subject and able-bodied subjects.

CLINICAL RELEVANCE

Adaptive control using real-time prediction learning has the potential to help decrease both the time and the cognitive load required by amputees in real-world functional situations when using myoelectric prostheses.

More with less: A comparative kinematical analysis of Django Reinhardt's adaptations to hand injury

BACKGROUND

At the age of 18 years, jazz guitarist Django Reinhardt (1910-1953) sustained significant burns to his left-hand ring and little fingers; yet, subsequently, he relearned to play and achieved international fame, despite his injuries.

CASE DESCRIPTION AND METHODS

Archive film footage and novel motion analysis software were used to compare movements of Django's fretting hand with that of six other guitarists of the same genre.

FINDINGS AND OUTCOMES

Django employed greater abduction of index and middle fingers (-9.11 ± 6.52° vs -5.78 ± 2.41°; p < 0.001) and more parallel alignment of fingers to the guitar neck (157.7 ± 3.37° vs 150.59 ± 2.67°; p < 0.001) compared to controls.

CONCLUSION

In response to debilitating hand injury, Django developed quantifiable compensatory adaptation of function of his remaining functional fingers by developing an original playing technique.

CLINICAL RELEVANCE

Hand function following injury may be optimized by maximizing latent degrees of freedom in remaining digits, rather than through extensive surgical reconstruction or complex prostheses. Further study of adaptation strategies may inform prosthesis design.

Towards Zero Retraining for Myoelectric Control Based on Common Model Component Analysis

In spite of several decades of intense research and development, the existing algorithms of myoelectric pattern recognition (MPR) are yet to satisfy the criteria that a practical upper extremity prostheses should fulfill. This study focuses on the criterion of the short, or even zero subject training. Due to the inherent nonstationarity in surface electromyography (sEMG) signals, current myoelectric control algorithms usually need to be retrained daily during a multiple days' usage. This study was conducted based on the hypothesis that there exist some invariant characteristics in the sEMG signals when a subject performs the same motion in different days. Therefore, given a set of classifiers (models) trained on several days, it is possible to find common characteristics among them. To this end, we proposed to use common model component analysis (CMCA) framework, in which an optimized projection was found to minimize the dissimilarity among multiple models of linear discriminant analysis (LDA) trained using data from different days. Five intact-limbed subjects and two transradial amputee subjects participated in an experiment including six sessions of sEMG data recording, which were performed in six different days, to simulate the application of MPR over multiple days. The results demonstrate that CMCA has a significant better generalization ability with unseen data (not included in the training data), leading to classification accuracy improvement and increase of completion rate in a motion test simulation, when comparing with the baseline reference method. The results indicate that CMCA holds a great potential in the effort of developing zero retraining of MPR.

Adaptive myoelectric pattern recognition toward improved multifunctional prosthesis control

The non-stationary property of electromyography (EMG) signals in real life settings usually hinders the clinical application of the myoelectric pattern recognition for prosthesis control. The classical EMG pattern recognition approach consists of two separate steps: training and testing, without considering the changes between training and testing data induced by electrode shift, fatigue, impedance changes and psychological factors, and often results in performance degradation. The aim of this study was to develop an adaptive myoelectric pattern recognition system, aiming to retrain the classifier online with the testing data without supervision, providing a self-correction mechanism for suppressing misclassifications. This paper presents an adaptive unsupervised classifier based on support vector machine (SVM) to improve the classification performance. Experimental data from 15 healthy subjects were used to evaluate performance. Preliminary study on intra-session and inter-session EMG data was conducted to verify the performance of the unsupervised adaptive SVM classifier. The unsupervised adaptive SVM classifier outperformed the conventional SVM by 3.3% and 8.0% for the combination of time-domain and autoregressive features in the intra-session and inter-session tests, respectively. The proposed approach is capable of incorporating the useful information in testing data to the classification model by taking into account the overtime changes in the testing data with respect to the training data to retrain the original classifier, therefore providing a self-correction mechanism for suppressing misclassifications.

A real-time, practical sensor fault-tolerant module for robust EMG pattern recognition

BACKGROUND

Unreliability of surface EMG recordings over time is a challenge for applying the EMG pattern recognition (PR)-controlled prostheses in clinical practice. Our previous study proposed a sensor fault-tolerant module (SFTM) by utilizing redundant information in multiple EMG signals. The SFTM consists of multiple sensor fault detectors and a self-recovery mechanism that can identify anomaly in EMG signals and remove the recordings of the disturbed signals from the input of the pattern classifier to recover the PR performance. While the proposed SFTM has shown great promise, the previous design is impractical. A practical SFTM has to be fast enough, lightweight, automatic, and robust under different conditions with or without disturbances.

METHODS

This paper presented a real-time, practical SFTM towards robust EMG PR. A novel fast LDA retraining algorithm and a fully automatic sensor fault detector based on outlier detection were developed, which allowed the SFTM to promptly detect disturbances and recover the PR performance immediately. These components of SFTM were then integrated with the EMG PR module and tested on five able-bodied subjects and a transradial amputee in real-time for classifying multiple hand and wrist motions under different conditions with different disturbance types and levels.

RESULTS

The proposed fast LDA retraining algorithm significantly shortened the retraining time from nearly 1 s to less than 4 ms when tested on the embedded system prototype, which demonstrated the feasibility of a nearly "zero-delay" SFTM that is imperceptible to the users. The results of the real-time tests suggested that the SFTM was able to handle different types of disturbances investigated in this study and significantly improve the classification performance when one or multiple EMG signals were disturbed. In addition, the SFTM could also maintain the system's classification performance when there was no disturbance.

CONCLUSIONS

This paper presented a real-time, lightweight, and automatic SFTM, which paved the way for reliable and robust EMG PR for prosthesis control.

Reduced Daily Recalibration of Myoelectric Prosthesis Classifiers Based on Domain Adaptation

Control scheme design based on surface electromyography (sEMG) pattern recognition has been the focus of much research on a myoelectric prosthesis (MP) technology. Due to inherent nonstationarity in sEMG signals, prosthesis systems may need to be recalibrated day after day in daily use applications; thereby, hindering MP usability. In order to reduce the recalibration time in the subsequent days following the initial training, we propose a domain adaptation (DA) framework, which automatically reuses the models trained in earlier days as input for two baseline classifiers: a polynomial classifier (PC) and a linear discriminant analysis (LDA). Two novel algorithms of DA are introduced, one for PC and the other one for LDA. Five intact-limbed subjects and two transradial-amputee subjects participated in an experiment lasting ten days, to simulate the application of a MP over multiple days. The experiment results of four methods were compared: PC-DA (PC with DA), PC-BL (baseline PC), LDA-DA (LDA with DA), and LDA-BL (baseline LDA). In a new day, the DA methods reuse nine pretrained models, which were calibrated by 40 s training data per class in nine previous days. We show that the proposed DA methods significantly outperform nonadaptive baseline methods. The improvement in classification accuracy ranges from 5.49% to 28.48%, when the recording time per class is 2 s. For example, the average classification rates of PC-BL and PC-DA are 83.70% and 92.99%, respectively, for intact-limbed subjects with a nine-motions classification task. These results indicate that DA has the potential to improve the usability of MPs based on pattern recognition, by reducing the calibration time.

Toward real-time automated detection of turns during gait using wearable inertial measurement units

Previous studies have presented algorithms for detection of turns during gait using wearable sensors, but those algorithms were not built for real-time use. This paper therefore investigates the optimal approach for real-time detection of planned turns during gait using wearable inertial measurement units. Several different sensor positions (head, back and legs) and three different detection criteria (orientation, angular velocity and both) are compared with regard to their ability to correctly detect turn onset. Furthermore, the different sensor positions are compared with regard to their ability to predict the turn direction and amplitude. The evaluation was performed on ten healthy subjects who performed left/right turns at three amplitudes (22, 45 and 90 degrees). Results showed that turn onset can be most accurately detected with sensors on the back and using a combination of orientation and angular velocity. The same setup also gives the best prediction of turn direction and amplitude. Preliminary measurements with a single amputee were also performed and highlighted important differences such as slower turning that need to be taken into account.

Self-correcting pattern recognition system of surface EMG signals for upper limb prosthesis control

Pattern recognition methods for classifying user motion intent based on surface electromyography developed by research groups in well-controlled laboratory conditions are not yet clinically viable for upper limb prosthesis control, due to their limited robustness in users' real-life situations. To address this problem, a novel postprocessing algorithm, aiming to detect and remove misclassifications of a pattern recognition system of forearm and hand motions, is proposed. Using the maximum likelihood calculated by a classifier and the mean global muscle activity of the forearm, an artificial neural network was trained to detect potentially erroneous classification decisions. This system was compared to four previously proposed classification postprocessing methods, in both able-bodied and amputee subjects. Various nonstationarities were included in the experimental protocol to account for challenges posed in real-life settings, such as different contraction levels, static and dynamic motion phases, and effects induced by day-to-day transfers, such as electrode shifts, impedance changes, and psychometric user variability. The improvement in classification accuracy with respect to the unprocessed classifier ranged from 4.8% to 31.6%, depending on the scenarios investigated. The system significantly reduced misclassifications to wrong active classes and is thus a promising approach for improving the robustness of hand prosthesis controllability.

Proceedings of the first workshop on Peripheral Machine Interfaces: going beyond traditional surface electromyography

One of the hottest topics in rehabilitation robotics is that of proper control of prosthetic devices. Despite decades of research, the state of the art is dramatically behind the expectations. To shed light on this issue, in June, 2013 the first international workshop on Present and future of non-invasive peripheral nervous system (PNS)-Machine Interfaces (MI; PMI) was convened, hosted by the International Conference on Rehabilitation Robotics. The keyword PMI has been selected to denote human-machine interfaces targeted at the limb-deficient, mainly upper-limb amputees, dealing with signals gathered from the PNS in a non-invasive way, that is, from the surface of the residuum. The workshop was intended to provide an overview of the state of the art and future perspectives of such interfaces; this paper represents is a collection of opinions expressed by each and every researcher/group involved in it.

Support vector regression for improved real-time, simultaneous myoelectric control

This study describes the first application of a support vector machine (SVM) based scheme for real-time simultaneous and proportional myoelectric control of multiple degrees of freedom (DOFs). Three DOFs including wrist flexion-extension, abduction-adduction and forearm pronation-supination were investigated with 10 able-bodied subjects and two individuals with transradial limb deficiency (LD). A Fitts' law test involving real-time target acquisition tasks was conducted to compare the usability of the SVM-based control system to that of an artificial neural network (ANN) based method. Performance was assessed using the Fitts' law throughput value as well as additional metrics including completion rate, path efficiency and overshoot. The SVM-based approach outperformed the ANN-based system in every performance measure for able-bodied subjects. The SVM outperformed the ANN in path efficiency and throughput with the first LD subject and in throughput with the second LD subject. The superior performance of the SVM-based system appears to be due to its higher estimation accuracy of all DOFs during inactive and low amplitude segments (these periods were frequent during real-time control). Another advantage of the SVM-based method was that it substantially reduced the processing time for both training and real time control.

Correlation analysis of electromyogram signals for multiuser myoelectric interfaces

An inability to adapt myoelectric interfaces to a novel user's unique style of hand motion, or even to adapt to the motion style of an opposite limb upon which the interface is trained, are important factors inhibiting the practical application of myoelectric interfaces. This is mainly attributed to the individual differences in the exhibited electromyogram (EMG) signals generated by the muscles of different limbs. We propose in this paper a multiuser myoelectric interface which easily adapts to novel users and maintains good movement recognition performance. The main contribution is a framework for implementing style-independent feature transformation by using canonical correlation analysis (CCA) in which different users' data is projected onto a unified-style space. The proposed idea is summarized into three steps: 1) train a myoelectric pattern classifier on the set of style-independent features extracted from multiple users using the proposed CCA-based mapping; 2) create a new set of features describing the movements of a novel user during a quick calibration session; and 3) project the novel user's features onto a lower dimensional unified-style space with features maximally correlated with training data and classify accordingly. The proposed method has been validated on a set of eight intact-limbed subjects, left-and-right handed, performing ten classes of bilateral synchronous fingers movements with four electrodes on each forearm. The method was able to overcome individual differences through the style-independent framework with accuracies of > 83% across multiple users. Testing was also performed on a set of ten intact-limbed and six below-elbow amputee subjects as they performed finger and thumb movements. The proposed framework allowed us to train the classifier on a normal subject's data while subsequently testing it on an amputee's data after calibration with a performance of > 82% on average across all amputees.

Evidence of muscle synergies during human grasping

Motor synergies have been investigated since the 1980s as a simplifying representation of motor control by the nervous system. This way of representing finger positional data is in particular useful to represent the kinematics of the human hand. Whereas, so far, the focus has been on kinematic synergies, that is common patterns in the motion of the hand and fingers, we hereby also investigate their force aspects, evaluated through surface electromyography (sEMG). We especially show that force-related motor synergies exist, i.e. that muscle activation during grasping, as described by the sEMG signal, can be grouped synergistically; that these synergies are largely comparable to one another across human subjects notwithstanding the disturbances and inaccuracies typical of sEMG; and that they are physiologically feasible representations of muscular activity during grasping. Potential applications of this work include force control of mechanical hands, especially when many degrees of freedom must be simultaneously controlled.