A Multi-Variate Approach to Predicting Myoelectric Control Usability

EMG-based wake gestures eliminate false activations during out-of-set activities of daily living: an online myoelectric control study

Objective.While myoelectric control has been commercialized in prosthetics for decades, its adoption for more general human-machine interaction has been slow. Although high accuracies can be achieved across many gestures, current control approaches are prone to false activations in real-world conditions. This is because the same electromyogram (EMG) signals generated during the elicitation of gestures are also naturally activated when performing activities of daily living (ADLs), such as when driving to work or while typing on a keyboard. This can lead the myoelectric control system, which is trained on a closed set of gestures and thus unaware of the muscle activity associated with these ADLs, to be falsely activated, leading to erroneous inputs and user frustration.Approach.To overcome this problem, the concept of wake gestures, whereby users could switch between a dedicated control mode and a sleep mode by snapping their fingers, was explored. Using a simple dynamic time warping model, the real-world user-in-the-loop efficacy of wake gestures as a toggle for myoelectric interfaces was demonstrated through two online ubiquitous control tasks with varying levels of difficulty: (1) dismissing an alarm and (2) controlling a robot.Main results.During these online evaluations, the designed system ignored almost all (>99.9%) non-target EMG activity generated during a set of ADLs (i.e. walking, typing, writing, phone use, and driving), ignored all control gestures (i.e. wrist flexion, wrist extension, hand open, and hand close), and enabled reliable mode switching during intentional wake gesture elicitation. Additionally, questionnaires revealed that participants responded well to the use of wake gestures and generally preferred false negatives over false positives, providing valuable insights into the future design of these systems.Significance.These results highlight the real-world viability of wake gestures for enabling the intermittent use of myoelectric control, opening up new interaction possibilities for EMG-based inputs.

Screen Guided Training Does Not Capture Goal-Oriented Behaviours: Learning Myoelectric Control Mappings From Scratch Using Context Informed Incremental Learning

Human-machine interfaces based on myoelectric signals typically use screen-guided training (SGT) for model calibration, but this approach fails to capture realistic user behaviors. This study evaluates a user-in-the-loop context-informed incremental learning (CIIL) framework, comparing SGT, SGT followed by CIIL adaptation (SGT-A), and a novel zero-shot adaptation (ZS-A) CIIL approach that begins adapting with no prior training. Sixteen participants completed a Fitts' Law targeting task using these control schemes, with performance measured via online throughput and offline classification accuracy. Despite lower offline accuracy, the ZS-A model achieved the highest online throughput (1.47 ± 0.46 bits/s), significantly outperforming the SGT baseline (1.15 ± 0.37 bits/s) and reached competitive performance within 200 seconds. To further enhance control performance, a novel adaptive sigmoid-based proportional control mapping was introduced, dynamically adjusting control signals to allow precise control near neutral positions and rapid movements at higher activation levels, better aligning with natural user behaviors. These findings demonstrate that CIIL can surpass traditional SGT methods in online performance and emphasize the value of real-time user-in-the-loop data for developing adaptable and intuitive myoelectric interfaces, with implications for prosthetics, rehabilitation, and telerobotics.

Big data in myoelectric control: large multi-user models enable robust zero-shot EMG-based discrete gesture recognition

Myoelectric control, the use of electromyogram (EMG) signals generated during muscle contractions to control a system or device, is a promising input, enabling always-available control for emerging ubiquitous computing applications. However, its widespread use has historically been limited by the need for user-specific machine learning models because of behavioural and physiological differences between users. Leveraging the publicly available 612-user EMG-EPN612 dataset, this work dispels this notion, showing that true zero-shot cross-user myoelectric control is achievable without user-specific training. By taking a discrete approach to classification (i.e., recognizing the entire dynamic gesture as a single event), a classification accuracy of 93.0% for six gestures was achieved on a set of 306 unseen users, showing that big data approaches can enable robust cross-user myoelectric control. By organizing the results into a series of mini-studies, this work provides an in-depth analysis of discrete cross-user models to answer unknown questions and uncover new research directions. In particular, this work explores the number of participants required to build cross-user models, the impact of transfer learning for fine-tuning these models, and the effects of under-represented end-user demographics in the training data, among other issues. Additionally, in order to further evaluate the performance of the developed cross-user models, a completely new dataset was created (using the same recording device) that includes known covariate factors such as cross-day use and limb-position variability. The results show that the large data models can effectively generalize to new datasets and mitigate the impact of common confounding factors that have historically limited the adoption of EMG-based inputs.

Comparing online wrist and forearm EMG-based control using a rhythm game-inspired evaluation environment

Objective.The use of electromyogram (EMG) signals recorded from the wrist is emerging as a desirable input modality for human-machine interaction (HMI). Although forearm-based EMG has been used for decades in prosthetics, there has been comparatively little prior work evaluating the performance of wrist-based control, especially in online, user-in-the-loop studies. Furthermore, despite different motivating use cases for wrist-based control, research has mostly adopted legacy prosthesis control evaluation frameworks.Approach.Gaining inspiration from rhythm games and the Schmidt's law speed-accuracy tradeoff, this work proposes a new temporally constrained evaluation environment with a linearly increasing difficulty to compare the online usability of wrist and forearm EMG. Compared to the more commonly used Fitts' Law-style testing, the proposed environment may offer different insights for emerging use cases of EMG as it decouples the machine learning algorithm's performance from proportional control, is easily generalizable to different gesture sets, and enables the extraction of a wide set of usability metrics that describe a users ability to successfully accomplish a task at a certain time with different levels of induced stress.Main results.The results suggest that wrist EMG-based control is comparable to that of forearm EMG when using traditional prosthesis control gestures and can even be better when using fine finger gestures. Additionally, the results suggest that as the difficulty of the environment increased, the online metrics and their correlation to the offline metrics decreased, highlighting the importance of evaluating myoelectric control in real-time evaluations over a range of difficulties.Significance.This work provides valuable insights into the future design and evaluation of myoelectric control systems for emerging HMI applications.

Unravelling Influence Factors in Pattern Recognition Myoelectric Control Systems: The Impact of Limb Positions and Electrode Shifts

Pattern recognition (PR)-based myoelectric control systems can naturally provide multifunctional and intuitive control of upper limb prostheses and restore lost limb function, but understanding their robustness remains an open scientific question. This study investigates how limb positions and electrode shifts-two factors that have been suggested to cause classification deterioration-affect classifiers' performance by quantifying changes in the class distribution using each factor as a class and computing the repeatability and modified separability indices. Ten intact-limb participants took part in the study. Linear discriminant analysis (LDA) was used as the classifier. The results confirmed previous studies that limb positions and electrode shifts deteriorate classification performance (14-21% decrease) with no difference between factors (p > 0.05). When considering limb positions and electrode shifts as classes, we could classify them with an accuracy of 96.13 ± 1.44% and 65.40 ± 8.23% for single and all motions, respectively. Testing on five amputees corroborated the above findings. We have demonstrated that each factor introduces changes in the feature space that are statistically new class instances. Thus, the feature space contains two statistically classifiable clusters when the same motion is collected in two different limb positions or electrode shifts. Our results are a step forward in understanding PR schemes' challenges for myoelectric control of prostheses and further validation needs be conducted on more amputee-related datasets.

BioPoint: Enhancing Human-Computer Interaction through Single-Site, Multi-Sensor Gesture Recognition

Advancements in human-computer interaction (HCI) and machine learning are seen as key avenues to help individuals living with upper limb disabilities in accomplishing their activities of daily living. Multi-channel myoelectric systems are a promising approach for HCI due to their intuitive and accurate capture of user intent through muscle activity. However, such systems are still bulky compared to widely accepted smartwatches-like devices and as such pose a challenge for seamless integration into daily life. Consequently, this work presents the first single-site EMG-IMU-PPG system for multi-gesture recognition from the wrist. Overall, when evaluated on eight able-bodied participant, the system achieves an average accuracy of 65.28%±10.45% on 17 finger and wrist gestures and 90.98%±5.65% on eight hand gestures. This study also introduces "Myo Hero", an online environment inspired by "Guitar Hero." Myo Hero provides a dynamic method for testing multi-gesture recognition systems, improving the evaluation process's appeal. Online evaluation of the multimodal system substantially outperformed random play, signifying EMG-IMU-PPG's potential for human-computer interaction.

Understanding the influence of confounding factors in myoelectric control for discrete gesture recognition

Discrete myoelectric control-based gesture recognition has recently gained interest as a possible input modality for many emerging ubiquitous computing applications. Unlike the continuous control commonly employed in powered prostheses, discrete systems seek to recognize the dynamic sequences associated with gestures to generate event-based inputs. More akin to those used in general-purpose human-computer interaction, these could include, for example, a flick of the wrist to dismiss a phone call or a double tap of the index finger and thumb to silence an alarm. Moelectric control systems have been shown to achieve near-perfect classification accuracy, but in highly constrained offline settings. Real-world, online systems are subject to 'confounding factors' (i.e. factors that hinder the real-world robustness of myoelectric control that are not accounted for during typical offline analyses), which inevitably degrade system performance, limiting their practical use. Although these factors have been widely studied in continuous prosthesis control, there has been little exploration of their impacts on discrete myoelectric control systems for emerging applications and use cases. Correspondingly, this work examines, for the first time, three confounding factors and their effect on the robustness of discrete myoelectric control: (1)limb position variability, (2)cross-day use, and a newly identified confound faced by discrete systems (3)gesture elicitation speed. Results from four different discrete myoelectric control architectures: (1) Majority Vote LDA, (2) Dynamic Time Warping, (3) an LSTM network trained with Cross Entropy, and (4) an LSTM network trained with Contrastive Learning, show that classification accuracy is significantly degraded (p<0.05) as a result of each of these confounds. This work establishes that confounding factors are a critical barrier that must be addressed to enable the real-world adoption of discrete myoelectric control for robust and reliable gesture recognition.

Context-informed incremental learning improves both the performance and resilience of myoelectric control

Despite its rich history of success in controlling powered prostheses and emerging commercial interests in ubiquitous computing, myoelectric control continues to suffer from a lack of robustness. In particular, EMG-based systems often degrade over prolonged use resulting in tedious recalibration sessions, user frustration, and device abandonment. Unsupervised adaptation is one proposed solution that updates a model's parameters over time based on its own predictions during real-time use to maintain robustness without requiring additional user input or dedicated recalibration. However, these strategies can actually accelerate performance deterioration when they begin to classify (and thus adapt) incorrectly, defeating their own purpose. To overcome these limitations, we propose a novel adaptive learning strategy, Context-Informed Incremental Learning (CIIL), that leverages in situ context to better inform the prediction of pseudo-labels. In this work, we evaluate these CIIL strategies in an online target acquisition task for two use cases: (1) when there is a lack of training data and (2) when a drastic and enduring alteration in the input space has occurred. A total of 32 participants were evaluated across the two experiments. The results show that the CIIL strategies significantly outperform the current state-of-the-art unsupervised high-confidence adaptation and outperform models trained with the conventional screen-guided training approach, even after a 45-degree electrode shift (p < 0.05). Consequently, CIIL has substantial implications for the future of myoelectric control, potentially reducing the training burden while bolstering model robustness, and leading to improved real-time control.

Current status and clinical perspectives of extended reality for myoelectric prostheses: review

Training with "Extended Reality" or X-Reality (XR) systems can undoubtedly enhance the control of the myoelectric prostheses. However, there is no consensus on which factors improve the efficiency of skill transfer from virtual training to actual prosthesis abilities. This review examines the current status and clinical applications of XR in the field of myoelectric prosthesis training and analyses possible influences on skill migration. We have conducted a thorough search on databases in the field of prostheses using keywords such as extended reality, virtual reality and serious gaming. Our scoping review encompassed relevant applications, control methods, performance evaluation and assessment metrics. Our findings indicate that the implementation of XR technology for myoelectric rehabilitative training on prostheses provides considerable benefits. Additionally, there are numerous standardised methods available for evaluating training effectiveness. Recently, there has been a surge in the number of XR-based training tools for myoelectric prostheses, with an emphasis on user engagement and virtual training evaluation. Insufficient attention has been paid to significant limitations in the behaviour, functionality, and usage patterns of XR and myoelectric prostheses, potentially obstructing the transfer of skills and prospects for clinical application. Improvements are recommended in four critical areas: activities of daily living, training strategies, feedback, and the alignment of the virtual environment with the physical devices.

Virtual regression-based myoelectric hand-wrist prosthesis control and electrode site selection using no force feedback

Most transradial prosthesis users with conventional "Sequential" myoelectric control have two electrode sites which control one degree of freedom (DoF) at a time. Rapid EMG co-activation toggles control between DoFs (e.g., hand and wrist), providing limited function. We implemented a regression-based EMG control method which achieved simultaneous and proportional control of two DoFs in a virtual task. We automated electrode site selection using a short-duration (90 s) calibration period, without force feedback. Backward stepwise selection located the best electrodes for either six or 12 electrodes (selected from a pool of 16). We additionally studied two, 2-DoF controllers: "Intuitive" control (hand open-close and wrist pronation-supination controlled virtual target size and rotation, respectively) and "Mapping" control (wrist flexion-extension and ulnar-radial deviation controlled virtual target left-right and up-down movement, respectively). In practice, a Mapping controller would be mapped to control prosthesis hand open-close and wrist pronation-supination. Eleven able-bodied subjects and 4 limb-absent subjects completed virtual target matching tasks (fixed target moves to a new location after being "matched," and subject immediately pursues) and fixed (static) target tasks. For all subjects, both 2-DoF controllers with 6 optimally-sited electrodes had statistically better target matching performance than Sequential control in number of matches (average of 4-7 vs. 2 matches, p< 0.001) and throughput (average of 0.75-1.25 vs. 0.4 bits/s, p< 0.001), but not overshoot rate and path efficiency. There were no statistical differences between 6 and 12 optimally-sited electrodes for both 2-DoF controllers. These results support the feasibility of 2-DoF simultaneous, proportional myoelectric control.

Application of CT Multimodal Images in Rehabilitation Monitoring of Long-Distance Running

In order to monitor the rehabilitation of athletes injured in long-distance running, the author proposes a method for rehabilitation monitoring of long-distance running based on CT multimodal images. This method combines the latest multimodal image technology, integrates multimodal technology into CT images to improve the accuracy, performs image segmentation on CT multimodal images through medical segmentation methods, and analyzes the segmented images; finally, it can achieve the effect of rehabilitation treatment for athletes in long-distance running. Experimental results show that the total time taken by the authors' method is 10.9 hours, with an average time of 8 seconds, which is much shorter than the other two control methods. In conclusion, the authors' method allows for better rehabilitation monitoring of long-distance running sports injuries.

Myoelectric Control Performance of Two Degree of Freedom Hand-Wrist Prosthesis by Able-Bodied and Limb-Absent Subjects

Recent research has advanced two degree-of-freedom (DoF), simultaneous, independent and proportional control of hand-wrist prostheses using surface electromyogram signals from remnant muscles as the control input. We evaluated two such regression-based controllers, along with conventional, sequential two-site control with co-contraction mode switching (SeqCon), in box-block, refined-clothespin and door-knob tasks, on 10 able-bodied and 4 limb-absent subjects. Subjects operated a commercial hand and wrist using a socket bypass harness. One 2-DoF controller (DirCon) related the intuitive hand actions of open-close and pronation-supination to the associated prosthesis hand-wrist actions, respectively. The other (MapCon) mapped myoelectrically more distinct, but less intuitive, actions of wrist flexion-extension and ulnar-radial deviation. Each 2-DoF controller was calibrated from separate 90 s calibration contractions. SeqCon performed better statistically than MapCon in the predominantly 1-DoF box-block task (>20 blocks/minute vs. 8-18 blocks/minute, on average). In this task, SeqCon likely benefited from an ability to easily focus on 1-DoF and not inadvertently trigger co-contraction for mode switching. The remaining two tasks require 2-DoFs, and both 2-DoF controllers each performed better (factor of 2-4) than SeqCon. We also compared the use of 12 vs. 6 optimally-selected EMG electrodes as inputs, finding no statistical difference. Overall, we provide further evidence of the benefits of regression-based EMG prosthesis control of 2-DoFs in the hand-wrist.

The Influence of Training with Visual Biofeedback on the Predictability of Myoelectric Control Usability

Studies have shown that closed-loop myoelectric control schemes can lead to changes in user performance and behavior compared to open-loop systems. When users are placed within the control loop, such as during real-time use, they must correct for errors made by the controller and learn what behavior is necessary to produce desired outcomes. Augmented feedback, consequently, has been used to incorporate the user throughout the training process and to facilitate learning. This work explores the effect of visual feedback presented during user training on both the performance and predictability of a myoelectric classification-based control system. Our results suggest that properly designed feedback mechanisms and training tasks can influence the quality of the training data and the ability to predict usability using linear combinations of metrics derived from feature space. Furthermore, our results confirm that the most common in-lab training protocol, screen guided training, may yield training data that are less representative of online use than training protocols that incorporate the user in the loop. These results suggest that training protocols should be designed that better parallel the testing environment to more effectively prepare both the algorithms and users for real-time control.