Virtual biomechanics: a new method for online reconstruction of force from EMG recordings

Intuitive movement-based prosthesis control enables arm amputees to reach naturally in virtual reality

Impressive progress is being made in bionic limbs design and control. Yet, controlling the numerous joints of a prosthetic arm necessary to place the hand at a correct position and orientation to grasp objects remains challenging. Here, we designed an intuitive, movement-based prosthesis control that leverages natural arm coordination to predict distal joints missing in people with transhumeral limb loss based on proximal residual limb motion and knowledge of the movement goal. This control was validated on 29 participants, including seven with above-elbow limb loss, who picked and placed bottles in a wide range of locations in virtual reality, with median success rates over 99% and movement times identical to those of natural movements. This control also enabled 15 participants, including three with limb differences, to reach and grasp real objects with a robotic arm operated according to the same principle. Remarkably, this was achieved without any prior training, indicating that this control is intuitive and instantaneously usable. It could be used for phantom limb pain management in virtual reality, or to augment the reaching capabilities of invasive neural interfaces usually more focused on hand and grasp control.

Deep learning and session-specific rapid recalibration for dynamic hand gesture recognition from EMG

We anticipate wide adoption of wrist and forearm electomyographic (EMG) interface devices worn daily by the same user. This presents unique challenges that are not yet well addressed in the EMG literature, such as adapting for session-specific differences while learning a longer-term model of the specific user. In this manuscript we present two contributions toward this goal. First, we present the MiSDIREKt (Multi-Session Dynamic Interaction Recordings of EMG and Kinematics) dataset acquired using a novel hardware design. A single participant performed four kinds of hand interaction tasks in virtual reality for 43 distinct sessions over 12 days, totaling 814 min. Second, we analyze this data using a non-linear encoder-decoder for dimensionality reduction in gesture classification. We find that an architecture which recalibrates with a small amount of single session data performs at an accuracy of 79.5% on that session, as opposed to architectures which learn solely from the single session (49.6%) or learn only from the training data (55.2%).

Model and experiments to optimize co-adaptation in a simplified myoelectric control system

OBJECTIVE

To compensate for a limb lost in an amputation, myoelectric prostheses use surface electromyography (EMG) from the remaining muscles to control the prosthesis. Despite considerable progress, myoelectric controls remain markedly different from the way we normally control movements, and require intense user adaptation. To overcome this, our goal is to explore concurrent machine co-adaptation techniques that are developed in the field of brain-machine interface, and that are beginning to be used in myoelectric controls.

APPROACH

We combined a simplified myoelectric control with a perturbation for which human adaptation is well characterized and modeled, in order to explore co-adaptation settings in a principled manner.

RESULTS

First, we reproduced results obtained in a classical visuomotor rotation paradigm in our simplified myoelectric context, where we rotate the muscle pulling vectors used to reconstruct wrist force from EMG. Then, a model of human adaptation in response to directional error was used to simulate various co-adaptation settings, where perturbations and machine co-adaptation are both applied on muscle pulling vectors. These simulations established that a relatively low gain of machine co-adaptation that minimizes final errors generates slow and incomplete adaptation, while higher gains increase adaptation rate but also errors by amplifying noise. After experimental verification on real subjects, we tested a variable gain that cumulates the advantages of both, and implemented it with directionally tuned neurons similar to those used to model human adaptation. This enables machine co-adaptation to locally improve myoelectric control, and to absorb more challenging perturbations.

SIGNIFICANCE

The simplified context used here enabled to explore co-adaptation settings in both simulations and experiments, and to raise important considerations such as the need for a variable gain encoded locally. The benefits and limits of extending this approach to more complex and functional myoelectric contexts are discussed.

Elucidating Sensorimotor Control Principles with Myoelectric Musculoskeletal Models

There is an old saying that you must walk a mile in someone's shoes to truly understand them. This mini-review will synthesize and discuss recent research that attempts to make humans "walk a mile" in an artificial musculoskeletal system to gain insight into the principles governing human movement control. In this approach, electromyography (EMG) is used to sample human motor commands; these commands serve as inputs to mathematical models of muscular dynamics, which in turn act on a model of skeletal dynamics to produce a simulated motor action in real-time (i.e., the model's state is updated fast enough produce smooth motion without noticeable transitions; Manal et al., 2002). In this mini-review, these are termed myoelectric musculoskeletal models (MMMs). After a brief overview of typical MMM design and operation principles, the review will highlight how MMMs have been used for understanding human sensorimotor control and learning by evoking apparent alterations in a user's biomechanics, neural control, and sensory feedback experiences.

Performance and Usability of Various Robotic Arm Control Modes from Human Force Signals

Elaborating an efficient and usable mapping between input commands and output movements is still a key challenge for the design of robotic arm prostheses. In order to address this issue, we present and compare three different control modes, by assessing them in terms of performance as well as general usability. Using an isometric force transducer as the command device, these modes convert the force input signal into either a position or a velocity vector, whose magnitude is linearly or quadratically related to force input magnitude. With the robotic arm from the open source 3D-printed Poppy Humanoid platform simulating a mobile prosthesis, an experiment was carried out with eighteen able-bodied subjects performing a 3-D target-reaching task using each of the three modes. The subjects were given questionnaires to evaluate the quality of their experience with each mode, providing an assessment of their global usability in the context of the task. According to performance metrics and questionnaire results, velocity control modes were found to perform better than position control mode in terms of accuracy and quality of control as well as user satisfaction and comfort. Subjects also seemed to favor quadratic velocity control over linear (proportional) velocity control, even if these two modes did not clearly distinguish from one another when it comes to performance and usability assessment. These results highlight the need to take into account user experience as one of the key criteria for the design of control modes intended to operate limb prostheses.

Toward EMG-controlled force field generation for training and rehabilitation: From movement data to muscle geometry

EMG signals are often used to control prostheses or assistive devices, but have been rarely used in rehabilitation. We propose a novel approach to personalised rehabilitation, based on EMG-driven force field adaptation. As a step toward this direction, here we show how EMG activity and movement data during a robot-assisted motor task can be used to estimate muscle geometry. We compare three different models of muscle geometry, characterised by (i) constant moment arms (CM); (ii) a normative model, based on polynomial functions of joint angles with fixed coefficients (normative polynomial, NP); and (iii) a person-adaptive model, in which the same polynomial model is fitted to individual subjects data (fitted polynomial, FP). We found that the FP model has the best performance, specially for subjects whose size is farther from 'average'. The fitting results also emphasise the adverse effect of muscles co-contraction.

Differences in adaptation rates after virtual surgeries provide direct evidence for modularity

Whether the nervous system relies on modularity to simplify acquisition and control of complex motor skills remains controversial. To date, evidence for modularity has been indirect, based on statistical regularities in the motor commands captured by muscle synergies. Here we provide direct evidence by testing the prediction that in a truly modular controller it must be harder to adapt to perturbations that are incompatible with the modules. We investigated a reaching task in which human subjects used myoelectric control to move a mass in a virtual environment. In this environment we could perturb the normal muscle-to-force mapping, as in a complex surgical rearrangement of the tendons, by altering the mapping between recorded muscle activity and simulated force applied on the mass. After identifying muscle synergies, we performed two types of virtual surgeries. After compatible virtual surgeries, a full range of movements could still be achieved recombining the synergies, whereas after incompatible virtual surgeries, new or modified synergies would be required. Adaptation rates after the two types of surgery were compared. If synergies were only a parsimonious description of the regularities in the muscle patterns generated by a nonmodular controller, we would expect adaptation rates to be similar, as both types of surgeries could be compensated with similar changes in the muscle patterns. In contrast, as predicted by modularity, we found strikingly faster adaptation after compatible surgeries than after incompatible ones. These results indicate that muscle synergies are key elements of a modular architecture underlying motor control and adaptation.

Are muscle synergies useful for neural control?

The observation that the activity of multiple muscles can be well approximated by a few linear synergies is viewed by some as a sign that such low-dimensional modules constitute a key component of the neural control system. Here, we argue that the usefulness of muscle synergies as a control principle should be evaluated in terms of errors produced not only in muscle space, but also in task space. We used data from a force-aiming task in two dimensions at the wrist, using an electromyograms (EMG)-driven virtual biomechanics technique that overcomes typical errors in predicting force from recorded EMG, to illustrate through simulation how synergy decomposition inevitably introduces substantial task space errors. Then, we computed the optimal pattern of muscle activation that minimizes summed-squared muscle activities, and demonstrated that synergy decomposition produced similar results on real and simulated data. We further assessed the influence of synergy decomposition on aiming errors (AEs) in a more redundant system, using the optimal muscle pattern computed for the elbow-joint complex (i.e., 13 muscles acting in two dimensions). Because EMG records are typically not available from all contributing muscles, we also explored reconstructions from incomplete sets of muscles. The redundancy of a given set of muscles had opposite effects on the goodness of muscle reconstruction and on task achievement; higher redundancy is associated with better EMG approximation (lower residuals), but with higher AEs. Finally, we showed that the number of synergies required to approximate the optimal muscle pattern for an arbitrary biomechanical system increases with task-space dimensionality, which indicates that the capacity of synergy decomposition to explain behavior depends critically on the scope of the original database. These results have implications regarding the viability of muscle synergy as a putative neural control mechanism, and also as a control algorithm to restore movements.

Wrist torque estimation during simultaneous and continuously changing movements: surface vs. untargeted intramuscular EMG

In this paper, the predictive capability of surface and untargeted intramuscular electromyography (EMG) was compared with respect to wrist-joint torque to quantify which type of measurement better represents joint torque during multiple degrees-of-freedom (DoF) movements for possible application in prosthetic control. Ten able-bodied subjects participated in the study. Surface and intramuscular EMG was recorded concurrently from the right forearm. The subjects were instructed to track continuous contraction profiles using single and combined DoF in two trials. The association between torque and EMG was assessed using an artificial neural network. Results showed a significant difference between the two types of EMG (P < 0.007) for all performance metrics: coefficient of determination (R(2)), Pearson correlation coefficient (PCC), and root mean square error (RMSE). The performance of surface EMG (R(2) = 0.93 ± 0.03; PCC = 0.98 ± 0.01; RMSE = 8.7 ± 2.1%) was found to be superior compared with intramuscular EMG (R(2) = 0.80 ± 0.07; PCC = 0.93 ± 0.03; RMSE = 14.5 ± 2.9%). The higher values of PCC compared with R(2) indicate that both methods are able to track the torque profile well but have some trouble (particularly intramuscular EMG) in estimating the exact amplitude. The possible cause for the difference, thus the low performance of intramuscular EMG, may be attributed to the very high selectivity of the recordings used in this study.