Technical Details of the Implantable Myoelectric Sensor (IMES) System for Multifunction Prosthesis Control

Intent Prediction Based on Biomechanical Coordination of EMG and Vision-Filtered Gaze for End-Point Control of an Arm Prosthesis

We propose a novel controller for powered prosthetic arms, where fused EMG and gaze data predict the desired end-point for a full arm prosthesis, which could drive the forward motion of individual joints. We recorded EMG, gaze, and motion-tracking during pick-and-place trials with 7 able-bodied subjects. Subjects positioned an object above a random target on a virtual interface, each completing around 600 trials. On average across all trials and subjects gaze preceded EMG and followed a repeatable pattern that allowed for prediction. A computer vision algorithm was used to extract the initial and target fixations and estimate the target position in 2D space. Two SVRs were trained with EMG data to predict the x- and y- position of the hand; results showed that the y-estimate was significantly better than the x-estimate. The EMG and gaze predictions were fused using a Kalman Filter-based approach, and the positional error from using EMG-only was significantly higher than the fusion of EMG and gaze. The final target position Root Mean Squared Error (RMSE) decreased from 9.28 cm with an EMG-only prediction to 6.94 cm when using a gaze-EMG fusion. This error also increased significantly when removing some or all arm muscle signals. However, using fused EMG and gaze, there were no significant difference between predictors that included all muscles, or only a subset of muscles.

Decoding of grasping tasks from intraneural recordings in trans-radial amputee

OBJECTIVE

A major challenge in neuroprosthetics is the restoration of sensory-motor hand functions in upper-limb amputees. Neuroprostheses based on the direct re-connection of the peripheral nerves may be an interesting approach for re-establishing the natural and effective bidirectional control of hand prostheses. Recent results have shown that transverse intrafascicular multi-channel electrodes (TIMEs) can restore natural and sophisticated sensory feedback. However, the potential of using TIME-recorded motor intraneural signals to decode grasping tasks has not as yet been explored.

APPROACH

In this study, we show that several hand-movement intentions can be decoded from intraneural signals recorded using four TIMEs implanted in the median and ulnar nerves of an upper limb amputee. Experimental sessions were performed over a week, from day 16 to day 23 after the surgical operation. Intraneural activity was recorded during several hand motor tasks imagined by the subject and processed offline.

MAIN RESULTS

We obtained a very high decoding accuracy considering 11 class states (up to 83%). These results confirm that neural signals recorded by multi-channel intraneural electrodes can be used to decode several movement intentions with high accuracy. Moreover, we were able to use same TIME channels for decoding over one week within the first month, even if the stability has to be confirmed during long-term experiments.

SIGNIFICANCE

Therefore, TIMEs could be used in the future to achieve a complete bidirectional approach exploiting neural pathways, to make a more natural and intuitive new generation of hand prostheses that have a closer resemblance to a healthy hand.

A Biomechatronic EPP upper-limb prosthesis teleoperation system implementation using Bluetooth Low Energy

In this paper a real time, stand-alone wireless Biomechatronic Extended Physiological Proprioception (EPP) teleoperation system was implemented using two Bluetooth Low Energy (BLE) wireless Systems on Chip (SoCs). This system is designed to achieve kinesthetic coupling between the amputee and prosthetic arm without the use of the classic EPP mechanical linkage, but with the use of a wireless implementation of a Master/Slave teleoperation topology. The experimental real-time implementation achieved a high level of transparency with minuscule time delays.

New developments in prosthetic arm systems

Absence of an upper limb leads to severe impairments in everyday life, which can further influence the social and mental state. For these reasons, early developments in cosmetic and body-driven prostheses date some centuries ago, and they have been evolving ever since. Following the end of the Second World War, rapid developments in technology resulted in powered myoelectric hand prosthetics. In the years to come, these devices were common on the market, though they still suffered high user abandonment rates. The reasons for rejection were trifold - insufficient functionality of the hardware, fragile design, and cumbersome control. In the last decade, both academia and industry have reached major improvements concerning technical features of upper limb prosthetics and methods for their interfacing and control. Advanced robotic hands are offered by several vendors and research groups, with a variety of active and passive wrist options that can be articulated across several degrees of freedom. Nowadays, elbow joint designs include active solutions with different weight and power options. Control features are getting progressively more sophisticated, offering options for multiple sensor integration and multi-joint articulation. Latest developments in socket designs are capable of facilitating implantable and multiple surface electromyography sensors in both traditional and osseointegration-based systems. Novel surgical techniques in combination with modern, sophisticated hardware are enabling restoration of dexterous upper limb functionality. This article is aimed at reviewing the latest state of the upper limb prosthetic market, offering insights on the accompanying technologies and techniques. We also examine the capabilities and features of some of academia's flagship solutions and methods.

Low-power polling mode of the next-generation IMES2 implantable wireless EMG sensor

The IMES1 Implantable MyoElectric Sensor device is currently in human clinical trials led by the Alfred Mann Foundation. The IMES is implanted in a residual limb and is powered wirelessly using a magnetic field. EMG signals resulting from the amputee's voluntary movement are amplified and transmitted wirelessly by the IMES to an external controller which controls movement of an external motorized prosthesis. Development of the IMES technology is on-going, producing the next-generation IMES2. Among various improvements, a new feature of the IMES2 is a low-power polling mode. In this low-power mode, the IMES2 power consumption can be dramatically reduced when the limb is inactive through the use of a polled sampling. With the onset of EMG activity, the IMES2 system can switch to the normal higher sample rate to allow the acquisition of high-fidelity EMG data for prosthesis control.

Surveying the interest of individuals with upper limb loss in novel prosthetic control techniques

Background

Novel techniques for the control of upper limb prostheses may allow users to operate more complex prostheses than those that are currently available. Because many of these techniques are surgically invasive, it is important to understand whether individuals with upper limb loss would accept the associated risks in order to use a prosthesis.

Methods

An online survey of individuals with upper limb loss was conducted. Participants read descriptions of four prosthetic control techniques. One technique was noninvasive (myoelectric) and three were invasive (targeted muscle reinnervation, peripheral nerve interfaces, cortical interfaces). Participants rated how likely they were to try each technique if it offered each of six different functional features. They also rated their general interest in each of the six features. A two-way repeated measures analysis of variance with Greenhouse-Geisser corrections was used to examine the effect of the technique type and feature on participants’ interest in each technique.

Results

Responses from 104 individuals were analyzed. Many participants were interested in trying the techniques – 83 % responded positively toward myoelectric control, 63 % toward targeted muscle reinnervation, 68 % toward peripheral nerve interfaces, and 39 % toward cortical interfaces. Common concerns about myoelectric control were weight, cost, durability, and difficulty of use, while the most common concern about the invasive techniques was surgical risk. Participants expressed greatest interest in basic prosthesis features (e.g., opening and closing the hand slowly), as opposed to advanced features like fine motor control and touch sensation.

Conclusions

The results of these investigations may be used to inform the development of future prosthetic technologies that are appealing to individuals with upper limb loss.

Electronic supplementary material

The online version of this article (doi:10.1186/s12984-015-0044-2) contains supplementary material, which is available to authorized users.

Evaluation of head orientation and neck muscle EMG signals as three-dimensional command sources

Background

High cervical spinal cord injuries result in significant functional impairments and affect both the injured individual as well as their family and care givers. To help restore function to these individuals, multiple user interfaces are available to enable command and control of external devices. However, little work has been performed to assess the 3D performance of these interfaces.

Methods

We investigated the performance of eight human subjects in using three user interfaces (head orientation, EMG from muscles of the head and neck, and a three-axis joystick) to command the endpoint position of a multi-axis robotic arm within a 3D workspace to perform a novel out-to-center 3D Fitts’ Law style task. Two of these interfaces (head orientation, EMG from muscles of the head and neck) could realistically be used by individuals with high tetraplegia, while the joystick was evaluated as a standard of high performance. Performance metrics were developed to assess the aspects of command source performance. Data were analyzed using a mixed model design ANOVA. Fixed effects were investigated between sources as well as for interactions between index of difficulty, command source, and the five performance measures used. A 5% threshold for statistical significance was used in the analysis.

Results

The performances of the three command interfaces were rather similar, though significant differences between command sources were observed. The apparent similarity is due in large part to the sequential command strategy (i.e., one dimension of movement at a time) typically adopted by the subjects. EMG-based commands were particularly pulsatile in nature. The use of sequential commands had a significant impact on each command source’s performance for movements in two or three dimensions.

Conclusions

While the sequential nature of the commands produced by the user did not fit with Fitts’ Law, the other performance measures used were able to illustrate the properties of each command source. Though pulsatile, given the overall similarity between head orientation and the EMG interface, (which also could be readily included in a future implanted neuroprosthesis) the use of EMG as a command source for controlling an arm in 3D space is an attractive choice.

Design and evaluation of prosthetic shoulder controller

We developed a 2-degree-of-freedom (DOF) shoulder position transducer (sensing shoulder protraction-retraction and elevation-depression) that can be used to control two of a powered prosthetic humerus' DOFs. We also developed an evaluation protocol based on Fitts' law to assess the performance of our device. The primary motivation for this work was to support development of powered prosthetic shoulder joints of a new generation of prosthetic arms for people with shoulder disarticulation and very high-level transhumeral amputation. We found that transducers that provided resistance to shoulder movement performed better than those providing no resistance. We also found that a position control scheme, where effector position is proportional to shoulder position, performed better than a velocity control scheme, where effector velocity is proportional to shoulder position. More generally, our transducer can be used to control motion along any two DOFs under a proportional control scheme. It can also be used in a more general 4-DOF control scheme by sequentially controlling two DOFs at a time. The evaluation protocol has general applicability for researchers and practitioners. Researchers can employ it to compare different prosthesis designs and control schemes, while practitioners may find the evaluation protocol useful in evaluating and training people with amputation in the use of prostheses.

Multichannel wireless ECoG array ASIC devices

Surgical resection of epileptogenic foci is often a beneficial treatment for patients suffering debilitating seizures arising from intractable epilepsy [1], [2], [3]. Electrodes placed subdurally on the surface of the brain in the form of an ECoG array is one of the multiple methods for localizing epileptogenic zones for the purpose of defining the region for surgical resection. Currently, transcutaneous wires from ECoG grids limit the duration of time that implanted grids can be used for diagnosis. A wireless ECoG recording and stimulation system may be a solution to extend the diagnostic period. To avoid the transcutaneous connections, a 64-channel wireless silicon recording/stimulating ASIC was developed as the electronic component of a wireless ECoG array that uses SIROF electrodes on a polyimide substrate[4]. Here we describe two new ASIC devices that have been developed and tested as part of the on-going wireless ECoG system design.

On the viability of implantable electrodes for the natural control of artificial limbs: review and discussion

The control of robotic prostheses based on pattern recognition algorithms is a widely studied subject that has shown promising results in acute experiments. The long-term implementation of this technology, however, has not yet been achieved due to practical issues that can be mainly attributed to the use of surface electrodes and their highly environmental dependency. This paper describes several implantable electrodes and discusses them as a solution for the natural control of artificial limbs. In this context "natural" is defined as producing control over limb movement analogous to that of an intact physiological system. This includes coordinated and simultaneous movements of different degrees of freedom. It also implies that the input signals must come from nerves or muscles that were originally meant to produce the intended movement and that feedback is perceived as originating in the missing limb without requiring burdensome levels of concentration. After scrutinizing different electrode designs and their clinical implementation, we concluded that the epimysial and cuff electrodes are currently promising candidates to achieving a long-term stable and natural control of robotic prosthetics, provided that communication from the electrodes to the outside of the body is guaranteed.

Simultaneous and proportional force estimation in multiple degrees of freedom from intramuscular EMG

This letter investigates simultaneous and proportional estimation of force in 2 degree-of-freedoms (DoFs) from intramuscular electromyography (EMG). Intramuscular EMG signals from three able-bodied subjects were recorded along with isometric forces in multiple DoF from the right arm. The association between five EMG features and force profiles was modeled using an artificial neural network. Correlation coefficients between the measured and the estimated forces were 0.85 ± 0.056 and 0.88 ± 0.05 without and with post processing, respectively. The results showed that force can be estimated in 2 DoFs with high accuracy and that the degree of performance depended on the force function (task) to be estimated.

Development of an implantable myoelectric sensor for advanced prosthesis control

Modern hand and wrist prostheses afford a high level of mechanical sophistication, but the ability to control them in an intuitive and repeatable manner lags. Commercially available systems using surface electromyographic (EMG) or myoelectric control can supply at best two degrees of freedom (DOF), most often sequentially controlled. This limitation is partially due to the nature of surface-recorded EMG, for which the signal contains components from multiple muscle sources. We report here on the development of an implantable myoelectric sensor using EMG sensors that can be chronically implanted into an amputee's residual muscles. Because sensing occurs at the source of muscle contraction, a single principal component of EMG is detected by each sensor, corresponding to intent to move a particular effector. This system can potentially provide independent signal sources for control of individual effectors within a limb prosthesis. The use of implanted devices supports inter-day signal repeatability. We report on efforts in preparation for human clinical trials, including animal testing, and a first-in-human proof of principle demonstration where the subject was able to intuitively and simultaneously control two DOF in a hand and wrist prosthesis.

Multi-scale recordings for neuroprosthetic control of finger movements

We trained a rhesus monkey to perform individuated and combined finger flexions and extensions of the thumb, index, and middle finger. A Utah Electrode Array (UEA) was implanted into the hand region of the motor cortex contralateral to the monkey's trained hand. We also implanted a microwire electrocorticography grid (microECoG) epidurally so that it covered the UEA. The microECoG grid spanned the arm and hand regions of both the primary motor and somatosensory cortices. Previously this monkey had Implantable MyoElectric Sensors (IMES) surgically implanted into the finger muscles of the monkey's forearm. Action potentials (APs), local field potentials (LFPs), and microECoG signals were recorded from wired head-stage connectors for the UEA and microECoG grids, while EMG was recorded wirelessly. The monkey performed a finger flexion/extension task while neural and EMG data were acquired. We wrote an algorithm that uses the spike data from the UEA to perform a real-time decode of the monkey's finger movements. Also, analyses of the LFP and microECoG data indicate that these data show trial-averaged differences between different finger movements, indicating the data are potentially decodeable.

Continuous detection and decoding of dexterous finger flexions with implantable myoelectric sensors

A rhesus monkey was trained to perform individuated and combined finger flexions of the thumb, index, and middle finger. Nine implantable myoelectric sensors (IMES) were then surgically implanted into the finger muscles of the monkey's forearm, without any adverse effects over two years postimplantation. Using an inductive link, EMG was wirelessly recorded from the IMES as the monkey performed a finger flexion task. The EMG from the different IMES implants showed very little cross correlation. An offline parallel linear discriminant analysis (LDA) based algorithm was used to decode finger activity based on features extracted from continuously presented frames of recorded EMG. The offline parallel LDA was run on intraday sessions as well as on sessions where the algorithm was trained on one day and tested on following days. The performance of the algorithm was evaluated continuously by comparing classification output by the algorithm to the current state of the finger switches. The algorithm detected and classified seven different finger movements, including individual and combined finger flexions, and a no-movement state (chance performance = 12.5%) . When the algorithm was trained and tested on data collected the same day, the average performance was 43.8+/-3.6% n=10. When the training-testing separation period was five months, the average performance of the algorithm was 46.5+/-3.4% n=8. These results demonstrated that using EMG recorded and wirelessly transmitted by IMES offers a promising approach for providing intuitive, dexterous control of artificial limbs where human patients have sufficient, functional residual muscle following amputation.

Implantable myoelectric sensors (IMESs) for intramuscular electromyogram recording

We have developed a multichannel electrogmyography sensor system capable of receiving and processing signals from up to 32 implanted myoelectric sensors (IMES). The appeal of implanted sensors for myoelectric control is that electromyography (EMG) signals can be measured at their source providing relatively cross-talk-free signals that can be treated as independent control sites. An external telemetry controller receives telemetry sent over a transcutaneous magnetic link by the implanted electrodes. The same link provides power and commands to the implanted electrodes. Wireless telemetry of EMG signals from sensors implanted in the residual musculature eliminates the problems associated with percutaneous wires, such as infection, breakage, and marsupialization. Each implantable sensor consists of a custom-designed application-specified integrated circuit that is packaged into a biocompatible RF BION capsule from the Alfred E. Mann Foundation. Implants are designed for permanent long-term implantation with no servicing requirements. We have a fully operational system. The system has been tested in animals. Implants have been chronically implanted in the legs of three cats and are still completely operational four months after implantation.

Control strategies for smart prosthetic hand technology: an overview

A chronological overview of the applications of control theory to prosthetic hand is presented. The overview focuses on hard computing or control techniques such as multivariable feedback, optimal, nonlinear, adaptive and robust and soft computing or control techniques such as artificial intelligence, neural networks, fuzzy logic, genetic algorithms and on the fusion of hard and soft control techniques. This overview is not intended to be an exhaustive survey on this topic and any omissions of other works is purely unintentional.

Decoding individuated finger flexions with Implantable MyoElectric Sensors

We trained a rhesus monkey to perform randomly cued, individuated finger flexions of the thumb, index, and middle finger. Nine Implantable MyoElectric Sensors (IMES) were then surgically implanted into the finger muscles of the monkey's forearm, without any observable adverse chronic effects. Using an inductive link, we wirelessly recorded EMG from the IMES as the monkey performed a finger flexion task. A principal components analysis (PCA) based algorithm was used to decode which finger switch was pressed based on the recorded EMG. This algorithm correctly decoded which finger was moved 89% of the time. These results demonstrate that IMES offer a safe and highly promising approach for providing intuitive, dexterous control of artificial limbs and hands after amputation.

A comparison of the effects of electrode implantation and targeting on pattern classification accuracy for prosthesis control

The use of surface versus intramuscular electrodes as well as the effect of electrode targeting on pattern-recognition-based multifunctional prosthesis control was explored. Surface electrodes are touted for their ability to record activity from relatively large portions of muscle tissue. Intramuscular electromyograms (EMGs) can provide focal recordings from deep muscles of the forearm and independent signals relatively free of crosstalk. However, little work has been done to compare the two. Additionally, while previous investigations have either targeted electrodes to specific muscles or used untargeted (symmetric) electrode arrays, no work has compared these approaches to determine if one is superior. The classification accuracies of pattern-recognition-based classifiers utilizing surface and intramuscular as well as targeted and untargeted electrodes were compared across 11 subjects. A repeated-measures analysis of variance revealed that when only EMG amplitude information was used from all available EMG channels, the targeted surface, targeted intramuscular, and untargeted surface electrodes produced similar classification accuracies while the untargeted intramuscular electrodes produced significantly lower accuracies. However, no statistical differences were observed between any of the electrode conditions when additional features were extracted from the EMG signal. It was concluded that the choice of electrode should be driven by clinical factors, such as signal robustness/stability, cost, etc., instead of by classification accuracy.

Controlling a prosthetic arm with a throat microphone

The aim of the present paper is to illustrate the phases of design and application of an innovative input source for an EMG upper limb prosthetic arm: a laryngophone, otherwise called throat microphone (t-mic). In the last years several different input sources were explored, from the implantable myoelectric sensors to the mechanomyographic sensors. The idea of controlling a prosthesis with vocal commands is quite recent but seems to be promising in helping users to better control their devices, improving the quality of their life.