Reconnecting the Hand and Arm to the Brain: Efficacy of Neural Interfaces for Sensorimotor Restoration After Tetraplegia

BACKGROUND AND OBJECTIVES

Paralysis after spinal cord injury involves damage to pathways that connect neurons in the brain to peripheral nerves in the limbs. Re-establishing this communication using neural interfaces has the potential to bridge the gap and restore upper extremity function to people with high tetraplegia. We report a novel approach for restoring upper extremity function using selective peripheral nerve stimulation controlled by intracortical microelectrode recordings from sensorimotor networks, along with restoration of tactile sensation of the hand using intracortical microstimulation.

METHODS

A 27-year-old right-handed man with AIS-B (motor-complete, sensory-incomplete) C3-C4 tetraplegia was enrolled into the clinical trial. Six 64-channel intracortical microelectrode arrays were implanted into left hemisphere regions involved in upper extremity function, including primary motor and sensory cortices, inferior frontal gyrus, and anterior intraparietal area. Nine 16-channel extraneural peripheral nerve electrodes were implanted to allow targeted stimulation of right median, ulnar (2), radial, axillary, musculocutaneous, suprascapular, lateral pectoral, and long thoracic nerves, to produce selective muscle contractions on demand. Proof-of-concept studies were performed to demonstrate feasibility of using a brain-machine interface to read from and write to the brain for restoring motor and sensory functions of the participant's own arm and hand.

RESULTS

Multiunit neural activity that correlated with intended motor action was successfully recorded from intracortical arrays. Microstimulation of electrodes in somatosensory cortex produced repeatable sensory percepts of individual fingers for restoration of touch sensation. Selective electrical activation of peripheral nerves produced antigravity muscle contractions, resulting in functional movements that the participant was able to command under brain control to perform virtual and actual arm and hand movements. The system was well tolerated with no operative complications.

CONCLUSION

The combination of implanted cortical electrodes and nerve cuff electrodes has the potential to create bidirectional restoration of motor and sensory functions of the arm and hand after neurological injury.

Peripheral nerve blocks of wrist and finger flexors can increase hand opening in chronic hemiparetic stroke

Introduction

Hand opening is reduced by abnormal wrist and finger flexor activity in many individuals with stroke. This flexor activity also limits hand opening produced by functional electrical stimulation (FES) of finger and wrist extensor muscles. Recent advances in electrical nerve block technologies have the potential to mitigate this abnormal flexor behavior, but the actual impact of nerve block on hand opening in stroke has not yet been investigated.

Methods

In this study, we applied the local anesthetic ropivacaine to the median and ulnar nerve to induce a complete motor block in 9 individuals with stroke and observed the impact of this block on hand opening as measured by hand pentagonal area. Volitional hand opening and FES-driven hand opening were measured, both while the arm was fully supported on a haptic table (Unloaded) and while lifting against gravity (Loaded). Linear mixed effect regression (LMER) modeling was used to determine the effect of Block.

Results

The ropivacaine block allowed increased hand opening, both volitional and FES-driven, and for both unloaded and loaded conditions. Notably, only the FES-driven and Loaded condition's improvement in hand opening with the block was statistically significant. Hand opening in the FES and Loaded condition improved following nerve block by nearly 20%.

Conclusion

Our results suggest that many individuals with stroke would see improved hand-opening with wrist and finger flexor activity curtailed by nerve block, especially when FES is used to drive the typically paretic finger and wrist extensor muscles. Such a nerve block (potentially produced by aforementioned emerging electrical nerve block technologies) could thus significantly address prior observed shortcomings of FES interventions for individuals with stroke.

Grand Challenges at the Interface of Engineering and Medicine

Over the past two decades Biomedical Engineering has emerged as a major discipline that bridges societal needs of human health care with the development of novel technologies. Every medical institution is now equipped at varying degrees of sophistication with the ability to monitor human health in both non-invasive and invasive modes. The multiple scales at which human physiology can be interrogated provide a profound perspective on health and disease. We are at the nexus of creating "avatars" (herein defined as an extension of "digital twins") of human patho/physiology to serve as paradigms for interrogation and potential intervention. Motivated by the emergence of these new capabilities, the IEEE Engineering in Medicine and Biology Society, the Departments of Biomedical Engineering at Johns Hopkins University and Bioengineering at University of California at San Diego sponsored an interdisciplinary workshop to define the grand challenges that face biomedical engineering and the mechanisms to address these challenges. The Workshop identified five grand challenges with cross-cutting themes and provided a roadmap for new technologies, identified new training needs, and defined the types of interdisciplinary teams needed for addressing these challenges. The themes presented in this paper include: 1) accumedicine through creation of avatars of cells, tissues, organs and whole human; 2) development of smart and responsive devices for human function augmentation; 3) exocortical technologies to understand brain function and treat neuropathologies; 4) the development of approaches to harness the human immune system for health and wellness; and 5) new strategies to engineer genomes and cells.

Implanted Electrodes for Functional Electrical Stimulation to Restore Upper and Lower Extremity Function: History and Future Directions

Functional electrical stimulation (FES) to activate nerves and muscles in paralyzed extremities has considerable promise to improve outcome after neurological disease or injury, especially in individuals who have upper motor nerve dysfunction due to central nervous system pathology. Because technology has improved, a wide variety of methods for providing electrical stimulation to create functional movements have been developed, including muscle stimulating electrodes, nerve stimulating electrodes, and hybrid constructs. However, in spite of decades of success in experimental settings with clear functional improvements for individuals with paralysis, the technology has not yet reached widespread clinical translation. In this review, we outline the history of FES techniques and approaches and describe future directions in evolution of the technology.

Tessellation of artificial touch via microstimulation of human somatosensory cortex

When we interact with objects, we rely on signals from the hand that convey information about the object and our interaction with it. A basic feature of these interactions, the locations of contacts between the hand and object, is often only available via the sense of touch. Information about locations of contact between a brain-controlled bionic hand and an object can be signaled via intracortical microstimulation (ICMS) of somatosensory cortex (S1), which evokes touch sensations that are localized to a specific patch of skin. To provide intuitive location information, tactile sensors on the robotic hand drive ICMS through electrodes that evoke sensations at skin locations matching sensor locations. This approach requires that ICMS-evoked sensations be focal, stable, and distributed over the hand. To systematically investigate the localization of ICMS-evoked sensations, we analyzed the projected fields (PFs) of ICMS-evoked sensations - their location and spatial extent - from reports obtained over multiple years from three participants implanted with microelectrode arrays in S1. First, we found that PFs vary widely in their size across electrodes, are highly stable within electrode, are distributed over large swaths of each participant's hand, and increase in size as the amplitude or frequency of ICMS increases. Second, while PF locations match the locations of the receptive fields (RFs) of the neurons near the stimulating electrode, PFs tend to be subsumed by the corresponding RFs. Third, multi-channel stimulation gives rise to a PF that reflects the conjunction of the PFs of the component channels. By stimulating through electrodes with largely overlapping PFs, then, we can evoke a sensation that is experienced primarily at the intersection of the component PFs. To assess the functional consequence of this phenomenon, we implemented multichannel ICMS-based feedback in a bionic hand and demonstrated that the resulting sensations are more localizable than are those evoked via single-channel ICMS.

The Impact of Hill-Type Actuator Components on the Performance of Reinforcement Learning Controllers to Reverse Upper-Limb Paralysis

Functional electrical stimulation (FES) may allow people who are paralyzed due to spinal cord injuries (SCIs) to regain the ability to move. Deep neural networks (DNNs) trained with reinforcement learning (RL) have been recently explored as a promising methodology to control FES systems to restore upper-limb movements. However, previous studies suggested that large asymmetries in antagonistic upper-limb muscle strengths could impair RL controller performance. In this work, we investigated the underlying causes of asymmetry-associated decreases in controller performance by comparing different Hill-type models of muscle atrophy, and by characterizing RL controller sensitivity to passive mechanical properties of the arm. Simulations indicated that RL controller performance is relatively insensitive to moderate (up to 50%) changes in tendon stiffness and in flexor muscle stiffness. However, the viable workspace for RL control was substantially affected by flexor muscle weakness and by extensor muscle stiffness. Furthermore, we uncovered that RL controller performance issues previously attributed to asymmetrical antagonistic muscle strength resulted from flexor muscle active forces that were insufficient to counteract extensor muscle passive resistance. The simulations supported the adoption of rehabilitation protocols for reaching tasks that prioritize decreasing muscle passive resistance, and counteracting passive resistance with increased antagonistic muscle strength.

Reconnecting the Hand and Arm to the Brain: Efficacy of Neural Interfaces for Sensorimotor Restoration after Tetraplegia

Background

Paralysis after spinal cord injury involves damage to pathways that connect neurons in the brain to peripheral nerves in the limbs. Re-establishing this communication using neural interfaces has the potential to bridge the gap and restore upper extremity function to people with high tetraplegia.

Objective

We report a novel approach for restoring upper extremity function using selective peripheral nerve stimulation controlled by intracortical microelectrode recordings from sensorimotor networks, along with restoration of tactile sensation of the hand using intracortical microstimulation.

Methods

A right-handed man with motor-complete C3-C4 tetraplegia was enrolled into the clinical trial. Six 64-channel intracortical microelectrode arrays were implanted into left hemisphere regions involved in upper extremity function, including primary motor and sensory cortices, inferior frontal gyrus, and anterior intraparietal area. Nine 16-channel extraneural peripheral nerve electrodes were implanted to allow targeted stimulation of right median, ulnar (2), radial, axillary, musculocutaneous, suprascapular, lateral pectoral, and long thoracic nerves, to produce selective muscle contractions on demand. Proof-of-concept studies were performed to demonstrate feasibility of a bidirectional brain-machine interface to restore function of the participant's own arm and hand.

Results

Multi-unit neural activity that correlated with intended motor action was successfully recorded from intracortical arrays. Microstimulation of electrodes in somatosensory cortex produced repeatable sensory percepts of individual fingers for restoration of touch sensation. Selective electrical activation of peripheral nerves produced antigravity muscle contractions. The system was well tolerated with no operative complications.

Conclusion

The combination of implanted cortical electrodes and nerve cuff electrodes has the potential to allow restoration of motor and sensory functions of the arm and hand after neurological injury.

Interim Safety Profile From the Feasibility Study of the BrainGate Neural Interface System

BACKGROUND AND OBJECTIVES

Brain-computer interfaces (BCIs) are being developed to restore mobility, communication, and functional independence to people with paralysis. Though supported by decades of preclinical data, the safety of chronically implanted microelectrode array BCIs in humans is unknown. We report safety results from the prospective, open-label, nonrandomized BrainGate feasibility study (NCT00912041), the largest and longest-running clinical trial of an implanted BCI.

METHODS

Adults aged 18-75 years with quadriparesis from spinal cord injury, brainstem stroke, or motor neuron disease were enrolled through 7 clinical sites in the United States. Participants underwent surgical implantation of 1 or 2 microelectrode arrays in the motor cortex of the dominant cerebral hemisphere. The primary safety outcome was device-related serious adverse events (SAEs) requiring device explantation or resulting in death or permanently increased disability during the 1-year postimplant evaluation period. The secondary outcomes included the type and frequency of other adverse events and the feasibility of the BrainGate system for controlling a computer or other assistive technologies.

RESULTS

From 2004 to 2021, 14 adults enrolled in the BrainGate trial had devices surgically implanted. The average duration of device implantation was 872 days, yielding 12,203 days of safety experience. There were 68 device-related adverse events, including 6 device-related SAEs. The most common device-related adverse event was skin irritation around the percutaneous pedestal. There were no safety events that required device explantation, no unanticipated adverse device events, no intracranial infections, and no participant deaths or adverse events resulting in permanently increased disability related to the investigational device.

DISCUSSION

The BrainGate Neural Interface system has a safety record comparable with other chronically implanted medical devices. Given rapid recent advances in this technology and continued performance gains, these data suggest a favorable risk/benefit ratio in appropriately selected individuals to support ongoing research and development.

TRIAL REGISTRATION INFORMATION

ClinicalTrials.gov Identifier: NCT00912041.

CLASSIFICATION OF EVIDENCE

This study provides Class IV evidence that the neurosurgically placed BrainGate Neural Interface system is associated with a low rate of SAEs defined as those requiring device explantation, resulting in death, or resulting in permanently increased disability during the 1-year postimplant period.

Deep Reinforcement Learning for Control of Time-Varying Musculoskeletal Systems With High Fatigability: A Feasibility Study

Functional electrical stimulation (FES) can be used to restore motor function to people with paralysis caused by spinal cord injuries (SCIs). However, chronically-paralyzed FES-stimulated muscles can fatigue quickly, which may decrease FES controller performance. In this work, we explored the feasibility of using deep neural network (DNN) controllers trained with reinforcement learning (RL) to control FES of upper-limb muscles after SCI. We developed upper-limb biomechanical models that exhibited increased muscle fatigability, decreased muscle recovery, and decreased muscle strength, as observed in people with chronic SCIs. Simulations confirmed that controller training time and controller performance are impaired to varying degrees by muscle fatigability. Also, the simulations showed that large muscle strength asymmetries between opposing muscles can substantially impair controller performance. However, the results of this study suggest that controller performance for highly-fatigable musculoskeletal systems can be preserved by allowing for rest between movements. Overall, the results suggest that RL can be used to successfully train FES controllers, even for highly-fatigable musculoskeletal systems.

Improving the Learning Rate, Accuracy, and Workspace of Reinforcement Learning Controllers for a Musculoskeletal Model of the Human Arm

Cervical spinal cord injuries frequently cause paralysis of all four limbs - a medical condition known as tetraplegia. Functional electrical stimulation (FES), when combined with an appropriate controller, can be used to restore motor function by electrically stimulating the neuromuscular system. Previous works have demonstrated that reinforcement learning can be used to successfully train FES controllers. Here, we demonstrate that transfer learning and curriculum learning can be used to improve the learning rates, accuracies, and workspaces of FES controllers that are trained using reinforcement learning.

Hindsight Experience Replay Improves Reinforcement Learning for Control of a MIMO Musculoskeletal Model of the Human Arm

High-level spinal cord injuries often result in paralysis of all four limbs, leading to decreased patient independence and quality of life. Coordinated functional electrical stimulation (FES) of paralyzed muscles can be used to restore some motor function in the upper extremity. To coordinate functional movements, FES controllers should be developed to exploit the complex characteristics of human movement and produce the intended movement kinematics and/or kinetics. Here, we demonstrate the ability of a controller trained using reinforcement learning to generate desired movements of a horizontal planar musculoskeletal model of the human arm with 2 degrees of freedom and 6 actuators.The controller is given information about the kinematics of the arm, but not the internal state of the actuators.In particular,we demonstrate that a technique called “hindsight experience replay” can improve controller performance while also decreasing controller training time.

Stable, simultaneous and proportional 4-DoF prosthetic hand control via synergy-inspired linear interpolation: a case series

BACKGROUND

Current commercial prosthetic hand controllers limit patients' ability to fully engage high Degree-of-Freedom (DoF) prosthetic hands. Available feedforward controllers rely on large training data sets for controller setup and a need for recalibration upon prosthesis donning. Recently, an intuitive, proportional, simultaneous, regression-based 3-DoF controller remained stable for several months without retraining by combining chronically implanted electromyography (ciEMG) electrodes with a K-Nearest-Neighbor (KNN) mapping technique. The training dataset requirements for simultaneous KNN controllers increase exponentially with DoF, limiting the realistic development of KNN controllers in more than three DoF. We hypothesize that a controller combining linear interpolation, the muscle synergy framework, and a sufficient number of ciEMG channels (at least two per DoF), can allow stable, high-DoF control.

METHODS

Two trans-radial amputee subjects, S6 and S8, were implanted with percutaneously interfaced bipolar intramuscular electrodes. At the time of the study, S6 and S8 had 6 and 8 bipolar EMG electrodes, respectively. A Virtual Reality (VR) system guided users through single and paired training movements in one 3-DoF and four different 4-DoF cases. A linear model of user activity was built by partitioning EMG feature space into regions bounded by vectors of steady state movement EMG patterns. The controller evaluated online EMG signals by linearly interpolating the movement class labels for surrounding trained EMG movements. This yields a simultaneous, continuous, intuitive, and proportional controller. Controllers were evaluated in 3-DoF and 4-DoF through a target-matching task in which subjects controlled a virtual hand to match 80 targets spanning the available movement space. Match Percentage, Time-To-Target, and Path Efficiency were evaluated over a 10-month period based on subject availability.

RESULTS AND CONCLUSIONS

In 3-DoF, S6 and S8 matched most targets and demonstrated stable control after 8 and 10 months, respectively. In 4-DoF, both subjects initially found two of four 4-DoF controllers usable, matching most targets. S8 4-DoF controllers were stable, and showed improving trends over 7-9 months without retraining or at-home practice. S6 4-DoF controllers were unstable after 7 months without retraining. These results indicate that the performance of the controller proposed in this study may remain stable, or even improve, provided initial viability and a sufficient number of EMG channels. Overall, this study demonstrates a controller capable of stable, simultaneous, proportional, intuitive, and continuous control in 3-DoF for up to ten months and in 4-DoF for up to nine months without retraining or at-home use with minimal training times.

Neural ensemble dynamics in dorsal motor cortex during speech in people with paralysis

Speaking is a sensorimotor behavior whose neural basis is difficult to study with single neuron resolution due to the scarcity of human intracortical measurements. We used electrode arrays to record from the motor cortex 'hand knob' in two people with tetraplegia, an area not previously implicated in speech. Neurons modulated during speaking and during non-speaking movements of the tongue, lips, and jaw. This challenges whether the conventional model of a 'motor homunculus' division by major body regions extends to the single-neuron scale. Spoken words and syllables could be decoded from single trials, demonstrating the potential of intracortical recordings for brain-computer interfaces to restore speech. Two neural population dynamics features previously reported for arm movements were also present during speaking: a component that was mostly invariant across initiating different words, followed by rotatory dynamics during speaking. This suggests that common neural dynamical motifs may underlie movement of arm and speech articulators.

The Reconnecting the Hand and Arm with Brain (ReHAB) Commentary on "An Integrated Brain-Machine Interface Platform With Thousands of Channels"

Intracortical brain-machine interfaces are a promising technology for allowing people with chronic and severe neurological disorders that resulted in loss of function to potentially regain those functions through neuroprosthetic devices. The penetrating microelectrode arrays used in almost all previous studies of intracortical brain-machine interfaces in people had a limited recording life (potentially due to issues with long-term biocompatibility), as well as a limited number of recording electrodes with limited distribution in the brain. Significant advances are required in this array interface to deal with the issues of long-term biocompatibility and lack of distributed recordings. The Musk and Neuralink manuscript proposes a novel and potentially disruptive approach to advancing the brain-electrode interface technology, with the potential of addressing many of these hurdles. Our commentary addresses the potential advantages of the proposed approach, as well as the remaining challenges to be addressed.

Rapid calibration of an intracortical brain-computer interface for people with tetraplegia

OBJECTIVE

Brain-computer interfaces (BCIs) can enable individuals with tetraplegia to communicate and control external devices. Though much progress has been made in improving the speed and robustness of neural control provided by intracortical BCIs, little research has been devoted to minimizing the amount of time spent on decoder calibration.

APPROACH

We investigated the amount of time users needed to calibrate decoders and achieve performance saturation using two markedly different decoding algorithms: the steady-state Kalman filter, and a novel technique using Gaussian process regression (GP-DKF).

MAIN RESULTS

Three people with tetraplegia gained rapid closed-loop neural cursor control and peak, plateaued decoder performance within 3 min of initializing calibration. We also show that a BCI-naïve user (T5) was able to rapidly attain closed-loop neural cursor control with the GP-DKF using self-selected movement imagery on his first-ever day of closed-loop BCI use, acquiring a target 37 s after initiating calibration.

SIGNIFICANCE

These results demonstrate the potential for an intracortical BCI to be used immediately after deployment by people with paralysis, without the need for user learning or extensive system calibration.

Signal processing methods for reducing artifacts in microelectrode brain recordings caused by functional electrical stimulation

OBJECTIVE

Functional electrical stimulation (FES) is a promising technology for restoring movement to paralyzed limbs. Intracortical brain-computer interfaces (iBCIs) have enabled intuitive control over virtual and robotic movements, and more recently over upper extremity FES neuroprostheses. However, electrical stimulation of muscles creates artifacts in intracortical microelectrode recordings that could degrade iBCI performance. Here, we investigate methods for reducing the cortically recorded artifacts that result from peripheral electrical stimulation.

APPROACH

One participant in the BrainGate2 pilot clinical trial had two intracortical microelectrode arrays placed in the motor cortex, and thirty-six stimulating intramuscular electrodes placed in the muscles of the contralateral limb. We characterized intracortically recorded electrical artifacts during both intramuscular and surface stimulation. We compared the performance of three artifact reduction methods: blanking, common average reference (CAR) and linear regression reference (LRR), which creates channel-specific reference signals, composed of weighted sums of other channels.

MAIN RESULTS

Electrical artifacts resulting from surface stimulation were 175  ×  larger than baseline neural recordings (which were 110 µV peak-to-peak), while intramuscular stimulation artifacts were only 4  ×  larger. The artifact waveforms were highly consistent across electrodes within each array. Application of LRR reduced artifact magnitudes to less than 10 µV and largely preserved the original neural feature values used for decoding. Unmitigated stimulation artifacts decreased iBCI decoding performance, but performance was almost completely recovered using LRR, which outperformed CAR and blanking and extracted useful neural information during stimulation artifact periods.

SIGNIFICANCE

The LRR method was effective at reducing electrical artifacts resulting from both intramuscular and surface FES, and almost completely restored iBCI decoding performance (>90% recovery for surface stimulation and full recovery for intramuscular stimulation). The results demonstrate that FES-induced artifacts can be easily mitigated in FES  +  iBCI systems by using LRR for artifact reduction, and suggest that the LRR method may also be useful in other noise reduction applications.

Principled BCI Decoder Design and Parameter Selection Using a Feedback Control Model

Decoders optimized offline to reconstruct intended movements from neural recordings sometimes fail to achieve optimal performance online when they are used in closed-loop as part of an intracortical brain-computer interface (iBCI). This is because typical decoder calibration routines do not model the emergent interactions between the decoder, the user, and the task parameters (e.g. target size). Here, we investigated the feasibility of simulating online performance to better guide decoder parameter selection and design. Three participants in the BrainGate2 pilot clinical trial controlled a computer cursor using a linear velocity decoder under different gain (speed scaling) and temporal smoothing parameters and acquired targets with different radii and distances. We show that a user-specific iBCI feedback control model can predict how performance changes under these different decoder and task parameters in held-out data. We also used the model to optimize a nonlinear speed scaling function for the decoder. When used online with two participants, it increased the dynamic range of decoded speeds and decreased the time taken to acquire targets (compared to an optimized standard decoder). These results suggest that it is feasible to simulate iBCI performance accurately enough to be useful for quantitative decoder optimization and design.

Closed-loop cortical control of virtual reach and posture using Cartesian and joint velocity commands

OBJECTIVE

Brain-computer interfaces (BCIs) are a promising technology for the restoration of function to people with paralysis, especially for controlling coordinated reaching. Typical BCI studies decode Cartesian endpoint velocities as commands, but human arm movements might be better controlled in a joint-based coordinate frame, which may match underlying movement encoding in the motor cortex. A better understanding of BCI controlled reaching by people with paralysis may lead to performance improvements in brain-controlled assistive devices.

APPROACH

Two intracortical BCI participants in the BrainGate2 pilot clinical trial performed a visual 3D endpoint virtual reality reaching task using two decoders: Cartesian and joint velocity. Task performance metrics (i.e. success rate and path efficiency) and single feature and population tuning were compared across the two decoder conditions. The participants also demonstrated the first BCI control of a fourth dimension of reaching, the arm's swivel angle, in a 4D posture matching task.

MAIN RESULTS

Both users achieved significantly higher success rates using Cartesian velocity control, and joint controlled trajectories were more variable and significantly more curved. Neural tuning analyses showed that most single feature activity was best described by a Cartesian kinematic encoding model, and population analyses revealed only slight differences in aggregate activity between the decoder conditions. Simulations of a BCI user reproduced trajectory features seen during closed-loop joint control when assuming only Cartesian-tuned features passed through a joint decoder. With minimal training, both participants controlled the virtual arm's swivel angle to complete a 4D posture matching task, and achieved significantly higher success using a Cartesian  +  swivel velocity decoder compared to a joint velocity decoder.

SIGNIFICANCE

These results suggest that Cartesian velocity command interfaces may provide better BCI control of arm movements than other kinematic variables, even in 4D posture tasks with swivel angle targets.

A Comparison of Intention Estimation Methods for Decoder Calibration in Intracortical Brain-Computer Interfaces

OBJECTIVE

Recent reports indicate that making better assumptions about the user's intended movement can improve the accuracy of decoder calibration for intracortical brain-computer interfaces. Several methods now exist for estimating user intent, including an optimal feedback control model, a piecewise-linear feedback control model, ReFIT, and other heuristics. Which of these methods yields the best decoding performance?

METHODS

Using data from the BrainGate2 pilot clinical trial, we measured how a steady-state velocity Kalman filter decoder was affected by the choice of intention estimation method. We examined three separate components of the Kalman filter: dimensionality reduction, temporal smoothing, and output gain (speed scaling).

RESULTS

The decoder's dimensionality reduction properties were largely unaffected by the intention estimation method. Decoded velocity vectors differed by <5% in terms of angular error and speed vs. target distance curves across methods. In contrast, the smoothing and gain properties of the decoder were greatly affected (> 50% difference in average values). Since the optimal gain and smoothing properties are task-specific (e.g. lower gains are better for smaller targets but worse for larger targets), no one method was better for all tasks.

CONCLUSION

Our results show that, when gain and smoothing differences are accounted for, current intention estimation methods yield nearly equivalent decoders and that simple models of user intent, such as a position error vector (target position minus cursor position), perform comparably to more elaborate models. Our results also highlight that simple differences in gain and smoothing properties have a large effect on online performance and can confound decoder comparisons.

Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration

BACKGROUND

People with chronic tetraplegia, due to high-cervical spinal cord injury, can regain limb movements through coordinated electrical stimulation of peripheral muscles and nerves, known as functional electrical stimulation (FES). Users typically command FES systems through other preserved, but unrelated and limited in number, volitional movements (eg, facial muscle activity, head movements, shoulder shrugs). We report the findings of an individual with traumatic high-cervical spinal cord injury who coordinated reaching and grasping movements using his own paralysed arm and hand, reanimated through implanted FES, and commanded using his own cortical signals through an intracortical brain-computer interface (iBCI).

METHODS

We recruited a participant into the BrainGate2 clinical trial, an ongoing study that obtains safety information regarding an intracortical neural interface device, and investigates the feasibility of people with tetraplegia controlling assistive devices using their cortical signals. Surgical procedures were performed at University Hospitals Cleveland Medical Center (Cleveland, OH, USA). Study procedures and data analyses were performed at Case Western Reserve University (Cleveland, OH, USA) and the US Department of Veterans Affairs, Louis Stokes Cleveland Veterans Affairs Medical Center (Cleveland, OH, USA). The study participant was a 53-year-old man with a spinal cord injury (cervical level 4, American Spinal Injury Association Impairment Scale category A). He received two intracortical microelectrode arrays in the hand area of his motor cortex, and 4 months and 9 months later received a total of 36 implanted percutaneous electrodes in his right upper and lower arm to electrically stimulate his hand, elbow, and shoulder muscles. The participant used a motorised mobile arm support for gravitational assistance and to provide humeral abduction and adduction under cortical control. We assessed the participant's ability to cortically command his paralysed arm to perform simple single-joint arm and hand movements and functionally meaningful multi-joint movements. We compared iBCI control of his paralysed arm with that of a virtual three-dimensional arm. This study is registered with ClinicalTrials.gov, number NCT00912041.

FINDINGS

The intracortical implant occurred on Dec 1, 2014, and we are continuing to study the participant. The last session included in this report was Nov 7, 2016. The point-to-point target acquisition sessions began on Oct 8, 2015 (311 days after implant). The participant successfully cortically commanded single-joint and coordinated multi-joint arm movements for point-to-point target acquisitions (80-100% accuracy), using first a virtual arm and second his own arm animated by FES. Using his paralysed arm, the participant volitionally performed self-paced reaches to drink a mug of coffee (successfully completing 11 of 12 attempts within a single session 463 days after implant) and feed himself (717 days after implant).

INTERPRETATION

To our knowledge, this is the first report of a combined implanted FES+iBCI neuroprosthesis for restoring both reaching and grasping movements to people with chronic tetraplegia due to spinal cord injury, and represents a major advance, with a clear translational path, for clinically viable neuroprostheses for restoration of reaching and grasping after paralysis.

FUNDING

National Institutes of Health, Department of Veterans Affairs.

Training an Actor-Critic Reinforcement Learning Controller for Arm Movement Using Human-Generated Rewards

Functional Electrical Stimulation (FES) employs neuroprostheses to apply electrical current to the nerves and muscles of individuals paralyzed by spinal cord injury to restore voluntary movement. Neuroprosthesis controllers calculate stimulation patterns to produce desired actions. To date, no existing controller is able to efficiently adapt its control strategy to the wide range of possible physiological arm characteristics, reaching movements, and user preferences that vary over time. Reinforcement learning (RL) is a control strategy that can incorporate human reward signals as inputs to allow human users to shape controller behavior. In this paper, ten neurologically intact human participants assigned subjective numerical rewards to train RL controllers, evaluating animations of goal-oriented reaching tasks performed using a planar musculoskeletal human arm simulation. The RL controller learning achieved using human trainers was compared with learning accomplished using human-like rewards generated by an algorithm; metrics included success at reaching the specified target; time required to reach the target; and target overshoot. Both sets of controllers learned efficiently and with minimal differences, significantly outperforming standard controllers. Reward positivity and consistency were found to be unrelated to learning success. These results suggest that human rewards can be used effectively to train RL-based FES controllers.

Signal-independent noise in intracortical brain-computer interfaces causes movement time properties inconsistent with Fitts' law

OBJECTIVE

Do movements made with an intracortical BCI (iBCI) have the same movement time properties as able-bodied movements? Able-bodied movement times typically obey Fitts' law: [Formula: see text] (where MT is movement time, D is target distance, R is target radius, and [Formula: see text] are parameters). Fitts' law expresses two properties of natural movement that would be ideal for iBCIs to restore: (1) that movement times are insensitive to the absolute scale of the task (since movement time depends only on the ratio [Formula: see text]) and (2) that movements have a large dynamic range of accuracy (since movement time is logarithmically proportional to [Formula: see text]).

APPROACH

Two participants in the BrainGate2 pilot clinical trial made cortically controlled cursor movements with a linear velocity decoder and acquired targets by dwelling on them. We investigated whether the movement times were well described by Fitts' law.

MAIN RESULTS

We found that movement times were better described by the equation [Formula: see text], which captures how movement time increases sharply as the target radius becomes smaller, independently of distance. In contrast to able-bodied movements, the iBCI movements we studied had a low dynamic range of accuracy (absence of logarithmic proportionality) and were sensitive to the absolute scale of the task (small targets had long movement times regardless of the [Formula: see text] ratio). We argue that this relationship emerges due to noise in the decoder output whose magnitude is largely independent of the user's motor command (signal-independent noise). Signal-independent noise creates a baseline level of variability that cannot be decreased by trying to move slowly or hold still, making targets below a certain size very hard to acquire with a standard decoder.

SIGNIFICANCE

The results give new insight into how iBCI movements currently differ from able-bodied movements and suggest that restoring a Fitts' law-like relationship to iBCI movements may require non-linear decoding strategies.

Semiparametric Identification of Human Arm Dynamics for Flexible Control of a Functional Electrical Stimulation Neuroprosthesis

We present a method to identify the dynamics of a human arm controlled by an implanted functional electrical stimulation neuroprosthesis. The method uses Gaussian process regression to predict shoulder and elbow torques given the shoulder and elbow joint positions and velocities and the electrical stimulation inputs to muscles. We compare the accuracy of torque predictions of nonparametric, semiparametric, and parametric model types. The most accurate of the three model types is a semiparametric Gaussian process model that combines the flexibility of a black box function approximator with the generalization power of a parameterized model. The semiparametric model predicted torques during stimulation of multiple muscles with errors less than 20% of the total muscle torque and passive torque needed to drive the arm. The identified model allows us to define an arbitrary reaching trajectory and approximately determine the muscle stimulations required to drive the arm along that trajectory.

Feedback control policies employed by people using intracortical brain-computer interfaces

OBJECTIVE

When using an intracortical BCI (iBCI), users modulate their neural population activity to move an effector towards a target, stop accurately, and correct for movement errors. We call the rules that govern this modulation a 'feedback control policy'. A better understanding of these policies may inform the design of higher-performing neural decoders.

APPROACH

We studied how three participants in the BrainGate2 pilot clinical trial used an iBCI to control a cursor in a 2D target acquisition task. Participants used a velocity decoder with exponential smoothing dynamics. Through offline analyses, we characterized the users' feedback control policies by modeling their neural activity as a function of cursor state and target position. We also tested whether users could adapt their policy to different decoder dynamics by varying the gain (speed scaling) and temporal smoothing parameters of the iBCI.

MAIN RESULTS

We demonstrate that control policy assumptions made in previous studies do not fully describe the policies of our participants. To account for these discrepancies, we propose a new model that captures (1) how the user's neural population activity gradually declines as the cursor approaches the target from afar, then decreases more sharply as the cursor comes into contact with the target, (2) how the user makes constant feedback corrections even when the cursor is on top of the target, and (3) how the user actively accounts for the cursor's current velocity to avoid overshooting the target. Further, we show that users can adapt their control policy to decoder dynamics by attenuating neural modulation when the cursor gain is high and by damping the cursor velocity more strongly when the smoothing dynamics are high.

SIGNIFICANCE

Our control policy model may help to build better decoders, understand how neural activity varies during active iBCI control, and produce better simulations of closed-loop iBCI movements.

Case study: Head orientation and neck electromyography for cursor control in persons with high cervical tetraplegia

We evaluated the ability of an individual with a high cervical spinal cord injury (SCI) to control a cursor on a computer screen using two different user interfaces: (1) head movements measured via a head-mounted orientation sensor and (2) electromyography (EMG) signals from four head and neck muscles acquired using a 4-channel implanted upper-limb neuroprosthesis that had been deployed in an earlier study. The subject moved the cursor to a set of targets on the screen in a two-dimensional, center-out, target-acquisition task, and his performance was evaluated with a variety of performance measures to assess both position and velocity control accuracy. The subject's performance with both command sources was also compared with the performance of a group of nondisabled subjects. Head orientation provided more accurate performance but was less responsive than EMG. Both command sources showed some directionally dependent performance, with movement to diagonally located targets being performed by a series of sequential motions rather than via straight paths. Overall, the SCI subject's performance with each command source was similar to that reported for a nondisabled population using the same interfaces and performing the same task.

An optimized proportional-derivative controller for the human upper extremity with gravity

When Functional Electrical Stimulation (FES) is used to restore movement in subjects with spinal cord injury (SCI), muscle stimulation patterns should be selected to generate accurate and efficient movements. Ideally, the controller for such a neuroprosthesis will have the simplest architecture possible, to facilitate translation into a clinical setting. In this study, we used the simulated annealing algorithm to optimize two proportional-derivative (PD) feedback controller gain sets for a 3-dimensional arm model that includes musculoskeletal dynamics and has 5 degrees of freedom and 22 muscles, performing goal-oriented reaching movements. Controller gains were optimized by minimizing a weighted sum of position errors, orientation errors, and muscle activations. After optimization, gain performance was evaluated on the basis of accuracy and efficiency of reaching movements, along with three other benchmark gain sets not optimized for our system, on a large set of dynamic reaching movements for which the controllers had not been optimized, to test ability to generalize. Robustness in the presence of weakened muscles was also tested. The two optimized gain sets were found to have very similar performance to each other on all metrics, and to exhibit significantly better accuracy, compared with the three standard gain sets. All gain sets investigated used physiologically acceptable amounts of muscular activation. It was concluded that optimization can yield significant improvements in controller performance while still maintaining muscular efficiency, and that optimization should be considered as a strategy for future neuroprosthesis controller design.

Velocity neurons improve performance more than goal or position neurons do in a simulated closed-loop BCI arm-reaching task

Brain-Computer Interfaces (BCIs) that convert brain-recorded neural signals into intended movement commands could eventually be combined with Functional Electrical Stimulation to allow individuals with Spinal Cord Injury to regain effective and intuitive control of their paralyzed limbs. To accelerate the development of such an approach, we developed a model of closed-loop BCI control of arm movements that (1) generates realistic arm movements (based on experimentally measured, visually-guided movements with real-time error correction), (2) simulates cortical neurons with firing properties consistent with literature reports, and (3) decodes intended movements from the noisy neural ensemble. With this model we explored (1) the relative utility of neurons tuned for different movement parameters (position, velocity, and goal) and (2) the utility of recording from larger numbers of neurons-critical issues for technology development and for determining appropriate brain areas for recording. We simulated arm movements that could be practically restored to individuals with severe paralysis, i.e., movements from an armrest to a volume in front of the person. Performance was evaluated by calculating the smallest movement endpoint target radius within which the decoded cursor position could dwell for 1 s. Our results show that goal, position, and velocity neurons all contribute to improve performance. However, velocity neurons enabled smaller targets to be reached in shorter amounts of time than goal or position neurons. Increasing the number of neurons also improved performance, although performance saturated at 30-50 neurons for most neuron types. Overall, our work presents a closed-loop BCI simulator that models error corrections and the firing properties of various movement-related neurons that can be easily modified to incorporate different neural properties. We anticipate that this kind of tool will be important for development of future BCIs.

Real-time simulation of three-dimensional shoulder girdle and arm dynamics

Electrical stimulation is a promising technology for the restoration of arm function in paralyzed individuals. Control of the paralyzed arm under electrical stimulation, however, is a challenging problem that requires advanced controllers and command interfaces for the user. A real-time model describing the complex dynamics of the arm would allow user-in-the-loop type experiments where the command interface and controller could be assessed. Real-time models of the arm previously described have not included the ability to model the independently controlled scapula and clavicle, limiting their utility for clinical applications of this nature. The goal of this study therefore was to evaluate the performance and mechanical behavior of a real-time, dynamic model of the arm and shoulder girdle. The model comprises seven segments linked by eleven degrees of freedom and actuated by 138 muscle elements. Polynomials were generated to describe the muscle lines of action to reduce computation time, and an implicit, first-order Rosenbrock formulation of the equations of motion was used to increase simulation step-size. The model simulated flexion of the arm faster than real time, simulation time being 92% of actual movement time on standard desktop hardware. Modeled maximum isometric torque values agreed well with values from the literature, showing that the model simulates the moment-generating behavior of a real human arm. The speed of the model enables experiments where the user controls the virtual arm and receives visual feedback in real time. The ability to optimize potential solutions in simulation greatly reduces the burden on the user during development.

Evaluation of head orientation and neck muscle EMG signals as three-dimensional command sources

Background

High cervical spinal cord injuries result in significant functional impairments and affect both the injured individual as well as their family and care givers. To help restore function to these individuals, multiple user interfaces are available to enable command and control of external devices. However, little work has been performed to assess the 3D performance of these interfaces.

Methods

We investigated the performance of eight human subjects in using three user interfaces (head orientation, EMG from muscles of the head and neck, and a three-axis joystick) to command the endpoint position of a multi-axis robotic arm within a 3D workspace to perform a novel out-to-center 3D Fitts’ Law style task. Two of these interfaces (head orientation, EMG from muscles of the head and neck) could realistically be used by individuals with high tetraplegia, while the joystick was evaluated as a standard of high performance. Performance metrics were developed to assess the aspects of command source performance. Data were analyzed using a mixed model design ANOVA. Fixed effects were investigated between sources as well as for interactions between index of difficulty, command source, and the five performance measures used. A 5% threshold for statistical significance was used in the analysis.

Results

The performances of the three command interfaces were rather similar, though significant differences between command sources were observed. The apparent similarity is due in large part to the sequential command strategy (i.e., one dimension of movement at a time) typically adopted by the subjects. EMG-based commands were particularly pulsatile in nature. The use of sequential commands had a significant impact on each command source’s performance for movements in two or three dimensions.

Conclusions

While the sequential nature of the commands produced by the user did not fit with Fitts’ Law, the other performance measures used were able to illustrate the properties of each command source. Though pulsatile, given the overall similarity between head orientation and the EMG interface, (which also could be readily included in a future implanted neuroprosthesis) the use of EMG as a command source for controlling an arm in 3D space is an attractive choice.

A fully implanted intramuscular bipolar myoelectric signal recording electrode

OBJECTIVES

The objective of this study is to develop a fully implanted, intramuscular, bipolar, myoelectric signal (IM-MES) recording electrode for functional electrical stimulation (FES), prosthetic myoelectric control, and other permanently implantable systems.

MATERIALS AND METHODS

An existing fully implanted intramuscular stimulating electrode was modified at each end to allow bipolar recording. The design change also required a modification of the implantation method. Mechanical and in vivo testing was performed on the novel components of the electrode. The first clinical application also is described.

RESULTS

The electrode design modifications did not create any areas of excess mechanical strain on the wires at the distal end where the leads were wound into electrode surfaces. In vivo testing showed that the IM-MES electrode recorded myoelectric signals that were equivalent to an existing epimysial MES electrode. The modified implantation method was simple to implement. The IM-MES electrode was used in an upper extremity FES system in an individual with a spinal cord injury and provided signals that were suitable for a command signal.

CONCLUSIONS

A fully implanted, bipolar intramuscular recording electrode (IM-MES) was developed. Implantation of the IM-MES is straightforward, and almost any muscle can be targeted. Testing has been performed to demonstrate the suitability of the IM-MES electrode for clinical use. Initial clinical applications were successful.

Characterizing and predicting submovements during human three-dimensional arm reaches

We have demonstrated that 3D target-oriented human arm reaches can be represented as linear combinations of discrete submovements, where the submovements are a set of minimum-jerk basis functions for the reaches. We have also demonstrated the ability of deterministic feed-forward Artificial Neural Networks (ANNs) to predict the parameters of the submovements. ANNs were trained using kinematic data obtained experimentally from five human participants making target-directed movements that were decomposed offline into minimum-jerk submovements using an optimization algorithm. Under cross-validation, the ANNs were able to accurately predict the parameters (initiation-time, amplitude, and duration) of the individual submovements. We also demonstrated that the ANNs can together form a closed-loop model of human reaching capable of predicting 3D trajectories with VAF >95.9% and RMSE ≤4.32 cm relative to the actual recorded trajectories. This closed-loop model is a step towards a practical arm trajectory generator based on submovements, and should be useful for the development of future arm prosthetic devices that are controlled by brain computer interfaces or other user interfaces.

Miniature low-power inertial sensors: promising technology for implantable motion capture systems

Inertial and magnetic sensors are valuable for untethered, self-contained human movement analysis. Very recently, complete integration of inertial sensors, magnetic sensors, and processing into single packages, has resulted in miniature, low power devices that could feasibly be employed in an implantable motion capture system. We developed a wearable sensor system based on a commercially available system-in-package inertial and magnetic sensor. We characterized the accuracy of the system in measuring 3-D orientation-with and without magnetometer-based heading compensation-relative to a research grade optical motion capture system. The root mean square error was less than 4° in dynamic and static conditions about all axes. Using four sensors, recording from seven degrees-of-freedom of the upper limb (shoulder, elbow, wrist) was demonstrated in one subject during reaching motions. Very high correlation and low error was found across all joints relative to the optical motion capture system. Findings were similar to previous publications using inertial sensors, but at a fraction of the power consumption and size of the sensors. Such ultra-small, low power sensors provide exciting new avenues for movement monitoring for various movement disorders, movement-based command interfaces for assistive devices, and implementation of kinematic feedback systems for assistive interventions like functional electrical stimulation.

Selection of muscle and nerve-cuff electrodes for neuroprostheses using customizable musculoskeletal model

Neuroprosthetic systems based on functional electrical stimulation aim to restore motor function to individuals with paralysis following spinal cord injury. Identifying the optimal electrode set for the neuroprosthesis is complicated because it depends on the characteristics of the individual (such as injury level), the force capacities of the muscles, the movements the system aims to restore, and the hardware limitations (number and type of electrodes available). An electrode-selection method has been developed that uses a customized musculoskeletal model. Candidate electrode sets are created based on desired functional outcomes and the hard ware limitations of the proposed system. Inverse-dynamic simulations are performed to determine the proportion of target movements that can be accomplished with each set; the set that allows the most movements to be performed is chosen as the optimal set. The technique is demonstrated here for a system recently developed by our research group to restore whole-arm movement to individuals with high-level tetraplegia. The optimal set included selective nerve-cuff electrodes for the radial and musculocutaneous nerves; single-channel cuffs for the axillary, suprascapular, upper subscapular, and long-thoracic nerves; and muscle-based electrodes for the remaining channels. The importance of functional goals, hardware limitations, muscle and nerve anatomy, and surgical feasibility are highlighted.

Multi-muscle FES force control of the human arm for arbitrary goals

We present a method for controlling a neuroprosthesis for a paralyzed human arm using functional electrical stimulation (FES) and characterize the errors of the controller. The subject has surgically implanted electrodes for stimulating muscles in her shoulder and arm. Using input/output data, a model mapping muscle stimulations to isometric endpoint forces measured at the subject's hand was identified. We inverted the model of this redundant and coupled multiple-input multiple-output system by minimizing muscle activations and used this inverse for feedforward control. The magnitude of the total root mean square error over a grid in the volume of achievable isometric endpoint force targets was 11% of the total range of achievable forces. Major sources of error were random error due to trial-to-trial variability and model bias due to nonstationary system properties. Because the muscles working collectively are the actuators of the skeletal system, the quantification of errors in force control guides designs of motion controllers for multi-joint, multi-muscle FES systems that can achieve arbitrary goals.

Predicting the initiation of minimum-jerk submovements in three-dimensional target-oriented human arm trajectories

Target-oriented human arm trajectories can be represented as a series of summed minimum-jerk submovements. Under this framework, corrections for errors in reaching trajectories could be implemented by adding another submovement to the ongoing trajectory. It has been proposed that a feedback-feedforward error-detection process continuously evaluates trajectory error, but this process initiates corrections at discrete points in time. The present study demonstrates the ability of a feed-forward Artificial Neural Network (ANN) to learn the function of this error-detection process. Experimentally recorded human target-oriented arm trajectories were decomposed into submovements. It was assumed that the parameters of each submovement are known at their onset. Trained on these parameters, for each of three participants, an ANN can predict presence of corrections with sensitivity and specificity > 80%, and can predict their timing with R(2) > 40%.

Center of mass acceleration feedback control for standing by functional neuromuscular stimulation: a simulation study

The potential efficacy of total body center of mass (COM) acceleration for feedback control of standing balance by functional neuromuscular stimulation (FNS) following spinal cord injury (SCI) was investigated. COM acceleration may be a viable alternative to conventional joint kinematics because of its rapid responsiveness, focal representation of COM dynamics, and ease of measurement. A computational procedure was developed using an anatomically realistic, three-dimensional, bipedal biomechanical model to determine optimal patterns of muscle excitations to produce targeted effects upon COM acceleration from erect stance. The procedure was verified with electromyographic data collected from standing nondisabled subjects undergoing systematic perturbations. Using 16 muscle groups targeted by existing implantable neuroprostheses, we generated data to train an artificial neural network (ANN)-based controller in simulation. During forward simulations, proportional feedback of COM acceleration drove the ANN to produce muscle excitation patterns countering the effects of applied perturbations. Feedback gains were optimized to minimize upper-limb (UL) loading required to stabilize against disturbances. Compared with the clinical case of maximum constant excitation, the controller reduced UL loading by 43% in resisting external perturbations and by 51% during simulated one-arm reaching. Future work includes performance assessment against expected measurement errors and development of user-specific control systems.

Prediction of imagined single-joint movements in a person with high-level tetraplegia

Cortical neuroprostheses for movement restoration require developing models for relating neural activity to desired movement. Previous studies have focused on correlating single-unit activities (SUA) in primary motor cortex to volitional arm movements in able-bodied primates. The extent of the cortical information relevant to arm movements remaining in severely paralyzed individuals is largely unknown. We record intracortical signals using a microelectrode array chronically implanted in the precentral gyrus of a person with tetraplegia, and estimate positions of imagined single-joint arm movements. Using visually guided motor imagery, the participant imagined performing eight distinct single-joint arm movements, while SUA, multispike trains (MSP), multiunit activity, and local field potential time (LFPrms), and frequency signals (LFPstft) were recorded. Using linear system identification, imagined joint trajectories were estimated with 20-60% variance explained, with wrist flexion/extension predicted the best and pronation/supination the poorest. Statistically, decoding of MSP and LFPstft yielded estimates that equaled those of SUA. Including multiple signal types in a decoder increased prediction accuracy in all cases. We conclude that signals recorded from a single restricted region of the precentral gyrus in this person with tetraplegia contained useful information regarding the intended movements of upper extremity joints.

Electromyogram-based neural network control of transhumeral prostheses

Upper-limb amputation can cause a great deal of functional impairment for patients, particularly for those with amputation at or above the elbow. Our long-term objective is to improve functional outcomes for patients with amputation by integrating a fully implanted electromyographic (EMG) recording system with a wireless telemetry system that communicates with the patient's prosthesis. We believe that this should generate a scheme that will allow patients to robustly control multiple degrees of freedom simultaneously. The goal of this study is to evaluate the feasibility of predicting dynamic arm movements (both flexion/extension and pronation/supination) based on EMG signals from a set of muscles that would likely be intact in patients with transhumeral amputation. We recorded movement kinematics and EMG signals from seven muscles during a variety of movements with different complexities. Time-delayed artificial neural networks were then trained offline to predict the measured arm trajectories based on features extracted from the measured EMG signals. We evaluated the relative effectiveness of various muscle subsets. Predicted movement trajectories had average root-mean-square errors of approximately 15.7° and 24.9° and average R(2) values of approximately 0.81 and 0.46 for elbow flexion/extension and forearm pronation/supination, respectively.

Control of a time-delayed 5 degrees of freedom arm model for use in upper extremity functional electrical stimulation

The goal of this work is to design a controller for a functional electrical stimulation (FES) neuroprosthesis aimed at restoring shoulder and elbow function in individuals who have suffered a high-level cervical (C3-C4) spinal cord injury (SCI). The controller is a mathematical algorithm that coordinates the electrical stimulations applied to the paralyzed muscles such that the arm closely tracks a given desired trajectory. An issue that so far has received little attention is that of time-delays. These delays arise from two sources: (1) the muscle excitation-activation dynamics (10-30 ms) and (2) the sampling of the electrical stimulation (80 ms at the typical 12 Hz stimulation frequency). Using a 5 degrees of freedom (5 DOF) arm model we designed and evaluated a novel controller capable of maintaining stable and accurate tracking performance in the presence of time-delays. For a desired trajectory consisting of 10 randomized reaches, the controller achieved excellent tracking performance as measured by the root-mean-square error (RMSE) between the desired and simulated joint angles (RMSE= [1.48°; 0.81°; 2.14°; 3.11°; 2.29°]).

User-in-the-loop continuous and proportional control of a virtual prosthesis in a posture matching task

As the development of dexterous prosthetic hand and wrist units continues, there is a need for command interfaces that will enable a user to operate these multi-joint devices in a natural, coordinated manner. In previous work, we have demonstrated that it is possible to simultaneously decode hand and wrist kinematics from myoelectric signals recorded from the forearm in an offline manner. The goal of this study was to quantify the performance of this command interface during real-time control of a kinematic prosthesis. One subject with intact limbs controlled a virtual prosthesis and attempted to match a series of target postures using the proposed control scheme as well as using the movements of the intact limb. Initial results indicate that subjects can complete these target matching tasks in the virtual environment. Future work will evaluate the controllability of the proposed strategy relative to traditional control schemes.

Evaluation of volitional control of hand with vertical force assist device for high tetraplegia

Individuals with high tetraplegia have lost motor and sensory function below the neck. This population is highly disabled and requires some method of assistance to accomplish activities of daily living. To date, rehabilitative interventions available in the clinic are limited and have not been designed to exploit residual volitional control. The goal of this study was to validate the presence of volitional control at the hand in individuals with high tetraplegia when the arm is supported against gravity with a vertical assist device. Any volitional control is pertinent to the development of rehabilitative interventions for high tetraplegia because it can be utilized to improve the overall performance and user acceptance of the device. The results from this study provide evidence toward the possibility of an individual with high tetraplegia being able to position their hand and generate forces volitionally in a functional workspace if the weight of their arm is supported by a vertical assist device.

Standard task set for evaluating rehabilitation interventions for individuals with arm paralysis

We have developed a set of upper-limb functional tasks to guide the design and test the performance of rehabilitation technologies that restore arm motion in people with high tetraplegia. Our goal was to develop a short set of tasks that would be representative of a much larger set of activities of daily living (ADLs), while also being feasible for a user of a unilateral, implanted functional electrical stimulation (FES) system. To compile this list of tasks, we reviewed existing clinical outcome measures related to arm and hand function and were further informed by surveys of patient desires. We ultimately selected a set of five tasks that captured the most common components of movement seen in ADLs and is therefore highly relevant for assessing FES-restored unilateral arm function in individuals with high cervical spinal cord injury. The tasks are intended to be used when setting design specifications and for evaluating and standardizing rehabilitation technologies under development. While not unique, this set of tasks will provide a common basis for comparing different interventions (e.g., FES, powered orthoses, robotic assistants) and testing different user command interfaces (e.g., sip-and-puff, head joysticks, brain-computer interfaces).

Continuous neuronal ensemble control of simulated arm reaching by a human with tetraplegia

Functional electrical stimulation (FES), the coordinated electrical activation of multiple muscles, has been used to restore arm and hand function in people with paralysis. User interfaces for such systems typically derive commands from mechanically unrelated parts of the body with retained volitional control, and are unnatural and unable to simultaneously command the various joints of the arm. Neural interface systems, based on spiking intracortical signals recorded from the arm area of motor cortex, have shown the ability to control computer cursors, robotic arms and individual muscles in intact non-human primates. Such neural interface systems may thus offer a more natural source of commands for restoring dexterous movements via FES. However, the ability to use decoded neural signals to control the complex mechanical dynamics of a reanimated human limb, rather than the kinematics of a computer mouse, has not been demonstrated. This study demonstrates the ability of an individual with long-standing tetraplegia to use cortical neuron recordings to command the real-time movements of a simulated dynamic arm. This virtual arm replicates the dynamics associated with arm mass and muscle contractile properties, as well as those of an FES feedback controller that converts user commands into the required muscle activation patterns. An individual with long-standing tetraplegia was thus able to control a virtual, two-joint, dynamic arm in real time using commands derived from an existing human intracortical interface technology. These results show the feasibility of combining such an intracortical interface with existing FES systems to provide a high-performance, natural system for restoring arm and hand function in individuals with extensive paralysis.

Virtual reality environment for simulating tasks with a myoelectric prosthesis: an assessment and training tool

Intuitively and efficiently controlling multiple degrees of freedom is a major hurdle in the field of upper limb prosthetics. A virtual reality myoelectric transhumeral prosthesis simulator has been developed for cost-effectively testing novel control algorithms and devices. The system acquires EMG commands and residual limb kinematics, simulates the prosthesis dynamics, and displays the combined residual limb and prosthesis movements in a virtual reality environment that includes force-based interactions with virtual objects. A virtual Box and Block Test is demonstrated. Three normally-limbed subjects performed the simulated test using a sequential and a synchronous control method. With the sequential method, subjects moved on average 6.7±1.9 blocks in 120 seconds, similar to the number of blocks transhumeral amputees are able to move with their physical prostheses during clinical evaluation. With the synchronous method, subjects moved 6.7±2.2 blocks. The virtual reality prosthesis simulator is thus a promising tool for developing and evaluating control methods, prototyping novel prostheses, and training amputees.

Application of system identification methods for decoding imagined single-joint movements in an individual with high tetraplegia

This study investigated the decoding of imagined arm movements from M1 in an individual with high level tetraplegia. The participant was instructed to imagine herself performing a series of single-joint arm movements, aided by the visual cue of an animate character performing these movements. System identification was used offline to predict the trajectories of the imagined movements and compare these predictions to the trajectories of the actual movements. We report rates of 25 - 50% for predicting completely imagined arm movements in the absence of a priori movements to aid in decoder building.

Command of an upper extremity FES system using a simple set of commands

The purpose of this project is to develop a method of providing a complete set of movement commands to an upper extremity neuroprosthesis when the number of command signals available is fewer than required by the mechanical system. The functional relevance of the developed command interface will be tested in a virtual reality environment, where users-able bodied and paralyzed-will use a limited number of commands to control a simulated neuroprosthesis capable of executing a set of important daily activities that require coordinated motion of a larger number of mechanical degrees of freedom.

Comprehensive joint feedback control for standing by functional neuromuscular stimulation-a simulation study

Previous investigations of feedback control of standing after spinal cord injury (SCI) using functional neuromuscular stimulation (FNS) have primarily targeted individual joints. This study assesses the potential efficacy of comprehensive (trunk, hips, knees, and ankles) joint feedback control against postural disturbances using a bipedal, 3-D computer model of SCI stance. Proportional-derivative feedback drove an artificial neural network trained to produce muscle excitation patterns consistent with maximal joint stiffness values achievable about neutral stance given typical SCI muscle properties. Feedback gains were optimized to minimize upper extremity (UE) loading required to stabilize against disturbances. Compared to the baseline case of maximum constant muscle excitations used clinically, the controller reduced UE loading by 55% in resisting external force perturbations and by 84% during simulated one-arm functional tasks. Performance was most sensitive to inaccurate measurements of ankle plantar/dorsiflexion position and hip ab/adduction velocity feedback. In conclusion, comprehensive joint feedback demonstrates potential to markedly improve FNS standing function. However, alternative control structures capable of effective performance with fewer sensor-based feedback parameters may better facilitate clinical usage.

Musculoskeletal model of trunk and hips for development of seated-posture-control neuroprosthesis

The paralysis resulting from spinal cord injury severely limits voluntary seated-posture control and increases predisposition to a number of health risks. We developed and verified a musculoskeletal model of the hips and lumbar spine using published data. We then used the model to select the optimal muscles for-and evaluate the likely functional recovery benefit of-an 8-channel seated-posture-control neuroprosthesis based on functional electrical stimulation (FES). We found that the model-predicted optimal muscle set included the erector spinae, oblique abdominals, gluteus maximus, and iliopsoas. We mapped muscle excitations to seated trunk posture so that the required excitations at any posture could be approximated using a static map. Using the optimal muscle set, the model predicted a maximum stimulated range of motion of 49 degrees flexion, 9 degrees extension, and 16 degrees lateral bend. In the nominal upright posture, the modeled user could hold almost 15 kg with arms at sides and elbows bent. We discuss in this article the practicality of using FES with the oblique abdominals. A seated-posture-control neuroprosthesis would increase the user's bimanual work space and include several secondary benefits.

Stimulation stability and selectivity of chronically implanted multicontact nerve cuff electrodes in the human upper extremity

Nine spiral nerve cuff electrodes were implanted in two human subjects for up to three years with no adverse functional effects. The objective of this study was to look at the long term nerve and muscle response to stimulation through nerve cuff electrodes. The nerve conduction velocity remained within the clinically accepted range for the entire testing period. The stimulation thresholds stabilized after approximately 20 weeks. The variability in the activation over time was not different from muscle-based electrodes used in implanted functional electrical stimulation systems. Three electrodes had multiple, independent contacts to evaluate selective recruitment of muscles. A single muscle could be selectively activated from each electrode using single-contact stimulation and the selectivity was increased with the use of field steering techniques. The selectivity after three years was consistent with selectivity measured during the implant surgery. Nerve cuff electrodes are effective for chronic muscle activation and multichannel functional electrical stimulation in humans.

Toward the restoration of hand use to a paralyzed monkey: brain-controlled functional electrical stimulation of forearm muscles

Loss of hand use is considered by many spinal cord injury survivors to be the most devastating consequence of their injury. Functional electrical stimulation (FES) of forearm and hand muscles has been used to provide basic, voluntary hand grasp to hundreds of human patients. Current approaches typically grade pre-programmed patterns of muscle activation using simple control signals, such as those derived from residual movement or muscle activity. However, the use of such fixed stimulation patterns limits hand function to the few tasks programmed into the controller. In contrast, we are developing a system that uses neural signals recorded from a multi-electrode array implanted in the motor cortex; this system has the potential to provide independent control of multiple muscles over a broad range of functional tasks. Two monkeys were able to use this cortically controlled FES system to control the contraction of four forearm muscles despite temporary limb paralysis. The amount of wrist force the monkeys were able to produce in a one-dimensional force tracking task was significantly increased. Furthermore, the monkeys were able to control the magnitude and time course of the force with sufficient accuracy to track visually displayed force targets at speeds reduced by only one-third to one-half of normal. Although these results were achieved by controlling only four muscles, there is no fundamental reason why the same methods could not be scaled up to control a larger number of muscles. We believe these results provide an important proof of concept that brain-controlled FES prostheses could ultimately be of great benefit to paralyzed patients with injuries in the mid-cervical spinal cord.