Simultaneous and independent control of a brain-computer interface and contralateral limb movement

Principles of human movement augmentation and the challenges in making it a reality

Augmenting the body with artificial limbs controlled concurrently to one's natural limbs has long appeared in science fiction, but recent technological and neuroscientific advances have begun to make this possible. By allowing individuals to achieve otherwise impossible actions, movement augmentation could revolutionize medical and industrial applications and profoundly change the way humans interact with the environment. Here, we construct a movement augmentation taxonomy through what is augmented and how it is achieved. With this framework, we analyze augmentation that extends the number of degrees-of-freedom, discuss critical features of effective augmentation such as physiological control signals, sensory feedback and learning as well as application scenarios, and propose a vision for the field.

Simultaneous control of natural and extra degrees of freedom by isometric force and electromyographic activity in the muscle-to-force null space

OBJECTIVE

Muscle activation patterns in the muscle-to-force null space, i.e., patterns that do not generate task-relevant forces, may provide an opportunity for motor augmentation by allowing to control additional end-effectors simultaneously to natural limbs. Here we tested the feasibility of muscular null space control for augmentation by assessing simultaneous control of natural and extra degrees of freedom.

APPROACH

We instructed eight participants to control translation and rotation of a virtual 3D end-effector by simultaneous generation of isometric force at the hand and null space activity extracted in real-time from the electromyographic signals recorded from 15 shoulder and arm muscles. First, we identified the null space components that each participant could control more naturally by voluntary co-contraction. Then, participants performed several blocks of a reaching and holding task. They displaced an ellipsoidal cursor to reach one of nine targets by generating force, and simultaneously rotated the cursor to match the target orientation by activating null space components. We developed an information-theoretic metric, an index of difficulty defined as the sum of a spatial and a temporal term, to assess individual null space control ability for both reaching and holding.

MAIN RESULTS

On average, participants could reach the targets in most trials already in the first block (72%) and they improved with practice (maximum 93%) but holding performance remained lower (maximum 43%). As there was a high inter-individual variability in performance, we performed a simulation with different spatial and temporal task conditions to estimate those for which each individual participants would have performed best.

SIGNIFICANCE

Muscular null space control is feasible and may be used to control additional virtual or robotics end-effectors. However, decoding of motor commands must be optimized according to individual null space control ability.

Reconfiguring Motor Circuits for a Joint Manual and BCI Task

Designing brain-computer interfaces (BCIs) that can be used in conjunction with ongoing motor behavior requires an understanding of how neural activity co-opted for brain control interacts with existing neural circuits. For example, BCIs may be used to regain lost motor function after stroke. This requires that neural activity controlling unaffected limbs is dissociated from activity controlling the BCI. In this study we investigated how primary motor cortex accomplishes simultaneous BCI control and motor control in a task that explicitly required both activities to be driven from the same brain region (i.e. a dual-control task). Single-unit activity was recorded from intracortical, multi-electrode arrays while a non-human primate performed this dual-control task. Compared to activity observed during naturalistic motor control, we found that both units used to drive the BCI directly (control units) and units that did not directly control the BCI (non-control units) significantly changed their tuning to wrist torque. Using a measure of effective connectivity, we observed that control units decrease their connectivity. Through an analysis of variance we found that the intrinsic variability of the control units has a significant effect on task proficiency. When this variance is accounted for, motor cortical activity is flexible enough to perform novel BCI tasks that require active decoupling of natural associations to wrist motion. This study provides insight into the neural activity that enables a dual-control brain-computer interface.

Nonhuman Primate Studies to Advance Vision Science and Prevent Blindness

Most primate behavior is dependent on high acuity vision. Optimal visual performance in primates depends heavily upon frontally placed eyes, retinal specializations, and binocular vision. To see an object clearly its image must be placed on or near the fovea of each eye. The oculomotor system is responsible for maintaining precise eye alignment during fixation and generating eye movements to track moving targets. The visual system of nonhuman primates has a similar anatomical organization and functional capability to that of humans. This allows results obtained in nonhuman primates to be applied to humans. The visual and oculomotor systems of primates are immature at birth and sensitive to the quality of binocular visual and eye movement experience during the first months of life. Disruption of postnatal experience can lead to problems in eye alignment (strabismus), amblyopia, unsteady gaze (nystagmus), and defective eye movements. Recent studies in nonhuman primates have begun to discover the neural mechanisms associated with these conditions. In addition, genetic defects that target the retina can lead to blindness. A variety of approaches including gene therapy, stem cell treatment, neuroprosthetics, and optogenetics are currently being used to restore function associated with retinal diseases. Nonhuman primates often provide the best animal model for advancing fundamental knowledge and developing new treatments and cures for blinding diseases.

Changes in cortical network connectivity with long-term brain-machine interface exposure after chronic amputation

Studies on neural plasticity associated with brain–machine interface (BMI) exposure have primarily documented changes in single neuron activity, and largely in intact subjects. Here, we demonstrate significant changes in ensemble-level functional connectivity among primary motor cortical (MI) neurons of chronically amputated monkeys exposed to control a multiple-degree-of-freedom robot arm. A multi-electrode array was implanted in M1 contralateral or ipsilateral to the amputation in three animals. Two clusters of stably recorded neurons were arbitrarily assigned to control reach and grasp movements, respectively. With exposure, network density increased in a nearly monotonic fashion in the contralateral monkeys, whereas the ipsilateral monkey pruned the existing network before re-forming a denser connectivity. Excitatory connections among neurons within a cluster were denser, whereas inhibitory connections were denser among neurons across the two clusters. These results indicate that cortical network connectivity can be modified with BMI learning, even among neurons that have been chronically de-efferented and de-afferented due to amputation.

Previous studies have shown short-term plasticity in single neurons or local field potentials during brain-machine interface (BMI) training. Here the authors report long-term changes in functional connectivity of motor cortex neuronal ensemble activity as chronically amputated monkeys learn to operate a BMI.

Intracortical Brain-Machine Interfaces Advance Sensorimotor Neuroscience

Brain-machine interfaces (BMIs) decode brain activity to control external devices. Over the past two decades, the BMI community has grown tremendously and reached some impressive milestones, including the first human clinical trials using chronically implanted intracortical electrodes. It has also contributed experimental paradigms and important findings to basic neuroscience. In this review, we discuss neuroscience achievements stemming from BMI research, specifically that based upon upper limb prosthetic control with intracortical microelectrodes. We will focus on three main areas: first, we discuss progress in neural coding of reaches in motor cortex, describing recent results linking high dimensional representations of cortical activity to muscle activation. Next, we describe recent findings on learning and plasticity in motor cortex on various time scales. Finally, we discuss how bidirectional BMIs have led to better understanding of somatosensation in and related to motor cortex.