Motor cortex latent dynamics encode spatial and temporal arm movement parameters independently

The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: Each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, but also argue that not all parameters of movement are defined by different trajectories of population activity.

De novo motor learning creates structure in neural activity that shapes adaptation

Animals can quickly adapt learned movements to external perturbations, and their existing motor repertoire likely influences their ease of adaptation. Long-term learning causes lasting changes in neural connectivity, which shapes the activity patterns that can be produced during adaptation. Here, we examined how a neural population's existing activity patterns, acquired through de novo learning, affect subsequent adaptation by modeling motor cortical neural population dynamics with recurrent neural networks. We trained networks on different motor repertoires comprising varying numbers of movements, which they acquired following various learning experiences. Networks with multiple movements had more constrained and robust dynamics, which were associated with more defined neural 'structure'-organization in the available population activity patterns. This structure facilitated adaptation, but only when the changes imposed by the perturbation were congruent with the organization of the inputs and the structure in neural activity acquired during de novo learning. These results highlight trade-offs in skill acquisition and demonstrate how different learning experiences can shape the geometrical properties of neural population activity and subsequent adaptation.

Nonlinear manifolds underlie neural population activity during behaviour

There is rich variety in the activity of single neurons recorded during behaviour. Yet, these diverse single neuron responses can be well described by relatively few patterns of neural co-modulation. The study of such low-dimensional structure of neural population activity has provided important insights into how the brain generates behaviour. Virtually all of these studies have used linear dimensionality reduction techniques to estimate these population-wide co-modulation patterns, constraining them to a flat "neural manifold". Here, we hypothesised that since neurons have nonlinear responses and make thousands of distributed and recurrent connections that likely amplify such nonlinearities, neural manifolds should be intrinsically nonlinear. Combining neural population recordings from monkey, mouse, and human motor cortex, and mouse striatum, we show that: 1) neural manifolds are intrinsically nonlinear; 2) their nonlinearity becomes more evident during complex tasks that require more varied activity patterns; and 3) manifold nonlinearity varies across architecturally distinct brain regions. Simulations using recurrent neural network models confirmed the proposed relationship between circuit connectivity and manifold nonlinearity, including the differences across architecturally distinct regions. Thus, neural manifolds underlying the generation of behaviour are inherently nonlinear, and properly accounting for such nonlinearities will be critical as neuroscientists move towards studying numerous brain regions involved in increasingly complex and naturalistic behaviours.

A spinal cord neuroprosthesis for locomotor deficits due to Parkinson's disease

People with late-stage Parkinson's disease (PD) often suffer from debilitating locomotor deficits that are resistant to currently available therapies. To alleviate these deficits, we developed a neuroprosthesis operating in closed loop that targets the dorsal root entry zones innervating lumbosacral segments to reproduce the natural spatiotemporal activation of the lumbosacral spinal cord during walking. We first developed this neuroprosthesis in a non-human primate model that replicates locomotor deficits due to PD. This neuroprosthesis not only alleviated locomotor deficits but also restored skilled walking in this model. We then implanted the neuroprosthesis in a 62-year-old male with a 30-year history of PD who presented with severe gait impairments and frequent falls that were medically refractory to currently available therapies. We found that the neuroprosthesis interacted synergistically with deep brain stimulation of the subthalamic nucleus and dopaminergic replacement therapies to alleviate asymmetry and promote longer steps, improve balance and reduce freezing of gait. This neuroprosthesis opens new perspectives to reduce the severity of locomotor deficits in people with PD.

Preserved neural dynamics across animals performing similar behaviour

Animals of the same species exhibit similar behaviours that are advantageously adapted to their body and environment. These behaviours are shaped at the species level by selection pressures over evolutionary timescales. Yet, it remains unclear how these common behavioural adaptations emerge from the idiosyncratic neural circuitry of each individual. The overall organization of neural circuits is preserved across individuals1 because of their common evolutionarily specified developmental programme2-4. Such organization at the circuit level may constrain neural activity5-8, leading to low-dimensional latent dynamics across the neural population9-11. Accordingly, here we suggested that the shared circuit-level constraints within a species would lead to suitably preserved latent dynamics across individuals. We analysed recordings of neural populations from monkey and mouse motor cortex to demonstrate that neural dynamics in individuals from the same species are surprisingly preserved when they perform similar behaviour. Neural population dynamics were also preserved when animals consciously planned future movements without overt behaviour12 and enabled the decoding of planned and ongoing movement across different individuals. Furthermore, we found that preserved neural dynamics extend beyond cortical regions to the dorsal striatum, an evolutionarily older structure13,14. Finally, we used neural network models to demonstrate that behavioural similarity is necessary but not sufficient for this preservation. We posit that these emergent dynamics result from evolutionary constraints on brain development and thus reflect fundamental properties of the neural basis of behaviour.

A Unified, Scalable Framework for Neural Population Decoding

Our ability to use deep learning approaches to decipher neural activity would likely benefit from greater scale, in terms of both model size and datasets. However, the integration of many neural recordings into one unified model is challenging, as each recording contains the activity of different neurons from different individual animals. In this paper, we introduce a training framework and architecture designed to model the population dynamics of neural activity across diverse, large-scale neural recordings. Our method first tokenizes individual spikes within the dataset to build an efficient representation of neural events that captures the fine temporal structure of neural activity. We then employ cross-attention and a PerceiverIO backbone to further construct a latent tokenization of neural population activities. Utilizing this architecture and training framework, we construct a large-scale multi-session model trained on large datasets from seven nonhuman primates, spanning over 158 different sessions of recording from over 27,373 neural units and over 100 hours of recordings. In a number of different tasks, we demonstrate that our pretrained model can be rapidly adapted to new, unseen sessions with unspecified neuron correspondence, enabling few-shot performance with minimal labels. This work presents a powerful new approach for building deep learning tools to analyze neural data and stakes out a clear path to training at scale.

De novo motor learning creates structure in neural activity space that shapes adaptation

Animals can quickly adapt learned movements in response to external perturbations. Motor adaptation is likely influenced by an animal's existing movement repertoire, but the nature of this influence is unclear. Long-term learning causes lasting changes in neural connectivity which determine the activity patterns that can be produced. Here, we sought to understand how a neural population's activity repertoire, acquired through long-term learning, affects short-term adaptation by modeling motor cortical neural population dynamics during de novo learning and subsequent adaptation using recurrent neural networks. We trained these networks on different motor repertoires comprising varying numbers of movements. Networks with multiple movements had more constrained and robust dynamics, which were associated with more defined neural 'structure'-organization created by the neural population activity patterns corresponding to each movement. This structure facilitated adaptation, but only when small changes in motor output were required, and when the structure of the network inputs, the neural activity space, and the perturbation were congruent. These results highlight trade-offs in skill acquisition and demonstrate how prior experience and external cues during learning can shape the geometrical properties of neural population activity as well as subsequent adaptation.

Rapid adaptation of brain-computer interfaces to new neuronal ensembles or participants via generative modelling

For brain-computer interfaces (BCIs), obtaining sufficient training data for algorithms that map neural signals onto actions can be difficult, expensive or even impossible. Here we report the development and use of a generative model-a model that synthesizes a virtually unlimited number of new data distributions from a learned data distribution-that learns mappings between hand kinematics and the associated neural spike trains. The generative spike-train synthesizer is trained on data from one recording session with a monkey performing a reaching task and can be rapidly adapted to new sessions or monkeys by using limited additional neural data. We show that the model can be adapted to synthesize new spike trains, accelerating the training and improving the generalization of BCI decoders. The approach is fully data-driven, and hence, applicable to applications of BCIs beyond motor control.

Local field potentials reflect cortical population dynamics in a region-specific and frequency-dependent manner

The spiking activity of populations of cortical neurons is well described by the dynamics of a small number of population-wide covariance patterns, the 'latent dynamics'. These latent dynamics are largely driven by the same correlated synaptic currents across the circuit that determine the generation of local field potentials (LFP). Yet, the relationship between latent dynamics and LFPs remains largely unexplored. Here, we characterised this relationship for three different regions of primate sensorimotor cortex during reaching. The correlation between latent dynamics and LFPs was frequency-dependent and varied across regions. However, for any given region, this relationship remained stable throughout the behaviour: in each of primary motor and premotor cortices, the LFP-latent dynamics correlation profile was remarkably similar between movement planning and execution. These robust associations between LFPs and neural population latent dynamics help bridge the wealth of studies reporting neural correlates of behaviour using either type of recordings.

Epidural electrical stimulation of the cervical dorsal roots restores voluntary upper limb control in paralyzed monkeys

Regaining arm control is a top priority for people with paralysis. Unfortunately, the complexity of the neural mechanisms underlying arm control has limited the effectiveness of neurotechnology approaches. Here, we exploited the neural function of surviving spinal circuits to restore voluntary arm and hand control in three monkeys with spinal cord injury, using spinal cord stimulation. Our neural interface leverages the functional organization of the dorsal roots to convey artificial excitation via electrical stimulation to relevant spinal segments at appropriate movement phases. Stimulation bursts targeting specific spinal segments produced sustained arm movements, enabling monkeys with arm paralysis to perform an unconstrained reach-and-grasp task. Stimulation specifically improved strength, task performances and movement quality. Electrophysiology suggested that residual descending inputs were necessary to produce coordinated movements. The efficacy and reliability of our approach hold realistic promises of clinical translation.

Rethinking brain-wide interactions through multi-region 'network of networks' models

The neural control of behavior is distributed across many functionally and anatomically distinct brain regions even in small nervous systems. While classical neuroscience models treated these regions as a set of hierarchically isolated nodes, the brain comprises a recurrently interconnected network in which each region is intimately modulated by many others. Uncovering these interactions is now possible through experimental techniques that access large neural populations from many brain regions simultaneously. Harnessing these large-scale datasets, however, requires new theoretical approaches. Here, we review recent work to understand brain-wide interactions using multi-region 'network of networks' models and discuss how they can guide future experiments. We also emphasize the importance of multi-region recordings, and posit that studying individual components in isolation will be insufficient to understand the neural basis of behavior.

Long-term stability of cortical population dynamics underlying consistent behavior

Animals readily execute learned behaviors in a consistent manner over long periods of time, and yet no equally stable neural correlate has been demonstrated. How does the cortex achieve this stable control? Using the sensorimotor system as a model of cortical processing, we investigated the hypothesis that the dynamics of neural latent activity, which captures the dominant co-variation patterns within the neural population, must be preserved across time. We recorded from populations of neurons in premotor, primary motor and somatosensory cortices as monkeys performed a reaching task, for up to 2 years. Intriguingly, despite a steady turnover in the recorded neurons, the low-dimensional latent dynamics remained stable. The stability allowed reliable decoding of behavioral features for the entire timespan, while fixed decoders based directly on the recorded neural activity degraded substantially. We posit that stable latent cortical dynamics within the manifold are the fundamental building blocks underlying consistent behavioral execution.

A versatile robotic platform for the design of natural, three-dimensional reaching and grasping tasks in monkeys

OBJECTIVE

Translational studies on motor control and neurological disorders require detailed monitoring of sensorimotor components of natural limb movements in relevant animal models. However, available experimental tools do not provide a sufficiently rich repertoire of behavioral signals. Here, we developed a robotic platform that enables the monitoring of kinematics, interaction forces, and neurophysiological signals during user-defined upper limb tasks for monkeys.

APPROACH

We configured the platform to position instrumented objects in a three-dimensional workspace and provide an interactive dynamic force-field.

MAIN RESULTS

We show the relevance of our platform for fundamental and translational studies with three example applications. First, we study the kinematics of natural grasp in response to variable interaction forces. We then show simultaneous and independent encoding of kinematic and forces in single unit intra-cortical recordings from sensorimotor cortical areas. Lastly, we demonstrate the relevance of our platform to develop clinically relevant brain computer interfaces in a kinematically unconstrained motor task.

SIGNIFICANCE

Our versatile control structure does not depend on the specific robotic arm used and allows for the design and implementation of a variety of tasks that can support both fundamental and translational studies of motor control.

A Neural Population Mechanism for Rapid Learning

Long-term learning of language, mathematics, and motor skills likely requires cortical plasticity, but behavior often requires much faster changes, sometimes even after single errors. Here, we propose one neural mechanism to rapidly develop new motor output without altering the functional connectivity within or between cortical areas. We tested cortico-cortical models relating the activity of hundreds of neurons in the premotor (PMd) and primary motor (M1) cortices throughout adaptation to reaching movement perturbations. We found a signature of learning in the "output-null" subspace of PMd with respect to M1 reflecting the ability of premotor cortex to alter preparatory activity without directly influencing M1. The output-null subspace planning activity evolved with adaptation, yet the "output-potent" mapping that captures information sent to M1 was preserved. Our results illustrate a population-level cortical mechanism to progressively adjust the output from one brain area to its downstream structures that could be exploited for rapid behavioral adaptation.

Cortical population activity within a preserved neural manifold underlies multiple motor behaviors

Populations of cortical neurons flexibly perform different functions; for the primary motor cortex (M1) this means a rich repertoire of motor behaviors. We investigate the flexibility of M1 movement control by analyzing neural population activity during a variety of skilled wrist and reach-to-grasp tasks. We compare across tasks the neural modes that capture dominant neural covariance patterns during each task. While each task requires different patterns of muscle and single unit activity, we find unexpected similarities at the neural population level: the structure and activity of the neural modes is largely preserved across tasks. Furthermore, we find two sets of neural modes with task-independent activity that capture, respectively, generic temporal features of the set of tasks and a task-independent mapping onto muscle activity. This system of flexibly combined, well-preserved neural modes may underlie the ability of M1 to learn and generate a wide-ranging behavioral repertoire.

Linear-nonlinear-time-warp-poisson models of neural activity

Prominent models of spike trains assume only one source of variability - stochastic (Poisson) spiking - when stimuli and behavior are fixed. However, spike trains may also reflect variability due to internal processes such as planning. For example, we can plan a movement at one point in time and execute it at some arbitrary later time. Neurons involved in planning may thus share an underlying time course that is not precisely locked to the actual movement. Here we combine the standard Linear-Nonlinear-Poisson (LNP) model with Dynamic Time Warping (DTW) to account for shared temporal variability. When applied to recordings from macaque premotor cortex, we find that time warping considerably improves predictions of neural activity. We suggest that such temporal variability is a widespread phenomenon in the brain which should be modeled.

A cryptography-based approach for movement decoding

Brain decoders use neural recordings to infer the activity or intent of a user. To train a decoder, one generally needs to infer the measured variables of interest (covariates) from simultaneously measured neural activity. However, there are cases for which obtaining supervised data is difficult or impossible. Here, we describe an approach for movement decoding that does not require access to simultaneously measured neural activity and motor outputs. We use the statistics of movement-much like cryptographers use the statistics of language-to find a mapping between neural activity and motor variables, and then align the distribution of decoder outputs with the typical distribution of motor outputs by minimizing their Kullback-Leibler divergence. By using datasets collected from the motor cortex of three non-human primates performing either a reaching task or an isometric force-production task, we show that the performance of such a distribution-alignment decoding algorithm is comparable to the performance of supervised approaches. Distribution-alignment decoding promises to broaden the set of potential applications of brain decoding.

Altered tuning in primary motor cortex does not account for behavioral adaptation during force field learning

Although primary motor cortex (M1) is intimately involved in the dynamics of limb movement, its inputs may be more closely related to higher-order aspects of movement and multi-modal sensory feedback. Motor learning is thought to result from the adaption of internal models that compute transformations between these representations. While the psychophysics of motor learning has been studied in many experiments, the particular role of M1 in the process remains the subject of debate. Studies of learning-related changes in the spatial tuning of M1 neurons have yielded conflicting results. To resolve the discrepancies, we recorded from M1 during curl field adaptation in a reaching task. Our results suggest that aside from the addition of the load itself, the relation of M1 to movement dynamics remains unchanged as monkeys adapt behaviorally. Accordingly, we implemented a musculoskeletal model to generate synthetic neural activity having a fixed dynamical relation to movement and showed that these simulated neurons reproduced the observed behavior of the recorded M1 neurons. The stable representation of movement dynamics in M1 suggests that behavioral changes are mediated through progressively altered recruitment of M1 neurons, while the output effect of those neurons remained largely unchanged.

Neural Manifolds for the Control of Movement

The analysis of neural dynamics in several brain cortices has consistently uncovered low-dimensional manifolds that capture a significant fraction of neural variability. These neural manifolds are spanned by specific patterns of correlated neural activity, the "neural modes." We discuss a model for neural control of movement in which the time-dependent activation of these neural modes is the generator of motor behavior. This manifold-based view of motor cortex may lead to a better understanding of how the brain controls movement.

Representation of Stimulus Speed and Direction in Vibrissal-Sensitive Regions of the Trigeminal Nuclei: A Comparison of Single Unit and Population Responses

The rat vibrissal (whisker) system is one of the oldest and most important models for the study of active tactile sensing and sensorimotor integration. It is well established that primary sensory neurons in the trigeminal ganglion respond to deflections of one and only one whisker, and that these neurons are strongly tuned for both the speed and direction of individual whisker deflections. During active whisking behavior, however, multiple whiskers will be deflected simultaneously. Very little is known about how neurons at central levels of the trigeminal pathway integrate direction and speed information across multiple whiskers. In the present work, we investigated speed and direction coding in the trigeminal brainstem nuclei, the first stage of neural processing that exhibits multi-whisker receptive fields. Specifically, we recorded both single-unit spikes and local field potentials from fifteen sites in spinal trigeminal nucleus interpolaris and oralis while systematically varying the speed and direction of coherent whisker deflections delivered across the whisker array. For 12/15 neurons, spike rate was higher when the whisker array was stimulated from caudal to rostral rather than rostral to caudal. In addition, 10/15 neurons exhibited higher firing rates for slower stimulus speeds. Interestingly, using a simple decoding strategy for the local field potentials and spike trains, classification of speed and direction was higher for field potentials than for single unit spike trains, suggesting that the field potential is a robust reflection of population activity. Taken together, these results point to the idea that population responses in these brainstem regions in the awake animal will be strongest during behaviors that stimulate a population of whiskers with a directionally coherent motion.