Cortical population activity within a preserved neural manifold underlies multiple motor behaviors

The cortical critical power law balances energy and information in an optimal fashion

A recent study has suggested that the stimulus responses of cortical neural populations follow a critical power law. More precisely, the power spectrum of the covariance matrix of neural responses follows a power law with an exponent indicating that the neural manifold lies on the edge of differentiability. This criticality is hypothesized to balance expressivity and robustness in neural encoding, as population responses on a nondifferential fractal manifold are thought to be overly sensitive to perturbations. However, contrary to this hypothesis, we prove that neural coding is far more robust than previously assumed. We develop a theoretical framework that provides an analytical expression for the Fisher information of population coding under the small noise assumption. Our results reveal that, due to its intrinsic high dimensionality, population coding maintains reliability even on a nondifferentiable fractal manifold, despite its sensitivity to perturbations. Furthermore, the theory reveals that the trade-off between energetic cost and information makes the critical power-law coding the optimal neural encoding of sensory information for a wide range of conditions. In this derivation, we highlight the essential role of a neural correlation, known as differential correlation, in power-law population coding. By uncovering the nontrivial nature of high-dimensional information coding, this work deepens our understanding of criticality and power laws in both biological and artificial neural computation.

Population coding of distinct categories of behavior in the frontal eye field

Brain regions frequently contribute to the control of a range of behaviors. To understand how a brain area controls multiple behaviors, we examined how the frontal eye field (FEF) encodes different eye movements by recording the activity of 1,200 neurons during smooth pursuit, pursuit suppression, and saccade tasks in two female Macaca fascicularis monkeys. Single neurons tended to respond on all tasks. In the absence of task-specific clusters, we analyzed the relationships in directional preference between tasks. The tuning curves during the pursuit and suppression tasks were strongly correlated, unlike the correlations between the pursuit and saccade tasks that were considerably weaker. To study the implications of single neuron coding, we examined the patterns of population activity on the three tasks. We identified the low-dimensional subspaces that captured the most variance in population activity during each task and quantified the extent of overlap between these spaces. The absence of overlap between the subspaces spanned by population activity on the pursuit and saccades tasks prompted an independent linear readout of these tasks. Conversely, pursuit and pursuit suppression showed substantial overlap in their population activity subspaces. This overlap emphasized the predominance of visual motion in pursuit encoding and indicated that the linear readouts accounting for a large part of the variability in the pursuit tasks cannot completely attenuate the activity during suppression. Overall, these results imply that at the population level, FEF is organized predominantly along sensory rather than motor parameters.NEW & NOTEWORTHY We investigated how the frontal eye field (FEF) encodes smooth pursuit, pursuit suppression, and saccade in monkeys. Tuning curves were highly correlated between pursuit and suppression, but the correlation was much weaker in pursuit and saccade. Pursuit and saccade occupied orthogonal subspaces, indicating independent tuning, whereas pursuit and suppression overlapped substantially, emphasizing the predominance of visual motion in pursuit encoding. These findings contribute to a better understanding of the mechanisms underlying the FEF's ability to control multiple behaviors.

Assistive sensory-motor perturbations influence learned neural representations

Task errors are used to learn and refine motor skills. We investigated how task assistance influences learned neural representations using Brain-Computer Interfaces (BCIs), which map neural activity into movement via a decoder. We analyzed motor cortex activity as monkeys practiced BCI with a decoder that adapted to improve or maintain performance over days. Over time, task-relevant information became concentrated in fewer neurons, unlike with fixed decoders. At the population level, task information also became largely confined to a few neural modes that accounted for an unexpectedly small fraction of the population variance. A neural network model suggests the adaptive decoders directly contribute to forming these more compact neural representations. Our findings show that assistive decoders manipulate error information used for long-term learning computations like credit assignment, which informs our understanding of motor learning and has implications for designing real-world BCIs.

TRACKING THE TOPOLOGY OF NEURAL MANIFOLDS ACROSS POPULATIONS

Neural manifolds summarize the intrinsic structure of the information encoded by a population of neurons. Advances in experimental techniques have made simultaneous recordings from multiple brain regions increasingly commonplace, raising the possibility of studying how these manifolds relate across populations. However, when the manifolds are nonlinear and possibly code for multiple unknown variables, it is challenging to extract robust and falsifiable information about their relationships. We introduce a framework, called the method of analogous cycles, for matching topological features of neural manifolds using only observed dissimilarity matrices within and between neural populations. We demonstrate via analysis of simulations and in vivo experimental data that this method can be used to correctly identify multiple shared circular coordinate systems across both stimuli and inferred neural manifolds. Conversely, the method rejects matching features that are not intrinsic to one of the systems. Further, as this method is deterministic and does not rely on dimensionality reduction or optimization methods, it is amenable to direct mathematical investigation and interpretation in terms of the underlying neural activity. We thus propose the method of analogous cycles as a suitable foundation for a theory of cross-population analysis via neural manifolds.

Cerebellar output shapes cortical preparatory activity during motor adaptation

The cerebellum plays a key role in motor adaptation by driving trial-to-trial recalibration of movements based on previous errors. In primates, cortical correlates of adaptation are encoded already in the pre-movement motor plan, but these early cortical signals could be driven by a cerebellar-to-cortical information flow or evolve independently through intracortical mechanisms. To address this question, we trained female macaque monkeys to reach against a viscous force field (FF) while blocking cerebellar outflow. The cerebellar block led to impaired FF adaptation and a compensatory, re-aiming-like shift in motor cortical preparatory activity. In the null-field conditions, the cerebellar block altered neural preparatory activity by increasing task-representation dimensionality and impeding generalization. A computational model indicated that low-dimensional (cerebellar-like) feedback is sufficient to replicate these findings. We conclude that cerebellar signals carry task structure information that constrains the dimensionality of the cortical preparatory manifold and promotes generalization. In the absence of these signals, cortical mechanisms are harnessed to partially restore adaptation.

Facial gestures are enacted via a cortical hierarchy of dynamic and stable codes

Successful communication requires the generation and perception of a shared set of signals. Facial gestures are one fundamental set of communicative behaviors in primates, generated through the dynamic arrangement of dozens of fine muscles. While much progress has been made uncovering the neural mechanisms of face perception, little is known about those controlling facial gesture production. Commensurate with the importance of facial gestures in daily social life, anatomical work has shown that facial muscles are under direct control from multiple cortical regions, including primary and premotor in lateral frontal cortex, and cingulate in medial frontal cortex. Furthermore, neuropsychological evidence from focal lesion patients has suggested that lateral cortex controls voluntary movements, and medial emotional expressions. Here we show that lateral and medial cortical face motor regions encode both types of gestures. They do so through unique temporal activity patterns, distinguishable well-prior to movement onset. During gesture production, cortical regions encoded facial kinematics in a context-dependent manner. Our results show how cortical regions projecting in parallel downstream, but each situated at a different level of a posterior-anterior hierarchy form a continuum of gesture coding from dynamic to temporally stable, in order to produce context-related, coherent motor outputs during social communication.

Developmental decorrelation of local cortical activity through adolescence supports high-dimensional encoding and working memory

Adolescence is a key period for the maturation of cognitive control during which cortical circuitry is refined through processes such as synaptic pruning, but how these refinements modulate local functional dynamics to support cognition remains only partially characterized. Here, we used data from a longitudinal, adolescent cohort (N = 134 individuals ages 10-31 years, N = 202 total sessions) that completed MRI scans at ultra-high field (7 Tesla). We used resting state fMRI data to compute surface-based regional homogeneity (ReHo)-a measure of time-dependent correlations in fMRI activity between a vertex and its immediate neighbors-as an index of local functional connectivity across the cortex. We found widespread decreases in ReHo, suggesting increasing heterogeneity and specialization of functional circuits through adolescence. Decreases in ReHo included a spatial component which overlapped with sensorimotor and cingulo-opercular networks, in which ReHo decreases were associated with developmental stabilization of working memory performance. We show that decreases in ReHo are associated with higher intrinsic coding dimensionality, demonstrating how functional specialization of these circuits may confer computational benefits by facilitating increased capacity for encoding information. These results suggest a remodeling of cortical activity in adolescence through which local functional circuits become increasingly specialized, higher-dimensional, and more capable of supporting adult-like cognitive functioning.

Cerebellar output shapes cortical preparatory activity during motor adaptation

The cerebellum plays a key role in motor adaptation by driving trial-to-trial recalibration of movements based on previous errors. In primates, cortical correlates of adaptation are encoded already in the pre-movement motor plan, but these early cortical signals could be driven by a cerebellar-to-cortical information flow or evolve independently through intracortical mechanisms. To address this question, we trained female macaque monkeys to reach against a viscous force field (FF) while blocking cerebellar outflow. The cerebellar block led to impaired FF adaptation and a compensatory, re-aiming-like shift in motor cortical preparatory activity. In the null-field conditions, the cerebellar block altered neural preparatory activity by increasing task-representation dimensionality and impeding generalization. A computational model indicated that low-dimensional (cerebellar-like) feedback is sufficient to replicate these findings. We conclude that cerebellar signals carry task structure information that constrains the dimensionality of the cortical preparatory manifold and promotes generalization. In the absence of these signals, cortical mechanisms are harnessed to partially restore adaptation.

Conjoint specification of action by neocortex and striatum

The interplay between two major forebrain structures-cortex and subcortical striatum-is critical for flexible, goal-directed action. Traditionally, it has been proposed that striatum is critical for selecting what type of action is initiated, while the primary motor cortex is involved in specifying the continuous parameters of an upcoming/ongoing movement. Recent data indicate that striatum may also be involved in specification. These alternatives have been difficult to reconcile because comparing very distinct actions, as is often done, makes essentially indistinguishable predictions. Here, we develop quantitative models to reveal a somewhat paradoxical insight: only comparing neural activity across similar actions makes strongly distinguishing predictions. We thus developed a novel reach-to-pull task in which mice reliably selected between two similar but distinct reach targets and pull forces. Simultaneous cortical and subcortical recordings were uniquely consistent with a model in which cortex and striatum jointly specify continuous parameters governing movement execution.

Orthogonal neural representations support perceptual judgments of natural stimuli

In natural visually guided behavior, observers must separate relevant information from a barrage of irrelevant information. Many studies have investigated the neural underpinnings of this ability using artificial stimuli presented on blank backgrounds. Natural images, however, contain task-irrelevant background elements that might interfere with the perception of object features. Recent studies suggest that visual feature estimation can be modeled through the linear decoding of task-relevant information from visual cortex. So, if the representations of task-relevant and irrelevant features are not orthogonal in the neural population, then variation in the task-irrelevant features would impair task performance. We tested this hypothesis using human psychophysics and monkey neurophysiology combined with parametrically variable naturalistic stimuli. We demonstrate that (1) the neural representation of one feature (the position of an object) in visual area V4 is orthogonal to those of several background features, (2) the ability of human observers to precisely judge object position was largely unaffected by those background features, and (3) many features of the object and the background (and of objects from a separate stimulus set) are orthogonally represented in V4 neural population responses. Our observations are consistent with the hypothesis that orthogonal neural representations can support stable perception of object features despite the richness of natural visual scenes.

Contextual neural dynamics during time perception in the primate ventral premotor cortex

Understanding how time perception adapts to cognitive demands remains a significant challenge. In some contexts, the brain encodes time categorically (as "long" or "short"), while in others, it encodes precise time intervals on a continuous scale. Although the ventral premotor cortex (VPC) is known for its role in complex temporal processes, such as speech, its specific involvement in time estimation remains underexplored. In this study, we investigated how the VPC processes temporal information during a time interval comparison task (TICT) and a time interval categorization task (TCT) in primates. We found a notable heterogeneity in neuronal responses associated with time perception across both tasks. While most neurons responded during time interval presentation, a smaller subset retained this information during the working memory periods. Population-level analysis revealed distinct dynamics between tasks: In the TICT, population activity exhibited a linear and parametric relationship with interval duration, whereas in the TCT, neuronal activity diverged into two distinct dynamics corresponding to the interval categories. During delay periods, these categorical or parametric representations remained consistent within each task context. This contextual shift underscores the VPC's adaptive role in interval estimation and highlights how temporal representations are modulated by cognitive demands.

An emerging view of neural geometry in motor cortex supports high-performance decoding

Decoders for brain-computer interfaces (BCIs) assume constraints on neural activity, chosen to reflect scientific beliefs while yielding tractable computations. Recent scientific advances suggest that the true constraints on neural activity, especially its geometry, may be quite different from those assumed by most decoders. We designed a decoder, MINT, to embrace statistical constraints that are potentially more appropriate. If those constraints are accurate, MINT should outperform standard methods that explicitly make different assumptions. Additionally, MINT should be competitive with expressive machine learning methods that can implicitly learn constraints from data. MINT performed well across tasks, suggesting its assumptions are well-matched to the data. MINT outperformed other interpretable methods in every comparison we made. MINT outperformed expressive machine learning methods in 37 of 42 comparisons. MINT's computations are simple, scale favorably with increasing neuron counts, and yield interpretable quantities such as data likelihoods. MINT's performance and simplicity suggest it may be a strong candidate for many BCI applications.

Exploring Synergies in Brain-Machine Interfaces: Compression vs. Performance

Individuals with severe neurological injuries often rely on assistive technologies, but current methods have limitations in accurately decoding multi-degree-of-freedom (DoF) movements. Intracortical brain-machine interfaces (iBMIs) use neural signals to provide a more natural control method, but currently struggle with higher-DoF movements-something the brain handles effortlessly. It has been theorized that the brain simplifies high-DoF movement through muscle synergies, which link multiple muscles to function as a single unit. These synergies have been studied using dimensionality reduction techniques like principal component analysis (PCA), non-negative matrix factorization (NMF), and demixed PCA (dPCA) and successfully used to reduce noise and improve offline decoder stability in non-invasive applications. However, their effectiveness in improving decoding and generalizability for implanted recordings across varied tasks is unclear. Here, we evaluated if brain and muscle synergies can enhance iBMI performance in non-human primates performing a two-DoF finger task. Specifically, we tested if PCA, dPCA, and NMF could compress and denoise brain and muscle data and improve decoder generalization across tasks. Our results showed that while all methods effectively compressed data with minimal loss in decoding accuracy, none improved performance through denoising. Additionally, none of the methods enhanced generalization across tasks. These findings suggest that while dimensionality reduction can aid data compression, alone it may not reveal the "true" control space needed to improve decoder performance or generalizability. Further research is required to determine whether synergies are the optimal control framework or if alternative approaches are required to enhance decoder robustness in iBMI applications.

Sensorimotor environment but not task rule reconfigures population dynamics in rhesus monkey posterior parietal cortex

Primates excel at mapping sensory inputs flexibly onto motor outcomes. We asked if the neural dynamics to support context-sensitive sensorimotor mapping generalizes or differs between different behavioral contexts that demand such flexibility. We compared reaching under mirror-reversed vision, a case of adaptation to a modified sensorimotor environment (SE), with anti reaching, a case of applying an abstract task rule (TR). While neural dynamics in monkey posterior parietal cortex show shifted initial states and non-aligned low-dimensional neural subspaces in the SE task, remapping is achieved in overlapping subspaces in the TR task. A recurrent neural network model demonstrates how output constraints mimicking SE and TR tasks are sufficient to generate the two fundamentally different neural computational dynamics. We conclude that sensorimotor remapping to implement an abstract task rule happens within the existing repertoire of neural dynamics, while compensation of perturbed sensory feedback requires exploration of independent neural dynamics in parietal cortex.

Unsupervised, piecewise linear decoding enables an accurate prediction of muscle activity in a multi-task brain computer interface

Objective.Creating an intracortical brain computer interface (iBCI) capable of seamless transitions between tasks and contexts would greatly enhance user experience. However, the nonlinearity in neural activity presents challenges to computing a global iBCI decoder. We aimed to develop a method that differs from a globally optimized decoder to address this issue.Approach.We devised an unsupervised approach that relies on the structure of a low-dimensional neural manifold to implement a piecewise linear decoder. We created a distinctive dataset in which monkeys performed a diverse set of tasks, some trained, others innate, while we recorded neural signals from the motor cortex (M1) and electromyographs (EMGs) from upper limb muscles. We used both linear and nonlinear dimensionality reduction techniques to discover neural manifolds and applied unsupervised algorithms to identify clusters within those spaces. Finally, we fit a linear decoder of EMG for each cluster. A specific decoder was activated corresponding to the cluster each new neural data point belonged to.Main results.We found clusters in the neural manifolds corresponding with the different tasks or task sub-phases. The performance of piecewise decoding improved as the number of clusters increased and plateaued gradually. With only two clusters it already outperformed a global linear decoder, and unexpectedly, it outperformed even a global recurrent neural network decoder with 10-12 clusters.Significance.This study introduced a computationally lightweight solution for creating iBCI decoders that can function effectively across a broad range of tasks. EMG decoding is particularly challenging, as muscle activity is used, under varying contexts, to control interaction forces and limb stiffness, as well as motion. The results suggest that a piecewise linear decoder can provide a good approximation to the nonlinearity between neural activity and motor outputs, a result of our increased understanding of the structure of neural manifolds in motor cortex.

Neuroethology of natural actions in freely moving monkeys

The current understanding of primate natural action organization derives from laboratory experiments in restrained contexts (RCs) under the assumption that this knowledge generalizes to freely moving contexts (FMCs). In this work, we developed a neurobehavioral platform to enable wireless recording of the same premotor neurons in both RCs and FMCs. Neurons often encoded the same hand and mouth actions differently in RCs and FMCs. Furthermore, in FMCs, we identified cells that selectively encoded actions untestable during RCs and others that displayed mixed selectivity for multiple actions, which is compatible with an organization based on cortical motor synergies at different levels of complexity. Cross-context decoding demonstrated that neural activity in FMCs is richer and more generalizable than in RCs, which suggests that neuroethological approaches are better suited to unveil the neural bases of behavior.

Experience-driven development of decision-related representations in the auditory cortex

Associating sensory stimuli with behavioral significance induces substantial changes in stimulus representations. Recent studies suggest that primary sensory cortices not only adjust representations of task-relevant stimuli, but actively participate in encoding features of the decision-making process. We sought to determine whether this trait is innate in sensory cortices or if choice representation develops with time and experience. To trace choice representation development, we perform chronic two-photon calcium imaging in the primary auditory cortex of head-fixed mice while they gain experience in a tone detection task with a delayed decision window. Our results reveal a progressive increase in choice-dependent activity within a specific subpopulation of neurons, aligning with growing task familiarity and adapting to changing task rules. Furthermore, task experience correlates with heightened synchronized activity in these populations and the ability to differentiate between different types of behavioral decisions. Notably, the activity of this subpopulation accurately decodes the same action at different task phases. Our findings establish a dynamic restructuring of population activity in the auditory cortex to encode features of the decision-making process that develop over time and refines with experience.

Latent learning drives sleep-dependent plasticity in distinct CA1 subpopulations

Latent learning is a process that enables the brain to transform experiences into "cognitive maps," a form of implicit memory, without requiring reinforced training. To investigate its neural mechanisms, we record from hippocampal neurons in mice during latent learning of spatial maps and observe that the high-dimensional neural state space gradually transforms into a low-dimensional manifold that closely resembles the physical environment. This transformation process is associated with the neural reactivation of navigational experiences during sleep. Additionally, we identify a subset of hippocampal neurons that, rather than forming place fields in a novel environment, maintain weak spatial tuning but gradually develop correlated activity with other neurons. The elevated correlation introduces redundancy into the ensemble code, transforming the neural state space into a low-dimensional manifold that effectively links discrete place fields of place cells into a map-like structure. These results suggest a potential mechanism for latent learning of spatial maps in the hippocampus.

Spatial context non-uniformly modulates inter-laminar information flow in the primary visual cortex

Our visual experience is a result of the concerted activity of neuronal ensembles in the sensory hierarchy. Yet, how the spatial organization of objects influences this activity remains poorly understood. We investigate how inter-laminar information flow within the primary visual cortex (V1) is affected by visual stimuli in isolation or with flankers at spatial configurations that are known to cause non-uniform degradation of perception. By employing dimensionality reduction approaches to simultaneous, layer-specific population recordings, we establish that information propagation between cortical layers occurs along a structurally stable communication subspace. The spatial configuration of contextual stimuli differentially modulates inter-laminar communication efficacy, the balance of feedforward and effective feedback signaling, and contextual signaling in the superficial layers. Remarkably, these modulations mirror the spatially non-uniform aspects of perceptual degradation. Our results suggest a model of retinotopically non-uniform cortical connectivity in the output layers of V1 that influences information flow in the sensory hierarchy.

Aligned and oblique dynamics in recurrent neural networks

The relation between neural activity and behaviorally relevant variables is at the heart of neuroscience research. When strong, this relation is termed a neural representation. There is increasing evidence, however, for partial dissociations between activity in an area and relevant external variables. While many explanations have been proposed, a theoretical framework for the relationship between external and internal variables is lacking. Here, we utilize recurrent neural networks (RNNs) to explore the question of when and how neural dynamics and the network's output are related from a geometrical point of view. We find that training RNNs can lead to two dynamical regimes: dynamics can either be aligned with the directions that generate output variables, or oblique to them. We show that the choice of readout weight magnitude before training can serve as a control knob between the regimes, similar to recent findings in feedforward networks. These regimes are functionally distinct. Oblique networks are more heterogeneous and suppress noise in their output directions. They are furthermore more robust to perturbations along the output directions. Crucially, the oblique regime is specific to recurrent (but not feedforward) networks, arising from dynamical stability considerations. Finally, we show that tendencies toward the aligned or the oblique regime can be dissociated in neural recordings. Altogether, our results open a new perspective for interpreting neural activity by relating network dynamics and their output.

Tracking the topology of neural manifolds across populations

Neural manifolds summarize the intrinsic structure of the information encoded by a population of neurons. Advances in experimental techniques have made simultaneous recordings from multiple brain regions increasingly commonplace, raising the possibility of studying how these manifolds relate across populations. However, when the manifolds are nonlinear and possibly code for multiple unknown variables, it is challenging to extract robust and falsifiable information about their relationships. We introduce a framework, called the method of analogous cycles, for matching topological features of neural manifolds using only observed dissimilarity matrices within and between neural populations. We demonstrate via analysis of simulations and in vivo experimental data that this method can be used to correctly identify multiple shared circular coordinate systems across both stimuli and inferred neural manifolds. Conversely, the method rejects matching features that are not intrinsic to one of the systems. Further, as this method is deterministic and does not rely on dimensionality reduction or optimization methods, it is amenable to direct mathematical investigation and interpretation in terms of the underlying neural activity. We thus propose the method of analogous cycles as a suitable foundation for a theory of cross-population analysis via neural manifolds.

Few-shot Algorithms for Consistent Neural Decoding (FALCON) Benchmark

Intracortical brain-computer interfaces (iBCIs) can restore movement and communication abilities to individuals with paralysis by decoding their intended behavior from neural activity recorded with an implanted device. While this activity yields high-performance decoding over short timescales, neural data are often nonstationary, which can lead to decoder failure if not accounted for. To maintain performance, users must frequently recalibrate decoders, which requires the arduous collection of new neural and behavioral data. Aiming to reduce this burden, several approaches have been developed that either limit recalibration data requirements (few-shot approaches) or eliminate explicit recalibration entirely (zero-shot approaches). However, progress is limited by a lack of standardized datasets and comparison metrics, causing methods to be compared in an ad hoc manner. Here we introduce the FALCON benchmark suite (Few-shot Algorithms for COnsistent Neural decoding) to standardize evaluation of iBCI robustness. FALCON curates five datasets of neural and behavioral data that span movement and communication tasks to focus on behaviors of interest to modern-day iBCIs. Each dataset includes calibration data, optional few-shot recalibration data, and private evaluation data. We implement a flexible evaluation platform which only requires user-submitted code to return behavioral predictions on unseen data. We also seed the benchmark by applying baseline methods spanning several classes of possible approaches. FALCON aims to provide rigorous selection criteria for robust iBCI decoders, easing their translation to real-world devices.

Flexible control of sequence working memory in the macaque frontal cortex

To memorize a sequence, one must serially bind each item to its rank order. How the brain controls a given input to bind its associated order in sequence working memory (SWM) remains unexplored. Here, we investigated the neural representations underlying SWM control using electrophysiological recordings in the frontal cortex of macaque monkeys performing forward and backward SWM tasks. Separate and generalizable low-dimensional subspaces for sensory and memory information were found within the same frontal circuitry, and SWM control was reflected in these neural subspaces' organized dynamics. Each item at each rank was sequentially entered into a common sensory subspace and, depending on forward or backward task requirement, flexibly and timely sent into rank-selective SWM subspaces. Neural activity in these SWM subspaces faithfully predicted the recalled item and order information in single error trials. Thus, compositional neural population codes with well-orchestrated dynamics in frontal cortex support the flexible control of SWM.

Global motor dynamics - Invariant neural representations of motor behavior in distributed brain-wide recordings

Objective.Motor-related neural activity is more widespread than previously thought, as pervasive brain-wide neural correlates of motor behavior have been reported in various animal species. Brain-wide movement-related neural activity have been observed in individual brain areas in humans as well, but it is unknown to what extent global patterns exist.Approach.Here, we use a decoding approach to capture and characterize brain-wide neural correlates of movement. We recorded invasive electrophysiological data from stereotactic electroencephalographic electrodes implanted in eight epilepsy patients who performed both an executed and imagined grasping task. Combined, these electrodes cover the whole brain, including deeper structures such as the hippocampus, insula and basal ganglia. We extract a low-dimensional representation and classify movement from rest trials using a Riemannian decoder.Main results.We reveal global neural dynamics that are predictive across tasks and participants. Using an ablation analysis, we demonstrate that these dynamics remain remarkably stable under loss of information. Similarly, the dynamics remain stable across participants, as we were able to predict movement across participants using transfer learning.Significance.Our results show that decodable global motor-related neural dynamics exist within a low-dimensional space. The dynamics are predictive of movement, nearly brain-wide and present in all our participants. The results broaden the scope to brain-wide investigations, and may allow combining datasets of multiple participants with varying electrode locations or calibrationless neural decoder.

Measuring instability in chronic human intracortical neural recordings towards stable, long-term brain-computer interfaces

Intracortical brain-computer interfaces (iBCIs) enable people with tetraplegia to gain intuitive cursor control from movement intentions. To translate to practical use, iBCIs should provide reliable performance for extended periods of time. However, performance begins to degrade as the relationship between kinematic intention and recorded neural activity shifts compared to when the decoder was initially trained. In addition to developing decoders to better handle long-term instability, identifying when to recalibrate will also optimize performance. We propose a method, "MINDFUL", to measure instabilities in neural data for useful long-term iBCI, without needing labels of user intentions. Longitudinal data were analyzed from two BrainGate2 participants with tetraplegia as they used fixed decoders to control a computer cursor spanning 142 days and 28 days, respectively. We demonstrate a measure of instability that correlates with changes in closed-loop cursor performance solely based on the recorded neural activity (Pearson r = 0.93 and 0.72, respectively). This result suggests a strategy to infer online iBCI performance from neural data alone and to determine when recalibration should take place for practical long-term use.

Exploiting correlations across trials and behavioral sessions to improve neural decoding

Traditional neural decoders model the relationship between neural activity and behavior within individual trials of a single experimental session, neglecting correlations across trials and sessions. However, animals exhibit similar neural activities when performing the same behavioral task, and their behaviors are influenced by past experiences from previous trials. To exploit these informative correlations in large datasets, we introduce two complementary models: a multi-session reduced-rank model that shares similar behaviorally-relevant statistical structure in neural activity across sessions to improve decoding, and a multi-session state-space model that shares similar behavioral statistical structure across trials and sessions. Applied across 433 sessions spanning 270 brain regions in the International Brain Laboratory public mouse Neuropixels dataset, our decoders demonstrate improved decoding accuracy for four distinct behaviors compared to traditional approaches. Unlike existing deep learning approaches, our models are interpretable and efficient, uncovering latent behavioral dynamics that govern animal decision-making, quantifying single-neuron contributions to decoding behaviors, and identifying different activation timescales of neural activity across the brain. Code: https://github.com/yzhang511/neural_decoding.

De novo sensorimotor learning through reuse of movement components

From tying one's shoelaces to driving a car, complex skills involving the coordination of multiple muscles are common in everyday life; yet relatively little is known about how these skills are learned. Recent studies have shown that new sensorimotor skills involving re-mapping familiar body movements to unfamiliar outputs cannot be learned by adjusting pre-existing controllers, and that new task-specific controllers must instead be learned "de novo". To date, however, few studies have investigated de novo learning in scenarios requiring continuous and coordinated control of relatively unpractised body movements. In this study, we used a myoelectric interface to investigate how a novel controller is learned when the task involves an unpractised combination of relatively untrained continuous muscle contractions. Over five sessions on five consecutive days, participants learned to trace a series of trajectories using a computer cursor controlled by the activation of two muscles. The timing of the generated cursor trajectory and its shape relative to the target improved for conditions trained with post-trial visual feedback. Improvements in timing transferred to all untrained conditions, but improvements in shape transferred less robustly to untrained conditions requiring the trained order of muscle activation. All muscle outputs in the final session could already be generated during the first session, suggesting that participants learned the new task by improving the selection of existing motor commands. These results suggest that the novel controllers acquired during de novo learning can, in some circumstances, be constructed from components of existing controllers.

Rapid context inference in a thalamocortical model using recurrent neural networks

Cognitive flexibility is a fundamental ability that enables humans and animals to exhibit appropriate behaviors in various contexts. The thalamocortical interactions between the prefrontal cortex (PFC) and the mediodorsal thalamus (MD) have been identified as crucial for inferring temporal context, a critical component of cognitive flexibility. However, the neural mechanism responsible for context inference remains unknown. To address this issue, we propose a PFC-MD neural circuit model that utilizes a Hebbian plasticity rule to support rapid, online context inference. Specifically, the model MD thalamus can infer temporal contexts from prefrontal inputs within a few trials. This is achieved through the use of PFC-to-MD synaptic plasticity with pre-synaptic traces and adaptive thresholding, along with winner-take-all normalization in the MD. Furthermore, our model thalamus gates context-irrelevant neurons in the PFC, thus facilitating continual learning. We evaluate our model performance by having it sequentially learn various cognitive tasks. Incorporating an MD-like component alleviates catastrophic forgetting of previously learned contexts and demonstrates the transfer of knowledge to future contexts. Our work provides insight into how biological properties of thalamocortical circuits can be leveraged to achieve rapid context inference and continual learning.

A computational mechanism of cue-stimulus integration for pain in the brain

The brain integrates information from pain-predictive cues and noxious inputs to construct the pain experience. Although previous studies have identified neural encodings of individual pain components, how they are integrated remains elusive. Here, using a cue-induced pain task, we examined temporal functional magnetic resonance imaging activities within the state space, where axes represent individual voxel activities. By analyzing the features of these activities at the large-scale network level, we demonstrated that overall brain networks preserve both cue and stimulus information in their respective subspaces within the state space. However, only higher-order brain networks, including limbic and default mode networks, could reconstruct the pattern of participants' reported pain by linear summation of subspace activities, providing evidence for the integration of cue and stimulus information. These results suggest a hierarchical organization of the brain for processing pain components and elucidate the mechanism for their integration underlying our pain perception.

Unsupervised, piecewise linear decoding enables an accurate prediction of muscle activity in a multi-task brain computer interface

Objective

Creating an intracortical brain-computer interface (iBCI) capable of seamless transitions between tasks and contexts would greatly enhance user experience. However, the nonlinearity in neural activity presents challenges to computing a global iBCI decoder. We aimed to develop a method that differs from a globally optimized decoder to address this issue.

Approach

We devised an unsupervised approach that relies on the structure of a low-dimensional neural manifold to implement a piecewise linear decoder. We created a distinctive dataset in which monkeys performed a diverse set of tasks, some trained, others innate, while we recorded neural signals from the motor cortex (M1) and electromyographs (EMGs) from upper limb muscles. We used both linear and nonlinear dimensionality reduction techniques to discover neural manifolds and applied unsupervised algorithms to identify clusters within those spaces. Finally, we fit a linear decoder of EMG for each cluster. A specific decoder was activated corresponding to the cluster each new neural data point belonged to.

Main results

We found clusters in the neural manifolds corresponding with the different tasks or task sub-phases. The performance of piecewise decoding improved as the number of clusters increased and plateaued gradually. With only two clusters it already outperformed a global linear decoder, and unexpectedly, it outperformed even a global recurrent neural network (RNN) decoder with 10-12 clusters.

Significance

This study introduced a computationally lightweight solution for creating iBCI decoders that can function effectively across a broad range of tasks. EMG decoding is particularly challenging, as muscle activity is used, under varying contexts, to control interaction forces and limb stiffness, as well as motion. The results suggest that a piecewise linear decoder can provide a good approximation to the nonlinearity between neural activity and motor outputs, a result of our increased understanding of the structure of neural manifolds in motor cortex.

Motor Cortex Latent Dynamics Encode Spatial and Temporal Arm Movement Parameters Independently

The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where three male monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show that this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, and also argue that not all parameters of movement are defined by different trajectories of population activity.

Revealing unexpected complex encoding but simple decoding mechanisms in motor cortex via separating behaviorally relevant neural signals

In motor cortex, behaviorally relevant neural responses are entangled with irrelevant signals, which complicates the study of encoding and decoding mechanisms. It remains unclear whether behaviorally irrelevant signals could conceal some critical truth. One solution is to accurately separate behaviorally relevant and irrelevant signals at both single-neuron and single-trial levels, but this approach remains elusive due to the unknown ground truth of behaviorally relevant signals. Therefore, we propose a framework to define, extract, and validate behaviorally relevant signals. Analyzing separated signals in three monkeys performing different reaching tasks, we found neural responses previously considered to contain little information actually encode rich behavioral information in complex nonlinear ways. These responses are critical for neuronal redundancy and reveal movement behaviors occupy a higher-dimensional neural space than previously expected. Surprisingly, when incorporating often-ignored neural dimensions, behaviorally relevant signals can be decoded linearly with comparable performance to nonlinear decoding, suggesting linear readout may be performed in motor cortex. Our findings prompt that separating behaviorally relevant signals may help uncover more hidden cortical mechanisms.

Neural Manifold Constraint for Spike Prediction Models Under Behavioral Reinforcement

Spike prediction models effectively predict downstream spike trains from upstream neural activity for neural prostheses. Such prostheses could potentially restore damaged neural communication pathways using predicted patterns to guide electrical stimulations on downstream. Since the ground truth of downstream neural activity is unavailable for subjects with the damage, reinforcement learning (RL) with behavior-level rewards becomes necessary for model training. However, existing models do not involve any constraint on the generated firing patterns and neglect the correlations among neural activities. Thus, the model outputs can greatly deviate from the natural range of neural activities, causing concerns for clinical usage. This study proposes the neural manifold constraint to solve this problem, shaping RL-generated spike trains in the feature space. The constraint terms describe the first and second order statistics of the neural manifold estimated from neural recordings during subjects' freely moving period. Then, the models can be optimized within the neural manifold by behavioral reinforcement. We test the method to predict primary motor cortex (M1) spikes from medial prefrontal (mPFC) spikes when rats perform the two-lever discrimination task. Results show that the neural activity generated by constrained models resembles the real M1 recordings. Compared with models without constraints, our approach achieves similar behavioral success rates, but reduces the mean squared error of neural firing by 61%. The constraints also increase the model's robustness across data segments and induce realistic neural correlations. Our method provides a promising tool to restore transregional communication with high behavioral performance and more realistic microscopic patterns.

Neural manifolds in V1 change with top-down signals from V4 targeting the foveal region

High-dimensional brain activity is often organized into lower-dimensional neural manifolds. However, the neural manifolds of the visual cortex remain understudied. Here, we study large-scale multi-electrode electrophysiological recordings of macaque (Macaca mulatta) areas V1, V4, and DP with a high spatiotemporal resolution. We find that the population activity of V1 contains two separate neural manifolds, which correlate strongly with eye closure (eyes open/closed) and have distinct dimensionalities. Moreover, we find strong top-down signals from V4 to V1, particularly to the foveal region of V1, which are significantly stronger during the eyes-open periods. Finally, in silico simulations of a balanced spiking neuron network qualitatively reproduce the experimental findings. Taken together, our analyses and simulations suggest that top-down signals modulate the population activity of V1. We postulate that the top-down modulation during the eyes-open periods prepares V1 for fast and efficient visual responses, resulting in a type of visual stand-by state.

Neural population dynamics in songbird RA and HVC during learned motor-vocal behavior

Complex, learned motor behaviors involve the coordination of large-scale neural activity across multiple brain regions, but our understanding of the population-level dynamics within different regions tied to the same behavior remains limited. Here, we investigate the neural population dynamics underlying learned vocal production in awake-singing songbirds. We use Neuropixels probes to record the simultaneous extracellular activity of populations of neurons in two regions of the vocal motor pathway. In line with observations made in non-human primates during limb-based motor tasks, we show that the population-level activity in both the premotor nucleus HVC and the motor nucleus RA is organized on low-dimensional neural manifolds upon which coordinated neural activity is well described by temporally structured trajectories during singing behavior. Both the HVC and RA latent trajectories provide relevant information to predict vocal sequence transitions between song syllables. However, the dynamics of these latent trajectories differ between regions. Our state-space models suggest a unique and continuous-over-time correspondence between the latent space of RA and vocal output, whereas the corresponding relationship for HVC exhibits a higher degree of neural variability. We then demonstrate that comparable high-fidelity reconstruction of continuous vocal outputs can be achieved from HVC and RA neural latents and spiking activity. Unlike those that use spiking activity, however, decoding models using neural latents generalize to novel sub-populations in each region, consistent with the existence of preserved manifolds that confine vocal-motor activity in HVC and RA.

Similar neural states, but dissimilar decoding patterns for motor control in parietal cortex

Discrete neural states are associated with reaching movements across the fronto-parietal network. Here, the Hidden Markov Model (HMM) applied to spiking activity of the somato-motor parietal area PE revealed a sequence of states similar to those of the contiguous visuomotor areas PEc and V6A. Using a coupled clustering and decoding approach, we proved that these neural states carried spatiotemporal information regarding behaviour in all three posterior parietal areas. However, comparing decoding accuracy, PE was less informative than V6A and PEc. In addition, V6A outperformed PEc in target inference, indicating functional differences among the parietal areas. To check the consistency of these differences, we used both a supervised and an unsupervised variant of the HMM, and compared its performance with two more common classifiers, Support Vector Machine and Long-Short Term Memory. The differences in decoding between areas were invariant to the algorithm used, still showing the dissimilarities found with HMM, thus indicating that these dissimilarities are intrinsic in the information encoded by parietal neurons. These results highlight that, when decoding from the parietal cortex, for example, in brain machine interface implementations, attention should be paid in selecting the most suitable source of neural signals, given the great heterogeneity of this cortical sector.

Orthogonal neural representations support perceptual judgements of natural stimuli

In natural behavior, observers must separate relevant information from a barrage of irrelevant information. Many studies have investigated the neural underpinnings of this ability using artificial stimuli presented on simple backgrounds. Natural viewing, however, carries a set of challenges that are inaccessible using artificial stimuli, including neural responses to background objects that are task-irrelevant. An emerging body of evidence suggests that the visual abilities of humans and animals can be modeled through the linear decoding of task-relevant information from visual cortex. This idea suggests the hypothesis that irrelevant features of a natural scene should impair performance on a visual task only if their neural representations intrude on the linear readout of the task relevant feature, as would occur if the representations of task-relevant and irrelevant features are not orthogonal in the underlying neural population. We tested this hypothesis using human psychophysics and monkey neurophysiology, in response to parametrically variable naturalistic stimuli. We demonstrate that 1) the neural representation of one feature (the position of a central object) in visual area V4 is orthogonal to those of several background features, 2) the ability of human observers to precisely judge object position was largely unaffected by task-irrelevant variation in those background features, and 3) many features of the object and the background are orthogonally represented by V4 neural responses. Our observations are consistent with the hypothesis that orthogonal neural representations can support stable perception of objects and features despite the tremendous richness of natural visual scenes.

Probing latent brain dynamics in Alzheimer's disease via recurrent neural network

The impairment of cognitive function in Alzheimer's disease (AD) is clearly correlated to abnormal changes in cortical rhythm. However, the mechanisms underlying this correlation are still poorly understood. Here, we investigate how network structure and dynamical characteristics alter their abnormal changes in cortical rhythm. To that end, biological data of AD and normal participates are collected. By extracting the energy characteristics of different sub-bands in EEG signals, we find that the rhythm of AD patients is special particularly in theta and alpha bands. The cortical rhythm of normal state is mainly at alpha band, while that of AD state shift to the theta band. Furthermore, recurrent neural network (RNN) is trained to explore the rhythm formation and transformation between two neural states from the perspective view of neurocomputation. It is found that the neural coupling strength decreases significantly under AD state when compared with normal state, which weakens the ability of information transmission in AD state. Besides, the low-dimensional properties of RNN are obtained. By analyzing the relationship between the cortical rhythm transition and the low-dimensional trajectory, it is concluded that the low-dimensional trajectory update is slower and the communication cost is higher in AD state, which explains the abnormal synchronization of AD brain network. Our work reveals the causes for the formation of abnormal brain synchronous functional network status, which may expand our understanding of the mechanism of cognitive impairment in AD and provide an EEG biomarker for early AD.

Synergy quality assessment of muscle modules for determining learning performance using a realistic musculoskeletal model

How our central nervous system efficiently controls our complex musculoskeletal system is still debated. The muscle synergy hypothesis is proposed to simplify this complex system by assuming the existence of functional neural modules that coordinate several muscles. Modularity based on muscle synergies can facilitate motor learning without compromising task performance. However, the effectiveness of modularity in motor control remains debated. This ambiguity can, in part, stem from overlooking that the performance of modularity depends on the mechanical aspects of modules of interest, such as the torque the modules exert. To address this issue, this study introduces two criteria to evaluate the quality of module sets based on commonly used performance metrics in motor learning studies: the accuracy of torque production and learning speed. One evaluates the regularity in the direction of mechanical torque the modules exert, while the other evaluates the evenness of its magnitude. For verification of our criteria, we simulated motor learning of torque production tasks in a realistic musculoskeletal system of the upper arm using feed-forward neural networks while changing the control conditions. We found that the proposed criteria successfully explain the tendency of learning performance in various control conditions. These result suggest that regularity in the direction of and evenness in magnitude of mechanical torque of utilized modules are significant factor for determining learning performance. Although the criteria were originally conceived for an error-based learning scheme, the approach to pursue which set of modules is better for motor control can have significant implications in other studies of modularity in general.

Motor cortex latent dynamics encode spatial and temporal arm movement parameters independently

The fluid movement of an arm requires multiple spatiotemporal parameters to be set independently. Recent studies have argued that arm movements are generated by the collective dynamics of neurons in motor cortex. An untested prediction of this hypothesis is that independent parameters of movement must map to independent components of the neural dynamics. Using a task where monkeys made a sequence of reaching movements to randomly placed targets, we show that the spatial and temporal parameters of arm movements are independently encoded in the low-dimensional trajectories of population activity in motor cortex: Each movement's direction corresponds to a fixed neural trajectory through neural state space and its speed to how quickly that trajectory is traversed. Recurrent neural network models show this coding allows independent control over the spatial and temporal parameters of movement by separate network parameters. Our results support a key prediction of the dynamical systems view of motor cortex, but also argue that not all parameters of movement are defined by different trajectories of population activity.

De novo motor learning creates structure in neural activity that shapes adaptation

Animals can quickly adapt learned movements to external perturbations, and their existing motor repertoire likely influences their ease of adaptation. Long-term learning causes lasting changes in neural connectivity, which shapes the activity patterns that can be produced during adaptation. Here, we examined how a neural population's existing activity patterns, acquired through de novo learning, affect subsequent adaptation by modeling motor cortical neural population dynamics with recurrent neural networks. We trained networks on different motor repertoires comprising varying numbers of movements, which they acquired following various learning experiences. Networks with multiple movements had more constrained and robust dynamics, which were associated with more defined neural 'structure'-organization in the available population activity patterns. This structure facilitated adaptation, but only when the changes imposed by the perturbation were congruent with the organization of the inputs and the structure in neural activity acquired during de novo learning. These results highlight trade-offs in skill acquisition and demonstrate how different learning experiences can shape the geometrical properties of neural population activity and subsequent adaptation.

A brain machine interface framework for exploring proactive control of smart environments

Brain machine interfaces (BMIs) can substantially improve the quality of life of elderly or disabled people. However, performing complex action sequences with a BMI system is onerous because it requires issuing commands sequentially. Fundamentally different from this, we have designed a BMI system that reads out mental planning activity and issues commands in a proactive manner. To demonstrate this, we recorded brain activity from freely-moving monkeys performing an instructed task and decoded it with an energy-efficient, small and mobile field-programmable gate array hardware decoder triggering real-time action execution on smart devices. Core of this is an adaptive decoding algorithm that can compensate for the day-by-day neuronal signal fluctuations with minimal re-calibration effort. We show that open-loop planning-ahead control is possible using signals from primary and pre-motor areas leading to significant time-gain in the execution of action sequences. This novel approach provides, thus, a stepping stone towards improved and more humane control of different smart environments with mobile brain machine interfaces.

Cortico-cerebellar coordination facilitates neuroprosthetic control

Temporally coordinated neural activity is central to nervous system function and purposeful behavior. Still, there is a paucity of evidence demonstrating how this coordinated activity within cortical and subcortical regions governs behavior. We investigated this between the primary motor (M1) and contralateral cerebellar cortex as rats learned a neuroprosthetic/brain-machine interface (BMI) task. In neuroprosthetic task, actuator movements are causally linked to M1 "direct" neurons that drive the decoder for successful task execution. However, it is unknown how task-related M1 activity interacts with the cerebellum. We observed a notable 3 to 6 hertz coherence that emerged between these regions' local field potentials (LFPs) with learning that also modulated task-related spiking. We identified robust task-related indirect modulation in the cerebellum, which developed a preferential relationship with M1 task-related activity. Inhibiting cerebellar cortical and deep nuclei activity through optogenetics led to performance impairments in M1-driven neuroprosthetic control. Together, these results demonstrate that cerebellar influence is necessary for M1-driven neuroprosthetic control.

Learning leaves a memory trace in motor cortex

How are we able to learn new behaviors without disrupting previously learned ones? To understand how the brain achieves this, we used a brain-computer interface (BCI) learning paradigm, which enables us to detect the presence of a memory of one behavior while performing another. We found that learning to use a new BCI map altered the neural activity that monkeys produced when they returned to using a familiar BCI map in a way that was specific to the learning experience. That is, learning left a "memory trace" in the primary motor cortex. This memory trace coexisted with proficient performance under the familiar map, primarily by altering neural activity in dimensions that did not impact behavior. Forming memory traces might be how the brain is able to provide for the joint learning of multiple behaviors without interference.

What does the mean mean? A simple test for neuroscience

Trial-averaged metrics, e.g. tuning curves or population response vectors, are a ubiquitous way of characterizing neuronal activity. But how relevant are such trial-averaged responses to neuronal computation itself? Here we present a simple test to estimate whether average responses reflect aspects of neuronal activity that contribute to neuronal processing. The test probes two assumptions implicitly made whenever average metrics are treated as meaningful representations of neuronal activity: Reliability: Neuronal responses repeat consistently enough across trials that they convey a recognizable reflection of the average response to downstream regions.Behavioural relevance: If a single-trial response is more similar to the average template, it is more likely to evoke correct behavioural responses. We apply this test to two data sets: (1) Two-photon recordings in primary somatosensory cortices (S1 and S2) of mice trained to detect optogenetic stimulation in S1; and (2) Electrophysiological recordings from 71 brain areas in mice performing a contrast discrimination task. Under the highly controlled settings of Data set 1, both assumptions were largely fulfilled. In contrast, the less restrictive paradigm of Data set 2 met neither assumption. Simulations predict that the larger diversity of neuronal response preferences, rather than higher cross-trial reliability, drives the better performance of Data set 1. We conclude that when behaviour is less tightly restricted, average responses do not seem particularly relevant to neuronal computation, potentially because information is encoded more dynamically. Most importantly, we encourage researchers to apply this simple test of computational relevance whenever using trial-averaged neuronal metrics, in order to gauge how representative cross-trial averages are in a given context.

Linking fast and slow: The case for generative models

A pervasive challenge in neuroscience is testing whether neuronal connectivity changes over time due to specific causes, such as stimuli, events, or clinical interventions. Recent hardware innovations and falling data storage costs enable longer, more naturalistic neuronal recordings. The implicit opportunity for understanding the self-organised brain calls for new analysis methods that link temporal scales: from the order of milliseconds over which neuronal dynamics evolve, to the order of minutes, days, or even years over which experimental observations unfold. This review article demonstrates how hierarchical generative models and Bayesian inference help to characterise neuronal activity across different time scales. Crucially, these methods go beyond describing statistical associations among observations and enable inference about underlying mechanisms. We offer an overview of fundamental concepts in state-space modeling and suggest a taxonomy for these methods. Additionally, we introduce key mathematical principles that underscore a separation of temporal scales, such as the slaving principle, and review Bayesian methods that are being used to test hypotheses about the brain with multiscale data. We hope that this review will serve as a useful primer for experimental and computational neuroscientists on the state of the art and current directions of travel in the complex systems modelling literature.

Preparatory activity and the expansive null-space

The study of the cortical control of movement experienced a conceptual shift over recent decades, as the basic currency of understanding shifted from single-neuron tuning towards population-level factors and their dynamics. This transition was informed by a maturing understanding of recurrent networks, where mechanism is often characterized in terms of population-level factors. By estimating factors from data, experimenters could test network-inspired hypotheses. Central to such hypotheses are 'output-null' factors that do not directly drive motor outputs yet are essential to the overall computation. In this Review, we highlight how the hypothesis of output-null factors was motivated by the venerable observation that motor-cortex neurons are active during movement preparation, well before movement begins. We discuss how output-null factors then became similarly central to understanding neural activity during movement. We discuss how this conceptual framework provided key analysis tools, making it possible for experimenters to address long-standing questions regarding motor control. We highlight an intriguing trend: as experimental and theoretical discoveries accumulate, the range of computational roles hypothesized to be subserved by output-null factors continues to expand.

Neurobiologically realistic neural network enables cross-scale modeling of neural dynamics

Fundamental principles underlying computation in multi-scale brain networks illustrate how multiple brain areas and their coordinated activity give rise to complex cognitive functions. Whereas brain activity has been studied at the micro- to meso-scale to reveal the connections between the dynamical patterns and the behaviors, investigations of neural population dynamics are mainly limited to single-scale analysis. Our goal is to develop a cross-scale dynamical model for the collective activity of neuronal populations. Here we introduce a bio-inspired deep learning approach, termed NeuroBondGraph Network (NBGNet), to capture cross-scale dynamics that can infer and map the neural data from multiple scales. Our model not only exhibits more than an 11-fold improvement in reconstruction accuracy, but also predicts synchronous neural activity and preserves correlated low-dimensional latent dynamics. We also show that the NBGNet robustly predicts held-out data across a long time scale (2 weeks) without retraining. We further validate the effective connectivity defined from our model by demonstrating that neural connectivity during motor behaviour agrees with the established neuroanatomical hierarchy of motor control in the literature. The NBGNet approach opens the door to revealing a comprehensive understanding of brain computation, where network mechanisms of multi-scale activity are critical.

Multimodal subspace identification for modeling discrete-continuous spiking and field potential population activity

Objective.Learning dynamical latent state models for multimodal spiking and field potential activity can reveal their collective low-dimensional dynamics and enable better decoding of behavior through multimodal fusion. Toward this goal, developing unsupervised learning methods that are computationally efficient is important, especially for real-time learning applications such as brain-machine interfaces (BMIs). However, efficient learning remains elusive for multimodal spike-field data due to their heterogeneous discrete-continuous distributions and different timescales.Approach.Here, we develop a multiscale subspace identification (multiscale SID) algorithm that enables computationally efficient learning for modeling and dimensionality reduction for multimodal discrete-continuous spike-field data. We describe the spike-field activity as combined Poisson and Gaussian observations, for which we derive a new analytical SID method. Importantly, we also introduce a novel constrained optimization approach to learn valid noise statistics, which is critical for multimodal statistical inference of the latent state, neural activity, and behavior. We validate the method using numerical simulations and with spiking and local field potential population activity recorded during a naturalistic reach and grasp behavior.Main results.We find that multiscale SID accurately learned dynamical models of spike-field signals and extracted low-dimensional dynamics from these multimodal signals. Further, it fused multimodal information, thus better identifying the dynamical modes and predicting behavior compared to using a single modality. Finally, compared to existing multiscale expectation-maximization learning for Poisson-Gaussian observations, multiscale SID had a much lower training time while being better in identifying the dynamical modes and having a better or similar accuracy in predicting neural activity and behavior.Significance.Overall, multiscale SID is an accurate learning method that is particularly beneficial when efficient learning is of interest, such as for online adaptive BMIs to track non-stationary dynamics or for reducing offline training time in neuroscience investigations.

Neural population dynamics reveals disruption of spinal circuits' responses to proprioceptive input during electrical stimulation of sensory afferents

While neurostimulation technologies are rapidly approaching clinical applications for sensorimotor disorders, the impact of electrical stimulation on network dynamics is still unknown. Given the high degree of shared processing in neural structures, it is critical to understand if neurostimulation affects functions that are related to, but not targeted by, the intervention. Here, we approach this question by studying the effects of electrical stimulation of cutaneous afferents on unrelated processing of proprioceptive inputs. We recorded intraspinal neural activity in four monkeys while generating proprioceptive inputs from the radial nerve. We then applied continuous stimulation to the radial nerve cutaneous branch and quantified the impact of the stimulation on spinal processing of proprioceptive inputs via neural population dynamics. Proprioceptive pulses consistently produce neural trajectories that are disrupted by concurrent cutaneous stimulation. This disruption propagates to the somatosensory cortex, suggesting that electrical stimulation can perturb natural information processing across the neural axis.