Modeling statistical dependencies in multi-region spike train data

Modeling conditional distributions of neural and behavioral data with masked variational autoencoders

Extracting the relationship between high-dimensional neural recordings and complex behavior is a ubiquitous problem in neuroscience. Encoding and decoding models target the conditional distribution of neural activity given behavior and vice versa, while dimensionality reduction techniques extract low-dimensional representations thereof. Variational autoencoders (VAEs) are flexible tools for inferring such low-dimensional embeddings but struggle to accurately model arbitrary conditional distributions such as those arising in neural encoding and decoding, let alone simultaneously. Here, we present a VAE-based approach for calculating such conditional distributions. We first validate our approach on a task with known ground truth. Next, we retrieve conditional distributions over masked body parts of walking flies. Finally, we decode motor trajectories from neural activity in a monkey-reach task and query the same VAE for the encoding distribution. Our approach unifies dimensionality reduction and learning conditional distributions, allowing the scaling of common analyses in neuroscience to today's high-dimensional multi-modal datasets.

Neural Correlation Integrated Adaptive Point Process Filtering on Population Spike Trains

Brain encodes information through neural spiking activities that modulate external environmental stimuli and underlying internal states. Population of neurons coordinate through functional connectivity to plan movement trajectories and accurately activate neuromuscular activities. Motor Brain-machine interface (BMI) is a platform to study the relationship between behaviors and neural ensemble activities. In BMI, point process filters model directly on spike timings to extract underlying states such as motion intents from observed multi-neuron spike trains. However, these methods assume the encoded information from individual neurons is conditionally independent, which leads to less precise estimation. It is necessary to incorporate functional neural connectivity into a point process filter to improve the state estimation. In this paper, we propose a neural correlation integrated adaptive point process filter (CIPPF) that can incorporate the information from functional neural connectivity from population spike trains in a recursive Bayesian framework. Functional neural connectivity information is approximated by an artificial neural network to provide extra updating information for the posterior estimation. Gaussian approximation is applied on the probability distribution to obtain a closed-form solution. Our proposed method is validated on both simulation and real data collected from the rat two-lever discrimination task. Due to the simultaneous modeling of functional neural connectivity and single neuronal tuning properties, the proposed method shows better decoding performance. This suggests the possibility to improve BMI performance by processing the coordinated neural population activities.

Stimulus-invariant aspects of the retinal code drive discriminability of natural scenes

Everything that the brain sees must first be encoded by the retina, which maintains a reliable representation of the visual world in many different, complex natural scenes while also adapting to stimulus changes. This study quantifies whether and how the brain selectively encodes stimulus features about scene identity in complex naturalistic environments. While a wealth of previous work has dug into the static and dynamic features of the population code in retinal ganglion cells (RGCs), less is known about how populations form both flexible and reliable encoding in natural moving scenes. We record from the larval salamander retina responding to five different natural movies, over many repeats, and use these data to characterize the population code in terms of single-cell fluctuations in rate and pairwise couplings between cells. Decomposing the population code into independent and cell-cell interactions reveals how broad scene structure is encoded in the retinal output. while the single-cell activity adapts to different stimuli, the population structure captured in the sparse, strong couplings is consistent across natural movies as well as synthetic stimuli. We show that these interactions contribute to encoding scene identity. We also demonstrate that this structure likely arises in part from shared bipolar cell input as well as from gap junctions between RGCs and amacrine cells.

MotorNet, a Python toolbox for controlling differentiable biomechanical effectors with artificial neural networks

Artificial neural networks (ANNs) are a powerful class of computational models for unravelling neural mechanisms of brain function. However, for neural control of movement, they currently must be integrated with software simulating biomechanical effectors, leading to limiting impracticalities: (1) researchers must rely on two different platforms and (2) biomechanical effectors are not generally differentiable, constraining researchers to reinforcement learning algorithms despite the existence and potential biological relevance of faster training methods. To address these limitations, we developed MotorNet, an open-source Python toolbox for creating arbitrarily complex, differentiable, and biomechanically realistic effectors that can be trained on user-defined motor tasks using ANNs. MotorNet is designed to meet several goals: ease of installation, ease of use, a high-level user-friendly application programming interface, and a modular architecture to allow for flexibility in model building. MotorNet requires no dependencies outside Python, making it easy to get started with. For instance, it allows training ANNs on typically used motor control models such as a two joint, six muscle, planar arm within minutes on a typical desktop computer. MotorNet is built on PyTorch and therefore can implement any network architecture that is possible using the PyTorch framework. Consequently, it will immediately benefit from advances in artificial intelligence through PyTorch updates. Finally, it is open source, enabling users to create and share their own improvements, such as new effector and network architectures or custom task designs. MotorNet's focus on higher-order model and task design will alleviate overhead cost to initiate computational projects for new researchers by providing a standalone, ready-to-go framework, and speed up efforts of established computational teams by enabling a focus on concepts and ideas over implementation.

Towards a "universal translator" for neural dynamics at single-cell, single-spike resolution

Neuroscience research has made immense progress over the last decade, but our understanding of the brain remains fragmented and piecemeal: the dream of probing an arbitrary brain region and automatically reading out the information encoded in its neural activity remains out of reach. In this work, we build towards a first foundation model for neural spiking data that can solve a diverse set of tasks across multiple brain areas. We introduce a novel self-supervised modeling approach for population activity in which the model alternates between masking out and reconstructing neural activity across different time steps, neurons, and brain regions. To evaluate our approach, we design unsupervised and supervised prediction tasks using the International Brain Laboratory repeated site dataset, which is comprised of Neuropixels recordings targeting the same brain locations across 48 animals and experimental sessions. The prediction tasks include single-neuron and region-level activity prediction, forward prediction, and behavior decoding. We demonstrate that our multi-task-masking (MtM) approach significantly improves the performance of current state-of-the-art population models and enables multi-task learning. We also show that by training on multiple animals, we can improve the generalization ability of the model to unseen animals, paving the way for a foundation model of the brain at single-cell, single-spike resolution. Project page and code: https://ibl-mtm.github.io/.

Thalamocortical interactions shape hierarchical neural variability during stimulus perception

The brain is organized hierarchically to process sensory signals. But, how do functional connections within and across areas contribute to this hierarchical order? We addressed this problem in the thalamocortical network, while monkeys detected vibrotactile stimulus. During this task, we quantified neural variability and directed functional connectivity in simultaneously recorded neurons sharing the cutaneous receptive field within and across VPL and areas 3b and 1. Before stimulus onset, VPL and area 3b exhibited similar fast dynamics while area 1 showed slower timescales. During the stimulus presence, inter-trial neural variability increased along the network VPL-3b-1 while VPL established two main feedforward pathways with areas 3b and 1 to process the stimulus. This lower variability of VPL and area 3b was found to regulate feedforward thalamocortical pathways. Instead, intra-cortical interactions were only anticipated by higher intrinsic timescales in area 1. Overall, our results provide evidence of hierarchical functional roles along the thalamocortical network.

A multi-region recurrent circuit for evidence accumulation in rats

Decision-making based on noisy evidence requires accumulating evidence and categorizing it to form a choice. Here we evaluate a proposed feedforward and modular mapping of this process in rats: evidence accumulated in anterodorsal striatum (ADS) is categorized in prefrontal cortex (frontal orienting fields, FOF). Contrary to this, we show that both regions appear to be indistinguishable in their encoding/decoding of accumulator value and communicate this information bidirectionally. Consistent with a role for FOF in accumulation, silencing FOF to ADS projections impacted behavior throughout the accumulation period, even while nonselective FOF silencing did not. We synthesize these findings into a multi-region recurrent neural network trained with a novel approach. In-silico experiments reveal that multiple scales of recurrence in the cortico-striatal circuit rescue computation upon nonselective FOF perturbations. These results suggest that ADS and FOF accumulate evidence in a recurrent and distributed manner, yielding redundant representations and robustness to certain perturbations.

A multi-demand operating system underlying diverse cognitive tasks

The existence of a multiple-demand cortical system with an adaptive, domain-general, role in cognition has been proposed, but the underlying dynamic mechanisms and their links to cognitive control abilities are poorly understood. Here we use a probabilistic generative Bayesian model of brain circuit dynamics to determine dynamic brain states across multiple cognitive domains, independent datasets, and participant groups, including task fMRI data from Human Connectome Project, Dual Mechanisms of Cognitive Control study and a neurodevelopment study. We discover a shared brain state across seven distinct cognitive tasks and found that the dynamics of this shared brain state predicted cognitive control abilities in each task. Our findings reveal the flexible engagement of dynamic brain processes across multiple cognitive domains and participant groups, and uncover the generative mechanisms underlying the functioning of a domain-general cognitive operating system. Our computational framework opens promising avenues for probing neurocognitive function and dysfunction.

Time-varying generalized linear models: characterizing and decoding neuronal dynamics in higher visual areas

To create a behaviorally relevant representation of the visual world, neurons in higher visual areas exhibit dynamic response changes to account for the time-varying interactions between external (e.g., visual input) and internal (e.g., reward value) factors. The resulting high-dimensional representational space poses challenges for precisely quantifying individual factors' contributions to the representation and readout of sensory information during a behavior. The widely used point process generalized linear model (GLM) approach provides a powerful framework for a quantitative description of neuronal processing as a function of various sensory and non-sensory inputs (encoding) as well as linking particular response components to particular behaviors (decoding), at the level of single trials and individual neurons. However, most existing variations of GLMs assume the neural systems to be time-invariant, making them inadequate for modeling nonstationary characteristics of neuronal sensitivity in higher visual areas. In this review, we summarize some of the existing GLM variations, with a focus on time-varying extensions. We highlight their applications to understanding neural representations in higher visual areas and decoding transient neuronal sensitivity as well as linking physiology to behavior through manipulation of model components. This time-varying class of statistical models provide valuable insights into the neural basis of various visual behaviors in higher visual areas and hold significant potential for uncovering the fundamental computational principles that govern neuronal processing underlying various behaviors in different regions of the brain.

Identification of interacting neural populations: methods and statistical considerations

As improved recording technologies have created new opportunities for neurophysiological investigation, emphasis has shifted from individual neurons to multiple populations that form circuits, and it has become important to provide evidence of cross-population coordinated activity. We review various methods for doing so, placing them in six major categories while avoiding technical descriptions and instead focusing on high-level motivations and concerns. Our aim is to indicate what the methods can achieve and the circumstances under which they are likely to succeed. Toward this end, we include a discussion of four cross-cutting issues: the definition of neural populations, trial-to-trial variability and Poisson-like noise, time-varying dynamics, and causality.

Tracking the Dynamic Neural Connectivity via Conjugate Gradient Optimization. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2023;2023:1-4. [PMID: 38082697 DOI: 10.1109/embc40787.2023.10340664] [Citation(s) in RCA: 0] [Impact Index Per Article: 0]

Neural connectivity describes how neuron populations coordinate and create cognitive and behavioral functions. Neural connectivity performs dynamics where its population spiking responses to stimuli or intention change over time. Brain-machine interface (BMI) provides a framework for studying dynamical neural connectivity. In BMI, point process is a powerful technique in analyzing the single neuronal tuning. And generalized linear mode (GLM) as an encoding model can incorporate the tuning in kinematics and the neural connectivity. Quantification and tracking of dynamic neural connectivity can contribute to the elucidation of the generation of brain functions in a computational way. However, most of the previous work focused on single neuronal adaptation to kinematics. When a neuron is significantly modulated by some other neurons in some tasks, the shape of the log likelihood function for single neuronal observations can be narrowed in some dimensions. And the existing gradient-based methods are not able to reach the optimum in a fast and adaptive searching way. In this work, to maximize the likelihood of observations and obtain the dynamic neural connectivity tuning parameters, we proposed a conjugate gradient-based encoding model (CGE). We illustrate CGE for likelihood function using the real experimental data under manual control and brain control. The results show that the proposed CGE has better performance in tracking the dynamic neural connectivity tuning parameters and modeling neural encoding.Clinical Relevance- Not directly related.

Population burst propagation across interacting areas of the brain

For many perceptual and behavioral tasks, a prominent feature of neural spike trains involves high firing rates across relatively short intervals of time. We call these events "population bursts." Because during a population burst information is, presumably, transmitted from one part of the brain to another, burst timing should reveal activity related to the flow of information across neural circuits. We developed a statistical method (based on a point process model) of determining, accurately, the time of the maximum (peak) population firing rate on a trial-by-trial basis and used it to characterize burst propagation across areas. We then examined the tendency of peak firing rates in distinct brain areas to shift earlier or later in time, together, across repeated trials, and found this trial-to-trial coupling of peak times to be a sensitive indicator of interaction across populations. In the data we examined, from the Allen Brain Observatory, we found many very strong correlations (95% confidence intervals above 0.75) in cases where standard methods were unable to demonstrate cross-area correlation. The statistical model introduced cross-area covariation only through population-level trial-dependent time shifts and gain constants (values of which were learned from the data), yet it provided very good fits to data histograms, including histograms of spike count correlations within and across visual areas. Our results demonstrate the utility of carefully assessing timing and propagation, across brain regions, of transient bursts in neural population activity, based on multiple spike train recordings.NEW & NOTEWORTHY We developed a novel statistical method for identifying coordinated propagation of activity across populations of spiking neurons, with high temporal accuracy. Using simultaneous recordings from three visual areas we document precise timing relationships on a trial-by-trial basis, and we show how previously existing techniques can fail to discover coordinated activity in cases where the new approach finds very strong cross-area correlation.

Functional interactions among neurons within single columns of macaque V1

Recent developments in high-density neurophysiological tools now make it possible to record from hundreds of single neurons within local, highly interconnected neural networks. Among the many advantages of such recordings is that they dramatically increase the quantity of identifiable, functional interactions between neurons thereby providing an unprecedented view of local circuits. Using high-density, Neuropixels recordings from single neocortical columns of primary visual cortex in nonhuman primates, we identified 1000s of functionally interacting neuronal pairs using established crosscorrelation approaches. Our results reveal clear and systematic variations in the synchrony and strength of functional interactions within single cortical columns. Despite neurons residing within the same column, both measures of interactions depended heavily on the vertical distance separating neuronal pairs, as well as on the similarity of stimulus tuning. In addition, we leveraged the statistical power afforded by the large numbers of functionally interacting pairs to categorize interactions between neurons based on their crosscorrelation functions. These analyses identified distinct, putative classes of functional interactions within the full population. These classes of functional interactions were corroborated by their unique distributions across defined laminar compartments and were consistent with known properties of V1 cortical circuitry, such as the lead-lag relationship between simple and complex cells. Our results provide a clear proof-of-principle for the use of high-density neurophysiological recordings to assess circuit-level interactions within local neuronal networks.

Multiregion neuronal activity: the forest and the trees

The past decade has witnessed remarkable advances in the simultaneous measurement of neuronal activity across many brain regions, enabling fundamentally new explorations of the brain-spanning cellular dynamics that underlie sensation, cognition and action. These recently developed multiregion recording techniques have provided many experimental opportunities, but thoughtful consideration of methodological trade-offs is necessary, especially regarding field of view, temporal acquisition rate and ability to guarantee cellular resolution. When applied in concert with modern optogenetic and computational tools, multiregion recording has already made possible fundamental biological discoveries - in part via the unprecedented ability to perform unbiased neural activity screens for principles of brain function, spanning dozens of brain areas and from local to global scales.

Disentangling the flow of signals between populations of neurons

Technological advances now allow us to record from large populations of neurons across multiple brain areas. These recordings may illuminate how communication between areas contributes to brain function, yet a substantial barrier remains: how do we disentangle the concurrent, bidirectional flow of signals between populations of neurons? We propose here a dimensionality reduction framework, delayed latents across groups (DLAG), that disentangles signals relayed in each direction, identifies how these signals are represented by each population and characterizes how they evolve within and across trials. We demonstrate that DLAG performs well on synthetic datasets similar in scale to current neurophysiological recordings. Then we study simultaneously recorded populations in primate visual areas V1 and V2, where DLAG reveals signatures of bidirectional yet selective communication. Our framework lays a foundation for dissecting the intricate flow of signals across populations of neurons, and how this signalling contributes to cortical computation.

Modeling Neural Connectivity in a Point-Process Analogue of Kalman Filter

A neural encoding model describes how single neuron tunes to external stimuli as well as its connectivity with other neurons. The connectivity illustrates the neuronal interaction within populations in response to the shared latent brain states. Understanding these interactions is crucial to computationally predict the neural activity, which elucidates the neural encoding mechanism A computational analysis on the neural connectivity also facilitates developing more point process decoding model to interpret movement state from neural spike observations for brain machine interfaces (BMI). Most of the previous point process models only consider single neural tuning property and assumes conditional independence among multiple neurons. The connectivity among neurons is not considered in such a Bayesian approach to derive the state. In this work, we propose a point-process analogue of Kalman Filter to model the neural connectivity in a closed-form Bayesian filter. Neural connectivity corrects the posterior of the state given the multi-dimension observation, and a Gaussian distribution is used to approximate the updated posterior distribution. We validate the proposed method on simulation data and compared with traditional point process filtering with conditional independent assumption. The result shows that our method models the neural connectivity information and the single neuronal tuning property at the same time and achieves a better performance of the state decoding. Clinical Relevance - This paper proposes a closed-form derivation of a point process filter based on Gaussian approximations. It can model both single neuronal tuning property and the neural connectivity, which is potential to understanding the neural connectivity computationally.

Dynamic causal communication channels between neocortical areas

Processing of sensory information depends on the interactions between hierarchically connected neocortical regions, but it remains unclear how the activity in one area causally influences the activity dynamics in another and how rapidly such interactions change with time. Here, we show that the communication between the primary visual cortex (V1) and high-order visual area LM is context-dependent and surprisingly dynamic over time. By momentarily silencing one area while recording activity in the other, we find that both areas reliably affected changing subpopulations of target neurons within one hundred milliseconds while mice observed a visual stimulus. The influence of LM feedback on V1 responses became even more dynamic when the visual stimuli predicted a reward, causing fast changes in the geometry of V1 population activity and affecting stimulus coding in a context-dependent manner. Therefore, the functional interactions between cortical areas are not static but unfold through rapidly shifting communication subspaces whose dynamics depend on context when processing sensory information.

•

Optogenetic perturbations reveal the causal structure of long-range cortical influences

•

How visual areas influence each other changes dynamically over tens of milliseconds

•

Feedback to V1 improves visual stimulus encoding required for behavior

•

The dynamics of feedback influences depend on the behavioral context

The Hybrid Drive: a chronic implant device combining tetrode arrays with silicon probes for layer-resolved ensemble electrophysiology in freely moving mice

Objective. Understanding the function of brain cortices requires simultaneous investigation at multiple spatial and temporal scales and to link neural activity to an animal's behavior. A major challenge is to measure within- and across-layer information in actively behaving animals, in particular in mice that have become a major species in neuroscience due to an extensive genetic toolkit. Here we describe the Hybrid Drive, a new chronic implant for mice that combines tetrode arrays to record within-layer information with silicon probes to simultaneously measure across-layer information.Approach. The design of our device combines up to 14 tetrodes and 2 silicon probes, that can be arranged in custom arrays to generate unique areas-specific (and multi-area) layouts.Main Results. We show that large numbers of neurons and layer-resolved local field potentials can be recorded from the same brain region across weeks without loss in electrophysiological signal quality. The drive's lightweight structure (~3.5 g) leaves animal behavior largely unchanged, compared to other tetrode drives, during a variety of experimental paradigms. We demonstrate how the data collected with the Hybrid Drive allow state-of-the-art analysis in a series of experiments linking the spiking activity of CA1 pyramidal layer neurons to the oscillatory activity across hippocampal layers.Significance. Our new device fits a gap in the existing technology and increases the range and precision of questions that can be addressed about neural computations in freely behaving mice.