Dimensionality reduction for large-scale neural recordings

Multiscale organization of neuronal activity unifies scale-dependent theories of brain function

Brain recordings collected at different resolutions support distinct signatures of neural coding, leading to scale-dependent theories of brain function. Here, we show that these disparate signatures emerge from a heavy-tailed, multiscale functional organization of neuronal activity observed across calcium-imaging recordings collected from the whole brains of zebrafish and C. elegans as well as from sensory regions in Drosophila, mice, and macaques. Network simulations demonstrate that this conserved hierarchical structure enhances information processing. Finally, we find that this organization is maintained despite significant cross-scale reconfiguration of cellular coordination during behavior. Our findings suggest that this nonlinear organization of neuronal activity is a universal principle conserved for its ability to adaptively link behavior to neural dynamics across multiple spatiotemporal scales while balancing functional resiliency and information processing efficiency.

Deconstructing the Mapper algorithm to extract richer topological and temporal features from functional neuroimaging data

Capturing and tracking large-scale brain activity dynamics holds the potential to deepen our understanding of cognition. Previously, tools from topological data analysis, especially Mapper, have been successfully used to mine brain activity dynamics at the highest spatiotemporal resolutions. Even though it is a relatively established tool within the field of topological data analysis, Mapper results are highly impacted by parameter selection. Given that noninvasive human neuroimaging data (e.g., from fMRI) is typically fraught with artifacts and no gold standards exist regarding "true" state transitions, we argue for a thorough examination of Mapper parameter choices to better reveal their impact. Using synthetic data (with known transition structure) and real fMRI data, we explore a variety of parameter choices for each Mapper step, thereby providing guidance and heuristics for the field. We also release our parameter exploration toolbox as a software package to make it easier for scientists to investigate and apply Mapper to any dataset.

Interpretable deep learning for deconvolutional analysis of neural signals

The widespread adoption of deep learning to build models that capture the dynamics of neural populations is typically based on "black-box" approaches that lack an interpretable link between neural activity and network parameters. Here, we propose to apply algorithm unrolling, a method for interpretable deep learning, to design the architecture of sparse deconvolutional neural networks and obtain a direct interpretation of network weights in relation to stimulus-driven single-neuron activity through a generative model. We characterize our method, referred to as deconvolutional unrolled neural learning (DUNL), and show its versatility by applying it to deconvolve single-trial local signals across multiple brain areas and recording modalities. To exemplify use cases of our decomposition method, we uncover multiplexed salience and reward prediction error signals from midbrain dopamine neurons in an unbiased manner, perform simultaneous event detection and characterization in somatosensory thalamus recordings, and characterize the heterogeneity of neural responses in the piriform cortex and in the striatum during unstructured, naturalistic experiments. Our work leverages the advances in interpretable deep learning to gain a mechanistic understanding of neural activity.

The Dimensions of dimensionality

Cognitive scientists often infer multidimensional representations from data. Whether the data involve text, neuroimaging, neural networks, or human judgments, researchers frequently infer and analyze latent representational spaces (i.e., embeddings). However, the properties of a latent representation (e.g., prediction performance, interpretability, compactness) depend on the inference procedure, which can vary widely across endeavors. For example, dimensions are not always globally interpretable and the dimensionality of different embeddings may not be readily comparable. Moreover, the dichotomy between multidimensional spaces and purportedly richer representational formats, such as graph representations, is misleading. We review what the different notions of dimension in cognitive science imply for how these latent representations should be used and interpreted.

A reinforcement learning based software simulator for motor brain-computer interfaces

Intracortical motor brain-computer interfaces (BCIs) are expensive and time-consuming to design because accurate evaluation traditionally requires real-time experiments. In a BCI system, a user interacts with an imperfect decoder and continuously changes motor commands in response to unexpected decoded movements. This "closed-loop" nature of BCI leads to emergent interactions between the user and decoder that are challenging to model. The gold standard for BCI evaluation is therefore real-time experiments, which significantly limits the speed and community of BCI research. We present a new BCI simulator that enables researchers to accurately and quickly design BCIs for cursor control entirely in software. Our simulator replaces the BCI user with a deep reinforcement learning (RL) agent that interacts with a simulated BCI system and learns to optimally control it. We demonstrate that our simulator is accurate and versatile, reproducing the published results of three distinct types of BCI decoders: (1) a state-of-the-art linear decoder (FIT-KF), (2) a "two-stage" BCI decoder requiring closed-loop decoder adaptation (ReFIT-KF), and (3) a nonlinear recurrent neural network decoder (FORCE).

A universal hippocampal memory code across animals and environments

How learning is affected by context is a fundamental question of neuroscience, as the ability to generalize learning to different contexts is necessary for navigating the world. An example of swift contextual generalization is observed in conditioning tasks, where performance is quickly generalized from one context to another. A key question in identifying the neural substrate underlying this ability is how the hippocampus (HPC) represents task-related stimuli across different environments, given that HPC cells exhibit place-specific activity that changes across contexts (remapping). In this study, we used calcium imaging to monitor hippocampal neuron activity as rats performed a conditioning task across multiple spatial contexts. We investigated whether hippocampal cells, which encode both spatial locations (place cells) and task-related information, could maintain their task representation even when their spatial encoding remapped in a new spatial context. To assess the consistency of task representations, we used advanced dimensionality reduction techniques combined with machine learning to develop manifold representations of population level HPC activity. The results showed that task-related neural representations remained stable even as place cell representations of spatial context changed, thus demonstrating similar embedding geometries of neural representations of the task across different spatial contexts. Notably, these patterns were not only consistent within the same animal across different contexts but also significantly similar across different animals, suggesting a standardized neural encoding or 'neural syntax' in the hippocampus. These findings bridge a critical gap between memory and navigation research, revealing how the hippocampus maintains cognitive consistency across different spatial environments. These findings also suggest that hippocampal function is governed by a neural framework shared between animals, an observation that may have broad implications for understanding memory, learning, and related cognitive processes. Looking ahead, this work opens new avenues for exploring the fundamental principles underlying hippocampal encoding strategies.

MEA-NAP: A flexible network analysis pipeline for neuronal 2D and 3D organoid multielectrode recordings

Microelectrode array (MEA) recordings are commonly used to compare firing and burst rates in neuronal cultures. MEA recordings can also reveal microscale functional connectivity, topology, and network dynamics-patterns seen in brain networks across spatial scales. Network topology is frequently characterized in neuroimaging with graph theoretical metrics. However, few computational tools exist for analyzing microscale functional brain networks from MEA recordings. Here, we present a MATLAB MEA network analysis pipeline (MEA-NAP) for raw voltage time series acquired from single- or multi-well MEAs. Applications to 3D human cerebral organoids or 2D human-derived or murine cultures reveal differences in network development, including topology, node cartography, and dimensionality. MEA-NAP incorporates multi-unit template-based spike detection, probabilistic thresholding for determining significant functional connections, and normalization techniques for comparing networks. MEA-NAP can identify network-level effects of pharmacologic perturbation and/or disease-causing mutations and thus can provide a translational platform for revealing mechanistic insights and screening new therapeutic approaches. VIDEO ABSTRACT.

A data-driven analysis of the perceptual and neural responses to natural objects reveals organising principles of human visual cognition

A key challenge in understanding the functional organisation of visual cortex stems from the fact that only a small proportion of the objects experienced during natural viewing can be presented in a typical experiment. This constraint often leads to experimental designs that compare responses to objects from experimenter-defined stimulus conditions, potentially limiting the interpretation of the data. To overcome this issue, we used images from the THINGS initiative, which provides a systematic sampling of natural objects. A data-driven analysis was then applied to reveal the functional organisation of the visual brain, incorporating both perceptual and neural responses to these objects. Perceptual properties of the objects were taken from an analysis of similarity judgements, and neural properties were taken from whole brain fMRI responses to the same objects. Partial least squares regression (PLSR) was then used to predict neural responses across the brain from the perceptual properties while simultaneously applying dimensionality reduction. The PLSR model accurately predicted neural responses across visual cortex using only a small number of components. These components revealed smooth, graded neural topographies, which were similar in both hemispheres, and captured a variety of object properties including animacy, real-world size, and object category. However, they did not accord in any simple way with previous theoretical perspectives on object perception. Instead, our findings suggest that visual cortex encodes information in a statistically efficient manner, reflecting natural variability among objects.Significance statement The ability to recognise objects is fundamental to how we interact with our environment, yet the organising principles underlying neural representations of visual objects remain contentious. In this study, we sought to address this question by analysing perceptual and neural responses to a large, unbiased sample of objects. Using a data-driven approach, we leveraged perceptual properties of objects to predict neural responses using a small number of components. This model predicted neural responses with a high degree of accuracy across visual cortex. The components did not directly align with previous explanations of object perception. Instead, our findings suggest the organisation of the visual brain is based on the statistical properties of objects in the natural world.

Mesoscale neuronal granular trial variability in vivo illustrated by nonlinear recurrent network in silico

Large-scale neural recording with single-neuron resolution has revealed the functional complexity of the neural systems. However, even under well-designed task conditions, the cortex-wide network exhibits highly dynamic trial variability, posing challenges to the conventional trial-averaged analysis. To study mesoscale trial variability, we conducted a comparative study between fluorescence imaging of layer-2/3 neurons in vivo and network simulation in silico. We imaged up to 40,000 cortical neurons' triggered responses by deep brain stimulus (DBS). And we build an in silico network to reproduce the biological phenomena we observed in vivo. We proved the existence of ineluctable trial variability and found it influenced by input amplitude and range. Moreover, we demonstrated that a spatially heterogeneous coding community accounts for more reliable inter-trial coding despite single-unit trial variability. A deeper understanding of trial variability from the perspective of a dynamical system may lead to uncovering intellectual abilities such as parallel coding and creativity.

Nonnegative matrix factorization for analyzing state dependent neuronal network dynamics in calcium recordings

Calcium imaging allows recording from hundreds of neurons in vivo with the ability to resolve single cell activity. Evaluating and analyzing neuronal responses, while also considering all dimensions of the data set to make specific conclusions, is extremely difficult. Often, descriptive statistics are used to analyze these forms of data. These analyses, however, remove variance by averaging the responses of single neurons across recording sessions, or across combinations of neurons, to create single quantitative metrics, losing the temporal dynamics of neuronal activity, and their responses relative to each other. Dimensionally Reduction (DR) methods serve as a good foundation for these analyses because they reduce the dimensions of the data into components, while still maintaining the variance. Nonnegative Matrix Factorization (NMF) is an especially promising DR analysis method for analyzing activity recorded in calcium imaging because of its mathematical constraints, which include positivity and linearity. We adapt NMF for our analyses and compare its performance to alternative dimensionality reduction methods on both artificial and in vivo data. We find that NMF is well-suited for analyzing calcium imaging recordings, accurately capturing the underlying dynamics of the data, and outperforming alternative methods in common use.

Entropy of Neuronal Spike Patterns

Neuronal spike patterns are the fundamental units of neural communication in the brain, which is still not fully understood. Entropy measures offer a quantitative framework to assess the variability and information content of these spike patterns. By quantifying the uncertainty and informational content of neuronal patterns, entropy measures provide insights into neural coding strategies, synaptic plasticity, network dynamics, and cognitive processes. Here, we review basic entropy metrics and then we provide examples of recent advancements in using entropy as a tool to improve our understanding of neuronal processing. It focuses especially on studies on critical dynamics in neural networks and the relation of entropy to predictive coding and cortical communication. We highlight the necessity of expanding entropy measures from single neurons to encompass multi-neuronal activity patterns, as cortical circuits communicate through coordinated spatiotemporal activity patterns, called neuronal packets. We discuss how the sequential and partially stereotypical nature of neuronal packets influences the entropy of cortical communication. Stereotypy reduces entropy by enhancing reliability and predictability in neural signaling, while variability within packets increases entropy, allowing for greater information capacity. This balance between stereotypy and variability supports both robustness and flexibility in cortical information processing. We also review challenges in applying entropy to analyze such spatiotemporal neuronal spike patterns, notably, the "curse of dimensionality" in estimating entropy for high-dimensional neuronal data. Finally, we discuss strategies to overcome these challenges, including dimensionality reduction techniques, advanced entropy estimators, sparse coding schemes, and the integration of machine learning approaches. Thus, this work summarizes the most recent developments on how entropy measures contribute to our understanding of principles underlying neural coding.

Formation and retrieval of cell assemblies in a biologically realistic spiking neural network model of area CA3 in the mouse hippocampus

The hippocampal formation is critical for episodic memory, with area Cornu Ammonis 3 (CA3) a necessary substrate for auto-associative pattern completion. Recent theoretical and experimental evidence suggests that the formation and retrieval of cell assemblies enable these functions. Yet, how cell assemblies are formed and retrieved in a full-scale spiking neural network (SNN) of CA3 that incorporates the observed diversity of neurons and connections within this circuit is not well understood. Here, we demonstrate that a data-driven SNN model quantitatively reflecting the neuron type-specific population sizes, intrinsic electrophysiology, connectivity statistics, synaptic signaling, and long-term plasticity of the mouse CA3 is capable of robust auto-association and pattern completion via cell assemblies. Our results show that a broad range of assembly sizes could successfully and systematically retrieve patterns from heavily incomplete or corrupted cues after a limited number of presentations. Furthermore, performance was robust with respect to partial overlap of assemblies through shared cells, substantially enhancing memory capacity. These novel findings provide computational evidence that the specific biological properties of the CA3 circuit produce an effective neural substrate for associative learning in the mammalian brain.

Neural dynamics in the orbitofrontal cortex reveal cognitive strategies

Behavior is sloppy: a multitude of cognitive strategies can produce similar behavioral read-outs. An underutilized approach is to combine multifaceted behavioral analyses with neural recordings to resolve cognitive strategies. Here we show that rats performing a decision-making task exhibit distinct strategies over training, and these cognitive strategies are decipherable from orbitofrontal cortex (OFC) neural dynamics. We trained rats to perform a temporal wagering task with hidden reward states. While naive rats passively adapted to reward statistics, expert rats inferred reward states. Electrophysiological recordings and novel methods for characterizing population dynamics identified latent neural factors that reflected inferred states in expert but not naive rats. In experts, these factors showed abrupt changes following single trials that were informative of state transitions. These dynamics were driven by neurons whose firing rates reflected single trial inferences, and OFC inactivations showed they were causal to behavior. These results reveal the neural signatures of inference.

Between-area communication through the lens of within-area neuronal dynamics

A core problem in systems and circuits neuroscience is deciphering the origin of shared dynamics in neuronal activity: Do they emerge through local network interactions, or are they inherited from external sources? We explore this question with large-scale networks of spatially ordered spiking neuron models where a downstream network receives input from an upstream sender network. We show that linear measures of the communication between the sender and receiver networks can discriminate between emergent or inherited population dynamics. A match in the dimensionality of the sender and receiver population activities promotes faithful communication. In contrast, a nonlinear mapping between the sender to receiver activity, for example, through downstream emergent population-wide fluctuations, can impair linear communication. Our work exposes the benefits and limitations of linear measures when analyzing between-area communication in circuits with rich population-wide neuronal dynamics.

Exploiting correlations across trials and behavioral sessions to improve neural decoding

Traditional neural decoders model the relationship between neural activity and behavior within individual trials of a single experimental session, neglecting correlations across trials and sessions. However, animals exhibit similar neural activities when performing the same behavioral task, and their behaviors are influenced by past experiences from previous trials. To exploit these informative correlations in large datasets, we introduce two complementary models: a multi-session reduced-rank model that shares similar behaviorally-relevant statistical structure in neural activity across sessions to improve decoding, and a multi-session state-space model that shares similar behavioral statistical structure across trials and sessions. Applied across 433 sessions spanning 270 brain regions in the International Brain Laboratory public mouse Neuropixels dataset, our decoders demonstrate improved decoding accuracy for four distinct behaviors compared to traditional approaches. Unlike existing deep learning approaches, our models are interpretable and efficient, uncovering latent behavioral dynamics that govern animal decision-making, quantifying single-neuron contributions to decoding behaviors, and identifying different activation timescales of neural activity across the brain. Code: https://github.com/yzhang511/neural_decoding.

Recent Visual Experience Reshapes V4 Neuronal Activity and Improves Perceptual Performance

Recent visual experience heavily influences our visual perception, but how neuronal activity is reshaped to alter and improve perceptual discrimination remains unknown. We recorded from populations of neurons in visual cortical area V4 while two male rhesus macaque monkeys performed a natural image change detection task under different experience conditions. We found that maximizing the recent experience with a particular image led to an improvement in the ability to detect a change in that image. This improvement was associated with decreased neural responses to the image, consistent with neuronal changes previously seen in studies of adaptation and expectation. We found that the magnitude of behavioral improvement was correlated with the magnitude of response suppression. Furthermore, this suppression of activity led to an increase in signal separation, providing evidence that a reduction in activity can improve stimulus encoding. Within populations of neurons, greater recent experience was associated with decreased trial-to-trial shared variability, indicating that a reduction in variability is a key means by which experience influences perception. Taken together, the results of our study contribute to an understanding of how recent visual experience can shape our perception and behavior through modulating activity patterns in the mid-level visual cortex.

Understanding the neurobiology of social behavior through exploring brain-wide dynamics of neural activity

Social behavior is highly complex and adaptable. It can be divided into multiple temporal stages: detection, approach, and consummatory behavior. Each stage can be further divided into several cognitive and behavioral processes, such as perceiving social cues, evaluating the social and non-social contexts, and recognizing the internal/emotional state of others. Recent studies have identified numerous brain-wide circuits implicated in social behavior and suggested the existence of partially overlapping functional brain networks underlying various types of social and non-social behavior. However, understanding the brain-wide dynamics underlying social behavior remains challenging, and several brain-scale dynamics (macro-, meso-, and micro-scale levels) need to be integrated. Here, we suggest leveraging new tools and concepts to explore social brain networks and integrate those different levels. These include studying the expression of immediate-early genes throughout the entire brain to impartially define the structure of the neuronal networks involved in a given social behavior. Then, network dynamics could be investigated using electrode arrays or multi-channel fiber photometry. Finally, tools like high-density silicon probes and miniscopes can probe neural activity in specific areas and across neuronal populations at the single-cell level.

Dissociative and prioritized modeling of behaviorally relevant neural dynamics using recurrent neural networks

Understanding the dynamical transformation of neural activity to behavior requires new capabilities to nonlinearly model, dissociate and prioritize behaviorally relevant neural dynamics and test hypotheses about the origin of nonlinearity. We present dissociative prioritized analysis of dynamics (DPAD), a nonlinear dynamical modeling approach that enables these capabilities with a multisection neural network architecture and training approach. Analyzing cortical spiking and local field potential activity across four movement tasks, we demonstrate five use-cases. DPAD enabled more accurate neural-behavioral prediction. It identified nonlinear dynamical transformations of local field potentials that were more behavior predictive than traditional power features. Further, DPAD achieved behavior-predictive nonlinear neural dimensionality reduction. It enabled hypothesis testing regarding nonlinearities in neural-behavioral transformation, revealing that, in our datasets, nonlinearities could largely be isolated to the mapping from latent cortical dynamics to behavior. Finally, DPAD extended across continuous, intermittently sampled and categorical behaviors. DPAD provides a powerful tool for nonlinear dynamical modeling and investigation of neural-behavioral data.

Simultaneous interneuron labeling reveals cell type-specific, population-level interactions in cortex

Cortical interneurons shape network activity in cell type-specific ways, and interact with other cell types. These interactions are understudied, as current methods typically restrict in vivo labeling to one neuron type. Although post-hoc identification of many cell types has been accomplished, the method is not available to many labs. We present a method to distinguish two red fluorophores in vivo, allowing imaging of activity in somatostatin (SOM), parvalbumin (PV), and the rest of the neural population in mouse cortex. We compared population events in PV and SOM neurons and observed that local network states reflected the ratio of SOM to PV neuron activity, demonstrating the importance of simultaneous labeling to explain dynamics. Activity became sparser and less correlated when the ratio between SOM and PV activity was high. Our simple method can be flexibly applied to study interactions among any combination of distinct cell type populations across brain areas.

Neural Activity Differentiates Novel and Learned Event Boundaries

People parse continuous experiences at natural breakpoints called event boundaries, which is important for understanding an environment's causal structure and for responding to uncertainty within it. However, it remains unclear how different forms of uncertainty affect the parsing of continuous experiences and how such uncertainty influences the brain's processing of ongoing events. We exposed human participants of both sexes (N = 34) to a continuous sequence of semantically meaningless images. We generated sequences from random walks through a graph that grouped images into temporal communities. After learning, we asked participants to segment another sequence at natural breakpoints (event boundaries). Participants segmented the sequence at learned transitions between communities, as well as at novel transitions, suggesting that people can segment temporally extended experiences into events based on learned structure as well as prediction error. Greater segmentation at novel boundaries was associated with enhanced parietal scalp electroencephalography (EEG) activity between 250 and 450 ms after the stimulus onset. Multivariate classification of EEG activity showed that novel and learned boundaries evoked distinct patterns of neural activity, particularly theta band power in posterior electrodes. Learning also led to distinct neural representations for stimuli within the temporal communities, while neural activity at learned boundary nodes showed predictive evidence for the adjacent community. The data show that people segment experiences at both learned and novel boundaries and suggest that learned event boundaries trigger retrieval of information about the upcoming community that could underlie anticipation of the next event in a sequence.

Sex differences in neural representations of social and nonsocial reward in the medial prefrontal cortex

The reinforcing nature of social interactions is necessary for the maintenance of appropriate social behavior. However, the neural substrates underlying social reward processing and how they might differ based on the sex and internal state of the animal remains unknown. It is also unclear whether these neural substrates are shared with those involved in nonsocial rewarding processing. We developed a fully automated, two choice (social-sucrose) operant assay in which mice choose between social and nonsocial rewards to directly compare the reward-related behaviors associated with two competing stimuli. We performed cellular resolution calcium imaging of medial prefrontal cortex (mPFC) neurons in male and female mice across varying states of water restriction and social isolation. We found that mPFC neurons maintain largely non-overlapping, flexible representations of social and nonsocial reward that vary with internal state in a sex-dependent manner. Additionally, optogenetic manipulation of mPFC activity during the reward period of the assay disrupted reward-seeking behavior across male and female mice. Thus, using a two choice operant assay, we have identified sex-dependent, non-overlapping neural representations of social and nonsocial reward in the mPFC that vary with internal state and that are essential for appropriate reward-seeking behavior.

Neuroelectrophysiology-compatible electrolytic lesioning

Lesion studies have historically been instrumental for establishing causal connections between brain and behavior. They stand to provide additional insight if integrated with multielectrode techniques common in systems neuroscience. Here, we present and test a platform for creating electrolytic lesions through chronically implanted, intracortical multielectrode probes without compromising the ability to acquire neuroelectrophysiology. A custom-built current source provides stable current and allows for controlled, repeatable lesions in awake-behaving animals. Performance of this novel lesioning technique was validated using histology from ex vivo and in vivo testing, current and voltage traces from the device, and measurements of spiking activity before and after lesioning. This electrolytic lesioning method avoids disruptive procedures, provides millimeter precision over the extent and submillimeter precision over the location of the injury, and permits electrophysiological recording of single-unit activity from the remaining neuronal population after lesioning. This technique can be used in many areas of cortex, in several species, and theoretically with any multielectrode probe. The low-cost, external lesioning device can also easily be adopted into an existing electrophysiology recording setup. This technique is expected to enable future causal investigations of the recorded neuronal population's role in neuronal circuit function, while simultaneously providing new insight into local reorganization after neuron loss.

Modeling seizure networks in neuron-glia cultures using microelectrode arrays

Epilepsy is a common neurological disorder, affecting over 65 million people worldwide. Unfortunately, despite resective surgery, over 30 % of patients with drug-resistant epilepsy continue to experience seizures. Retrospective studies considering connectivity using intracranial electrocorticography (ECoG) obtained during neuromonitoring have shown that treatment failure is likely driven by failure to consider critical components of the seizure network, an idea first formally introduced in 2002. However, current studies only capture snapshots in time, precluding the ability to consider seizure network development. Over the past few years, multiwell microelectrode arrays have been increasingly used to study neuronal networks in vitro. As such, we sought to develop a novel in vitro MEA seizure model to allow for study of seizure networks. Specifically, we used 4-aminopyridine (4-AP) to capture hyperexcitable activity, and then show increased network changes after 2 days of chronic treatment. We characterize network changes using functional connectivity measures and a novel technique using dimensionality reduction. We find that 4-AP successfully captures persistently elevated mean firing rate and significant changes in underlying connectivity patterns. We believe this affords a robust in vitro seizure model from which longitudinal network changes can be studied, laying groundwork for future studies exploring seizure network development.

Automated customization of large-scale spiking network models to neuronal population activity

Understanding brain function is facilitated by constructing computational models that accurately reproduce aspects of brain activity. Networks of spiking neurons capture the underlying biophysics of neuronal circuits, yet their activity's dependence on model parameters is notoriously complex. As a result, heuristic methods have been used to configure spiking network models, which can lead to an inability to discover activity regimes complex enough to match large-scale neuronal recordings. Here we propose an automatic procedure, Spiking Network Optimization using Population Statistics (SNOPS), to customize spiking network models that reproduce the population-wide covariability of large-scale neuronal recordings. We first confirmed that SNOPS accurately recovers simulated neural activity statistics. Then, we applied SNOPS to recordings in macaque visual and prefrontal cortices and discovered previously unknown limitations of spiking network models. Taken together, SNOPS can guide the development of network models, thereby enabling deeper insight into how networks of neurons give rise to brain function.

Transformations in prefrontal ensemble activity underlying rapid threat avoidance learning

The capacity to learn cues that predict aversive outcomes, and understand how to avoid those outcomes, is critical for adaptive behavior. Naturalistic avoidance often means accessing a safe location, but whether a location is safe depends on the nature of the impending threat. These relationships must be rapidly learned if animals are to survive. The prelimbic subregion (PL) of the medial prefrontal cortex (mPFC) integrates learned associations to influence these threat avoidance strategies. Prior work has focused on the role of PL activity in avoidance behaviors that are fully established, leaving the prefrontal mechanisms that drive rapid avoidance learning poorly understood. To determine when and how these learning-related changes emerge, we recorded PL neural activity using miniscope calcium imaging as mice rapidly learned to avoid a threatening cue by accessing a safe location. Over the course of learning, we observed enhanced modulation of PL activity representing intersections of a threatening cue with safe or risky locations and movements between them. We observed rapid changes in PL population dynamics that preceded changes observable in the encoding of individual neurons. Successful avoidance could be predicted from cue-related population dynamics during early learning. Population dynamics during specific epochs of the conditioned tone period correlated with the modeled learning rates of individual animals. In contrast, changes in single-neuron encoding occurred later, once an avoidance strategy had stabilized. Together, our findings reveal the sequence of PL changes that characterize rapid threat avoidance learning.

A high-density multi-electrode platform examining the effects of radiation on in vitro cortical networks

Radiation therapy and stereotactic radiosurgery are common treatments for brain malignancies. However, the impact of radiation on underlying neuronal circuits is poorly understood. In the prefrontal cortex (PFC), neurons communicate via action potentials that control cognitive processes, thus it is important to understand the impact of radiation on these circuits. Here we present a novel protocol to investigate the effect of radiation on the activity and survival of PFC networks in vitro. Escalating doses of radiation were applied to PFC slices using a robotic radiosurgery platform at a standard dose rate of 10 Gy/min. High-density multielectrode array recordings of radiated slices were collected to capture extracellular activity across 4,096 channels. Radiated slices showed an increase in firing rate, functional connectivity, and complexity. Graph-theoretic measures of functional connectivity were altered following radiation. These results were compared to pharmacologically induced epileptic slices where neural complexity was markedly elevated, and functional connections were strong but remained spatially focused. Finally, propidium iodide staining revealed a dose-dependent effect of radiation on apoptosis. These findings provide a novel assay to investigate the impacts of clinically relevant doses of radiation on brain circuits and highlight the acute effects of escalating radiation doses on PFC neurons.

Neural subspaces of imagined movements in parietal cortex remain stable over several years in humans

Objective.A crucial goal in brain-machine interfacing is the long-term stability of neural decoding performance, ideally without regular retraining. Long-term stability has only been previously demonstrated in non-human primate experiments and only in primary sensorimotor cortices. Here we extend previous methods to determine long-term stability in humans by identifying and aligning low-dimensional structures in neural data.Approach.Over a period of 1106 and 871 d respectively, two participants completed an imagined center-out reaching task. The longitudinal accuracy between all day pairs was assessed by latent subspace alignment using principal components analysis and canonical correlations analysis of multi-unit intracortical recordings in different brain regions (Brodmann Area 5, Anterior Intraparietal Area and the junction of the postcentral and intraparietal sulcus).Main results.We show the long-term stable representation of neural activity in subspaces of intracortical recordings from higher-order association areas in humans.Significance.These results can be practically applied to significantly expand the longevity and generalizability of brain-computer interfaces.Clinical TrialsNCT01849822, NCT01958086, NCT01964261.

Focal seizures induce spatiotemporally organized spiking activity in the human cortex

Epileptic seizures are debilitating because of the clinical symptoms they produce. These symptoms, in turn, may stem directly from disruptions in neural coding. Recent evidence has suggested that the specific temporal order, or sequence, of spiking across a population of cortical neurons may encode information. Here, we investigate how seizures disrupt neuronal spiking sequences in the human brain by recording multi-unit activity from the cerebral cortex in five male participants undergoing monitoring for seizures. We find that pathological discharges during seizures are associated with bursts of spiking activity across a population of cortical neurons. These bursts are organized into highly consistent and stereotyped temporal sequences. As the seizure evolves, spiking sequences diverge from the sequences observed at baseline and become more spatially organized. The direction of this spatial organization matches the direction of the ictal discharges, which spread over the cortex as traveling waves. Our data therefore suggest that seizures can entrain cortical spiking sequences by changing the spatial organization of neuronal firing, providing a possible mechanism by which seizures create symptoms.

Deep learning based decoding of single local field potential events

How is information processed in the cerebral cortex? In most cases, recorded brain activity is averaged over many (stimulus) repetitions, which erases the fine-structure of the neural signal. However, the brain is obviously a single-trial processor. Thus, we here demonstrate that an unsupervised machine learning approach can be used to extract meaningful information from electro-physiological recordings on a single-trial basis. We use an auto-encoder network to reduce the dimensions of single local field potential (LFP) events to create interpretable clusters of different neural activity patterns. Strikingly, certain LFP shapes correspond to latency differences in different recording channels. Hence, LFP shapes can be used to determine the direction of information flux in the cerebral cortex. Furthermore, after clustering, we decoded the cluster centroids to reverse-engineer the underlying prototypical LFP event shapes. To evaluate our approach, we applied it to both extra-cellular neural recordings in rodents, and intra-cranial EEG recordings in humans. Finally, we find that single channel LFP event shapes during spontaneous activity sample from the realm of possible stimulus evoked event shapes. A finding which so far has only been demonstrated for multi-channel population coding.

A posture subspace in primary motor cortex

To generate movements, the brain must combine information about movement goal and body posture. Motor cortex (M1) is a key node for the convergence of these information streams. How are posture and goal information organized within M1's activity to permit the flexible generation of movement commands? To answer this question, we recorded M1 activity while monkeys performed a variety of tasks with the forearm in a range of postures. We found that posture- and goal-related components of neural population activity were separable and resided in nearly orthogonal subspaces. The posture subspace was stable across tasks. Within each task, neural trajectories for each goal had similar shapes across postures. Our results reveal a simpler organization of posture information in M1 than previously recognized. The compartmentalization of posture and goal information might allow the two to be flexibly combined in the service of our broad repertoire of actions.

Revealing unexpected complex encoding but simple decoding mechanisms in motor cortex via separating behaviorally relevant neural signals

In motor cortex, behaviorally relevant neural responses are entangled with irrelevant signals, which complicates the study of encoding and decoding mechanisms. It remains unclear whether behaviorally irrelevant signals could conceal some critical truth. One solution is to accurately separate behaviorally relevant and irrelevant signals at both single-neuron and single-trial levels, but this approach remains elusive due to the unknown ground truth of behaviorally relevant signals. Therefore, we propose a framework to define, extract, and validate behaviorally relevant signals. Analyzing separated signals in three monkeys performing different reaching tasks, we found neural responses previously considered to contain little information actually encode rich behavioral information in complex nonlinear ways. These responses are critical for neuronal redundancy and reveal movement behaviors occupy a higher-dimensional neural space than previously expected. Surprisingly, when incorporating often-ignored neural dimensions, behaviorally relevant signals can be decoded linearly with comparable performance to nonlinear decoding, suggesting linear readout may be performed in motor cortex. Our findings prompt that separating behaviorally relevant signals may help uncover more hidden cortical mechanisms.

Cortico-basal ganglia plasticity in motor learning

One key function of the brain is to control our body's movements, allowing us to interact with the world around us. Yet, many motor behaviors are not innate but require learning through repeated practice. Among the brain's motor regions, the cortico-basal ganglia circuit is particularly crucial for acquiring and executing motor skills, and neuronal activity in these regions is directly linked to movement parameters. Cell-type-specific adaptations of activity patterns and synaptic connectivity support the learning of new motor skills. Functionally, neuronal activity sequences become structured and associated with learned movements. On the synaptic level, specific connections become potentiated during learning through mechanisms such as long-term synaptic plasticity and dendritic spine dynamics, which are thought to mediate functional circuit plasticity. These synaptic and circuit adaptations within the cortico-basal ganglia circuitry are thus critical for motor skill acquisition, and disruptions in this plasticity can contribute to movement disorders.

Dynamic adaptation to novelty in the brain is related to arousal and intelligence

How does the human brain respond to novelty? Here, we address this question using fMRI data wherein human participants watch the same movie scene four times. On the first viewing, this movie scene is novel, and on later viewings it is not. We find that brain activity is lower-dimensional in response to novelty. At a finer scale, we find that this reduction in the dimensionality of brain activity is the result of increased coupling in specific brain systems, most specifically within and between the control and dorsal attention systems. Additionally, we found that novelty induced an increase in between-subject synchronization of brain activity in the same brain systems. We also find evidence that adaptation to novelty, herein operationalized as the difference between baseline coupling and novelty-response coupling, is related to fluid intelligence. Finally, using separately collected out-of-sample data, we find that the above results may be linked to psychological arousal.

Neural Manifold Constraint for Spike Prediction Models Under Behavioral Reinforcement

Spike prediction models effectively predict downstream spike trains from upstream neural activity for neural prostheses. Such prostheses could potentially restore damaged neural communication pathways using predicted patterns to guide electrical stimulations on downstream. Since the ground truth of downstream neural activity is unavailable for subjects with the damage, reinforcement learning (RL) with behavior-level rewards becomes necessary for model training. However, existing models do not involve any constraint on the generated firing patterns and neglect the correlations among neural activities. Thus, the model outputs can greatly deviate from the natural range of neural activities, causing concerns for clinical usage. This study proposes the neural manifold constraint to solve this problem, shaping RL-generated spike trains in the feature space. The constraint terms describe the first and second order statistics of the neural manifold estimated from neural recordings during subjects' freely moving period. Then, the models can be optimized within the neural manifold by behavioral reinforcement. We test the method to predict primary motor cortex (M1) spikes from medial prefrontal (mPFC) spikes when rats perform the two-lever discrimination task. Results show that the neural activity generated by constrained models resembles the real M1 recordings. Compared with models without constraints, our approach achieves similar behavioral success rates, but reduces the mean squared error of neural firing by 61%. The constraints also increase the model's robustness across data segments and induce realistic neural correlations. Our method provides a promising tool to restore transregional communication with high behavioral performance and more realistic microscopic patterns.

Dentate gyrus ensembles gate context-dependent neural states and memory retrieval

Memories of events are linked to the contexts in which they were encoded. This contextual linking ensures enhanced access to those memories that are most relevant to the context at hand, including specific associations that were previously learned in that context. This principle, referred to as encoding specificity, predicts that context-specific neural states should bias retrieval of particular associations over others, potentially allowing for the disambiguation of retrieval cues that may have multiple associations or meanings. Using a context-odor paired associate learning paradigm in mice, here, we show that chemogenetic manipulation of dentate gyrus ensembles corresponding to specific contexts reinstates context-specific neural states in downstream CA1 and biases retrieval toward context-specific associations.

Single-Trial Representations of Decision-Related Variables by Decomposed Frontal Corticostriatal Ensemble Activity

The frontal cortex-striatum circuit plays a pivotal role in adaptive goal-directed behaviors. However, it remains unclear how decision-related signals are mediated through cross-regional transmission between the medial frontal cortex and the striatum by neuronal ensembles in making decision based on outcomes of past action. Here, we analyzed neuronal ensemble activity obtained through simultaneous multiunit recordings in the secondary motor cortex (M2) and dorsal striatum (DS) in rats performing an outcome-based left-or-right choice task. By adopting tensor component analysis (TCA), a single-trial-based unsupervised dimensionality reduction approach, for concatenated ensembles of M2 and DS neurons, we identified distinct three spatiotemporal neural dynamics (TCA components) at the single-trial level specific to task-relevant variables. Choice-position-selective neural dynamics reflected the positions chosen and was correlated with the trial-to-trial fluctuation of behavioral variables. Intriguingly, choice-pattern-selective neural dynamics distinguished whether the incoming choice was a repetition or a switch from the previous choice before a response choice. Other neural dynamics was selective to outcome and increased within-trial activity following response. Our results demonstrate how the concatenated ensembles of M2 and DS process distinct features of decision-related signals at various points in time. Thereby, the M2 and DS collaboratively monitor action outcomes and determine the subsequent choice, whether to repeat or switch, for action selection.

Thalamocortical architectures for flexible cognition and efficient learning

The brain exhibits a remarkable ability to learn and execute context-appropriate behaviors. How it achieves such flexibility, without sacrificing learning efficiency, is an important open question. Neuroscience, psychology, and engineering suggest that reusing and repurposing computations are part of the answer. Here, we review evidence that thalamocortical architectures may have evolved to facilitate these objectives of flexibility and efficiency by coordinating distributed computations. Recent work suggests that distributed prefrontal cortical networks compute with flexible codes, and that the mediodorsal thalamus provides regularization to promote efficient reuse. Thalamocortical interactions resemble hierarchical Bayesian computations, and their network implementation can be related to existing gating, synchronization, and hub theories of thalamic function. By reviewing recent findings and providing a novel synthesis, we highlight key research horizons integrating computation, cognition, and systems neuroscience.

Integrator dynamics in the cortico-basal ganglia loop underlie flexible motor timing

Flexible control of motor timing is crucial for behavior. Before volitional movement begins, the frontal cortex and striatum exhibit ramping spiking activity, with variable ramp slopes anticipating movement onsets. This activity in the cortico-basal ganglia loop may function as an adjustable 'timer,' triggering actions at the desired timing. However, because the frontal cortex and striatum share similar ramping dynamics and are both necessary for timing behaviors, distinguishing their individual roles in this timer function remains challenging. To address this, we conducted perturbation experiments combined with multi-regional electrophysiology in mice performing a flexible lick-timing task. Following transient silencing of the frontal cortex, cortical and striatal activity swiftly returned to pre-silencing levels and resumed ramping, leading to a shift in lick timing close to the silencing duration. Conversely, briefly inhibiting the striatum caused a gradual decrease in ramping activity in both regions, with ramping resuming from post-inhibition levels, shifting lick timing beyond the inhibition duration. Thus, inhibiting the frontal cortex and striatum effectively paused and rewound the timer, respectively. These findings suggest the striatum is a part of the network that temporally integrates input from the frontal cortex and generates ramping activity that regulates motor timing.

Structured flexibility in recurrent neural networks via neuromodulation

The goal of theoretical neuroscience is to develop models that help us better understand biological intelligence. Such models range broadly in complexity and biological detail. For example, task-optimized recurrent neural networks (RNNs) have generated hypotheses about how the brain may perform various computations, but these models typically assume a fixed weight matrix representing the synaptic connectivity between neurons. From decades of neuroscience research, we know that synaptic weights are constantly changing, controlled in part by chemicals such as neuromodulators. In this work we explore the computational implications of synaptic gain scaling, a form of neuromodulation, using task-optimized low-rank RNNs. In our neuromodulated RNN (NM-RNN) model, a neuromodulatory subnetwork outputs a low-dimensional neuromodulatory signal that dynamically scales the low-rank recurrent weights of an output-generating RNN. In empirical experiments, we find that the structured flexibility in the NM-RNN allows it to both train and generalize with a higher degree of accuracy than low-rank RNNs on a set of canonical tasks. Additionally, via theoretical analyses we show how neuromodulatory gain scaling endows networks with gating mechanisms commonly found in artificial RNNs. We end by analyzing the low-rank dynamics of trained NM-RNNs, to show how task computations are distributed.

Neural manifolds in V1 change with top-down signals from V4 targeting the foveal region

High-dimensional brain activity is often organized into lower-dimensional neural manifolds. However, the neural manifolds of the visual cortex remain understudied. Here, we study large-scale multi-electrode electrophysiological recordings of macaque (Macaca mulatta) areas V1, V4, and DP with a high spatiotemporal resolution. We find that the population activity of V1 contains two separate neural manifolds, which correlate strongly with eye closure (eyes open/closed) and have distinct dimensionalities. Moreover, we find strong top-down signals from V4 to V1, particularly to the foveal region of V1, which are significantly stronger during the eyes-open periods. Finally, in silico simulations of a balanced spiking neuron network qualitatively reproduce the experimental findings. Taken together, our analyses and simulations suggest that top-down signals modulate the population activity of V1. We postulate that the top-down modulation during the eyes-open periods prepares V1 for fast and efficient visual responses, resulting in a type of visual stand-by state.

Anterior cingulate cortex provides the neural substrates for feedback-driven iteration of decision and value representation

Adjusting decision-making under uncertain and dynamic situations is the hallmark of intelligence. It requires a system capable of converting feedback information to renew the internal value. The anterior cingulate cortex (ACC) involves in error and reward events that prompt switching or maintenance of current decision strategies. However, it is unclear whether and how the changes of stimulus-action mapping during behavioral adaptation are encoded, nor how such computation drives decision adaptation. Here, we tracked ACC activity in male mice performing go/no-go auditory discrimination tasks with manipulated stimulus-reward contingencies. Individual ACC neurons integrate the outcome information to the value representation in the next-run trials. Dynamic recruitment of them determines the learning rate of error-guided value iteration and decision adaptation, forming a non-linear feedback-driven updating system to secure the appropriate decision switch. Optogenetically suppressing ACC significantly slowed down feedback-driven decision switching without interfering with the execution of the established strategy.

Recent visual experience reshapes V4 neuronal activity and improves perceptual performance

Recent visual experience heavily influences our visual perception, but how this is mediated by the reshaping of neuronal activity to alter and improve perceptual discrimination remains unknown. We recorded from populations of neurons in visual cortical area V4 while monkeys performed a natural image change detection task under different experience conditions. We found that maximizing the recent experience with a particular image led to an improvement in the ability to detect a change in that image. This improvement was associated with decreased neural responses to the image, consistent with neuronal changes previously seen in studies of adaptation and expectation. We found that the magnitude of behavioral improvement was correlated with the magnitude of response suppression. Furthermore, this suppression of activity led to an increase in signal separation, providing evidence that a reduction in activity can improve stimulus encoding. Within populations of neurons, greater recent experience was associated with decreased trial-to-trial shared variability, indicating that a reduction in variability is a key means by which experience influences perception. Taken together, the results of our study contribute to an understanding of how recent visual experience can shape our perception and behavior through modulating activity patterns in mid-level visual cortex.

Bridging model and experiment in systems neuroscience with Cleo: the Closed-Loop, Electrophysiology, and Optophysiology simulation testbed

Systems neuroscience has experienced an explosion of new tools for reading and writing neural activity, enabling exciting new experiments such as all-optical or closed-loop control that effect powerful causal interventions. At the same time, improved computational models are capable of reproducing behavior and neural activity with increasing fidelity. Unfortunately, these advances have drastically increased the complexity of integrating different lines of research, resulting in the missed opportunities and untapped potential of suboptimal experiments. Experiment simulation can help bridge this gap, allowing model and experiment to better inform each other by providing a low-cost testbed for experiment design, model validation, and methods engineering. Specifically, this can be achieved by incorporating the simulation of the experimental interface into our models, but no existing tool integrates optogenetics, two-photon calcium imaging, electrode recording, and flexible closed-loop processing with neural population simulations. To address this need, we have developed Cleo: the Closed-Loop, Electrophysiology, and Optophysiology experiment simulation testbed. Cleo is a Python package enabling injection of recording and stimulation devices as well as closed-loop control with realistic latency into a Brian spiking neural network model. It is the only publicly available tool currently supporting two-photon and multi-opsin/wavelength optogenetics. To facilitate adoption and extension by the community, Cleo is open-source, modular, tested, and documented, and can export results to various data formats. Here we describe the design and features of Cleo, validate output of individual components and integrated experiments, and demonstrate its utility for advancing optogenetic techniques in prospective experiments using previously published systems neuroscience models.

Unravelling consciousness and brain function through the lens of time, space, and information

Disentangling how cognitive functions emerge from the interplay of brain dynamics and network architecture is among the major challenges that neuroscientists face. Pharmacological and pathological perturbations of consciousness provide a lens to investigate these complex challenges. Here, we review how recent advances about consciousness and the brain's functional organisation have been driven by a common denominator: decomposing brain function into fundamental constituents of time, space, and information. Whereas unconsciousness increases structure-function coupling across scales, psychedelics may decouple brain function from structure. Convergent effects also emerge: anaesthetics, psychedelics, and disorders of consciousness can exhibit similar reconfigurations of the brain's unimodal-transmodal functional axis. Decomposition approaches reveal the potential to translate discoveries across species, with computational modelling providing a path towards mechanistic integration.

Computational role of structure in neural activity and connectivity

One major challenge of neuroscience is identifying structure in seemingly disorganized neural activity. Different types of structure have different computational implications that can help neuroscientists understand the functional role of a particular brain area. Here, we outline a unified approach to characterize structure by inspecting the representational geometry and the modularity properties of the recorded activity and show that a similar approach can also reveal structure in connectivity. We start by setting up a general framework for determining geometry and modularity in activity and connectivity and relating these properties with computations performed by the network. We then use this framework to review the types of structure found in recent studies of model networks performing three classes of computations.

Nondifferentiable activity in the brain

Spike raster plots of numerous neurons show vertical stripes, indicating that neurons exhibit synchronous activity in the brain. We seek to determine whether these coherent dynamics are caused by smooth brainwave activity or by something else. By analyzing biological data, we find that their cross-correlograms exhibit not only slow undulation but also a cusp at the origin, in addition to possible signs of monosynaptic connectivity. Here we show that undulation emerges if neurons are subject to smooth brainwave oscillations while a cusp results from nondifferentiable fluctuations. While modern analysis methods have achieved good connectivity estimation by adapting the models to slow undulation, they still make false inferences due to the cusp. We devise a new analysis method that may solve both problems. We also demonstrate that oscillations and nondifferentiable fluctuations may emerge in simulations of large-scale neural networks.

Investigating Internal Dynamics in Monkey's Primary Motor Cortex during Reaching

Neural dynamics are processes that describe how neurons in the brain change their activities through time in a coordinated manner. In motor control, neural dynamics, governed by both local dynamics of the motor cortex as well as inputs from other brain regions, drive the population neural state to evolve from an initial value. A notable feature is the emergence of rotation-like dynamics in neural state space. However, the causes of rotational dynamics in motor neural systems remain elusive. In this study, our objective is to investigate the impact of kinematics, specifically, the velocity and acceleration of the monkey's hand reaching movement, on rotational dynamics. We propose to employ a linear model to decompose the overall neural dynamics into one driven by the common input and the internal dynamics using single-trial data. Then, we assess the rotational features by comparing the power of internal dynamics with that of the overall dynamics, and quantifying the rotational strength of internal dynamics vs. the overall dynamics. We implement the proposed method on real M1 neural activities from the monkey's center-out reaching task. Our preliminary results demonstrate that the internal dynamics have much weaker rotational features than the overall dynamics. Given recent evidence from animal experiments showing the necessity of continuous common inputs to motor cortex during arm reaching, it indicates that the rotational dynamics in motor cortex may be mainly input driven when the subject is engaged in the movement task.

Reconfigurations of cortical manifold structure during reward-based motor learning

Adaptive motor behavior depends on the coordinated activity of multiple neural systems distributed across the brain. While the role of sensorimotor cortex in motor learning has been well established, how higher-order brain systems interact with sensorimotor cortex to guide learning is less well understood. Using functional MRI, we examined human brain activity during a reward-based motor task where subjects learned to shape their hand trajectories through reinforcement feedback. We projected patterns of cortical and striatal functional connectivity onto a low-dimensional manifold space and examined how regions expanded and contracted along the manifold during learning. During early learning, we found that several sensorimotor areas in the dorsal attention network exhibited increased covariance with areas of the salience/ventral attention network and reduced covariance with areas of the default mode network (DMN). During late learning, these effects reversed, with sensorimotor areas now exhibiting increased covariance with DMN areas. However, areas in posteromedial cortex showed the opposite pattern across learning phases, with its connectivity suggesting a role in coordinating activity across different networks over time. Our results establish the neural changes that support reward-based motor learning and identify distinct transitions in the functional coupling of sensorimotor to transmodal cortex when adapting behavior.

Neuronal representation of visual working memory content in the primate primary visual cortex

The human ability to perceive vivid memories as if they "float" before our eyes, even in the absence of actual visual stimuli, captivates the imagination. To determine the neural substrates underlying visual memories, we investigated the neuronal representation of working memory content in the primary visual cortex of monkeys. Our study revealed that neurons exhibit unique responses to different memory contents, using firing patterns distinct from those observed during the perception of external visual stimuli. Moreover, this neuronal representation evolves with alterations in the recalled content and extends beyond the retinotopic areas typically reserved for processing external visual input. These discoveries shed light on the visual encoding of memories and indicate avenues for understanding the remarkable power of the mind's eye.

Rule-based modulation of a sensorimotor transformation across cortical areas

Flexible responses to sensory stimuli based on changing rules are critical for adapting to a dynamic environment. However, it remains unclear how the brain encodes and uses rule information to guide behavior. Here, we made single-unit recordings while head-fixed mice performed a cross-modal sensory selection task where they switched between two rules: licking in response to tactile stimuli while rejecting visual stimuli, or vice versa. Along a cortical sensorimotor processing stream including the primary (S1) and secondary (S2) somatosensory areas, and the medial (MM) and anterolateral (ALM) motor areas, single-neuron activity distinguished between the two rules both prior to and in response to the tactile stimulus. We hypothesized that neural populations in these areas would show rule-dependent preparatory states, which would shape the subsequent sensory processing and behavior. This hypothesis was supported for the motor cortical areas (MM and ALM) by findings that (1) the current task rule could be decoded from pre-stimulus population activity; (2) neural subspaces containing the population activity differed between the two rules; and (3) optogenetic disruption of pre-stimulus states impaired task performance. Our findings indicate that flexible action selection in response to sensory input can occur via configuration of preparatory states in the motor cortex.