The implications of categorical and category-free mixed selectivity on representational geometries

Interpretable deep learning for deconvolutional analysis of neural signals

The widespread adoption of deep learning to model neural activity often relies on "black-box" approaches that lack an interpretable connection between neural activity and network parameters. Here, we propose using algorithm unrolling, a method for interpretable deep learning, to design the architecture of sparse deconvolutional neural networks and obtain a direct interpretation of network weights in relation to stimulus-driven single-neuron activity through a generative model. We introduce our method, deconvolutional unrolled neural learning (DUNL), and demonstrate its versatility by applying it to deconvolve single-trial local signals across multiple brain areas and recording modalities. We uncover multiplexed salience and reward prediction error signals from midbrain dopamine neurons, perform simultaneous event detection and characterization in somatosensory thalamus recordings, and characterize the heterogeneity of neural responses in the piriform cortex and across striatum during unstructured, naturalistic experiments. Our work leverages advances in interpretable deep learning to provide a mechanistic understanding of neural activity.

Cell-type-specific manifold analysis discloses independent geometric transformations in the hippocampal spatial code

Integrating analyses of genetically defined cell types with population-level approaches remains poorly explored. We investigated this question by focusing on hippocampal spatial maps and the contribution of two genetically defined pyramidal cell types in the deep and superficial CA1 sublayers. Using single- and dual-color miniscope imaging in mice running along a linear track, we found that population activity from these cells exhibited three-dimensional ring manifolds that encoded the animal position and running direction. Despite shared topology, sublayer-specific manifolds displayed distinct geometric features. Manipulating track orientation revealed rotational and translational changes in manifolds from deep cells, contrasting with more stable representations by superficial cells. These transformations were not observed in manifolds derived from the entire CA1 population. Instead, cell-type-specific chemogenetic silencing of either sublayer revealed independent geometric codes. Our results show how genetically specified subpopulations may underpin parallel spatial maps that can be manipulated independently.

Whisking and locomotion are jointly represented in superior colliculus neurons

Active sensation requires the brain to interpret external stimuli against an ongoing estimate of body position. While internal estimates of body position are often ascribed to the cerebral cortex, we examined the midbrain superior colliculus (SC), due to its close relationship with the sensory periphery as well as higher, motor-related brain regions. Using high-density electrophysiology and movement tracking, we discovered that the on-going kinematics of whisker motion and locomotion speed accurately predict the firing rate of mouse SC neurons. Neural activity was best predicted by movements occurring either in the past, present, or future, indicating that the SC population continuously estimates a trajectory of self-motion. A combined representation of slow and fast whisking features predicted absolute whisker angle at high temporal resolution. Sensory reafference played at least a partial role in shaping this feature tuning. Taken together, these data indicate that the SC contains a joint representation of whisking and locomotor features that is potentially useful in guiding complex orienting movements involving the face and limbs.

Prospective and retrospective representations of saccadic movements in primate prefrontal cortex

The dorso-lateral prefrontal cortex (dlPFC) contributes to flexible, goal-directed behaviors. However, a coherent picture of dlPFC function is lacking, as its activity is often studied only in relation to a few events within a fully learned behavioral task. Here we obtain a comprehensive description of dlPFC activity across different task epochs, saccade types, tasks, and learning stages. We consistently observe the strongest modulation of neural activity in relation to a retrospective representation of the most recent saccade. Prospective, planning-like activity is limited to task-related, delayed saccades directly eligible for a reward. The link between prospective and retrospective representations is highly structured, potentially reflecting a hard-wired feature of saccade responses. Only prospective representations are modulated by the recent behavioral history, but neither representation is modulated by day-to-day behavioral improvements. The dlPFC thus combines tightly linked flexible and rigid representations with a dominant contribution from retrospective signals maintaining the memory of past actions.

The cognitive critical brain: Modulation of criticality in perception-related cortical regions

The constantly evolving world necessitates a brain that can swiftly adapt and respond to rapid changes. The brain, conceptualized as a system performing cognitive functions through collective neural activity, has been shown to maintain a resting state characterized by near-critical neural dynamics, positioning it to effectively respond to external stimuli. However, how near-criticality is dynamically modulated during task performance remains insufficiently understood. In this study, we utilized the prototypical Ising Hamiltonian model to investigate the modulation of near-criticality in neural activity at the cortical subsystem level during perceptual tasks. Specifically, we simulated 2D-Ising models in silico using structural MRI data and empirically estimated the system's state in vivo using functional MRI data. We first replicated previous findings that the resting state is typically near-critical as captured by the Ising model. Importantly, we observed heterogeneous changes in criticality across cortical subsystems during a naturalistic movie-watching task, with visual and auditory regions fine-tuned closer to criticality. A more fine-grained analysis of the ventral temporal cortex during an object recognition task further revealed that only regions selectively responsive to a specific object category were tuned closer to criticality when processing that object category. In conclusion, our study provides empirical evidence from the domain of perception supporting the cognitive critical brain hypothesis that modulating the criticality of subsystems within the brain's hierarchical and modular organization may be a fundamental mechanism for achieving diverse cognitive functions.

A boundedly rational model for category learning

The computational modeling of category learning is typically evaluated in terms of the model's accuracy. For a model to accurately infer category membership of stimuli, it has to have sufficient representational precision. Thus, many category learning models infer category representations that guide decision-making and the model's fitness is evaluated by its ability to accurately choose. Substantial decision-making research, however, indicates that noise plays an important role. Specifically, noisy representations are assumed to introduce an element of stochasticity to decision-making. Noise can be minimized at the cost of cognitive resource expenditure. Thus, a more biologically plausible model of category learning should balance representational precision with costs. Here, we tested an autoencoder model that learns categories (the six category structures introduced by Roger Shepard and colleagues) by balancing the minimization of error with minimization of resource usage. By incorporating the goal of reducing category complexity, the currently proposed model biases category decisions toward previously learned central tendencies. We show that this model is still able to account for category learning performance in a traditional category learning benchmark. The currently proposed model additionally makes some novel predictions about category learning that future studies can test empirically. The goal of this paper is to make progress toward development of an ecologically and neurobiologically plausible model of category learning that can guide future studies and theoretical frameworks.

Interpretable deep learning for deconvolutional analysis of neural signals

The widespread adoption of deep learning to build models that capture the dynamics of neural populations is typically based on "black-box" approaches that lack an interpretable link between neural activity and network parameters. Here, we propose to apply algorithm unrolling, a method for interpretable deep learning, to design the architecture of sparse deconvolutional neural networks and obtain a direct interpretation of network weights in relation to stimulus-driven single-neuron activity through a generative model. We characterize our method, referred to as deconvolutional unrolled neural learning (DUNL), and show its versatility by applying it to deconvolve single-trial local signals across multiple brain areas and recording modalities. To exemplify use cases of our decomposition method, we uncover multiplexed salience and reward prediction error signals from midbrain dopamine neurons in an unbiased manner, perform simultaneous event detection and characterization in somatosensory thalamus recordings, and characterize the heterogeneity of neural responses in the piriform cortex and in the striatum during unstructured, naturalistic experiments. Our work leverages the advances in interpretable deep learning to gain a mechanistic understanding of neural activity.

Sensory History Drives Adaptive Neural Geometry in LP/Pulvinar-Prefrontal Cortex Circuits

Prior expectations guide attention and support perceptual filtering for efficient processing during decision-making. Here we show that during a visual discrimination task, mice adaptively use prior stimulus history to guide ongoing choices by estimating differences in evidence between consecutive trials (| Δ Dir |). The thalamic lateral posterior (LP)/pulvinar nucleus provides robust inputs to the Anterior Cingulate Cortex (ACC), which has been implicated in selective attention and predictive processing, but the function of the LP-ACC projection is unknown. We found that optogenetic manipulations of LP-ACC axons disrupted animals' ability to effectively estimate and use information across stimulus history, leading to | Δ Dir |-dependent ipsilateral biases. Two-photon calcium imaging of LP-ACC axons revealed an engagement-dependent low-dimensional organization of stimuli along a curved manifold. This representation was scaled by | Δ Dir | in a manner that emphasized greater deviations from prior evidence. Thus, our work identifies the LP-ACC pathway as essential for selecting and evaluating stimuli relative to prior evidence to guide decisions.

Modular representations emerge in neural networks trained to perform context-dependent tasks

The brain has large-scale modular structure in the form of brain regions, which are thought to arise from constraints on connectivity and the physical geometry of the cortical sheet. In contrast, experimental and theoretical work has argued both for and against the existence of specialized sub-populations of neurons (modules) within single brain regions. By studying artificial neural networks, we show that this local modularity emerges to support context-dependent behavior, but only when the input is low-dimensional. No anatomical constraints are required. We also show when modular specialization emerges at the population level (different modules correspond to orthogonal subspaces). Modularity yields abstract representations, allows for rapid learning and generalization on novel tasks, and facilitates the rapid learning of related contexts. Non-modular representations facilitate the rapid learning of unrelated contexts. Our findings reconcile conflicting experimental results and make predictions for future experiments.

A transient high-dimensional geometry affords stable conjunctive subspaces for efficient action selection

Flexible action selection requires cognitive control mechanisms capable of mapping the same inputs to different output actions depending on the context. From a neural state-space perspective, this requires a control representation that separates similar input neural states by context. Additionally, for action selection to be robust and time-invariant, information must be stable in time, enabling efficient readout. Here, using EEG decoding methods, we investigate how the geometry and dynamics of control representations constrain flexible action selection in the human brain. Participants performed a context-dependent action selection task. A forced response procedure probed action selection different states in neural trajectories. The result shows that before successful responses, there is a transient expansion of representational dimensionality that separated conjunctive subspaces. Further, the dynamics stabilizes in the same time window, with entry into this stable, high-dimensional state predictive of individual trial performance. These results establish the neural geometry and dynamics the human brain needs for flexible control over behavior.

A Transient High-dimensional Geometry Affords Stable Conjunctive Subspaces for Efficient Action Selection

Flexible action selection requires cognitive control mechanisms capable of mapping the same inputs to different output actions depending on the context. From a neural state-space perspective, this requires a control representation that separates similar input neural states by context. Additionally, for action selection to be robust and time-invariant, information must be stable in time, enabling efficient readout. Here, using EEG decoding methods, we investigate how the geometry and dynamics of control representations constrain flexible action selection in the human brain. Participants performed a context-dependent action selection task. A forced response procedure probed action selection different states in neural trajectories. The result shows that before successful responses, there is a transient expansion of representational dimensionality that separated conjunctive subspaces. Further, the dynamics stabilizes in the same time window, with entry into this stable, high-dimensional state predictive of individual trial performance. These results establish the neural geometry and dynamics the human brain needs for flexible control over behavior.

Neural representational geometries reflect behavioral differences in monkeys and recurrent neural networks

Animals likely use a variety of strategies to solve laboratory tasks. Traditionally, combined analysis of behavioral and neural recording data across subjects employing different strategies may obscure important signals and give confusing results. Hence, it is essential to develop techniques that can infer strategy at the single-subject level. We analyzed an experiment in which two male monkeys performed a visually cued rule-based task. The analysis of their performance shows no indication that they used a different strategy. However, when we examined the geometry of stimulus representations in the state space of the neural activities recorded in dorsolateral prefrontal cortex, we found striking differences between the two monkeys. Our purely neural results induced us to reanalyze the behavior. The new analysis showed that the differences in representational geometry are associated with differences in the reaction times, revealing behavioral differences we were unaware of. All these analyses suggest that the monkeys are using different strategies. Finally, using recurrent neural network models trained to perform the same task, we show that these strategies correlate with the amount of training, suggesting a possible explanation for the observed neural and behavioral differences.

Mixed selectivity: Cellular computations for complexity

The property of mixed selectivity has been discussed at a computational level and offers a strategy to maximize computational power by adding versatility to the functional role of each neuron. Here, we offer a biologically grounded implementational-level mechanistic explanation for mixed selectivity in neural circuits. We define pure, linear, and nonlinear mixed selectivity and discuss how these response properties can be obtained in simple neural circuits. Neurons that respond to multiple, statistically independent variables display mixed selectivity. If their activity can be expressed as a weighted sum, then they exhibit linear mixed selectivity; otherwise, they exhibit nonlinear mixed selectivity. Neural representations based on diverse nonlinear mixed selectivity are high dimensional; hence, they confer enormous flexibility to a simple downstream readout neural circuit. However, a simple neural circuit cannot possibly encode all possible mixtures of variables simultaneously, as this would require a combinatorially large number of mixed selectivity neurons. Gating mechanisms like oscillations and neuromodulation can solve this problem by dynamically selecting which variables are mixed and transmitted to the readout.

Coordinated Response Modulations Enable Flexible Use of Visual Information

We use sensory information in remarkably flexible ways. We can generalize by ignoring task-irrelevant features, report different features of a stimulus, and use different actions to report a perceptual judgment. These forms of flexible behavior are associated with small modulations of the responses of sensory neurons. While the existence of these response modulations is indisputable, efforts to understand their function have been largely relegated to theory, where they have been posited to change information coding or enable downstream neurons to read out different visual and cognitive information using flexible weights. Here, we tested these ideas using a rich, flexible behavioral paradigm, multi-neuron, multi-area recordings in primary visual cortex (V1) and mid-level visual area V4. We discovered that those response modulations in V4 (but not V1) contain the ingredients necessary to enable flexible behavior, but not via those previously hypothesized mechanisms. Instead, we demonstrated that these response modulations are precisely coordinated across the population such that downstream neurons have ready access to the correct information to flexibly guide behavior without making changes to information coding or synapses. Our results suggest a novel computational role for task-dependent response modulations: they enable flexible behavior by changing the information that gets out of a sensory area, not by changing information coding within it.

Evaluating theories of neural information integration during visual search

The brain routes and integrates information from many sources during behavior. A number of models explain this phenomenon within the framework of mixed selectivity theory, yet it is difficult to compare their predictions to understand how neurons and circuits integrate information. In this work, we apply time-series partial information decomposition [PID] to compare models of integration on a dataset of superior colliculus [SC] recordings collected during a multi-target visual search task. On this task, SC must integrate target guidance, bottom-up salience, and previous fixation signals to drive attention. We find evidence that SC neurons integrate these factors in diverse ways, including decision-variable selectivity to expected value, functional specialization to previous fixation, and code-switching (to incorporate new visual input).

Acquisition of non-olfactory encoding improves odour discrimination in olfactory cortex

Olfaction is influenced by contextual factors, past experiences, and the animal's internal state. Whether this information is integrated at the initial stages of cortical odour processing is not known, nor how these signals may influence odour encoding. Here we revealed multiple and diverse non-olfactory responses in the primary olfactory (piriform) cortex (PCx), which dynamically enhance PCx odour discrimination according to behavioural demands. We performed recordings of PCx neurons from mice trained in a virtual reality task to associate odours with visual contexts to obtain a reward. We found that learning shifts PCx activity from encoding solely odours to a regime in which positional, contextual, and associative responses emerge on odour-responsive neurons that become mixed-selective. The modulation of PCx activity by these non-olfactory signals was dynamic, improving odour decoding during task engagement and in rewarded contexts. This improvement relied on the acquired mixed-selectivity, demonstrating how integrating extra-sensory inputs in sensory cortices can enhance sensory processing while encoding the behavioural relevance of stimuli.

Computational role of structure in neural activity and connectivity

One major challenge of neuroscience is identifying structure in seemingly disorganized neural activity. Different types of structure have different computational implications that can help neuroscientists understand the functional role of a particular brain area. Here, we outline a unified approach to characterize structure by inspecting the representational geometry and the modularity properties of the recorded activity and show that a similar approach can also reveal structure in connectivity. We start by setting up a general framework for determining geometry and modularity in activity and connectivity and relating these properties with computations performed by the network. We then use this framework to review the types of structure found in recent studies of model networks performing three classes of computations.

Motor cortex retains and reorients neural dynamics during motor imagery

The most prominent characteristic of motor cortex is its activation during movement execution, but it is also active when we simply imagine movements in the absence of actual motor output. Despite decades of behavioural and imaging studies, it is unknown how the specific activity patterns and temporal dynamics in motor cortex during covert motor imagery relate to those during motor execution. Here we recorded intracortical activity from the motor cortex of two people who retain some residual wrist function following incomplete spinal cord injury as they performed both actual and imagined isometric wrist extensions. We found that we could decompose the population activity into three orthogonal subspaces, where one was similarly active during both action and imagery, and the others were active only during a single task type-action or imagery. Although they inhabited orthogonal neural dimensions, the action-unique and imagery-unique subspaces contained a strikingly similar set of dynamic features. Our results suggest that during motor imagery, motor cortex maintains the same overall population dynamics as during execution by reorienting the components related to motor output and/or feedback into a unique, output-null imagery subspace.

Sequential neuronal processing of number values, abstract decision, and action in the primate prefrontal cortex

Decision-making requires processing of sensory information, comparing the gathered evidence to make a judgment, and performing the action to communicate it. How neuronal representations transform during this cascade of representations remains a matter of debate. Here, we studied the succession of neuronal representations in the primate prefrontal cortex (PFC). We trained monkeys to judge whether a pair of sequentially presented displays had the same number of items. We used a combination of single neuron and population-level analyses and discovered a sequential transformation of represented information with trial progression. While numerical values were initially represented with high precision and in conjunction with detailed information such as order, the decision was encoded in a low-dimensional subspace of neural activity. This decision encoding was invariant to both retrospective numerical values and prospective motor plans, representing only the binary judgment of "same number" versus "different number," thus facilitating the generalization of decisions to novel number pairs. We conclude that this transformation of neuronal codes within the prefrontal cortex supports cognitive flexibility and generalizability of decisions to new conditions.

Anatomically distributed neural representations of instincts in the hypothalamus

Artificial activation of anatomically localized, genetically defined hypothalamic neuron populations is known to trigger distinct innate behaviors, suggesting a hypothalamic nucleus-centered organization of behavior control. To assess whether the encoding of behavior is similarly anatomically confined, we performed simultaneous neuron recordings across twenty hypothalamic regions in freely moving animals. Here we show that distinct but anatomically distributed neuron ensembles encode the social and fear behavior classes, primarily through mixed selectivity. While behavior class-encoding ensembles were spatially distributed, individual ensembles exhibited strong localization bias. Encoding models identified that behavior actions, but not motion-related variables, explained a large fraction of hypothalamic neuron activity variance. These results identify unexpected complexity in the hypothalamic encoding of instincts and provide a foundation for understanding the role of distributed neural representations in the expression of behaviors driven by hardwired circuits.

Widespread coding of navigational variables in prefrontal cortex

To navigate effectively, we must represent information about our location in the environment. Traditional research highlights the role of the hippocampal complex in this process. Spurred by recent research highlighting the widespread cortical encoding of cognitive and motor variables previously thought to have localized function, we hypothesized that navigational variables would be likewise encoded widely, especially in the prefrontal cortex, which is associated with volitional behavior. We recorded neural activity from six prefrontal regions while macaques performed a foraging task in an open enclosure. In all regions, we found strong encoding of allocentric position, allocentric head direction, boundary distance, and linear and angular velocity. These encodings were not accounted for by distance, time to reward, or motor factors. The strength of coding of all variables increased along a ventral-to-dorsal gradient. Together, these results argue that encoding of navigational variables is not localized to the hippocampus and support the hypothesis that navigation is continuous with other forms of flexible cognition in the service of action.

Electrophysiological Recordings from Identified Cell Types in the Olfactory Cortex of Awake Mice

Neural circuits consist of a myriad of distinct cell types, each with specific intrinsic properties and patterns of synaptic connectivity, which transform neural input and convey this information to downstream targets. Understanding how different features of an odor stimulus are encoded and relayed to their appropriate targets will require selective identification and manipulation of these different elements of the circuit. Here, we describe methods to obtain dense, extracellular electrophysiological recordings of odor-evoked activity in olfactory (piriform) cortex of awake, head-fixed mice, and optogenetic tools and procedures to identify genetically defined cell types within this circuit.