Multiple time-scales of decision-making in the hippocampus and prefrontal cortex

Spatial goal coding in the hippocampal formation

The mammalian hippocampal formation contains several distinct populations of neurons involved in representing self-position and orientation. These neurons, which include place, grid, head direction, and boundary-vector cells, are thought to collectively instantiate cognitive maps supporting flexible navigation. However, to flexibly navigate, it is necessary to also maintain internal representations of goal locations, such that goal-directed routes can be planned and executed. Although it has remained unclear how the mammalian brain represents goal locations, multiple neural candidates have recently been uncovered during different phases of navigation. For example, during planning, sequential activation of spatial cells may enable simulation of future routes toward the goal. During travel, modulation of spatial cells by the prospective route, or by distance and direction to the goal, may allow maintenance of route and goal-location information, supporting navigation on an ongoing basis. As the goal is approached, an increased activation of spatial cells may enable the goal location to become distinctly represented within cognitive maps, aiding goal localization. Lastly, after arrival at the goal, sequential activation of spatial cells may represent the just-taken route, enabling route learning and evaluation. Here, we review and synthesize these and other evidence for goal coding in mammalian brains, relate the experimental findings to predictions from computational models, and discuss outstanding questions and future challenges.

Rhythmic coordination and ensemble dynamics in the hippocampal-prefrontal network during odor-place associative memory and decision making

Memory-guided decision making involves long-range coordination across sensory and cognitive brain networks, with key roles for the hippocampus and prefrontal cortex (PFC). In order to investigate the mechanisms of such coordination, we monitored activity in hippocampus (CA1), PFC, and olfactory bulb (OB) in rats performing an odor-place associative memory guided decision task on a T-maze. During odor sampling, the beta (20-30 Hz) and respiratory (7-8 Hz) rhythms (RR) were prominent across the three regions, with beta and RR coherence between all pairs of regions enhanced during the odor-cued decision making period. Beta phase modulation of phase-locked CA1 and PFC neurons during this period was linked to accurate decisions, with a key role of CA1 interneurons in temporal coordination. Single neurons and ensembles in both CA1 and PFC encoded and predicted animals' upcoming choices, with different cell ensembles engaged during decision-making and decision execution on the maze. Our findings indicate that rhythmic coordination within the hippocampal-prefrontal-olfactory bulb network supports utilization of odor cues for memory-guided decision making.

The impact of stress on the hippocampal spatial code

Hippocampal function is severely compromised by prolonged, uncontrollable stress. However, how stress alters neural representations of our surroundings and events that occur within them remains less clear. We review hippocampal place cell studies that examine how spatial coding is affected by acute and chronic stress, as well as by stress accompanying fear conditioning. Emerging data suggest that chronic stress disrupts the acuity and specificity of CA1 spatial coding, both in familiar and novel contexts, and alters hippocampal oscillations. By contrast, acute stress may have a facilitatory impact on spatial representations. These findings encourage a fresh look at the documented stress-induced changes in hippocampal anatomy and in vitro excitability, and offer a new perspective on the links between stress and memory.

Hippocampal neuropathology in suicide: Gaps in our knowledge and opportunities for a breakthrough

Suicide is a major global hazard. There is a need for increasing suicide awareness and effective and evidence-based interventions, targeting both suicidal ideation and conduct. However, anti-suicide pharmacological effects are unsatisfactory. The human hippocampus is vulnerable to neuropsychiatric damages and subsequently releases psychobiological signals. Human hippocampal studies of suicide completers have shown mechanistic changes in neurobiology, which, however, could not reflect the neuropathological 'fingerprints' of fatal suicide ideations and suicide attempts. In this review, we provide several leading theories of suicide, including the serotoninergic system, Wnt pathway and brain-derived neurotrophic factor/tropomyosin receptor kinase B signalling, and discuss the evidence for their roles in suicide and treatment. Moreover, the cognitive dysfunctions associated with suicide risk are discussed, as well as the novel evidence on cognitive therapies that decrease suicidal ideation. We highlight the need to apply multi-omics techniques (including single-nucleus RNA sequencing and mass spectrometry histochemistry) on hippocampal samples from donors who died by suicide or legal euthanasia, to clarify the aetiology of suicide and propose novel therapeutic strategies.

Learning differentially shapes prefrontal and hippocampal activity during classical conditioning

The ability to use sensory cues to inform goal-directed actions is a critical component of behavior. To study how sounds guide anticipatory licking during classical conditioning, we employed high-density electrophysiological recordings from the hippocampal CA1 area and the prefrontal cortex (PFC) in mice. CA1 and PFC neurons undergo distinct learning-dependent changes at the single-cell level and maintain representations of cue identity at the population level. In addition, reactivation of task-related neuronal assemblies during hippocampal awake Sharp-Wave Ripples (aSWRs) changed within individual sessions in CA1 and over the course of multiple sessions in PFC. Despite both areas being highly engaged and synchronized during the task, we found no evidence for coordinated single cell or assembly activity during conditioning trials or aSWR. Taken together, our findings support the notion that persistent firing and reactivation of task-related neural activity patterns in CA1 and PFC support learning during classical conditioning.

Mixing memory and desire: How memory reactivation supports deliberative decision-making

Memories affect nearly every aspect of our mental life. They allow us to both resolve uncertainty in the present and to construct plans for the future. Recently, renewed interest in the role memory plays in adaptive behavior has led to new theoretical advances and empirical observations. We review key findings, with particular emphasis on how the retrieval of many kinds of memories affects deliberative action selection. These results are interpreted in a sequential inference framework, in which reinstatements from memory serve as "samples" of potential action outcomes. The resulting model suggests a central role for the dynamics of memory reactivation in determining the influence of different kinds of memory in decisions. We propose that representation-specific dynamics can implement a bottom-up "product of experts" rule that integrates multiple sets of action-outcome predictions weighted based on their uncertainty. We close by reviewing related findings and identifying areas for further research. This article is categorized under: Psychology > Reasoning and Decision Making Neuroscience > Cognition Neuroscience > Computation.

Neuronal sequences during theta rely on behavior-dependent spatial maps

Navigation through space involves learning and representing relationships between past, current, and future locations. In mammals, this might rely on the hippocampal theta phase code, where in each cycle of the theta oscillation, spatial representations provided by neuronal sequences start behind the animal's true location and then sweep forward. However, the exact relationship between theta phase, represented position and true location remains unclear and even paradoxical. Here, we formalize previous notions of 'spatial' or 'temporal' theta sweeps that have appeared in the literature. We analyze single-cell and population variables in unit recordings from rat CA1 place cells and compare them to model simulations based on each of these schemes. We show that neither spatial nor temporal sweeps quantitatively accounts for how all relevant variables change with running speed. To reconcile these schemes with our observations, we introduce 'behavior-dependent' sweeps, in which theta sweep length and place field properties, such as size and phase precession, vary across the environment depending on the running speed characteristic of each location. These behavior-dependent spatial maps provide a structured heterogeneity that is essential for understanding the hippocampal code.

Skipping ahead: A circuit for representing the past, present, and future

Envisioning the future is intuitively linked to our ability to remember the past. Within the memory system, substantial work has demonstrated the involvement of the prefrontal cortex and the hippocampus in representing the past and present. Recent data shows that both the prefrontal cortex and the hippocampus encode future trajectories, which are segregated in time by alternating cycles of the theta rhythm. Here, we discuss how information is temporally organized by these brain regions supported by the medial septum, nucleus reuniens, and parahippocampal regions. Finally, we highlight a brain circuit that we predict is essential for the temporal segregation of future scenarios.

Hippocampal replay reflects specific past experiences rather than a plan for subsequent choice

Executing memory-guided behavior requires storage of information about experience and later recall of that information to inform choices. Awake hippocampal replay, when hippocampal neural ensembles briefly reactivate a representation related to prior experience, has been proposed to critically contribute to these memory-related processes. However, it remains unclear whether awake replay contributes to memory function by promoting the storage of past experiences, facilitating planning based on evaluation of those experiences, or both. We designed a dynamic spatial task that promotes replay before a memory-based choice and assessed how the content of replay related to past and future behavior. We found that replay content was decoupled from subsequent choice and instead was enriched for representations of previously rewarded locations and places that had not been visited recently, indicating a role in memory storage rather than in directly guiding subsequent behavior.

Background synaptic input modulates the visuospatial working memory

It is generally thought that persistent firing of neurons in the prefrontal cortex underlies working memory. Previous studies have focused on the influence of recurrent synaptic connectivity in local circuits on memory storage. Given neurons in the neocortex are extensively connected, individual neural circuits should receive synaptic inputs from other areas. Here we explore how background synaptic inputs (BSIs) modulate the visuospatial working memory in an oculomotor delayed response task. In a local recurrent network composed of pyramidal cells and interneurons, a bump attractor persists across the delay period, encoding the cue location. Under independent BSIs, the spontaneous network state before the cue presentation can be classified as inactive, active, or overactive, occurring successively with increasing the BSI strength, and the active state facilitates the memory storage. Under spatially correlated BSIs, optimal scenarios, in terms of accuracy of representation and resistance to distraction, involve the BSIs with intermediate strength and low correlation or high strength and moderate correlation. Our results demonstrate how the memory storage is regulated via tuning the balance between local excitation and global inhibition in the network. The current work reveals the functional importance of background input and suggests that robust memory storage could be accomplished over a variety of network states.

The secret life of predictive brains: what's spontaneous activity for?

Brains at rest generate dynamical activity that is highly structured in space and time. We suggest that spontaneous activity, as in rest or dreaming, underlies top-down dynamics of generative models. During active tasks, generative models provide top-down predictive signals for perception, cognition, and action. When the brain is at rest and stimuli are weak or absent, top-down dynamics optimize the generative models for future interactions by maximizing the entropy of explanations and minimizing model complexity. Spontaneous fluctuations of correlated activity within and across brain regions may reflect transitions between 'generic priors' of the generative model: low dimensional latent variables and connectivity patterns of the most common perceptual, motor, cognitive, and interoceptive states. Even at rest, brains are proactive and predictive.

Spatial Representations in Rat Orbitofrontal Cortex

The orbitofrontal cortex (OFC) and hippocampus share striking cognitive and functional similarities. As a result, both structures have been proposed to encode "cognitive maps" that provide useful scaffolds for planning complex behaviors. However, while this function has been exemplified by spatial coding in neurons of hippocampal regions-particularly place and grid cells-spatial representations in the OFC have been investigated far less. Here we sought to address this by recording OFC neurons from male rats engaged in an open-field foraging task like that originally developed to characterize place fields in rodent hippocampal neurons. Single-unit activity was recorded as rats searched for food pellets scattered randomly throughout a large enclosure. In some sessions, particular flavors of food occurred more frequently in particular parts of the enclosure; in others, only a single flavor was used. OFC neurons showed spatially localized firing fields in both conditions, and representations changed between flavored and unflavored foraging periods in a manner reminiscent of remapping in the hippocampus. Compared with hippocampal recordings taken under similar behavioral conditions, OFC spatial representations were less temporally reliable, and there was no significant evidence of grid tuning in OFC neurons. These data confirm that OFC neurons show spatial firing fields in a large, two-dimensional environment in a manner similar to hippocampus. Consistent with the focus of the OFC on biological meaning and goals, spatial coding was weaker than in hippocampus and influenced by outcome identity.SIGNIFICANCE STATEMENT The orbitofrontal cortex (OFC) and hippocampus have both been proposed to encode "cognitive maps" that provide useful scaffolds for planning complex behaviors. This function is exemplified by place and grid cells identified in hippocampus, the activity of which maps spatial environments. The current study directly demonstrates very similar, though not identical, spatial representatives in OFC neurons, confirming that OFC-like hippocampus-can represent a spatial map under the appropriate experimental conditions.

Replay in minds and machines

Experience-related brain activity patterns reactivate during sleep, wakeful rest, and brief pauses from active behavior. In parallel, machine learning research has found that experience replay can lead to substantial performance improvements in artificial agents. Together, these lines of research suggest replay has a variety of computational benefits for decision-making and learning. Here, we provide an overview of putative computational functions of replay as suggested by machine learning and neuroscientific research. We show that replay can lead to faster learning, less forgetting, reorganization or augmentation of experiences, and support planning and generalization. In addition, we highlight the benefits of reactivating abstracted internal representations rather than veridical memories, and discuss how replay could provide a mechanism to build internal representations that improve learning and decision-making.

Navigating for reward

An organism's survival can depend on its ability to recall and navigate to spatial locations associated with rewards, such as food or a home. Accumulating research has revealed that computations of reward and its prediction occur on multiple levels across a complex set of interacting brain regions, including those that support memory and navigation. However, how the brain coordinates the encoding, recall and use of reward information to guide navigation remains incompletely understood. In this Review, we propose that the brain's classical navigation centres - the hippocampus and the entorhinal cortex - are ideally suited to coordinate this larger network by representing both physical and mental space as a series of states. These states may be linked to reward via neuromodulatory inputs to the hippocampus-entorhinal cortex system. Hippocampal outputs can then broadcast sequences of states to the rest of the brain to store reward associations or to facilitate decision-making, potentially engaging additional value signals downstream. This proposal is supported by recent advances in both experimental and theoretical neuroscience. By discussing the neural systems traditionally tied to navigation and reward at their intersection, we aim to offer an integrated framework for understanding navigation to reward as a fundamental feature of many cognitive processes.

Hippocampal place cell sequences differ during correct and error trials in a spatial memory task

Theta rhythms temporally coordinate sequences of hippocampal place cell ensembles during active behaviors, while sharp wave-ripples coordinate place cell sequences during rest. We investigated whether such coordination of hippocampal place cell sequences is disrupted during error trials in a delayed match-to-place task. As a reward location was learned across trials, place cell sequences developed that represented temporally compressed paths to the reward location during the approach to the reward location. Less compressed paths were represented on error trials as an incorrect stop location was approached. During rest periods of correct but not error trials, place cell sequences developed a bias to replay representations of paths ending at the correct reward location. These results support the hypothesis that coordination of place cell sequences by theta rhythms and sharp wave-ripples develops as a reward location is learned and may be important for the successful performance of a spatial memory task.