Neural ensembles in CA3 transiently encode paths forward of the animal at a decision point

A cognitive map for value-guided choice in the ventromedial prefrontal cortex

The prefrontal cortex (PFC) is crucial for economic decision-making. However, how PFC value representations facilitate flexible decisions remains unknown. We reframe economic decision-making as a navigation process through a cognitive map of choice values. We found rhesus macaques represented choices as navigation trajectories in a value space using a grid-like code. This occurred in ventromedial PFC (vmPFC) local field potential theta frequency across two datasets. vmPFC neurons deployed the same grid-like code and encoded chosen value. However, both signals depended on theta phase: occurring on theta troughs but on separate theta cycles. Finally, we found sharp-wave ripples-a key signature of planning and flexible behavior-in vmPFC. Thus, vmPFC utilizes cognitive map-based computations to organize and compare values, suggesting an alternative architecture for economic choice in PFC.

New information triggers prospective codes to adapt for flexible navigation

Navigating a dynamic world requires rapidly updating choices by integrating past experiences with new information. In hippocampus and prefrontal cortex, neural activity representing future goals is theorized to support navigational planning. However, it remains unknown how prospective goal representations incorporate new, pivotal information. Accordingly, we designed a navigation task that precisely introduces new information using virtual reality, and we recorded neural activity as male mice flexibly adapted their planned destinations. Here we show that new information triggered increased hippocampal prospective representations of both possible goals; while in prefrontal cortex, new information caused prospective representations of choices to rapidly shift to the new choice. When mice did not adapt, prefrontal choice codes failed to switch. Prospective codes were dependent on the amount of behavioral adaptation needed; the new goal arm was represented more strongly when animals needed to change their behavior more. Thus, we show how prospective codes update with new information to flexibly adapt ongoing navigational plans.

Touchscreen Response Precision Is Sensitive to the Explore/Exploit Trade-off

The explore/exploit trade-off is a fundamental property of choice selection during reward-guided decision making, where the "same" choice can reflect either of these internal cognitive states. An unanswered question is whether the execution of a decision provides an underexplored measure of internal cognitive states. Touchscreens are increasingly used across species for cognitive testing and afford the ability to measure the precise location of choice touch responses. We examined how male and female mice in a restless bandit decision making task interacted with a touchscreen to determine if the explore/exploit trade-off, prior reward, and/or sex differences change the variability in the kinetics of touchscreen choices. During exploit states, successive touch responses are closer together than those made in an explore state, suggesting exploit states reflect periods of increased motor stereotypy. Although exploit decisions might be expected to be rewarded more frequently than explore decisions, we find that immediate past reward reduces choice variability independently of explore/exploit state. Male mice are more variable in their interactions with the touchscreen than females, even in low-variability trials such as exploit or following reward. These results suggest that as exploit behavior emerges in reward-guided decision making, all mice become less variable and more automated in both their choice and the actions taken to make that choice, but this occurs on a background of increased male variability. These data uncover the hidden potential for touchscreen decision making tasks to uncover the latent neural states that unite cognition and movement.

Inflammation in Schizophrenia: The Role of Disordered Oscillatory Mechanisms

Schizophrenia is a chronic, debilitating disorder with diverse symptomatology, including disorganised cognition and behaviour. Despite considerable research effort, we have only a limited understanding of the underlying brain dysfunction. A significant proportion of individuals with schizophrenia exhibit high levels of inflammation, and inflammation associated with maternal immune system activation is a risk factor for the disorder. In this review, we outline the potential role of inflammation in the disorder, with a particular focus on how cytokine release might affect the development and function of GABAergic interneurons. One consequence of this change in inhibitory control is a disruption in oscillatory processes in the brain. These changes disrupt the spatial and temporal synchrony of neural activity in the brain, which, by disturbing representations of time and space, may underlie some of the disorganisation symptoms observed in the disorder.

Closed-loop modulation of remote hippocampal representations with neurofeedback

Humans can remember specific remote events without acting on them and influence which memories are retrieved based on internal goals. However, animal models typically present sensory cues to trigger memory retrieval and then assess retrieval based on action. Thus, it is difficult to determine whether measured neural activity patterns relate to the cue(s), the memory, or the behavior. We therefore asked whether retrieval-related neural activity could be generated in animals without cues or a behavioral report. We focused on hippocampal "place cells," which primarily represent the animal's current location (local representations) but can also represent locations away from the animal (remote representations). We developed a neurofeedback system to reward expression of remote representations and found that rats could learn to generate specific spatial representations that often jumped directly to the experimenter-defined target location. Thus, animals can deliberately engage remote representations, enabling direct study of retrieval-related activity in the brain.

Prefrontal default-mode network interactions with posterior hippocampus during exploration

Hippocampal maps and ventral prefrontal cortex (vPFC) value and goal representations support foraging in continuous spaces. How might hippocampal-vPFC interactions control the balance between behavioral exploration and exploitation? Using fMRI and reinforcement learning modeling, we investigated vPFC and hippocampal responses as humans explored and exploited a continuous one-dimensional space, with out-of-session and out-of-sample replication. The spatial distribution of rewards, or value landscape, modulated activity in the hippocampus and default network vPFC subregions, but not in ventrolateral prefrontal control subregions or medial orbitofrontal limbic subregions. While prefrontal default network and hippocampus displayed higher activity in less complex, easy-to-exploit value landscapes, vPFC-hippocampal connectivity increased in uncertain landscapes requiring exploration. Further, synchronization between prefrontal default network and posterior hippocampus scaled with behavioral exploration. Considered alongside electrophysiological studies, our findings suggest that locations to be explored are identified through coordinated activity binding prefrontal default network value representations to posterior hippocampal maps.

Choice Behaviors and Prefrontal-Hippocampal Coupling Are Disrupted in a Rat Model of Fetal Alcohol Spectrum Disorders

Fetal alcohol spectrum disorders (FASDs) are characterized by a range of physical, cognitive, and behavioral impairments. Determining how temporally specific alcohol exposure (AE) affects neural circuits is crucial to understanding the FASD phenotype. Third trimester AE can be modeled in rats by administering alcohol during the first two postnatal weeks, which damages the medial prefrontal cortex (mPFC) and hippocampus (HPC), structures whose functional interactions are required for working memory and executive function. Therefore, we hypothesized that AE during this period would impair working memory, disrupt choice behaviors, and alter mPFC-HPC oscillatory synchrony. To test this hypothesis, we recorded local field potentials from the mPFC and dorsal HPC as male and female AE and sham-intubated (SI) rats performed a spatial working memory task in adulthood and implemented algorithms to detect vicarious trial and errors (VTEs), behaviors associated with deliberative decision-making. We found that, compared with the SI group, the AE group performed fewer VTEs and demonstrated a disturbed relationship between VTEs and choice outcomes, while spatial working memory was unimpaired. This behavioral disruption was accompanied by alterations to mPFC and HPC oscillatory activity in the theta and beta bands, respectively, and a reduced prevalence of mPFC-HPC synchronous events. When trained on multiple behavioral variables, a machine learning algorithm could accurately predict whether rats were in the AE or SI group, thus characterizing a potential phenotype following third trimester AE. Together, these findings indicate that third trimester AE disrupts mPFC-HPC oscillatory interactions and choice behaviors.

Left-right-alternating theta sweeps in entorhinal-hippocampal maps of space

Place cells in the hippocampus and grid cells in the entorhinal cortex are elements of a neural map of self position1-5. For these cells to benefit navigation, their representation must be dynamically related to the surrounding locations2. A candidate mechanism for linking places along an animal's path has been described for place cells, in which the sequence of spikes in each cycle of the hippocampal theta oscillation encodes a trajectory from the animal's current location towards upcoming locations6-8. In mazes that bifurcate, such trajectories alternately traverse the two upcoming arms when the animal approaches the choice point9,10, raising the possibility that the trajectories express available forward paths encoded on previous trials10. However, to bridge the animal's path with the wider environment, beyond places previously or subsequently visited, an experience-independent spatial sampling mechanism might be required. Here we show in freely moving rats that in individual theta cycles, ensembles of grid cells and place cells encode a position signal that sweeps linearly outwards from the animal's location into the ambient environment, with sweep direction alternating stereotypically between left and right across successive theta cycles. These sweeps are accompanied by, and aligned with, a similarly alternating directional signal in a discrete population of parasubiculum cells that have putative connections to grid cells via conjunctive grid × direction cells. Sweeps extend into never-visited locations that are inaccessible to the animal. Sweeps persist during REM sleep. The sweep directions can be explained by an algorithm that maximizes the cumulative coverage of the surrounding manifold space. The sustained and unconditional expression of theta-patterned left-right-alternating sweeps in the entorhinal-hippocampal positioning system provides an efficient 'look around' mechanism for sampling locations beyond the travelled path.

Hippocampal neuronal activity is aligned with action plans

Neurons in the hippocampus are correlated with different variables, including space, time, sensory cues, rewards and actions, in which the extent of tuning depends on ongoing task demands1-8. However, it remains uncertain whether such diverse tuning corresponds to distinct functions within the hippocampal network or whether a more generic computation can account for these observations9. Here, to disentangle the contribution of externally driven cues versus internal computation, we developed a task in mice in which space, auditory tones, rewards and context were juxtaposed with changing relevance. High-density electrophysiological recordings revealed that neurons were tuned to each of these modalities. By comparing movement paths and action sequences, we observed that external variables had limited direct influence on hippocampal firing. Instead, spiking was influenced by online action plans and modulated by goal uncertainty. Our results suggest that internally generated cell assembly sequences are selected and updated by action plans towards deliberate goals. The apparent tuning of hippocampal neuronal spiking to different sensory modalities might emerge due to alignment to the afforded action progression within a task rather than representation of external cues.

Diminished State Space Theory of Human Aging

Many new technologies, such as smartphones, computers, or public-access systems (like ticket-vending machines), are a challenge for older adults. One feature that these technologies have in common is that they involve underlying, partially observable, structures (state spaces) that determine the actions that are necessary to reach a certain goal (e.g., to move from one menu to another, to change a function, or to activate a new service). In this work we provide a theoretical, neurocomputational account to explain these behavioral difficulties in older adults. Based on recent findings from age-comparative computational- and cognitive-neuroscience studies, we propose that age-related impairments in complex goal-directed behavior result from an underlying deficit in the representation of state spaces of cognitive tasks. Furthermore, we suggest that these age-related deficits in adaptive decision-making are due to impoverished neural representations in the orbitofrontal cortex and hippocampus.

Phase Precession Relative to Turning Angle in Theta-Modulated Head Direction Cells

Grid and place cells typically fire at progressively earlier phases within each cycle of the theta rhythm as rodents run across their firing fields, a phenomenon known as theta phase precession. Here, we report theta phase precession relative to turning angle in theta-modulated head direction cells within the anteroventral thalamic nucleus (AVN). As rodents turn their heads, these cells fire at progressively earlier phases as head direction sweeps over their preferred tuning direction. The degree of phase precession increases with angular head velocity. Moreover, phase precession is more pronounced in those theta-modulated head direction cells that exhibit theta skipping, with a stronger theta-skipping effect correlating with a higher degree of phase precession. These findings are consistent with a ring attractor model that integrates external theta input with internal firing rate adaptation-a phenomenon we identified in head direction cells within AVN. Our results broaden the range of information known to be subject to neural phase coding and enrich our understanding of the neural dynamics supporting spatial orientation and navigation.

Navigation: Scanning your future path

A new model of how we plan pathways through the world shows that populations of neurons that code our current position, including entorhinal grid cells and head direction cells, could interact to aid planning by scanning alternating forward trajectories through the environment.

A systems model of alternating theta sweeps via firing rate adaptation

Place and grid cells provide a neural system for self-location and tend to fire in sequences within each cycle of the hippocampal theta rhythm when rodents run on a linear track. These sequences correspond to the decoded location of the animal sweeping forward from its current location ("theta sweeps"). However, recent findings in open-field environments show alternating left-right theta sweeps and propose a circuit for their generation. Here, we present a computational model of this circuit, comprising theta-modulated head-direction cells, conjunctive grid × direction cells, and pure grid cells, based on continuous attractor dynamics, firing rate adaptation, and modulation by the medial-septal theta rhythm. Due to firing rate adaptation, the head-direction ring attractor exhibits left-right sweeps coding for internal direction, providing an input to the grid cell attractor network shifted along the internal direction, via an intermediate layer of conjunctive grid × direction cells, producing left-right sweeps of position by grid cells. Our model explains the empirical findings, including the alignment of internal position and direction sweeps and the dependence of sweep length on grid spacing. It makes predictions for theta-modulated head-direction cells, including relationships between theta phase precession during turning, theta skipping, anticipatory firing, and directional tuning width, several of which we verify in experimental data from anteroventral thalamus. The model also predicts relationships between position and direction sweeps, running speed, and dorsal-ventral location within the entorhinal cortex. Overall, a simple intrinsic mechanism explains the complex theta dynamics of an internal direction signal within the hippocampal formation, with testable predictions.

Exploring replay

Animals face uncertainty about their environments due to initial ignorance or subsequent changes. They therefore need to explore. However, the algorithmic structure of exploratory choices in the brain still remains largely elusive. Artificial agents face the same problem, and a venerable idea in reinforcement learning is that they can plan appropriate exploratory choices offline, during the equivalent of quiet wakefulness or sleep. Although offline processing in humans and other animals, in the form of hippocampal replay and preplay, has recently been the subject of highly informative modelling, existing methods only apply to known environments. Thus, they cannot predict exploratory replay choices during learning and/or behaviour in the face of uncertainty. Here, we extend an influential theory of hippocampal replay and examine its potential role in approximately optimal exploration, deriving testable predictions for the patterns of exploratory replay choices in a paradigmatic spatial navigation task. Our modelling provides a normative interpretation of the available experimental data suggestive of exploratory replay. Furthermore, we highlight the importance of sequence replay, and license a range of new experimental paradigms that should further our understanding of offline processing.

Proactive and reactive construction of memory-based preferences

We are often faced with decisions we have never encountered before, requiring us to infer possible outcomes before making a choice. Computational theories suggest that one way to make these types of decisions is by accessing and linking related experiences stored in memory. Past work has shown that such memory-based preference construction can occur at a number of different timepoints relative to the moment a decision is made. Some studies have found that memories are integrated at the time a decision is faced (reactively) while others found that memory integration happens earlier, when memories were initially encoded (proactively). Here we offer a resolution to this inconsistency, demonstrating that these two strategies tradeoff rationally as a function of the associative structure of memory. We use fMRI to decode patterns of brain responses unique to categories of images in memory and find that proactive memory access is more common and allows more efficient inference. However, we also find that participants use reactive access when choice options are linked to a larger number of memory associations. Together, these results indicate that the brain judiciously conducts proactive inference by accessing memories ahead of time when conditions make this strategy more favorable.

Mental exploration of future choices during immobility theta oscillations

Mental exploration enables flexible evaluation of potential future choices, guiding decision-making without requiring direct real-world iterations. Although the hippocampus is known to be active while imagining the future, the precise mechanisms that support mental exploration of future choices remain unclear. In the hippocampus, the theta rhythm (4-12 Hz) is prevalent during movement and supports memory coding during real-world exploration by organizing neuronal activity patterns into short virtual path segments (theta sequences) around the rat's location. We observed these theta-related neural activity patterns during movement in a hippocampus-dependent working memory task and also, unexpectedly, theta oscillations and theta-related neural activity during immobility. Compared to standard theta sequences during movement, theta sequences during immobility differed in that they occurred at a shifted theta phase and preferentially represented remote locations, in particular the next choice in the working memory task. Coding for future locations was also observed during awake sharp wave ripple, but these short-lasting events occurred rarely and were biased toward frequently visited locations. Therefore, our findings suggest that recurring bouts of theta oscillations during immobility, which are also observed in primates and humans, support the cognitive demands of mental exploration in the hippocampal network and facilitate ongoing predictions of future choices.

Cognitive maps and schizophrenia

Structured internal representations ('cognitive maps') shape cognition, from imagining the future and counterfactual past, to transferring knowledge to new settings. Our understanding of how such representations are formed and maintained in biological and artificial neural networks has grown enormously. The cognitive mapping hypothesis of schizophrenia extends this enquiry to psychiatry, proposing that diverse symptoms - from delusions to conceptual disorganization - stem from abnormalities in how the brain forms structured representations. These abnormalities may arise from a confluence of neurophysiological perturbations (excitation-inhibition imbalance, resulting in attractor instability and impaired representational capacity) and/or environmental factors such as early life psychosocial stressors (which impinge on representation learning). This proposal thus links knowledge of neural circuit abnormalities, environmental risk factors, and symptoms.

Neuroeconomically dissociable forms of mental accounting are altered in a mouse model of diabetes

Those with diabetes mellitus are at high-risk of developing psychiatric disorders, especially mood disorders, yet the link between hyperglycemia and altered motivation has not been thoroughly explored. Here, we characterized value-based decision-making behavior of a streptozocin-induced diabetic mouse model on Restaurant Row, a naturalistic neuroeconomic foraging paradigm capable of behaviorally capturing multiple decision systems known to depend on dissociable neural circuits. Mice made self-paced choices on a daily limited time-budget, accepting or rejecting reward offers based on cost (delays cued by tone pitch) and subjective value (flavors), in a closed-economy system tested across months. We found streptozocin-treated mice disproportionately undervalued less-preferred flavors and inverted their meal-consumption patterns shifted toward a more costly strategy overprioritizing high-value rewards. These foraging behaviors were driven by impairments in multiple decision-making processes, including the ability to deliberate when engaged in conflict and cache the value of the passage of time as sunk costs. Surprisingly, diabetes-induced changes in motivation depended not only on the type of choice being made, but also on the salience of reward-scarcity in the environment. These findings suggest that complex relationships between metabolic dysfunction and dissociable valuation algorithms underlying unique cognitive heuristics and sensitivity to opportunity costs can disrupt distinct computational processes leading to comorbid psychiatric vulnerabilities.

Distinct roles of dentate gyrus and medial entorhinal cortex inputs for phase precession and temporal correlations in the hippocampal CA3 area

The hippocampal CA3 subregion is a densely connected recurrent circuit that supports memory by generating and storing sequential neuronal activity patterns that reflect recent experience. While theta phase precession is thought to be critical for generating sequential activity during memory encoding, the circuit mechanisms that support this computation across hippocampal subregions are unknown. By analyzing CA3 network activity in the absence of each of its theta-modulated external excitatory inputs, we show necessary and unique contributions of the dentate gyrus (DG) and the medial entorhinal cortex (MEC) to phase precession. DG inputs are essential for preferential spiking of CA3 cells during late theta phases and for organizing the temporal order of neuronal firing, while MEC inputs sharpen the temporal precision throughout the theta cycle. A computational model that accounts for empirical findings suggests that the unique contribution of DG inputs to theta-related spike timing is supported by targeting precisely timed inhibitory oscillations. Our results thus identify a novel and unique functional role of the DG for sequence coding in the CA3 circuit.

Development of the SPEAR Model: Separate Phases of Encoding and Retrieval Are Necessary for Storing Multiple Overlapping Associative Memories

In keeping with the historical focus of this special issue of Hippocampus, this paper reviews the history of my development of the SPEAR model. The SPEAR model proposes that separate phases of encoding and retrieval (SPEAR) allow effective storage of multiple overlapping associative memories in the hippocampal formation and other cortical structures. The separate phases for encoding and retrieval are proposed to occur within different phases of theta rhythm with a cycle time on the order of 125 ms. The same framework applies to the slower transition between encoding and consolidation dynamics regulated by acetylcholine. The review includes description of the experimental data on acetylcholine and theta rhythm that motivated this model, the realization that existing associative memory models require these different dynamics, and the subsequent experimental data supporting these dynamics. The review also includes discussion of my work on the encoding of episodic memories as spatiotemporal trajectories, and some personal description of the episodic memories from my own spatiotemporal trajectory as I worked on this model.

Dorsal hippocampus represents locations to avoid as well as locations to approach during approach-avoidance conflict

Worrying about perceived threats is a hallmark of multiple psychological disorders including anxiety. This concern about future events is particularly important when an individual is faced with an approach-avoidance conflict. Potential goals to approach are known to be represented in the dorsal hippocampus during theta cycles. Similarly, important information that is distant from the animal's position is represented during hippocampal high-synchrony events (HSEs), which coincide with sharp-wave ripples (SWRs). It is likely that potential future threats may be similarly represented. We examined how threats and rewards were represented within the hippocampus during approach-avoidance conflicts in rats faced with a predator-like robot guarding a food reward. We found decoding of the pseudo-predator's location during HSEs when hesitating in the nest and during theta prior to retreating as the rats approached the pseudo-predator. After the first attack, we observed new place fields appearing at the location of the robot (not the location the rat was when attacked). The anxiolytic diazepam reduced anxiety-like behavior and altered hippocampal local field potentials (LFPs), including reducing SWRs, suggesting that one potential mechanism of diazepam's actions may be through altered representations of imagined threat. These results suggest that hippocampal representation of potential threats could be an important mechanism that underlies worry and a potential target for anxiolytics.

Mental Time Travel: A Retrospective

Because imagination activates the same neural circuits used in understanding the present, one can access that imagination even in non-linguistic animals through decoding techniques applied to large neural ensembles. This personal retrospective traces the history of the initial discovery that hippocampal theta sequences sweep forward to goals during moments of deliberation and discusses the history that was necessary to put ourselves in the position to recognize this signal. It also discusses how that discovery fits into the larger picture of hippocampal function and the concept of cognition as computation.

Environmental complexity modulates information processing and the balance between decision-making systems

Behavior in naturalistic scenarios occurs in diverse environments. Adaptive strategies rely on multiple neural circuits and competing decision systems. However, past studies of rodent decision making have largely measured behavior in simple environments. To fill this gap, we recorded neural ensembles from hippocampus (HC), dorsolateral striatum (DLS), and dorsomedial prefrontal cortex (dmPFC) while rats foraged for food under changing rules in environments with varying topological complexity. Environmental complexity increased behavioral variability, lengthened HC nonlocal sequences, and modulated action caching. We found contrasting representations between DLS and HC, supporting a competition between decision systems. dmPFC activity was indicative of setting this balance, in particular predicting the extent of HC non-local coding. Inactivating mPFC impaired short-term behavioral adaptation and produced long-term deficits in balancing decision systems. Our findings reveal the dynamic nature of decision-making systems and how environmental complexity modulates their engagement with implications for behavior in naturalistic environments.

Complexity demands more flexibility and the prefrontal cortex has an answer

Cognitive flexibility allows us to adapt our behavior to keep up with a changing environment. In this issue of Neuron, Mugan and colleagues1 manipulate the complexity of an environment to demonstrate how the medial prefrontal cortex controls a cognitive flexibility circuit featuring the dorsolateral striatum and hippocampus.

Proactive and reactive construction of memory-based preferences

We are often faced with decisions we have never encountered before, requiring us to infer possible outcomes before making a choice. Computational theories suggest that one way to make these types of decisions is by accessing and linking related experiences stored in memory. Past work has shown that such memory-based preference construction can occur at a number of different timepoints relative to the moment a decision is made. Some studies have found that memories are integrated at the time a decision is faced (reactively) while others found that memory integration happens earlier, when memories were initially encoded (proactively). Here we offer a resolution to this inconsistency, demonstrating that these two strategies tradeoff rationally as a function of the associative structure of memory. We use fMRI to decode patterns of brain responses unique to categories of images in memory and find that proactive memory access is more common and allows more efficient inference. However, we also find that participants use reactive access when choice options are linked to a larger number of memory associations. Together, these results indicate that the brain judiciously conducts proactive inference by accessing memories ahead of time when conditions make this strategy more favorable.

The perceived controllability of negatively-valenced episodic future thinking modulates delay discounting

Intertemporal decision-making is important for both economy and physical health. Nevertheless, in daily life, individuals tend to prefer immediate and smaller rewards to delayed and larger rewards, which is known as delay discounting (DD). Episodic future thinking (EFT) has been proven to influence DD. However, there is still no inconsistent conclusion on the effect of negative EFT on DD. Considering the perceived controllability of negative EFT may address the issue (Controllability refers to the extent to which progress and result of an event could be controlled by ourselves). In the current study, we manipulated EFT conditions (baseline, neutral EFT, negative-controllable EFT and negative-uncontrollable EFT), delayed time (i.e. 1 week, 1 month, 3 months, 6 months, 1 year and 3 years) and reward magnitude (small, large). We mainly found that when experiencing negative-uncontrollable EFT compared to negative-controllable EFT in the delayed time of 6 months with large rewards, individuals chose more delayed rewards, suggesting that negative-uncontrollable EFT effectively reduced DD under conditions of both large-magnitude reward and longer delayed time. The current study provides new insight for healthy groups on optimising EFT. In that case, individuals are able to gain long-term benefits in financial management and healthcare.

A cellular basis for mapping behavioural structure

To flexibly adapt to new situations, our brains must understand the regularities in the world, as well as those in our own patterns of behaviour. A wealth of findings is beginning to reveal the algorithms that we use to map the outside world1-6. However, the biological algorithms that map the complex structured behaviours that we compose to reach our goals remain unknown. Here we reveal a neuronal implementation of an algorithm for mapping abstract behavioural structure and transferring it to new scenarios. We trained mice on many tasks that shared a common structure (organizing a sequence of goals) but differed in the specific goal locations. The mice discovered the underlying task structure, enabling zero-shot inferences on the first trial of new tasks. The activity of most neurons in the medial frontal cortex tiled progress to goal, akin to how place cells map physical space. These 'goal-progress cells' generalized, stretching and compressing their tiling to accommodate different goal distances. By contrast, progress along the overall sequence of goals was not encoded explicitly. Instead, a subset of goal-progress cells was further tuned such that individual neurons fired with a fixed task lag from a particular behavioural step. Together, these cells acted as task-structured memory buffers, implementing an algorithm that instantaneously encoded the entire sequence of future behavioural steps, and whose dynamics automatically computed the appropriate action at each step. These dynamics mirrored the abstract task structure both on-task and during offline sleep. Our findings suggest that schemata of complex behavioural structures can be generated by sculpting progress-to-goal tuning into task-structured buffers of individual behavioural steps.

Measuring episodic memory and mental time travel: crossing the species gap

Mental time travel is the projection of the mind into the past or future, and relates to experiential aspects of episodic memory, and episodic future thinking. Framing episodic memory and future thinking in this way causes a challenge when studying memory in animals, where demonstration of this mental projection is prevented by the absence of language. However, there is good evidence that non-human animals pass tests of episodic memory that are based on behavioural criteria, meaning a better understanding needs to be had of the relationship between episodic memory and mental time travel. We argue that mental time travel and episodic memory are not synonymous, and that mental time travel is neither a requirement of, nor an irrelevance to, episodic memory. Mental time travel can allow improved behavioural choices based on episodic memory, and work in all species (including humans) should include careful consideration of the behavioural outputs being measured. This article is part of the theme issue 'Elements of episodic memory: lessons from 40 years of research'.

Is episodic-like memory like episodic memory?

Episodic memory involves the conscious recollection of personally experienced events and when absent, results in profound losses to the typical human conscious experience. Over the last 2.5 decades, the debate surrounding whether episodic memory is unique to humans has seen a lot of controversy and accordingly has received significant research attention. Various behavioural paradigms have been developed to test episodic-like memory; a term designed to reflect the behavioural characteristics of episodic memory in the absence of evidence for consciously experienced recall. In this review, we first outline the most influential paradigms that have been developed to assess episodic-like memory across a variety of non-human taxa (including mammals, birds and cephalopods), namely the what-where-when memory, incidental encoding and unexpected question, and source memory paradigms. Then, we examine whether various key features of human episodic memory are conceptually represented in episodic-like memory across phylogenetically and neurologically diverse taxa, identifying similarities, differences and gaps in the literature. We conclude that the evidence is mixed, and as episodic memory encompasses a variety of cognitive structures and processes, research on episodic-like memory in non-humans should follow this multifaceted approach and assess evidence across various behavioural paradigms that each target different aspects of human episodic memory.This article is part of the theme issue 'Elements of episodic memory: lessons from 40 years of research'.

Closed-loop modulation of remote hippocampal representations with neurofeedback

Humans can remember specific remote events without acting on them and influence which memories are retrieved based on internal goals. However, animal models typically present sensory cues to trigger memory retrieval and then assess retrieval based on action. Thus, it is difficult to determine whether measured neural activity patterns relate to the cue(s), the memory, or the behavior. We therefore asked whether retrieval-related neural activity could be generated in animals without cues or a behavioral report. We focused on hippocampal "place cells" which primarily represent the animal's current location (local representations) but can also represent locations away from the animal (remote representations). We developed a neurofeedback system to reward expression of remote representations and found that rats could learn to generate specific spatial representations that often jumped directly to the experimenter-defined target location. Thus, animals can deliberately engage remote representations, enabling direct study of retrieval-related activity in the brain.

Touchscreen response precision is sensitive to the explore/exploit tradeoff

The explore/exploit tradeoff is a fundamental property of choice selection during reward-guided decision making. In perceptual decision making, higher certainty decisions are more motorically precise, even when the decision does not require motor accuracy. However, while we can parametrically control uncertainty in perceptual tasks, we do not know what variables - if any - shape motor precision and reflect subjective certainty during reward-guided decision making. Touchscreens are increasingly used across species to measure choice, but provide no tactile feedback on whether an action is precise or not, and therefore provide a valuable opportunity to determine whether actions differ in precision due to explore/exploit state, reward, or individual variables. We find all three of these factors exert independent drives towards increased precision. During exploit states, successive touches to the same choice are closer together than those made in an explore state, consistent with exploit states reflecting higher certainty and/or motor stereotypy in responding. However, exploit decisions might be expected to be rewarded more frequently than explore decisions. We find that exploit choice precision is increased independently of a separate increase in precision due to immediate past reward, suggesting multiple mechanisms regulating choice precision. Finally, we see evidence that male mice in general are less precise in their interactions with the touchscreen than females, even when exploiting a choice. These results suggest that as exploit behavior emerges in reward-guided decision making, individuals become more motorically precise reflecting increased certainty, even when decision choice does not require additional motor accuracy, but this is influenced by individual differences and prior reward. These data uncover the hidden potential for touchscreen tasks in any species to uncover the latent neural states that unite cognition and movement.

Repulsion of CA3 / dentate gyrus representations is driven by distinct internal beliefs in the face of ambiguous sensory input

Recent human neuroimaging studies of episodic memory have revealed a counterintuitive phenomenon in the hippocampus: when events are highly similar, corresponding hippocampal activity patterns are sometimes less correlated than activity patterns associated with unrelated events. This phenomenon-repulsion-is not accounted for by most theories of the hippocampus, and the conditions that trigger repulsion remain poorly understood. Here, we used a spatial route-learning task and high-resolution fMRI in humans to test whether hippocampal repulsion is fundamentally driven by internal beliefs about the environment. By precisely measuring participants' internal beliefs and actively manipulating them, we show that repulsion selectively occurred in hippocampal subfields CA3 and dentate gyrus when visual input was ambiguous-or even identical-but internal beliefs were distinct. These findings firmly establish conditions that elicit repulsion and have broad relevance to theories of hippocampal function and to the fields of human episodic memory and rodent spatial navigation.

Integration of rate and phase codes by hippocampal cell-assemblies supports flexible encoding of spatiotemporal context

Spatial information is encoded by location-dependent hippocampal place cell firing rates and sub-second, rhythmic entrainment of spike times. These rate and temporal codes have primarily been characterized in low-dimensional environments under limited cognitive demands; but how is coding configured in complex environments when individual place cells signal several locations, individual locations contribute to multiple routes and functional demands vary? Quantifying CA1 population dynamics of male rats during a decision-making task, here we show that the phase of individual place cells' spikes relative to the local theta rhythm shifts to differentiate activity in different place fields. Theta phase coding also disambiguates repeated visits to the same location during different routes, particularly preceding spatial decisions. Using unsupervised detection of cell assemblies alongside theoretical simulation, we show that integrating rate and phase coding mechanisms dynamically recruits units to different assemblies, generating spiking sequences that disambiguate episodes of experience and multiplexing spatial information with cognitive context.

Hippocampal representations of alternative possibilities are flexibly generated to meet cognitive demands

The cognitive ability to go beyond the present to consider alternative possibilities, including potential futures and counterfactual pasts, can support adaptive decision making. Complex and changing real-world environments, however, have many possible alternatives. Whether and how the brain can select among them to represent alternatives that meet current cognitive needs remains unknown. We therefore examined neural representations of alternative spatial locations in the rat hippocampus during navigation in a complex patch foraging environment with changing reward probabilities. We found representations of multiple alternatives along paths ahead and behind the animal, including in distant alternative patches. Critically, these representations were modulated in distinct patterns across successive trials: alternative paths were represented proportionate to their evolving relative value and predicted subsequent decisions, whereas distant alternatives were prevalent during value updating. These results demonstrate that the brain modulates the generation of alternative possibilities in patterns that meet changing cognitive needs for adaptive behavior.

Hippocampal neuronal activity is aligned with action plans

Neurons in the hippocampus are correlated with different variables, including space, time, sensory cues, rewards, and actions, where the extent of tuning depends on ongoing task demands. However, it remains uncertain whether such diverse tuning corresponds to distinct functions within the hippocampal network or if a more generic computation can account for these observations. To disentangle the contribution of externally driven cues versus internal computation, we developed a task in mice where space, auditory tones, rewards, and context were juxtaposed with changing relevance. High-density electrophysiological recordings revealed that neurons were tuned to each of these modalities. By comparing movement paths and action sequences, we observed that external variables had limited direct influence on hippocampal firing. Instead, spiking was influenced by online action plans modulated by goal uncertainty. Our results suggest that internally generated cell assembly sequences are selected and updated by action plans toward deliberate goals. The apparent tuning of hippocampal neuronal spiking to different sensory modalities might emerge due to alignment to the afforded action progression within a task rather than representation of external cues.

Learning to Move and Plan like the Knight: Sequential Decision Making with a Novel Motor Mapping

Many skills that humans acquire throughout their lives, such as playing video games or sports, require substantial motor learning and multi-step planning. While both processes are typically studied separately, they are likely to interact during the acquisition of complex motor skills. In this work, we studied this interaction by assessing human performance in a sequential decision-making task that requires the learning of a non-trivial motor mapping. Participants were tasked to move a cursor from start to target locations in a grid world, using a standard keyboard. Notably, the specific keys were arbitrarily mapped to a movement rule resembling the Knight chess piece. In Experiment 1, we showed the learning of this mapping in the absence of planning, led to significant improvements in the task when presented with sequential decisions at a later stage. Computational modeling analysis revealed that such improvements resulted from an increased learning rate about the state transitions of the motor mapping, which also resulted in more flexible planning from trial to trial (less perseveration or habitual responses). In Experiment 2, we showed that incorporating mapping learning into the planning process, allows us to capture (1) differential task improvements for distinct planning horizons and (2) overall lower performance for longer horizons. Additionally, model analysis suggested that participants may limit their search to three steps ahead. We hypothesize that this limitation in planning horizon arises from capacity constraints in working memory, and may be the reason complex skills are often broken down into individual subroutines or components during learning.

Predictive sequence learning in the hippocampal formation

The hippocampus receives sequences of sensory inputs from the cortex during exploration and encodes the sequences with millisecond precision. We developed a predictive autoencoder model of the hippocampus including the trisynaptic and monosynaptic circuits from the entorhinal cortex (EC). CA3 was trained as a self-supervised recurrent neural network to predict its next input. We confirmed that CA3 is predicting ahead by analyzing the spike coupling between simultaneously recorded neurons in the dentate gyrus, CA3, and CA1 of the mouse hippocampus. In the model, CA1 neurons signal prediction errors by comparing CA3 predictions to the next direct EC input. The model exhibits the rapid appearance and slow fading of CA1 place cells and displays replay and phase precession from CA3. The model could be learned in a biologically plausible way with error-encoding neurons. Similarities between the hippocampal and thalamocortical circuits suggest that such computation motif could also underlie self-supervised sequence learning in the cortex.

Hierarchical Gradients of Encoded Spatial and Sensory Information in the Neocortex Are Attenuated by Dorsal Hippocampal Lesions

During navigation, the neocortex actively integrates learned spatial context with current sensory experience to guide behaviors. However, the relative encoding of spatial and sensorimotor information among cortical cells, and whether hippocampal feedback continues to modify these properties after learning, remains poorly understood. Thus, two-photon microscopy of male and female Thy1-GCaMP6s mice was used to longitudinally image neurons spanning superficial retrosplenial cortex and layers II-Va of primary and secondary motor cortices before and after bilateral dorsal hippocampal lesions. During behavior on a familiar cued treadmill, the locations of two obstacles were interchanged to decouple place-tuning from cue-tuning among position-correlated cells with fields at those locations. Subpopulations of place and cue cells each formed interareal gradients such that higher-level cortical regions exhibited higher fractions of place cells, whereas lower-level regions exhibited higher fractions of cue cells. Position-correlated cells in the motor cortex also formed translaminar gradients; more superficial cells were more likely to exhibit fields and were more sparsely and precisely tuned than deeper cells. After dorsal hippocampal lesions, a neural representation of the learned environment persisted, but retrosplenial cortex exhibited significantly increased cue-tuning, and, in motor cortices, both position-correlated cell recruitment and population activity at the unstable obstacle locations became more homogeneously elevated across laminae. Altogether, these results support that the hippocampus continues to modulate cortical responses in familiar environments, and the relative impact of descending feedback obeys hierarchical interareal and interlaminar gradients opposite to the flow of ascending sensory inputs.

Deliberative Behaviors and Prefrontal-Hippocampal Coupling are Disrupted in a Rat Model of Fetal Alcohol Spectrum Disorders

Fetal alcohol spectrum disorders (FASDs) are characterized by a range of physical, cognitive, and behavioral impairments. Determining how temporally specific alcohol exposure (AE) affects neural circuits is crucial to understanding the FASD phenotype. Third trimester AE can be modeled in rats by administering alcohol during the first two postnatal weeks, which damages the medial prefrontal cortex (mPFC), thalamic nucleus reuniens, and hippocampus (HPC), structures whose functional interactions are required for working memory and executive function. Therefore, we hypothesized that AE during this period would impair working memory, disrupt choice behaviors, and alter mPFC-HPC oscillatory synchrony. To test this hypothesis, we recorded local field potentials from the mPFC and dorsal HPC as AE and sham intubated (SI) rats performed a spatial working memory task in adulthood and implemented algorithms to detect vicarious trial and errors (VTEs), behaviors associated with deliberative decision-making. We found that, compared to the SI group, the AE group performed fewer VTEs and demonstrated a disturbed relationship between VTEs and choice outcomes, while spatial working memory was unimpaired. This behavioral disruption was accompanied by alterations to mPFC and HPC oscillatory activity in the theta and beta bands, respectively, and a reduced prevalence of mPFC-HPC synchronous events. When trained on multiple behavioral variables, a machine learning algorithm could accurately predict whether rats were in the AE or SI group, thus characterizing a potential phenotype following third trimester AE. Together, these findings indicate that third trimester AE disrupts mPFC-HPC oscillatory interactions and choice behaviors.

Entorhinal cortex represents task-relevant remote locations independent of CA1

Neurons can collectively represent the current sensory experience while an animal is exploring its environment or remote experiences while the animal is immobile. These remote representations can reflect learned associations1-3 and be required for learning4. Neurons in the medial entorhinal cortex (MEC) reflect the animal's current location during movement5, but little is known about what MEC neurons collectively represent during immobility. Here, we recorded thousands of neurons in superficial MEC and dorsal CA1 as mice learned to associate two pairs of rewarded locations. We found that during immobility, the MEC neural population frequently represented positions far from the animal's location, which we defined as 'non-local coding'. Cells with spatial firing fields at remote locations drove non-local coding, even as cells representing the current position remained active. While MEC non-local coding has been reported during sharp-wave ripples in downstream CA16, we observed non-local coding more often outside of ripples. In fact, CA1 activity was less coordinated with MEC during non-local coding. We further observed that non-local coding was pertinent to the task, as MEC preferentially represented remote task-relevant locations at appropriate times, while rarely representing task-irrelevant locations. Together, this work raises the possibility that MEC non-local coding could strengthen associations between locations independently from CA1.

Firing rate adaptation affords place cell theta sweeps, phase precession, and procession

Hippocampal place cells in freely moving rodents display both theta phase precession and procession, which is thought to play important roles in cognition, but the neural mechanism for producing theta phase shift remains largely unknown. Here, we show that firing rate adaptation within a continuous attractor neural network causes the neural activity bump to oscillate around the external input, resembling theta sweeps of decoded position during locomotion. These forward and backward sweeps naturally account for theta phase precession and procession of individual neurons, respectively. By tuning the adaptation strength, our model explains the difference between 'bimodal cells' showing interleaved phase precession and procession, and 'unimodal cells' in which phase precession predominates. Our model also explains the constant cycling of theta sweeps along different arms in a T-maze environment, the speed modulation of place cells' firing frequency, and the continued phase shift after transient silencing of the hippocampus. We hope that this study will aid an understanding of the neural mechanism supporting theta phase coding in the brain.

RECORD, a high-throughput, customizable system that unveils behavioral strategies leveraged by rodents during foraging-like decision-making

Translational studies benefit from experimental designs where laboratory organisms use human-relevant behaviors. One such behavior is decision-making, however studying complex decision-making in rodents is labor-intensive and typically restricted to two levels of cost/reward. We design a fully automated, inexpensive, high-throughput framework to study decision-making across multiple levels of rewards and costs: the REward-COst in Rodent Decision-making (RECORD) system. RECORD integrates three components: 1) 3D-printed arenas, 2) custom electronic hardware, and 3) software. We validated four behavioral protocols without employing any food or water restriction, highlighting the versatility of our system. RECORD data exposes heterogeneity in decision-making both within and across individuals that is quantifiably constrained. Using oxycodone self-administration and alcohol-consumption as test cases, we reveal how analytic approaches that incorporate behavioral heterogeneity are sensitive to detecting perturbations in decision-making. RECORD is a powerful approach to studying decision-making in rodents, with features that facilitate translational studies of decision-making in psychiatric disorders.

Sex differences in change-of-mind neuroeconomic decision-making is modulated by LINC00473 in medial prefrontal cortex

Changing one's mind is a complex cognitive phenomenon involving a continuous re-appraisal of the trade-off between past costs and future value. Recent work modeling this behavior across species has established associations between aspects of this choice process and their contributions to altered decision-making in psychopathology. Here, we investigated the actions in medial prefrontal cortex (mPFC) neurons of long intergenic non-coding RNA, LINC00473, known to induce stress resilience in a striking sex-dependent manner, but whose role in cognitive function is unknown. We characterized complex decision-making behavior in male and female mice longitudinally in our neuroeconomic foraging paradigm, Restaurant Row, following virus-mediated LINC00473 expression in mPFC neurons. On this task, mice foraged for their primary source of food among varying costs (delays) and subjective value (flavors) while on a limited time-budget during which decisions to accept and wait for rewards were separated into discrete stages of primary commitments and secondary re-evaluations. We discovered important differences in decision-making behavior between female and male mice. LINC00473 expression selectively influenced multiple features of re-evaluative choices, without affecting primary decisions, in female mice only. These behavioral effects included changing how mice (i) cached the value of the passage of time and (ii) weighed their history of economically disadvantageous choices. Both processes were uniquely linked to change-of-mind decisions and underlie the computational bases of distinct aspects of counterfactual thinking. These findings reveal a key bridge between a molecular driver of stress resilience and psychological mechanisms underlying sex-specific decision-making proclivities.

The homogenous hippocampus: How hippocampal cells process available and potential goals

We present here a view of the firing patterns of hippocampal cells that is contrary, both functionally and anatomically, to conventional wisdom. We argue that the hippocampus responds to efference copies of goals encoded elsewhere; and that it uses these to detect and resolve conflict or interference between goals in general. While goals can involve space, hippocampal cells do not encode spatial (or other special types of) memory, as such. We also argue that the transverse circuits of the hippocampus operate in an essentially homogeneous way along its length. The apparently different functions of different parts (e.g. memory retrieval versus anxiety) result from the different (situational/motivational) inputs on which those parts perform the same fundamental computational operations. On this view, the key role of the hippocampus is the iterative adjustment, via Papez-like circuits, of synaptic weights in cell assemblies elsewhere.

Flexible decision-making is related to strategy learning, vicarious trial and error, and medial prefrontal rhythms during spatial set-shifting

Flexible decision-making requires a balance between exploring features of an environment and exploiting prior knowledge. Behavioral flexibility is typically measured by how long it takes subjects to consistently make accurate choices after reward contingencies switch or task rules change. This measure, however, only allows for tracking flexibility across multiple trials, and does not assess the degree of flexibility. Plus, although increases in decision-making accuracy are strong indicators of learning, other decision-making behaviors have also been suggested as markers of flexibility, such as the on-the-fly decision reversals known as vicarious trial and error (VTE) or switches to a different, but incorrect, strategy. We sought to relate flexibility, learning, and neural activity by comparing choice history-derived evaluation of strategy use with changes in decision-making accuracy and VTE behavior while recording from the medial prefrontal cortex (mPFC) in rats. Using a set-shifting task that required rats to repeatedly switch between spatial decision-making strategies, we show that a previously developed strategy likelihood estimation procedure could identify putative learning points based on decision history. We confirm the efficacy of learning point estimation by showing increases in decision-making accuracy aligned to the learning point. Additionally, we show increases in the rate of VTE behavior surrounding identified learning points. By calculating changes in strategy likelihoods across trials, we tracked flexibility on a trial-by-trial basis and show that flexibility scores also increased around learning points. Further, we demonstrate that VTE behaviors could be separated into indecisive and deliberative subtypes depending on whether they occurred during periods of high or low flexibility and whether they led to correct or incorrect choice outcomes. Field potential recordings from the mPFC during decisions exhibited increased beta band activity on trials with VTE compared to non-VTE trials, as well as increased gamma during periods when learned strategies could be exploited compared to prelearning, exploratory periods. This study demonstrates that increased behavioral flexibility and VTE rates are often aligned to task learning. These relationships can break down, however, suggesting that VTE is not always an indicator of deliberative decision-making. Additionally, we further implicate the mPFC in decision-making and learning by showing increased beta-based activity on VTE trials and increased gamma after learning.

A Double Hit of Social and Economic Stress in Mice Precipitates Changes in Decision-Making Strategies

BACKGROUND

Economic stress can serve as a second hit for people who have already accumulated a history of adverse life experiences. How one recovers from a setback is a core feature of resilience but is seldom captured in animal studies.

METHODS

We challenged mice in a novel 2-hit stress model by first exposing them to chronic social defeat stress and then testing adaptations to increasing reward scarcity on a neuroeconomic task. Mice were tested across months on the Restaurant Row task, during which they foraged daily for their primary source of food while on a limited time budget in a closed-economy system. An abrupt transition into a reward-scarce environment elicits an economic challenge, precipitating a drop in food intake and body weight to which mice must respond to survive.

RESULTS

We found that mice with a history of social stress mounted a robust behavioral response to this economic challenge that was achieved through a complex redistribution of time allocation among competing opportunities. Interestingly, we found that mice with a history of social defeat displayed changes in the development of decision-making policies during the recovery process that are important not only for ensuring food security necessary for survival but also prioritizing subjective value and that these changes emerged only for certain types of choices.

CONCLUSIONS

These findings indicate that an individual's capacity to recover from economic challenges depends on that person's prior history of stress and can affect multiple decision-making aspects of subjective well-being, thus highlighting a motivational balance that may be altered in stress-related disorders such as depression.

A recurrent network model of planning explains hippocampal replay and human behavior

When faced with a novel situation, people often spend substantial periods of time contemplating possible futures. For such planning to be rational, the benefits to behavior must compensate for the time spent thinking. Here, we capture these features of behavior by developing a neural network model where planning itself is controlled by the prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences from its own policy, which we call 'rollouts'. In a spatial navigation task, the agent learns to plan when it is beneficial, which provides a normative explanation for empirical variability in human thinking times. Additionally, the patterns of policy rollouts used by the artificial agent closely resemble patterns of rodent hippocampal replays. Our work provides a theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by-and adaptively affect-prefrontal dynamics.

The role of the human hippocampus in decision-making under uncertainty

The role of the hippocampus in decision-making is beginning to be more understood. Because of its prospective and inferential functions, we hypothesized that it might be required specifically when decisions involve the evaluation of uncertain values. A group of individuals with autoimmune limbic encephalitis-a condition known to focally affect the hippocampus-were tested on how they evaluate reward against uncertainty compared to reward against another key attribute: physical effort. Across four experiments requiring participants to make trade-offs between reward, uncertainty and effort, patients with acute limbic encephalitis demonstrated blunted sensitivity to reward and effort whenever uncertainty was considered, despite demonstrating intact uncertainty sensitivity. By contrast, the valuation of these two attributes (reward and effort) was intact on uncertainty-free tasks. Reduced sensitivity to changes in reward under uncertainty correlated with the severity of hippocampal damage. Together, these findings provide evidence for a context-sensitive role of the hippocampus in value-based decision-making, apparent specifically under conditions of uncertainty.

Remapping revisited: how the hippocampus represents different spaces

The representation of distinct spaces by hippocampal place cells has been linked to changes in their place fields (the locations in the environment where the place cells discharge strongly), a phenomenon that has been termed 'remapping'. Remapping has been assumed to be accompanied by the reorganization of subsecond cofiring relationships among the place cells, potentially maximizing hippocampal information coding capacity. However, several observations challenge this standard view. For example, place cells exhibit mixed selectivity, encode non-positional variables, can have multiple place fields and exhibit unreliable discharge in fixed environments. Furthermore, recent evidence suggests that, when measured at subsecond timescales, the moment-to-moment cofiring of a pair of cells in one environment is remarkably similar in another environment, despite remapping. Here, I propose that remapping is a misnomer for the changes in place fields across environments and suggest instead that internally organized manifold representations of hippocampal activity are actively registered to different environments to enable navigation, promote memory and organize knowledge.

An Improved Dyna-Q Algorithm Inspired by the Forward Prediction Mechanism in the Rat Brain for Mobile Robot Path Planning

The traditional Model-Based Reinforcement Learning (MBRL) algorithm has high computational cost, poor convergence, and poor performance in robot spatial cognition and navigation tasks, and it cannot fully explain the ability of animals to quickly adapt to environmental changes and learn a variety of complex tasks. Studies have shown that vicarious trial and error (VTE) and the hippocampus forward prediction mechanism in rats and other mammals can be used as key components of action selection in MBRL to support "goal-oriented" behavior. Therefore, we propose an improved Dyna-Q algorithm inspired by the forward prediction mechanism of the hippocampus to solve the above problems and tackle the exploration-exploitation dilemma of Reinforcement Learning (RL). This algorithm alternately presents the potential path in the future for mobile robots and dynamically adjusts the sweep length according to the decision certainty, so as to determine action selection. We test the performance of the algorithm in a two-dimensional maze environment with static and dynamic obstacles, respectively. Compared with classic RL algorithms like State-Action-Reward-State-Action (SARSA) and Dyna-Q, the algorithm can speed up spatial cognition and improve the global search ability of path planning. In addition, our method reflects key features of how the brain organizes MBRL to effectively solve difficult tasks such as navigation, and it provides a new idea for spatial cognitive tasks from a biological perspective.