Task Demands Predict a Dynamic Switch in the Content of Awake Hippocampal Replay

Non-feature-specific elevated responses and feature-specific backward replay in human brain induced by visual sequence exposure

The ability of cortical circuits to adapt in response to experience is a fundamental property of the brain. After exposure to a moving dot sequence, flashing a dot as a cue at the starting point of the sequence can elicit successive elevated responses even in the absence of the sequence. These cue-triggered elevated responses have been shown to play a crucial role in predicting future events in dynamic environments. However, temporal sequences we are exposed to typically contain rich feature information. It remains unknown whether the elevated responses are feature-specific and, more crucially, how the brain organizes sequence information after exposure. To address these questions, participants were exposed to a predefined sequence of four motion directions for about 30 min, followed by the presentation of the start or end motion direction of the sequence as a cue. Surprisingly, we found that cue-triggered elevated responses were not specific to any motion direction. Interestingly, motion direction information was spontaneously reactivated, and the motion sequence was backward replayed in a time-compressed manner. These effects were observed even after brief exposure. Notably, no replay events were observed when the second or third motion direction of the sequence served as a cue. Further analyses revealed that activity in the medial temporal lobe (MTL) preceded the ripple power increase in visual cortex at the onset of replay, implying a coordinated relationship between the activities in the MTL and visual cortex. Together, these findings demonstrate that visual sequence exposure induces twofold brain plasticity that may simultaneously serve for different functional purposes. The non-feature-specific elevated responses may facilitate general processing of upcoming stimuli, whereas the feature-specific backward replay may underpin passive learning of visual sequences.

Awake replay: off the clock but on the job

Hippocampal replay is widely thought to support two key cognitive functions: online decision-making and offline memory consolidation. In this review, we take a closer look at the hypothesized link between awake replay and online decision-making in rodents, and find only marginal evidence in support of this role. By contrast, the consolidation view is bolstered by new computational ideas and recent data, suggesting that (i) replay performs offline fictive learning for later goal-oriented behavior; and (ii) replay tags memories prior to sleep, prioritizing them for consolidation. Based on these recent advances, we favor an updated and refined role for awake replay - that is, supporting prioritized offline learning and tagging outside the hippocampus - rather than a direct, online role in guiding behavior.

The time course and organization of hippocampal replay

The mechanisms by which the brain replays neural activity sequences remain unknown. Recording from large ensembles of hippocampal place cells in freely behaving rats, we observed that replay content is strictly organized over multiple timescales and governed by self-avoidance. After movement cessation, replays avoided the animal's previous path for 3 seconds. Chains of replays avoided self-repetition over a shorter timescale. We used a continuous attractor model of neural activity to demonstrate that neuronal fatigue both generates replay sequences and produces self-avoidance over the observed timescales. In addition, replay of past experience became predominant later into the stopping period, in a manner requiring cortical input. These results indicate a mechanism for replay generation that unexpectedly constrains which sequences can be produced across time.

Evaluating hippocampal replay without a ground truth

During rest and sleep, memory traces replay in the brain. The dialogue between brain regions during replay is thought to stabilize labile memory traces for long-term storage. However, because replay is an internally driven, spontaneous phenomenon, it does not have a ground truth - an external reference that can validate whether a memory has truly been replayed. Instead, replay detection is based on the similarity between the sequential neural activity comprising the replay event and the corresponding template of neural activity generated during active locomotion. If the statistical likelihood of observing such a match by chance is sufficiently low, the candidate replay event is inferred to be replaying that specific memory. However, without the ability to evaluate whether replay detection methods are successfully detecting true events and correctly rejecting non-events, the evaluation and comparison of different replay methods is challenging. To circumvent this problem, we present a new framework for evaluating replay, tested using hippocampal neural recordings from rats exploring two novel linear tracks. Using this two-track paradigm, our framework selects replay events based on their temporal fidelity (sequence-based detection), and evaluates the detection performance using each event's track discriminability, where sequenceless decoding across both tracks is used to quantify whether the track replaying is also the most likely track being reactivated.

Experience of Euclidean geometry sculpts the development and dynamics of rodent hippocampal sequential cell assemblies

Euclidean space is the fabric of the world we live in. Whether and how geometric experience shapes our spatial-temporal representations of the world remained unknown. We deprived male rats of experience with crucial features of Euclidean geometry by rearing them inside spheres, and compared activity of large hippocampal neuronal ensembles during navigation and sleep with that of cuboid cage-reared controls. Sphere-rearing from birth permitted emergence of accurate neuronal ensemble spatial codes and preconfigured and plastic time-compressed neuronal sequences. However, sphere-rearing led to diminished individual place cell tuning, more similar neuronal mapping of different track ends/corners, and impaired pattern separation and plasticity of multiple linear tracks, coupled with reduced preconfigured sleep network repertoires. Subsequent experience with multiple linear environments over four days largely reversed these effects. Thus, early-life experience with Euclidean geometry enriches the hippocampal repertoire of preconfigured neuronal patterns selected toward unique representation and discrimination of multiple linear environments.

Learning, Fast and Slow: Single- and Many-Shot Learning in the Hippocampus

The hippocampus is critical for memory and spatial navigation. The ability to map novel environments, as well as more abstract conceptual relationships, is fundamental to the cognitive flexibility that humans and other animals require to survive in a dynamic world. In this review, we survey recent advances in our understanding of how this flexibility is implemented anatomically and functionally by hippocampal circuitry, during both active exploration (online) and rest (offline). We discuss the advantages and limitations of spike timing-dependent plasticity and the more recently discovered behavioral timescale synaptic plasticity in supporting distinct learning modes in the hippocampus. Finally, we suggest complementary roles for these plasticity types in explaining many-shot and single-shot learning in the hippocampus and discuss how these rules could work together to support the learning of cognitive maps.

Self-avoidance dominates the selection of hippocampal replay

Spontaneous neural activity sequences are generated by the brain in the absence of external input 1-12 , yet how they are produced remains unknown. During immobility, hippocampal replay sequences depict spatial paths related to the animal's past experience or predicted future 13 . By recording from large ensembles of hippocampal place cells 14 in combination with optogenetic manipulation of cortical input in freely behaving rats, we show here that the selection of hippocampal replay is governed by a novel self-avoidance principle. Following movement cessation, replay of the animal's past path is strongly avoided, while replay of the future path predominates. Moreover, when the past and future paths overlap, early replays avoid both and depict entirely different trajectories. Further, replays avoid self-repetition, on a shorter timescale compared to the avoidance of previous behavioral trajectories. Eventually, several seconds into the stopping period, replay of the past trajectory dominates. This temporal organization contrasts with established and recent predictions 9,10,15,16 but is well-recapitulated by a symmetry-breaking attractor model of sequence generation in which individual neurons adapt their firing rates over time 26-35 . However, while the model is sufficient to produce avoidance of recently traversed or reactivated paths, it requires an additional excitatory input into recently activated cells to produce the later window of past-dominance. We performed optogenetic perturbations to demonstrate that this input is provided by medial entorhinal cortex, revealing its role in maintaining a memory of past experience that biases hippocampal replay. Together, these data provide specific evidence for how hippocampal replays are generated.

A recurrent network model of planning explains hippocampal replay and human behavior

When faced with a novel situation, people often spend substantial periods of time contemplating possible futures. For such planning to be rational, the benefits to behavior must compensate for the time spent thinking. Here, we capture these features of behavior by developing a neural network model where planning itself is controlled by the prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences from its own policy, which we call 'rollouts'. In a spatial navigation task, the agent learns to plan when it is beneficial, which provides a normative explanation for empirical variability in human thinking times. Additionally, the patterns of policy rollouts used by the artificial agent closely resemble patterns of rodent hippocampal replays. Our work provides a theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by-and adaptively affect-prefrontal dynamics.

Remapping revisited: how the hippocampus represents different spaces

The representation of distinct spaces by hippocampal place cells has been linked to changes in their place fields (the locations in the environment where the place cells discharge strongly), a phenomenon that has been termed 'remapping'. Remapping has been assumed to be accompanied by the reorganization of subsecond cofiring relationships among the place cells, potentially maximizing hippocampal information coding capacity. However, several observations challenge this standard view. For example, place cells exhibit mixed selectivity, encode non-positional variables, can have multiple place fields and exhibit unreliable discharge in fixed environments. Furthermore, recent evidence suggests that, when measured at subsecond timescales, the moment-to-moment cofiring of a pair of cells in one environment is remarkably similar in another environment, despite remapping. Here, I propose that remapping is a misnomer for the changes in place fields across environments and suggest instead that internally organized manifold representations of hippocampal activity are actively registered to different environments to enable navigation, promote memory and organize knowledge.

Paradoxical replay can protect contextual task representations from destructive interference when experience is unbalanced

Experience replay is a powerful mechanism to learn efficiently from limited experience. Despite several decades of compelling experimental results, the factors that determine which experiences are selected for replay remain unclear. A particular challenge for current theories is that on tasks that feature unbalanced experience, rats paradoxically replay the less-experienced trajectory. To understand why, we simulated a feedforward neural network with two regimes: rich learning (structured representations tailored to task demands) and lazy learning (unstructured, task-agnostic representations). Rich, but not lazy, representations degraded following unbalanced experience, an effect that could be reversed with paradoxical replay. To test if this computational principle can account for the experimental data, we examined the relationship between paradoxical replay and learned task representations in the rat hippocampus. Strikingly, we found a strong association between the richness of learned task representations and the paradoxicality of replay. Taken together, these results suggest that paradoxical replay specifically serves to protect rich representations from the destructive effects of unbalanced experience, and more generally demonstrate a novel interaction between the nature of task representations and the function of replay in artificial and biological systems.

Cross-strata co-occurrence of ripples with theta-frequency oscillations in the hippocampus of foraging rats

Brain rhythms have been postulated to play central roles in animal cognition. A prominently reported dichotomy of hippocampal rhythms links theta-frequency oscillations (4-12 Hz) and ripples (120-250 Hz) exclusively to preparatory and consummatory behaviours, respectively. However, because of the differential power expression of these two signals across hippocampal strata, such exclusivity requires validation through analyses of simultaneous multi-strata recordings. We assessed co-occurrence of theta-frequency oscillations with ripples in multi-channel recordings of extracellular potentials across hippocampal strata from foraging rats. We detected all ripple events from an identified stratum pyramidale (SP) channel. We then defined theta epochs based on theta oscillations detected from the stratum lacunosum-moleculare (SLM) or the stratum radiatum (SR). We found ∼20% of ripple events (in SP) to co-occur with theta epochs identified from SR/SLM channels, defined here as theta ripples. Strikingly, when theta epochs were instead identified from the SP channel, such co-occurrences were significantly reduced because of a progressive reduction in theta power along the SLM-SR-SP axis. Behaviourally, we found most theta ripples to occur during immobile periods, with comparable theta power during exploratory and immobile theta epochs. Furthermore, the progressive reduction in theta power along the SLM-SR-SP axis was common to exploratory and immobile periods. Finally, we found a strong theta-phase preference of theta ripples within the fourth quadrant [3π/2 - 2π] of the associated theta oscillation. The prevalence of theta ripples expands the potential roles of ripple-frequency oscillations to span the continuum of encoding, retrieval and consolidation, achieved through interactions with theta oscillations. KEY POINTS: The brain manifests oscillations in recorded electrical potentials, with different frequencies of oscillation associated with distinct behavioural states. A prominently reported dichotomy assigns theta-frequency oscillations (4-12 Hz) and ripples (120-250 Hz) recorded in the hippocampus to be exclusively associated with preparatory and consummatory behaviours, respectively. Our multi-strata recordings from the rodent hippocampus coupled with cross-strata analyses provide direct quantitative evidence for the occurrence of ripple events nested within theta oscillations. These results highlight the need for an analysis pipeline that explicitly accounts for the specific strata where individual oscillatory power is high, in analysing simultaneously recorded data from multiple strata. Our observations open avenues for investigations involving cross-strata interactions between theta oscillations and ripples across different behavioural states.

Episodic memory development: Bridging animal and human research

Human episodic memory is not functionally evident until about 2 years of age and continues to develop into the school years. Behavioral studies have elucidated this developmental timeline and its constituent processes. In tandem, lesion and neurophysiological studies in non-human primates and rodents have identified key neural substrates and circuit mechanisms that may underlie episodic memory development. Despite this progress, collaborative efforts between psychologists and neuroscientists remain limited, hindering progress. Here, we seek to bridge human and non-human episodic memory development research by offering a comparative review of studies using humans, non-human primates, and rodents. We highlight critical theoretical and methodological issues that limit cross-fertilization and propose a common research framework, adaptable to different species, that may facilitate cross-species research endeavors.

Disruption of awake sharp-wave ripples does not affect memorization of locations in repeated-acquisition spatial memory tasks

Storing and accessing memories is required to successfully perform day-to-day tasks, for example for engaging in a meaningful conversation. Previous studies in both rodents and primates have correlated hippocampal cellular activity with behavioral expression of memory. A key role has been attributed to awake hippocampal replay - a sequential reactivation of neurons representing a trajectory through space. However, it is unclear if awake replay impacts immediate future behavior, gradually creates and stabilizes long-term memories over a long period of time (hours and longer), or enables the temporary memorization of relevant events at an intermediate time scale (seconds to minutes). In this study, we aimed to address the uncertainty around the timeframe of impact of awake replay by collecting causal evidence from behaving rats. We detected and disrupted sharp wave ripples (SWRs) - signatures of putative replay events - using electrical stimulation of the ventral hippocampal commissure in rats that were trained on three different spatial memory tasks. In each task, rats were required to memorize a new set of locations in each trial or each daily session. Interestingly, the rats performed equally well with or without SWR disruptions. These data suggest that awake SWRs - and potentially replay - does not affect the immediate behavior nor the temporary memorization of relevant events at a short timescale that are required to successfully perform the spatial tasks. Based on these results, we hypothesize that the impact of awake replay on memory and behavior is long-term and cumulative over time.

Grid cells: the missing link in understanding Parkinson's disease?

The mechanisms underlying Parkinson's disease (PD) are complex and not fully understood, and the box-and-arrow model among other current models present significant challenges. This paper explores the potential role of the allocentric brain and especially its grid cells in several PD motor symptoms, including bradykinesia, kinesia paradoxa, freezing of gait, the bottleneck phenomenon, and their dependency on cueing. It is argued that central hubs, like the locus coeruleus and the pedunculopontine nucleus, often narrowly interpreted in the context of PD, play an equally important role in governing the allocentric brain as the basal ganglia. Consequently, the motor and secondary motor (e.g., spatially related) symptoms of PD linked with dopamine depletion may be more closely tied to erroneous computation by grid cells than to the basal ganglia alone. Because grid cells and their associated central hubs introduce both spatial and temporal information to the brain influencing velocity perception they may cause bradykinesia or hyperkinesia as well. In summary, PD motor symptoms may primarily be an allocentric disturbance resulting from virtual faulty computation by grid cells revealed by dopamine depletion in PD.

Memory consolidation: Building influence over the entorhinal cortex

Memory consolidation involves interactions between the hippocampus and other cortical areas. A new study identifies neurons in the medial entorhinal cortex that over learning increase their coordination with hippocampal replay events, suggesting a route for consolidation of spatial memories.

Theta-band phase locking during encoding leads to coordinated entorhinal-hippocampal replay

Precisely timed interactions between hippocampal and cortical neurons during replay epochs are thought to support learning. Indeed, research has shown that replay is associated with heightened hippocampal-cortical synchrony. Yet many caveats remain in our understanding. Namely, it remains unclear how this offline synchrony comes about, whether it is specific to particular behavioral states, and how-if at all-it relates to learning. In this study, we sought to address these questions by analyzing coordination between CA1 cells and neurons of the deep layers of the medial entorhinal cortex (dMEC) while rats learned a novel spatial task. During movement, we found a subset of dMEC cells that were particularly locked to hippocampal LFP theta-band oscillations and that were preferentially coordinated with hippocampal replay during offline periods. Further, dMEC synchrony with CA1 replay peaked ∼10 ms after replay initiation in CA1, suggesting that the distributed replay reflects extra-hippocampal information propagation and is specific to "offline" periods. Finally, theta-modulated dMEC cells showed a striking experience-dependent increase in synchronization with hippocampal replay trajectories, mirroring the animals' acquisition of the novel task and coupling to the hippocampal local field. Together, these findings provide strong support for the hypothesis that synergistic hippocampal-cortical replay supports learning and highlights phase locking to hippocampal theta oscillations as a potential mechanism by which such cross-structural synchrony comes about.

Hippocampo-cortical circuits for selective memory encoding, routing, and replay

Traditionally considered a homogeneous cell type, hippocampal pyramidal cells have been recently shown to be highly diverse. However, how this cellular diversity relates to the different hippocampal network computations that support memory-guided behavior is not yet known. We show that the anatomical identity of pyramidal cells is a major organizing principle of CA1 assembly dynamics, the emergence of memory replay, and cortical projection patterns in rats. Segregated pyramidal cell subpopulations encoded trajectory and choice-specific information or tracked changes in reward configuration respectively, and their activity was selectively read out by different cortical targets. Furthermore, distinct hippocampo-cortical assemblies coordinated the reactivation of complementary memory representations. These findings reveal the existence of specialized hippocampo-cortical subcircuits and provide a cellular mechanism that supports the computational flexibility and memory capacities of these structures.

Rethinking the hippocampal cognitive map as a meta-learning computational module

A hallmark of biological intelligence is the ability to adaptively draw on past experience to guide behaviour under novel situations. Yet, the neurobiological principles that underlie this form of meta-learning remain relatively unexplored. In this Opinion, we review the existing literature on hippocampal spatial representations and reinforcement learning theory and describe a novel theoretical framework that aims to account for biological meta-learning. We conjecture that so-called hippocampal cognitive maps of familiar environments are part of a larger meta-representation (meta-map) that encodes information states and sources, which support exploration and provides a foundation for learning. We also introduce concrete hypotheses on how these generic states can be encoded using a principle of superposition.

Spatiotemporal Precision of Neuroimaging in Psychiatry

Aberrant patterns of cognition, perception, and behavior seen in psychiatric disorders are thought to be driven by a complex interplay of neural processes that evolve at a rapid temporal scale. Understanding these dynamic processes in vivo in humans has been hampered by a trade-off between spatial and temporal resolutions inherent to current neuroimaging technology. A recent trend in psychiatric research has been the use of high temporal resolution imaging, particularly magnetoencephalography, often in conjunction with sophisticated machine learning decoding techniques. Developments here promise novel insights into the spatiotemporal dynamics of cognitive phenomena, including domains relevant to psychiatric illnesses such as reward and avoidance learning, memory, and planning. This review considers recent advances afforded by exploiting this increased spatiotemporal precision, with specific reference to applications that seek to drive a mechanistic understanding of psychopathology and the realization of preclinical translation.

Differential replay of reward and punishment paths predicts approach and avoidance

Neural replay is implicated in planning, where states relevant to a task goal are rapidly reactivated in sequence. It remains unclear whether, during planning, replay relates to an actual prospective choice. Here, using magnetoencephalography (MEG), we studied replay in human participants while they planned to either approach or avoid an uncertain environment containing paths leading to reward or punishment. We find evidence for forward sequential replay during planning, with rapid state-to-state transitions from 20 to 90 ms. Replay of rewarding paths was boosted, relative to aversive paths, before a decision to avoid and attenuated before a decision to approach. A trial-by-trial bias toward replaying prospective punishing paths predicted irrational decisions to approach riskier environments, an effect more pronounced in participants with higher trait anxiety. The findings indicate a coupling of replay with planned behavior, where replay prioritizes an online representation of a worst-case scenario for approaching or avoiding.

Replay and compositional computation

Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.

Distinct replay signatures for prospective decision-making and memory preservation

Theories of neural replay propose that it supports a range of functions, most prominently planning and memory consolidation. Here, we test the hypothesis that distinct signatures of replay in the same task are related to model-based decision-making ("planning") and memory preservation. We designed a reward learning task wherein participants utilized structure knowledge for model-based evaluation, while at the same time had to maintain knowledge of two independent and randomly alternating task environments. Using magnetoencephalography and multivariate analysis, we first identified temporally compressed sequential reactivation, or replay, both prior to choice and following reward feedback. Before choice, prospective replay strength was enhanced for the current task-relevant environment when a model-based planning strategy was beneficial. Following reward receipt, and consistent with a memory preservation role, replay for the alternative distal task environment was enhanced as a function of decreasing recency of experience with that environment. Critically, these planning and memory preservation relationships were selective to pre-choice and post-feedback periods, respectively. Our results provide support for key theoretical proposals regarding the functional role of replay and demonstrate that the relative strength of planning and memory-related signals are modulated by ongoing computational and task demands.

Neural ensembles in navigation: From single cells to population codes

The brain can represent behaviorally relevant information through the firing of individual neurons as well as the coordinated firing of ensembles of neurons. Neurons in the hippocampus and associated cortical regions participate in a variety of types of ensembles to support navigation. These ensemble types include single cell codes, population codes, time-compressed sequences, behavioral sequences, and engrams. We present the physiological basis and behavioral relevance of ensemble firing. We discuss how these traditional definitions of ensembles can constrain or expand potential analyses due to the underlying assumptions and abstractions made. We highlight how coding can change at the ensemble level while underlying single cell codes remain intact. Finally, we present how ensemble definitions could be broadened to better understand the full complexity of the brain.

Compositional Sequence Generation in the Entorhinal-Hippocampal System

Neurons in the medial entorhinal cortex exhibit multiple, periodically organized, firing fields which collectively appear to form an internal representation of space. Neuroimaging data suggest that this grid coding is also present in other cortical areas such as the prefrontal cortex, indicating that it may be a general principle of neural functionality in the brain. In a recent analysis through the lens of dynamical systems theory, we showed how grid coding can lead to the generation of a diversity of empirically observed sequential reactivations of hippocampal place cells corresponding to traversals of cognitive maps. Here, we extend this sequence generation model by describing how the synthesis of multiple dynamical systems can support compositional cognitive computations. To empirically validate the model, we simulate two experiments demonstrating compositionality in space or in time during sequence generation. Finally, we describe several neural network architectures supporting various types of compositionality based on grid coding and highlight connections to recent work in machine learning leveraging analogous techniques.

Neuronal signature of spatial decision-making during navigation by freely moving rats by using calcium imaging

A challenge in spatial memory is understanding how place cell firing contributes to decision-making in navigation. A spatial recency task was created in which freely moving rats first became familiar with a spatial context over several days and thereafter were required to encode and then selectively recall one of three specific locations within it that was chosen to be rewarded that day. Calcium imaging was used to record from more than 1,000 cells in area CA1 of the hippocampus of five rats during the exploration, sample, and choice phases of the daily task. The key finding was that neural activity in the startbox rose steadily in the short period prior to entry to the arena and that this selective population cell firing was predictive of the daily changing goal on correct trials but not on trials in which the animals made errors. Single-cell and population activity measures converged on the idea that prospective coding of neural activity can be involved in navigational decision-making.

Replay, the default mode network and the cascaded memory systems model

The spontaneous replay of patterns of activity related to past experiences and memories is a striking feature of brain activity, as is the coherent activation of sets of brain areas - particularly those comprising the default mode network (DMN) - during rest. We propose that these two phenomena are strongly intertwined and that their potential functions overlap. In the 'cascaded memory systems model' that we outline here, we hypothesize that the DMN forms the backbone for the propagation of replay, mediating interactions between the hippocampus and the neocortex that enable the consolidation of new memories. The DMN may also independently ignite replay cascades, which support reactivation of older memories or high-level semantic representations. We suggest that transient cortical activations, inducing long-range correlations across the neocortex, are a key mechanism supporting a hierarchy of representations that progresses from simple percepts to semantic representations of causes and, finally, to whole episodes.

Multiple-Timescale Representations of Space: Linking Memory to Navigation

When navigating through space, we must maintain a representation of our position in real time; when recalling a past episode, a memory can come back in a flash. Interestingly, the brain's spatial representation system, including the hippocampus, supports these two distinct timescale functions. How are neural representations of space used in the service of both real-world navigation and internal mnemonic processes? Recent progress has identified sequences of hippocampal place cells, evolving at multiple timescales in accordance with either navigational behaviors or internal oscillations, that underlie these functions. We review experimental findings on experience-dependent modulation of these sequential representations and consider how they link real-world navigation to time-compressed memories. We further discuss recent work suggesting the prevalence of these sequences beyond hippocampus and propose that these multiple-timescale mechanisms may represent a general algorithm for organizing cell assemblies, potentially unifying the dual roles of the spatial representation system in memory and navigation.

State transitions in the statistically stable place cell population correspond to rate of perceptual change

The hippocampus occupies a central role in mammalian navigation and memory. Yet an understanding of the rules that govern the statistics and granularity of the spatial code, as well as its interactions with perceptual stimuli, is lacking. We analyzed CA1 place cell activity recorded while rats foraged in different large-scale environments. We found that place cell activity was subject to an unexpected but precise homeostasis—the distribution of activity in the population as a whole being constant at all locations within and between environments. Using a virtual reconstruction of the largest environment, we showed that the rate of transition through this statistically stable population matches the rate of change in the animals’ visual scene. Thus, place fields near boundaries were small but numerous, while in the environment’s interior, they were larger but more dispersed. These results indicate that hippocampal spatial activity is governed by a small number of simple laws and, in particular, suggest the presence of an information-theoretic bound imposed by perception on the fidelity of the spatial memory system.

•

Neural activity in rodent CA1 place cell populations is homeostatically balanced

•

Hippocampal place field size and frequency are governed by proximity to boundaries

•

Transition rate through place cell population matches rate of change in visual scene

Spatial Learning Drives Rapid Goal Representation in Hippocampal Ripples without Place Field Accumulation or Goal-Oriented Theta Sequences

The hippocampus is critical for rapid acquisition of many forms of memory, although the circuit-level mechanisms through which the hippocampus rapidly consolidates novel information are unknown. Here, the activity of large ensembles of hippocampal neurons in adult male Long-Evans rats was monitored across a period of rapid spatial learning to assess how the network changes during the initial phases of memory formation and retrieval. In contrast to several reports, the hippocampal network did not display enhanced representation of the goal location via accumulation of place fields or elevated firing rates at the goal. Rather, population activity rates increased globally as a function of experience. These alterations in activity were mirrored in the power of the theta oscillation and in the quality of theta sequences, without preferential encoding of paths to the learned goal location. In contrast, during brief "offline" pauses in movement, representation of a novel goal location emerged rapidly in ripples, preceding other changes in network activity. These data demonstrate that the hippocampal network can facilitate active navigation without enhanced goal representation during periods of active movement, and further indicate that goal representation in hippocampal ripples before movement onset supports subsequent navigation, possibly through activation of downstream cortical networks.SIGNIFICANCE STATEMENT Understanding the mechanisms through which the networks of the brain rapidly assimilate information and use previously learned knowledge are fundamental areas of focus in neuroscience. In particular, the hippocampal circuit is a critical region for rapid formation and use of spatial memory. In this study, several circuit-level features of hippocampal function were quantified while rats performed a spatial navigation task requiring rapid memory formation and use. During periods of active navigation, a general increase in overall network activity is observed during memory acquisition, which plateaus during memory retrieval periods, without specific enhanced representation of the goal location. During pauses in navigation, rapid representation of the distant goal well emerges before either behavioral improvement or changes in online activity.

Decoding cognition from spontaneous neural activity

In human neuroscience, studies of cognition are rarely grounded in non-task-evoked, 'spontaneous' neural activity. Indeed, studies of spontaneous activity tend to focus predominantly on intrinsic neural patterns (for example, resting-state networks). Taking a 'representation-rich' approach bridges the gap between cognition and resting-state communities: this approach relies on decoding task-related representations from spontaneous neural activity, allowing quantification of the representational content and rich dynamics of such activity. For example, if we know the neural representation of an episodic memory, we can decode its subsequent replay during rest. We argue that such an approach advances cognitive research beyond a focus on immediate task demand and provides insight into the functional relevance of the intrinsic neural pattern (for example, the default mode network). This in turn enables a greater integration between human and animal neuroscience, facilitating experimental testing of theoretical accounts of intrinsic activity, and opening new avenues of research in psychiatry.

Spatial goal coding in the hippocampal formation

The mammalian hippocampal formation contains several distinct populations of neurons involved in representing self-position and orientation. These neurons, which include place, grid, head direction, and boundary-vector cells, are thought to collectively instantiate cognitive maps supporting flexible navigation. However, to flexibly navigate, it is necessary to also maintain internal representations of goal locations, such that goal-directed routes can be planned and executed. Although it has remained unclear how the mammalian brain represents goal locations, multiple neural candidates have recently been uncovered during different phases of navigation. For example, during planning, sequential activation of spatial cells may enable simulation of future routes toward the goal. During travel, modulation of spatial cells by the prospective route, or by distance and direction to the goal, may allow maintenance of route and goal-location information, supporting navigation on an ongoing basis. As the goal is approached, an increased activation of spatial cells may enable the goal location to become distinctly represented within cognitive maps, aiding goal localization. Lastly, after arrival at the goal, sequential activation of spatial cells may represent the just-taken route, enabling route learning and evaluation. Here, we review and synthesize these and other evidence for goal coding in mammalian brains, relate the experimental findings to predictions from computational models, and discuss outstanding questions and future challenges.

Adult hippocampal neurogenesis shapes adaptation and improves stress response: a mechanistic and integrative perspective

Adult hippocampal neurogenesis (AHN) represents a remarkable form of neuroplasticity that has increasingly been linked to the stress response in recent years. However, the hippocampus does not itself support the expression of the different dimensions of the stress response. Moreover, the main hippocampal functions are essentially preserved under AHN depletion and adult-born immature neurons (abGNs) have no extrahippocampal projections, which questions the mechanisms by which abGNs influence functions supported by brain areas far from the hippocampus. Within this framework, we propose that through its computational influences AHN is pivotal in shaping adaption to environmental demands, underlying its role in stress response. The hippocampus with its high input convergence and output divergence represents a computational hub, ideally positioned in the brain (1) to detect cues and contexts linked to past, current and predicted stressful experiences, and (2) to supervise the expression of the stress response at the cognitive, affective, behavioral, and physiological levels. AHN appears to bias hippocampal computations toward enhanced conjunctive encoding and pattern separation, promoting contextual discrimination and cognitive flexibility, reducing proactive interference and generalization of stressful experiences to safe contexts. These effects result in gating downstream brain areas with more accurate and contextualized information, enabling the different dimensions of the stress response to be more appropriately set with specific contexts. Here, we first provide an integrative perspective of the functional involvement of AHN in the hippocampus and a phenomenological overview of the stress response. We then examine the mechanistic underpinning of the role of AHN in the stress response and describe its potential implications in the different dimensions accompanying this response.

Reactivation predicts the consolidation of unbiased long-term cognitive maps

Spatial memories that can last a lifetime are thought to be encoded during 'online' periods of exploration and subsequently consolidated into stable cognitive maps through their 'offline' reactivation. However, the mechanisms and computational principles by which offline reactivation stabilize long-lasting spatial representations remain poorly understood. Here, we employed simultaneous fast calcium imaging and electrophysiology to track hippocampal place cells over 2 weeks of online spatial reward learning behavior and offline resting. We describe that recruitment to persistent network-level offline reactivation of spatial experiences in mice predicts the future representational stability of place cells days in advance of their online reinstatement. Moreover, while representations of reward-adjacent locations are generally more stable across days, offline-reactivation-related stability is, conversely, most prominent for reward-distal locations. Thus, while occurring on the tens of milliseconds timescale, offline reactivation is uniquely associated with the stability of multiday representations that counterbalance the overall reward-adjacency bias, thereby predicting the stabilization of cognitive maps that comprehensively reflect entire underlying spatial contexts. These findings suggest that post-learning offline-related memory consolidation plays a complimentary and computationally distinct role in learning compared to online encoding.

Ripple band phase precession of place cell firing during replay

Neuronal “replay,” in which place cell firing during rest recapitulates recently experienced trajectories, is thought to mediate the transmission of information from hippocampus to neocortex, but the mechanism for this transmission is unknown. Here, we show that replay uses a phase code to represent spatial trajectories by the phase of firing relative to the 150- to 250-Hz “ripple” oscillations that accompany replay events. This phase code is analogous to the theta phase precession of place cell firing during navigation, in which place cells fire at progressively earlier phases of the 6- to 12-Hz theta oscillation as their place field is traversed, providing information about self-location that is additional to the rate code and a necessary precursor of replay. Thus, during replay, each ripple cycle contains a “forward sweep” of decoded locations along the recapitulated trajectory. Our results indicate a novel encoding of trajectory information during replay and implicates phase coding as a general mechanism by which the hippocampus transmits experienced and replayed sequential information to downstream targets.

•

Place cells fire at successively earlier ripple band phases during replay

•

Ripple band firing phase during replay encodes location within the place field

•

This produces forward sweeps of place cell activity during each ripple cycle

Hippocampal replay reflects specific past experiences rather than a plan for subsequent choice

Executing memory-guided behavior requires storage of information about experience and later recall of that information to inform choices. Awake hippocampal replay, when hippocampal neural ensembles briefly reactivate a representation related to prior experience, has been proposed to critically contribute to these memory-related processes. However, it remains unclear whether awake replay contributes to memory function by promoting the storage of past experiences, facilitating planning based on evaluation of those experiences, or both. We designed a dynamic spatial task that promotes replay before a memory-based choice and assessed how the content of replay related to past and future behavior. We found that replay content was decoupled from subsequent choice and instead was enriched for representations of previously rewarded locations and places that had not been visited recently, indicating a role in memory storage rather than in directly guiding subsequent behavior.

Hippocampal replay of experience at real-world speeds

Representations related to past experiences play a critical role in memory and decision-making processes. The rat hippocampus expresses these types of representations during sharp-wave ripple (SWR) events, and previous work identified a minority of SWRs that contain ‘replay’ of spatial trajectories at ∼20x the movement speed of the animal. Efforts to understand replay typically make multiple assumptions about which events to examine and what sorts of representations constitute replay. We therefore lack a clear understanding of both the prevalence and the range of representational dynamics associated with replay. Here, we develop a state space model that uses a combination of movement dynamics of different speeds to capture the spatial content and time evolution of replay during SWRs. Using this model, we find that the large majority of replay events contain spatially coherent, interpretable content. Furthermore, many events progress at real-world, rather than accelerated, movement speeds, consistent with actual experiences.

Learning offline: memory replay in biological and artificial reinforcement learning

Learning to act in an environment to maximise rewards is among the brain's key functions. This process has often been conceptualised within the framework of reinforcement learning, which has also gained prominence in machine learning and artificial intelligence (AI) as a way to optimise decision making. A common aspect of both biological and machine reinforcement learning is the reactivation of previously experienced episodes, referred to as replay. Replay is important for memory consolidation in biological neural networks and is key to stabilising learning in deep neural networks. Here, we review recent developments concerning the functional roles of replay in the fields of neuroscience and AI. Complementary progress suggests how replay might support learning processes, including generalisation and continual learning, affording opportunities to transfer knowledge across the two fields to advance the understanding of biological and artificial learning and memory.

Replay in minds and machines

Experience-related brain activity patterns reactivate during sleep, wakeful rest, and brief pauses from active behavior. In parallel, machine learning research has found that experience replay can lead to substantial performance improvements in artificial agents. Together, these lines of research suggest replay has a variety of computational benefits for decision-making and learning. Here, we provide an overview of putative computational functions of replay as suggested by machine learning and neuroscientific research. We show that replay can lead to faster learning, less forgetting, reorganization or augmentation of experiences, and support planning and generalization. In addition, we highlight the benefits of reactivating abstracted internal representations rather than veridical memories, and discuss how replay could provide a mechanism to build internal representations that improve learning and decision-making.

Disrupting the medial prefrontal cortex with designer receptors exclusively activated by designer drug alters hippocampal sharp-wave ripples and their associated cognitive processes

The hippocampus and medial prefrontal cortex (mPFC) interact during a myriad of cognitive processes including decision-making and long-term memory consolidation. Exactly how the mPFC and hippocampus interact during goal-directed decision-making remains to be fully elucidated. During periods of rest, bursts of high-frequency oscillations, termed sharp-wave ripple (SWR), appear in the local field potential. Impairing SWRs on the maze or during post-learning rest can interfere with memory-guided decision-making and memory consolidation. We hypothesize that the hippocampus and mPFC bidirectionally interact during SWRs to support memory consolidation and decision-making. Rats were trained on the neuroeconomic spatial decision-making task, Restaurant Row, to make serial stay-skip decisions where the amount of effort (delay to reward) varied upon entry to each restaurant. Hippocampal cells and SWRs were recorded in rats with the mPFC transduced with inhibitory DREADDs. We found that disrupting the mPFC impaired consolidating SWRs in the hippocampus. Hippocampal SWR rates depended on the internalized value of the reward (derived from individual flavor preferences), a parameter important in decision-making, and disrupting the mPFC changed this relationship. Additionally, we found a dissociation between SWRs that occurred while rats were on the maze dependent upon whether those SWRs occurred while the rat was anticipating food reward or during post-reward consumption.

Multimodal neural recordings with Neuro-FITM uncover diverse patterns of cortical-hippocampal interactions

Many cognitive processes require communication between the neocortex and the hippocampus. However, coordination between large-scale cortical dynamics and hippocampal activity is not well understood, partially due to the difficulty in simultaneously recording from those regions. In the present study, we developed a flexible, insertable and transparent microelectrode array (Neuro-FITM) that enables investigation of cortical-hippocampal coordinations during hippocampal sharp-wave ripples (SWRs). Flexibility and transparency of Neuro-FITM allow simultaneous recordings of local field potentials and neural spiking from the hippocampus during wide-field calcium imaging. These experiments revealed that diverse cortical activity patterns accompanied SWRs and, in most cases, cortical activation preceded hippocampal SWRs. We demonstrated that, during SWRs, different hippocampal neural population activity was associated with distinct cortical activity patterns. These results suggest that hippocampus and large-scale cortical activity interact in a selective and diverse manner during SWRs underlying various cognitive functions. Our technology can be broadly applied to comprehensive investigations of interactions between the cortex and other subcortical structures.

Processing of Hippocampal Network Activity in the Receiver Network of the Medial Entorhinal Cortex Layer V

The interplay between hippocampus and medial entorhinal cortex (mEC) is of key importance for forming spatial representations. Within the hippocampal-entorhinal loop, the hippocampus receives context-specific signals from layers II/III of the mEC and feeds memory-associated activity back into layer V (LV). The processing of this output signal within the mEC, however, is largely unknown. We characterized the activation of the receiving mEC network by evoked and naturally occurring output patterns in mouse hippocampal-entorhinal cortex slices. Both types of glutamatergic neurons (mEC LVa and LVb) as well as fast-spiking inhibitory interneurons receive direct excitatory input from the intermediate/ventral hippocampus. Connections between the two types of excitatory neurons are sparse, and local processing of hippocampal output signals within mEC LV is asymmetric, favoring excitation of far projecting LVa neurons over locally projecting LVb neurons. These findings suggest a new role for mEC LV as a bifurcation gate for feedforward (telencephalic) and feedback (entorhinal-hippocampal) signal propagation.SIGNIFICANCE STATEMENT Patterned network activity in hippocampal networks plays a key role in the formation and consolidation of spatial memories. It is, however, largely unclear how information is transferred to the neocortex for long-term engrams. Here, we elucidate the propagation of network activity from the hippocampus to the medial entorhinal cortex. We show that patterned output from the hippocampus reaches both major cell types of deep entorhinal layers. These cells are, however, only weakly connected, giving rise to two parallel streams of activity for local and remote signal propagation, respectively. The relative weight of both pathways is regulated by local inhibitory interneurons. Our data reveal important insights into the hippocampal-neocortical dialogue, which is of key importance for memory consolidation in the mammalian brain.

Cortical reactivations of recent sensory experiences predict bidirectional network changes during learning

Salient experiences are often relived in the mind. Human neuroimaging studies suggest that such experiences drive activity patterns in visual association cortex that are subsequently reactivated during quiet waking. Nevertheless, the circuit-level consequences of such reactivations remain unclear. Here, we imaged hundreds of neurons in visual association cortex across days as mice learned a visual discrimination task. Distinct patterns of neurons were activated by different visual cues. These same patterns were subsequently reactivated during quiet waking in darkness, with higher reactivation rates during early learning and for food-predicting versus neutral cues. Reactivations involving ensembles of neurons encoding both the food cue and the reward predicted strengthening of next-day functional connectivity of participating neurons, while the converse was observed for reactivations involving ensembles encoding only the food cue. We propose that task-relevant neurons strengthen while task-irrelevant neurons weaken their dialog with the network via participation in distinct flavors of reactivation.

Different theta frameworks coexist in the rat hippocampus and are coordinated during memory-guided and novelty tasks

Hippocampal firing is organized in theta sequences controlled by internal memory processes and by external sensory cues, but how these computations are coordinated is not fully understood. Although theta activity is commonly studied as a unique coherent oscillation, it is the result of complex interactions between different rhythm generators. Here, by separating hippocampal theta activity in three different current generators, we found epochs with variable theta frequency and phase coupling, suggesting flexible interactions between theta generators. We found that epochs of highly synchronized theta rhythmicity preferentially occurred during behavioral tasks requiring coordination between internal memory representations and incoming sensory information. In addition, we found that gamma oscillations were associated with specific theta generators and the strength of theta-gamma coupling predicted the synchronization between theta generators. We propose a mechanism for segregating or integrating hippocampal computations based on the flexible coordination of different theta frameworks to accommodate the cognitive needs.

The roles of online and offline replay in planning

Animals and humans replay neural patterns encoding trajectories through their environment, both whilst they solve decision-making tasks and during rest. Both on-task and off-task replay are believed to contribute to flexible decision making, though how their relative contributions differ remains unclear. We investigated this question by using magnetoencephalography (MEG) to study human subjects while they performed a decision-making task that was designed to reveal the decision algorithms employed. We characterised subjects in terms of how flexibly each adjusted their choices to changes in temporal, spatial and reward structure. The more flexible a subject, the more they replayed trajectories during task performance, and this replay was coupled with re-planning of the encoded trajectories. The less flexible a subject, the more they replayed previously preferred trajectories during rest periods between task epochs. The data suggest that online and offline replay both participate in planning but support distinct decision strategies.

Three brain states in the hippocampus and cortex

Contemporary brain research seeks to understand how cognition is reducible to neural activity. Crucially, much of this effort is guided by a scientific paradigm that views neural activity as essentially driven by external stimuli. In contrast, recent perspectives argue that this paradigm is by itself inadequate and that understanding patterns of activity intrinsic to the brain is needed to explain cognition. Yet, despite this critique, the stimulus-driven paradigm still dominates-possibly because a convincing alternative has not been clear. Here, we review a series of findings suggesting such an alternative. These findings indicate that neural activity in the hippocampus occurs in one of three brain states that have radically different anatomical, physiological, representational, and behavioral correlates, together implying different functional roles in cognition. This three-state framework also indicates that neural representations in the hippocampus follow a surprising pattern of organization at the timescale of ∼1 s or longer. Lastly, beyond the hippocampus, recent breakthroughs indicate three parallel states in the cortex, suggesting shared principles and brain-wide organization of intrinsic neural activity.

Potential factors influencing replay across CA1 during sharp-wave ripples

Sharp-wave ripples are complex neurophysiological events recorded along the trisynaptic hippocampal circuit (i.e. from CA3 to CA1 and the subiculum) during slow-wave sleep and awake states. They arise locally but scale brain-wide to the hippocampal target regions at cortical and subcortical structures. During these events, neuronal firing sequences are replayed retrospectively or prospectively and in the forward or reverse order as defined by experience. They could reflect either pre-configured firing sequences, learned sequences or an option space to inform subsequent decisions. How can different sequences arise during sharp-wave ripples? Emerging data suggest the hippocampal circuit is organized in different loops across the proximal (close to dentate gyrus) and distal (close to entorhinal cortex) axis. These data also disclose a so-far neglected laminar organization of the hippocampal output during sharp-wave events. Here, I discuss whether by incorporating cell-type-specific mechanisms converging on deep and superficial CA1 sublayers along the proximodistal axis, some novel factors influencing the organization of hippocampal sequences could be unveiled. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.

Progress and issues in second-order analysis of hippocampal replay

Patterns of neural activity that occur spontaneously during sharp-wave ripple (SWR) events in the hippocampus are thought to play an important role in memory formation, consolidation and retrieval. Typical studies examining the content of SWRs seek to determine whether the identity and/or temporal order of cell firing is different from chance. Such 'first-order' analyses are focused on a single time point and template (map), and have been used to show, for instance, the existence of preplay. The major methodological challenge in first-order analyses is the construction and interpretation of different chance distributions. By contrast, 'second-order' analyses involve a comparison of SWR content between different time points, and/or between different templates. Typical second-order questions include tests of experience-dependence (replay) that compare SWR content before and after experience, and comparisons or replay between different arms of a maze. Such questions entail additional methodological challenges that can lead to biases in results and associated interpretations. We provide an inventory of analysis challenges for second-order questions about SWR content, and suggest ways of preventing, identifying and addressing possible analysis biases. Given evolving interest in understanding SWR content in more complex experimental scenarios and across different time scales, we expect these issues to become increasingly pervasive. This article is part of the Theo Murphy meeting issue 'Memory reactivation: replaying events past, present and future'.

Naïve to expert: Considering the role of previous knowledge in memory

In humans, most of our new memories are in some way or another related to what we have already experienced. However, in memory research, especially in non-human animal research, subjects are often mostly naïve to the world. But we know that previous knowledge will change how memories are processed and which brain areas are critical at which time point. Each process from encoding, consolidation, to memory retrieval will be affected. Here, we summarise previous knowledge effects on the neurobiology of memory in both humans and non-human animals, with a special focus on schemas - associative network structures. Furthermore, we propose a new theory on how there may be a continuous gradient from naïve to expert, which would modulate the importance and role of brain areas, such as the hippocampus and prefrontal cortex.

Dynamics of Awake Hippocampal-Prefrontal Replay for Spatial Learning and Memory-Guided Decision Making

Spatial learning requires remembering and choosing paths to goals. Hippocampal place cells replay spatial paths during immobility in reverse and forward order, offering a potential mechanism. However, how replay supports both goal-directed learning and memory-guided decision making is unclear. We therefore continuously tracked awake replay in the same hippocampal-prefrontal ensembles throughout learning of a spatial alternation task. We found that, during pauses between behavioral trajectories, reverse and forward hippocampal replay supports an internal cognitive search of available past and future possibilities and exhibits opposing learning gradients for prediction of past and future behavioral paths, respectively. Coordinated hippocampal-prefrontal replay distinguished correct past and future paths from alternative choices, suggesting a role in recall of past paths to guide planning of future decisions for spatial working memory. Our findings reveal a learning shift from hippocampal reverse-replay-based retrospective evaluation to forward-replay-based prospective planning, with prefrontal readout of memory-guided paths for learning and decision making.

Replay as wavefronts and theta sequences as bump oscillations in a grid cell attractor network

Grid cells fire in sequences that represent rapid trajectories in space. During locomotion, theta sequences encode sweeps in position starting slightly behind the animal and ending ahead of it. During quiescence and slow wave sleep, bouts of synchronized activity represent long trajectories called replays, which are well-established in place cells and have been recently reported in grid cells. Theta sequences and replay are hypothesized to facilitate many cognitive functions, but their underlying mechanisms are unknown. One mechanism proposed for grid cell formation is the continuous attractor network. We demonstrate that this established architecture naturally produces theta sequences and replay as distinct consequences of modulating external input. Driving inhibitory interneurons at the theta frequency causes attractor bumps to oscillate in speed and size, which gives rise to theta sequences and phase precession, respectively. Decreasing input drive to all neurons produces traveling wavefronts of activity that are decoded as replays.