1
|
Huang Q, Xiao Z, Yu Q, Luo Y, Xu J, Qu Y, Dolan R, Behrens T, Liu Y. Replay-triggered brain-wide activation in humans. Nat Commun 2024; 15:7185. [PMID: 39169063 PMCID: PMC11339350 DOI: 10.1038/s41467-024-51582-5] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/22/2023] [Accepted: 08/08/2024] [Indexed: 08/23/2024] Open
Abstract
The consolidation of discrete experiences into a coherent narrative shapes the cognitive map, providing structured mental representations of our experiences. In this process, past memories are reactivated and replayed in sequence, fostering hippocampal-cortical dialogue. However, brain-wide engagement coinciding with sequential reactivation (or replay) of memories remains largely unexplored. In this study, employing simultaneous EEG-fMRI, we capture both the spatial and temporal dynamics of memory replay. We find that during mental simulation, past memories are replayed in fast sequences as detected via EEG. These transient replay events are associated with heightened fMRI activity in the hippocampus and medial prefrontal cortex. Replay occurrence strengthens functional connectivity between the hippocampus and the default mode network, a set of brain regions key to representing the cognitive map. On the other hand, when subjects are at rest following learning, memory reactivation of task-related items is stronger than that of pre-learning rest, and is also associated with heightened hippocampal activation and augmented hippocampal connectivity to the entorhinal cortex. Together, our findings highlight a distributed, brain-wide engagement associated with transient memory reactivation and its sequential replay.
Collapse
Affiliation(s)
- Qi Huang
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Chinese Institute for Brain Research, Beijing, China
| | - Zhibing Xiao
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Chinese Institute for Brain Research, Beijing, China
| | - Qianqian Yu
- School of Psychology, Center for Brain Disorders and Cognitive Science, Shenzhen University, Shenzhen, China
| | - Yuejia Luo
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- School of Psychology, Center for Brain Disorders and Cognitive Science, Shenzhen University, Shenzhen, China
| | - Jiahua Xu
- Chinese Institute for Brain Research, Beijing, China
| | - Yukun Qu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Chinese Institute for Brain Research, Beijing, China
| | - Raymond Dolan
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, London, UK
- Wellcome Centre for Human Neuroimaging, UCL, London, UK
| | - Timothy Behrens
- Wellcome Centre for Human Neuroimaging, UCL, London, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
- Sainsbury Wellcome Centre for Neural Circuits and Behaviour, UCL, London, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.
- Chinese Institute for Brain Research, Beijing, China.
| |
Collapse
|
2
|
Jin SW, Ha HS, Lee I. Selective reactivation of value- and place-dependent information during sharp-wave ripples in the intermediate and dorsal hippocampus. SCIENCE ADVANCES 2024; 10:eadn0416. [PMID: 39110810 PMCID: PMC11305392 DOI: 10.1126/sciadv.adn0416] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/19/2023] [Accepted: 06/26/2024] [Indexed: 08/10/2024]
Abstract
Reactivating place cells during sharp-wave ripples in the hippocampus is important for memory consolidation. However, whether hippocampal reactivation is affected by the values of events experienced by the animal is largely unknown. Here, we investigated whether place cells in the dorsal (dHP) and intermediate hippocampus (iHP) of rats are differentially reactivated depending on the value associated with a place during the learning of places associated with higher-value rewards in a T-maze. Place cells in the iHP representing the high-value location were reactivated significantly more frequently than those representing the low-value location, characteristics not observed in the dHP. In contrast, the activities of place cells in the dHP coding the routes leading to high-value locations were replayed more than those in the iHP. Our findings suggest that value-based differential reactivation patterns along the septotemporal axis of the hippocampus may play essential roles in optimizing goal-directed spatial learning for maximal reward.
Collapse
Affiliation(s)
| | - Hee-Seung Ha
- Department of Brain and Cognitive Sciences, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Korea
| | - Inah Lee
- Department of Brain and Cognitive Sciences, Seoul National University, Gwanak-ro 1, Gwanak-gu, Seoul 08826, Korea
| |
Collapse
|
3
|
Liao Z, Losonczy A. Learning, Fast and Slow: Single- and Many-Shot Learning in the Hippocampus. Annu Rev Neurosci 2024; 47:187-209. [PMID: 38663090 DOI: 10.1146/annurev-neuro-102423-100258] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 08/09/2024]
Abstract
The hippocampus is critical for memory and spatial navigation. The ability to map novel environments, as well as more abstract conceptual relationships, is fundamental to the cognitive flexibility that humans and other animals require to survive in a dynamic world. In this review, we survey recent advances in our understanding of how this flexibility is implemented anatomically and functionally by hippocampal circuitry, during both active exploration (online) and rest (offline). We discuss the advantages and limitations of spike timing-dependent plasticity and the more recently discovered behavioral timescale synaptic plasticity in supporting distinct learning modes in the hippocampus. Finally, we suggest complementary roles for these plasticity types in explaining many-shot and single-shot learning in the hippocampus and discuss how these rules could work together to support the learning of cognitive maps.
Collapse
Affiliation(s)
- Zhenrui Liao
- Department of Neuroscience and Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA;
| | - Attila Losonczy
- Department of Neuroscience and Mortimer B. Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY, USA;
| |
Collapse
|
4
|
Mallory CS, Widloski J, Foster DJ. Self-avoidance dominates the selection of hippocampal replay. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.18.604185. [PMID: 39071427 PMCID: PMC11275714 DOI: 10.1101/2024.07.18.604185] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 07/30/2024]
Abstract
Spontaneous neural activity sequences are generated by the brain in the absence of external input 1-12 , yet how they are produced remains unknown. During immobility, hippocampal replay sequences depict spatial paths related to the animal's past experience or predicted future 13 . By recording from large ensembles of hippocampal place cells 14 in combination with optogenetic manipulation of cortical input in freely behaving rats, we show here that the selection of hippocampal replay is governed by a novel self-avoidance principle. Following movement cessation, replay of the animal's past path is strongly avoided, while replay of the future path predominates. Moreover, when the past and future paths overlap, early replays avoid both and depict entirely different trajectories. Further, replays avoid self-repetition, on a shorter timescale compared to the avoidance of previous behavioral trajectories. Eventually, several seconds into the stopping period, replay of the past trajectory dominates. This temporal organization contrasts with established and recent predictions 9,10,15,16 but is well-recapitulated by a symmetry-breaking attractor model of sequence generation in which individual neurons adapt their firing rates over time 26-35 . However, while the model is sufficient to produce avoidance of recently traversed or reactivated paths, it requires an additional excitatory input into recently activated cells to produce the later window of past-dominance. We performed optogenetic perturbations to demonstrate that this input is provided by medial entorhinal cortex, revealing its role in maintaining a memory of past experience that biases hippocampal replay. Together, these data provide specific evidence for how hippocampal replays are generated.
Collapse
|
5
|
Robson E, Donahue MM, Mably AJ, Demetrovich PG, Hewitt LT, Colgin LL. Social odors drive hippocampal CA2 place cell responses to social stimuli. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.07.16.603738. [PMID: 39071428 PMCID: PMC11275720 DOI: 10.1101/2024.07.16.603738] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 07/30/2024]
Abstract
Hippocampal region CA2 is essential for social memory processing. Interaction with social stimuli induces changes in CA2 place cell firing during active exploration and sharp wave-ripples during rest following a social interaction. However, it is unknown whether these changes in firing patterns are caused by integration of multimodal social stimuli or by a specific sensory modality associated with a social interaction. Rodents rely heavily on chemosensory cues in the form of olfactory signals for social recognition processes. To determine the extent to which olfactory signals contribute to CA2 place cell responses to social stimuli, we recorded CA2 place cells in rats freely exploring environments containing social stimuli that included or lacked olfactory content. We found that CA2 place cell firing patterns significantly changed only when social odors were prominent. Also, place cells that increased their firing in the presence of social odors alone preferentially increased their firing during subsequent sharp wave-ripples. Our results suggest that olfactory cues are essential for changing CA2 place cell firing patterns during and after social interactions. These results support prior work suggesting CA2 performs social functions and shed light on processes underlying CA2 responses to social stimuli.
Collapse
Affiliation(s)
- Emma Robson
- Center for Learning and Memory, The University of Texas at Austin, Austin, TX 78712
- Department of Neuroscience, The University of Texas at Austin, Austin, TX 78712
| | - Margaret M. Donahue
- Center for Learning and Memory, The University of Texas at Austin, Austin, TX 78712
- Institute for Neuroscience, The University of Texas at Austin, Austin, TX 78712
| | - Alexandra J. Mably
- Center for Learning and Memory, The University of Texas at Austin, Austin, TX 78712
- Department of Neuroscience, The University of Texas at Austin, Austin, TX 78712
| | - Peyton G. Demetrovich
- Center for Learning and Memory, The University of Texas at Austin, Austin, TX 78712
- Institute for Neuroscience, The University of Texas at Austin, Austin, TX 78712
| | - Lauren T. Hewitt
- Center for Learning and Memory, The University of Texas at Austin, Austin, TX 78712
- Institute for Neuroscience, The University of Texas at Austin, Austin, TX 78712
| | - Laura Lee Colgin
- Center for Learning and Memory, The University of Texas at Austin, Austin, TX 78712
- Department of Neuroscience, The University of Texas at Austin, Austin, TX 78712
- Institute for Neuroscience, The University of Texas at Austin, Austin, TX 78712
| |
Collapse
|
6
|
McNaughton N, Bannerman D. The homogenous hippocampus: How hippocampal cells process available and potential goals. Prog Neurobiol 2024; 240:102653. [PMID: 38960002 DOI: 10.1016/j.pneurobio.2024.102653] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/04/2024] [Revised: 04/25/2024] [Accepted: 06/24/2024] [Indexed: 07/05/2024]
Abstract
We present here a view of the firing patterns of hippocampal cells that is contrary, both functionally and anatomically, to conventional wisdom. We argue that the hippocampus responds to efference copies of goals encoded elsewhere; and that it uses these to detect and resolve conflict or interference between goals in general. While goals can involve space, hippocampal cells do not encode spatial (or other special types of) memory, as such. We also argue that the transverse circuits of the hippocampus operate in an essentially homogeneous way along its length. The apparently different functions of different parts (e.g. memory retrieval versus anxiety) result from the different (situational/motivational) inputs on which those parts perform the same fundamental computational operations. On this view, the key role of the hippocampus is the iterative adjustment, via Papez-like circuits, of synaptic weights in cell assemblies elsewhere.
Collapse
Affiliation(s)
- Neil McNaughton
- Department of Psychology and Brain Health Research Centre, University of Otago, POB56, Dunedin 9054, New Zealand.
| | - David Bannerman
- Department of Experimental Psychology, University of Oxford, South Parks Road, Oxford, England, UK
| |
Collapse
|
7
|
Jensen KT, Hennequin G, Mattar MG. A recurrent network model of planning explains hippocampal replay and human behavior. Nat Neurosci 2024; 27:1340-1348. [PMID: 38849521 PMCID: PMC11239510 DOI: 10.1038/s41593-024-01675-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Accepted: 05/07/2024] [Indexed: 06/09/2024]
Abstract
When faced with a novel situation, people often spend substantial periods of time contemplating possible futures. For such planning to be rational, the benefits to behavior must compensate for the time spent thinking. Here, we capture these features of behavior by developing a neural network model where planning itself is controlled by the prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences from its own policy, which we call 'rollouts'. In a spatial navigation task, the agent learns to plan when it is beneficial, which provides a normative explanation for empirical variability in human thinking times. Additionally, the patterns of policy rollouts used by the artificial agent closely resemble patterns of rodent hippocampal replays. Our work provides a theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by-and adaptively affect-prefrontal dynamics.
Collapse
Affiliation(s)
- Kristopher T Jensen
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK.
- Sainsbury Wellcome Centre, University College London, London, UK.
| | - Guillaume Hennequin
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
| | - Marcelo G Mattar
- Department of Cognitive Science, University of California, San Diego, CA, USA
- Department of Psychology, New York University, New York, NY, USA
| |
Collapse
|
8
|
Kleinman MR, Foster DJ. Spatial localization of hippocampal replay requires dopamine signaling. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.06.04.597435. [PMID: 38895442 PMCID: PMC11185723 DOI: 10.1101/2024.06.04.597435] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/21/2024]
Abstract
Sequenced reactivations of hippocampal neurons called replays, concomitant with sharp-wave ripples in the local field potential, are critical for the consolidation of episodic memory, but whether replays depend on the brain's reward or novelty signals is unknown. Here we combined chemogenetic silencing of dopamine neurons in ventral tegmental area (VTA) and simultaneous electrophysiological recordings in dorsal hippocampal CA1, in freely behaving rats experiencing changes to reward magnitude and environmental novelty. Surprisingly, VTA silencing did not prevent ripple increases where reward was increased, but caused dramatic, aberrant ripple increases where reward was unchanged. These increases were associated with increased reverse-ordered replays. On familiar tracks this effect disappeared, and ripples tracked reward prediction error, indicating that non-VTA reward signals were sufficient to direct replay. Our results reveal a novel dependence of hippocampal replay on dopamine, and a role for a VTA-independent reward prediction error signal that is reliable only in familiar environments.
Collapse
Affiliation(s)
- Matthew R Kleinman
- Helen Wills Neuroscience Institute and Department of Psychology, University of California, Berkeley, CA 94720, USA
| | - David J Foster
- Helen Wills Neuroscience Institute and Department of Psychology, University of California, Berkeley, CA 94720, USA
- Lead contact
| |
Collapse
|
9
|
Chettih SN, Mackevicius EL, Hale S, Aronov D. Barcoding of episodic memories in the hippocampus of a food-caching bird. Cell 2024; 187:1922-1935.e20. [PMID: 38554707 PMCID: PMC11015962 DOI: 10.1016/j.cell.2024.02.032] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/18/2023] [Revised: 11/28/2023] [Accepted: 02/23/2024] [Indexed: 04/02/2024]
Abstract
The hippocampus is critical for episodic memory. Although hippocampal activity represents place and other behaviorally relevant variables, it is unclear how it encodes numerous memories of specific events in life. To study episodic coding, we leveraged the specialized behavior of chickadees-food-caching birds that form memories at well-defined moments in time whenever they cache food for subsequent retrieval. Our recordings during caching revealed very sparse, transient barcode-like patterns of firing across hippocampal neurons. Each "barcode" uniquely represented a caching event and transiently reactivated during the retrieval of that specific cache. Barcodes co-occurred with the conventional activity of place cells but were uncorrelated even for nearby cache locations that had similar place codes. We propose that animals recall episodic memories by reactivating hippocampal barcodes. Similarly to computer hash codes, these patterns assign unique identifiers to different events and could be a mechanism for rapid formation and storage of many non-interfering memories.
Collapse
Affiliation(s)
- Selmaan N Chettih
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA
| | - Emily L Mackevicius
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA; Basis Research Institute, New York, NY 10027, USA
| | - Stephanie Hale
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA
| | - Dmitriy Aronov
- Zuckerman Mind Brain Behavior Institute, Columbia University, New York, NY 10027, USA.
| |
Collapse
|
10
|
McNamee DC. The generative neural microdynamics of cognitive processing. Curr Opin Neurobiol 2024; 85:102855. [PMID: 38428170 DOI: 10.1016/j.conb.2024.102855] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Revised: 02/06/2024] [Accepted: 02/07/2024] [Indexed: 03/03/2024]
Abstract
The entorhinal cortex and hippocampus form a recurrent network that informs many cognitive processes, including memory, planning, navigation, and imagination. Neural recordings from these regions reveal spatially organized population codes corresponding to external environments and abstract spaces. Aligning the former cognitive functionalities with the latter neural phenomena is a central challenge in understanding the entorhinal-hippocampal circuit (EHC). Disparate experiments demonstrate a surprising level of complexity and apparent disorder in the intricate spatiotemporal dynamics of sequential non-local hippocampal reactivations, which occur particularly, though not exclusively, during immobile pauses and rest. We review these phenomena with a particular focus on their apparent lack of physical simulative realism. These observations are then integrated within a theoretical framework and proposed neural circuit mechanisms that normatively characterize this neural complexity by conceiving different regimes of hippocampal microdynamics as neuromarkers of diverse cognitive computations.
Collapse
|
11
|
Yang W, Sun C, Huszár R, Hainmueller T, Kiselev K, Buzsáki G. Selection of experience for memory by hippocampal sharp wave ripples. Science 2024; 383:1478-1483. [PMID: 38547293 PMCID: PMC11068097 DOI: 10.1126/science.adk8261] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/12/2023] [Accepted: 02/23/2024] [Indexed: 04/02/2024]
Abstract
Experiences need to be tagged during learning for further consolidation. However, neurophysiological mechanisms that select experiences for lasting memory are not known. By combining large-scale neural recordings in mice with dimensionality reduction techniques, we observed that successive maze traversals were tracked by continuously drifting populations of neurons, providing neuronal signatures of both places visited and events encountered. When the brain state changed during reward consumption, sharp wave ripples (SPW-Rs) occurred on some trials, and their specific spike content decoded the trial blocks that surrounded them. During postexperience sleep, SPW-Rs continued to replay those trial blocks that were reactivated most frequently during waking SPW-Rs. Replay content of awake SPW-Rs may thus provide a neurophysiological tagging mechanism to select aspects of experience that are preserved and consolidated for future use.
Collapse
Affiliation(s)
- Wannan Yang
- Neuroscience Institute and Department of Neurology, NYU Grossman School of Medicine, New York University, New York City, NY, USA
- Center for Neural Science, New York University, New York City, NY, USA
| | - Chen Sun
- Mila - Quebec AI Institute, Montréal, Quebec, Canada
| | - Roman Huszár
- Neuroscience Institute and Department of Neurology, NYU Grossman School of Medicine, New York University, New York City, NY, USA
- Center for Neural Science, New York University, New York City, NY, USA
| | - Thomas Hainmueller
- Neuroscience Institute and Department of Neurology, NYU Grossman School of Medicine, New York University, New York City, NY, USA
- Department of Psychiatry, New York University Langone Medical Center, New York City, NY, USA
| | - Kirill Kiselev
- Center for Neural Science, New York University, New York City, NY, USA
| | - György Buzsáki
- Neuroscience Institute and Department of Neurology, NYU Grossman School of Medicine, New York University, New York City, NY, USA
- Center for Neural Science, New York University, New York City, NY, USA
| |
Collapse
|
12
|
Sagiv Y, Akam T, Witten IB, Daw ND. Prioritizing replay when future goals are unknown. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.02.29.582822. [PMID: 38496674 PMCID: PMC10942393 DOI: 10.1101/2024.02.29.582822] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/19/2024]
Abstract
Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. However, recent puzzling data appear to contradict this perspective by showing that replayed destinations lag current goals. These results may support an alternative hypothesis that replay updates route information to build a "cognitive map." Yet no similar theory exists to formalize this view, and it is unclear how such a map is represented or what role replay plays in computing it. We address these gaps by introducing a theory of replay that learns a map of routes to candidate goals, before reward is available or when its location may change. Our work extends the planning account to capture a general map-building function for replay, reconciling it with data, and revealing an unexpected relationship between the seemingly distinct hypotheses.
Collapse
Affiliation(s)
- Yotam Sagiv
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA
| | - Thomas Akam
- Department of Experimental Psychology, Oxford University, Oxford, UK
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA
| | - Nathaniel D Daw
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA
| |
Collapse
|
13
|
Jeffery KJ. The mosaic structure of the mammalian cognitive map. Learn Behav 2024; 52:19-34. [PMID: 38231426 PMCID: PMC10923978 DOI: 10.3758/s13420-023-00618-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 11/27/2023] [Indexed: 01/18/2024]
Abstract
The cognitive map, proposed by Tolman in the 1940s, is a hypothetical internal representation of space constructed by the brain to enable an animal to undertake flexible spatial behaviors such as navigation. The subsequent discovery of place cells in the hippocampus of rats suggested that such a map-like representation does exist, and also provided a tool with which to explore its properties. Single-neuron studies in rodents conducted in small singular spaces have suggested that the map is founded on a metric framework, preserving distances and directions in an abstract representational format. An open question is whether this metric structure pertains over extended, often complexly structured real-world space. The data reviewed here suggest that this is not the case. The emerging picture is that instead of being a single, unified construct, the map is a mosaic of fragments that are heterogeneous, variably metric, multiply scaled, and sometimes laid on top of each other. Important organizing factors within and between fragments include boundaries, context, compass direction, and gravity. The map functions not to provide a comprehensive and precise rendering of the environment but rather to support adaptive behavior, tailored to the species and situation.
Collapse
Affiliation(s)
- Kate J Jeffery
- School of Psychology and Neuroscience, College of Medical, Veterinary and Life Sciences, University of Glasgow, Glasgow, UK.
| |
Collapse
|
14
|
Lai AT, Espinosa G, Wink GE, Angeloni CF, Dombeck DA, MacIver MA. A robot-rodent interaction arena with adjustable spatial complexity for ethologically relevant behavioral studies. Cell Rep 2024; 43:113671. [PMID: 38280195 DOI: 10.1016/j.celrep.2023.113671] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 10/19/2023] [Accepted: 12/26/2023] [Indexed: 01/29/2024] Open
Abstract
Outside of the laboratory, animals behave in spaces where they can transition between open areas and coverage as they interact with others. Replicating these conditions in the laboratory can be difficult to control and record. This has led to a dominance of relatively simple, static behavioral paradigms that reduce the ethological relevance of behaviors and may alter the engagement of cognitive processes such as planning and decision-making. Therefore, we developed a method for controllable, repeatable interactions with others in a reconfigurable space. Mice navigate a large honeycomb lattice of adjustable obstacles as they interact with an autonomous robot coupled to their actions. We illustrate the system using the robot as a pseudo-predator, delivering airpuffs to the mice. The combination of obstacles and a mobile threat elicits a diverse set of behaviors, such as increased path diversity, peeking, and baiting, providing a method to explore ethologically relevant behaviors in the laboratory.
Collapse
Affiliation(s)
- Alexander T Lai
- Department of Biomedical Engineering, Technological Institute E311, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA
| | - German Espinosa
- Department of Computer Science, Northwestern University, Seeley Mudd 3219, 2233 Tech Drive, Evanston, IL 60208, USA
| | - Gabrielle E Wink
- Department of Mechanical Engineering, Technological Institute B224, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA
| | - Christopher F Angeloni
- Department of Neurobiology, Northwestern University, Hogan 2-160, 2205 Tech Drive, Evanston, IL 60208, USA
| | - Daniel A Dombeck
- Department of Neurobiology, Northwestern University, Hogan 2-160, 2205 Tech Drive, Evanston, IL 60208, USA.
| | - Malcolm A MacIver
- Department of Biomedical Engineering, Technological Institute E311, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA; Department of Computer Science, Northwestern University, Seeley Mudd 3219, 2233 Tech Drive, Evanston, IL 60208, USA; Department of Mechanical Engineering, Technological Institute B224, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA; Department of Neurobiology, Northwestern University, Hogan 2-160, 2205 Tech Drive, Evanston, IL 60208, USA.
| |
Collapse
|
15
|
Crombie KM, Azar A, Botsford C, Heilicher M, Jaeb M, Gruichich TS, Schomaker CM, Williams R, Stowe ZN, Dunsmoor JE, Cisler JM. Decoding context memories for threat in large-scale neural networks. Cereb Cortex 2024; 34:bhae018. [PMID: 38300181 PMCID: PMC10839849 DOI: 10.1093/cercor/bhae018] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/05/2023] [Revised: 01/08/2024] [Accepted: 01/10/2024] [Indexed: 02/02/2024] Open
Abstract
Humans are often tasked with determining the degree to which a given situation poses threat. Salient cues present during prior events help bring online memories for context, which plays an informative role in this process. However, it is relatively unknown whether and how individuals use features of the environment to retrieve context memories for threat, enabling accurate inferences about the current level of danger/threat (i.e. retrieve appropriate memory) when there is a degree of ambiguity surrounding the present context. We leveraged computational neuroscience approaches (i.e. independent component analysis and multivariate pattern analyses) to decode large-scale neural network activity patterns engaged during learning and inferring threat context during a novel functional magnetic resonance imaging task. Here, we report that individuals accurately infer threat contexts under ambiguous conditions through neural reinstatement of large-scale network activity patterns (specifically striatum, salience, and frontoparietal networks) that track the signal value of environmental cues, which, in turn, allows reinstatement of a mental representation, primarily within a ventral visual network, of the previously learned threat context. These results provide novel insight into distinct, but overlapping, neural mechanisms by which individuals may utilize prior learning to effectively make decisions about ambiguous threat-related contexts as they navigate the environment.
Collapse
Affiliation(s)
- Kevin M Crombie
- Department of Psychiatry and Behavioral Sciences, The University of Texas at Austin, 1601 Trinity Street, Building B, Austin, TX 78712, United States
- Department of Kinesiology, The University of Alabama, 620 Judy Bonner Drive, Box 870312, Tuscaloosa, AL 35487, United States
| | - Ameera Azar
- Department of Psychiatry and Behavioral Sciences, The University of Texas at Austin, 1601 Trinity Street, Building B, Austin, TX 78712, United States
| | - Chloe Botsford
- Department of Psychiatry, University of Wisconsin—Madison, 6001 Research Park Boulevard, Madison, WI 53719, United States
| | - Mickela Heilicher
- Department of Psychiatry, University of Wisconsin—Madison, 6001 Research Park Boulevard, Madison, WI 53719, United States
| | - Michael Jaeb
- Department of Psychiatry, University of Wisconsin—Madison, 6001 Research Park Boulevard, Madison, WI 53719, United States
| | - Tijana Sagorac Gruichich
- Department of Psychiatry, University of Wisconsin—Madison, 6001 Research Park Boulevard, Madison, WI 53719, United States
| | - Chloe M Schomaker
- Department of Psychiatry and Behavioral Sciences, The University of Texas at Austin, 1601 Trinity Street, Building B, Austin, TX 78712, United States
| | - Rachel Williams
- Department of Psychiatry, University of Wisconsin—Madison, 6001 Research Park Boulevard, Madison, WI 53719, United States
| | - Zachary N Stowe
- Department of Psychiatry, University of Wisconsin—Madison, 6001 Research Park Boulevard, Madison, WI 53719, United States
| | - Joseph E Dunsmoor
- Department of Psychiatry and Behavioral Sciences, The University of Texas at Austin, 1601 Trinity Street, Building B, Austin, TX 78712, United States
- Institute for Neuroscience, The University of Texas at Austin, Austin, TX 78712, United States
- Department of Neuroscience, The University of Texas at Austin, 1 University Station, Stop C7000, Austin, TX 78712, United States
| | - Josh M Cisler
- Department of Psychiatry and Behavioral Sciences, The University of Texas at Austin, 1601 Trinity Street, Building B, Austin, TX 78712, United States
- Institute for Early Life Adversity Research, The University of Texas at Austin Dell Medical School, 1601 Trinity Street, Building B, Austin, TX 78712, United States
| |
Collapse
|
16
|
Huelin Gorriz M, Takigawa M, Bendor D. The role of experience in prioritizing hippocampal replay. Nat Commun 2023; 14:8157. [PMID: 38071221 PMCID: PMC10710481 DOI: 10.1038/s41467-023-43939-z] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 11/24/2023] [Indexed: 12/18/2023] Open
Abstract
During sleep, recent memories are replayed by the hippocampus, leading to their consolidation, with a higher priority given to salient experiences. To examine the role of replay in the selective strengthening of memories, we recorded large ensembles of hippocampal place cells while male rats ran repeated spatial trajectories on two linear tracks, differing in either their familiarity or number of laps run. We observed that during sleep, the rate of replay events for a given track increased proportionally with the number of spatial trajectories run by the animal. In contrast, the rate of sleep replay events decreased if the animal was more familiar with the track. Furthermore, we find that the cumulative number of awake replay events occurring during behavior, influenced by both the novelty and duration of an experience, predicts which memories are prioritized for sleep replay, providing a more parsimonious neural correlate for the selective strengthening of memories.
Collapse
Affiliation(s)
- Marta Huelin Gorriz
- Institute of Behavioural Neuroscience (IBN), University College London (UCL), London, WC1H 0AP, UK
| | - Masahiro Takigawa
- Institute of Behavioural Neuroscience (IBN), University College London (UCL), London, WC1H 0AP, UK
| | - Daniel Bendor
- Institute of Behavioural Neuroscience (IBN), University College London (UCL), London, WC1H 0AP, UK.
| |
Collapse
|
17
|
Yang W, Sun C, Huszár R, Hainmueller T, Buzsáki G. Selection of experience for memory by hippocampal sharp wave ripples. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.11.07.565935. [PMID: 37987008 PMCID: PMC10659301 DOI: 10.1101/2023.11.07.565935] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/22/2023]
Abstract
A general wisdom is that experiences need to be tagged during learning for further consolidation. However, brain mechanisms that select experiences for lasting memory are not known. Combining large-scale neural recordings with a novel application of dimensionality reduction techniques, we observed that successive traversals in the maze were tracked by continuously drifting populations of neurons, providing neuronal signatures of both places visited and events encountered (trial number). When the brain state changed during reward consumption, sharp wave ripples (SPW-Rs) occurred on some trials and their unique spike content most often decoded the trial in which they occurred. In turn, during post-experience sleep, SPW-Rs continued to replay those trials that were reactivated most frequently during awake SPW-Rs. These findings suggest that replay content of awake SPW-Rs provides a tagging mechanism to select aspects of experience that are preserved and consolidated for future use.
Collapse
Affiliation(s)
- Wannan Yang
- Center for Neural Science, New York University, NY, USA
- Neuroscience Institute, NYU Grossman School of Medicine, New York University, New York, NY, USA
| | - Chen Sun
- Mila - Quebec AI Institute, Montréal, Canada
| | - Roman Huszár
- Neuroscience Institute, NYU Grossman School of Medicine, New York University, New York, NY, USA
| | - Thomas Hainmueller
- Neuroscience Institute, NYU Grossman School of Medicine, New York University, New York, NY, USA
- Department of Psychiatry, New York University Langone Medical Center, New York, NY, USA
| | - György Buzsáki
- Center for Neural Science, New York University, NY, USA
- Neuroscience Institute, NYU Grossman School of Medicine, New York University, New York, NY, USA
| |
Collapse
|
18
|
Lai C, Tanaka S, Harris TD, Lee AK. Volitional activation of remote place representations with a hippocampal brain-machine interface. Science 2023; 382:566-573. [PMID: 37917713 PMCID: PMC10683874 DOI: 10.1126/science.adh5206] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2023] [Accepted: 09/22/2023] [Indexed: 11/04/2023]
Abstract
The hippocampus is critical for recollecting and imagining experiences. This is believed to involve voluntarily drawing from hippocampal memory representations of people, events, and places, including maplike representations of familiar environments. However, whether representations in such "cognitive maps" can be volitionally accessed is unknown. We developed a brain-machine interface to test whether rats can do so by controlling their hippocampal activity in a flexible, goal-directed, and model-based manner. We found that rats can efficiently navigate or direct objects to arbitrary goal locations within a virtual reality arena solely by activating and sustaining appropriate hippocampal representations of remote places. This provides insight into the mechanisms underlying episodic memory recall, mental simulation and planning, and imagination and opens up possibilities for high-level neural prosthetics that use hippocampal representations.
Collapse
Affiliation(s)
- Chongxi Lai
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, USA
| | - Shinsuke Tanaka
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, USA
| | - Timothy D. Harris
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, USA
| | - Albert K. Lee
- Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, Virginia, USA
- Howard Hughes Medical Institute and Department of Neurology, Beth Israel Deaconess Medical Center, Boston, MA, USA
| |
Collapse
|
19
|
Liu C, Todorova R, Tang W, Oliva A, Fernandez-Ruiz A. Associative and predictive hippocampal codes support memory-guided behaviors. Science 2023; 382:eadi8237. [PMID: 37856604 PMCID: PMC10894649 DOI: 10.1126/science.adi8237] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 08/21/2023] [Indexed: 10/21/2023]
Abstract
Episodic memory involves learning and recalling associations between items and their spatiotemporal context. Those memories can be further used to generate internal models of the world that enable predictions to be made. The mechanisms that support these associative and predictive aspects of memory are not yet understood. In this study, we used an optogenetic manipulation to perturb the sequential structure, but not global network dynamics, of place cells as rats traversed specific spatial trajectories. This perturbation abolished replay of those trajectories and the development of predictive representations, leading to impaired learning of new optimal trajectories during memory-guided navigation. However, place cell assembly reactivation and reward-context associative learning were unaffected. Our results show a mechanistic dissociation between two complementary hippocampal codes: an associative code (through coactivity) and a predictive code (through sequences).
Collapse
Affiliation(s)
| | | | - Wenbo Tang
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, USA
| | - Azahara Oliva
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, USA
| | | |
Collapse
|
20
|
Newman JP, Zhang J, Cuevas-López A, Miller NJ, Honda T, van der Goes MSH, Leighton AH, Carvalho F, Lopes G, Lakunina A, Siegle JH, Harnett MT, Wilson MA, Voigts J. A unified open-source platform for multimodal neural recording and perturbation during naturalistic behavior. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.30.554672. [PMID: 37693443 PMCID: PMC10491150 DOI: 10.1101/2023.08.30.554672] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 09/12/2023]
Abstract
Behavioral neuroscience faces two conflicting demands: long-duration recordings from large neural populations and unimpeded animal behavior. To meet this challenge, we developed ONIX, an open-source data acquisition system with high data throughput (2GB/sec) and low closed-loop latencies (<1ms) that uses a novel 0.3 mm thin tether to minimize behavioral impact. Head position and rotation are tracked in 3D and used to drive active commutation without torque measurements. ONIX can acquire from combinations of passive electrodes, Neuropixels probes, head-mounted microscopes, cameras, 3D-trackers, and other data sources. We used ONIX to perform uninterrupted, long (~7 hours) neural recordings in mice as they traversed complex 3-dimensional terrain. ONIX allowed exploration with similar mobility as non-implanted animals, in contrast to conventional tethered systems which restricted movement. By combining long recordings with full mobility, our technology will enable new progress on questions that require high-quality neural recordings during ethologically grounded behaviors.
Collapse
Affiliation(s)
- Jonathan P Newman
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA
- Picower Institute for Learning and Memory, MIT, Cambridge, MA, USA
- Open Ephys Inc. Atlanta, GA, USA
| | - Jie Zhang
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA
- Picower Institute for Learning and Memory, MIT, Cambridge, MA, USA
| | - Aarón Cuevas-López
- Open Ephys Inc. Atlanta, GA, USA
- Dept. of Electrical Engineering, Polytechnic University of Valencia, Valencia, Spain
- Open Ephys Production Site, Lisbon, Portugal
| | - Nicholas J Miller
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA
- McGovern Institute for Brain Research, MIT, Cambridge, MA, USA
| | - Takato Honda
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA
- Picower Institute for Learning and Memory, MIT, Cambridge, MA, USA
| | - Marie-Sophie H van der Goes
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA
- McGovern Institute for Brain Research, MIT, Cambridge, MA, USA
| | | | | | | | - Anna Lakunina
- Allen Institute for Neural Dynamics, Seattle, Washington, USA
| | - Joshua H Siegle
- Allen Institute for Neural Dynamics, Seattle, Washington, USA
| | - Mark T Harnett
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA
- McGovern Institute for Brain Research, MIT, Cambridge, MA, USA
| | - Matthew A Wilson
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA
- Picower Institute for Learning and Memory, MIT, Cambridge, MA, USA
| | - Jakob Voigts
- Department of Brain and Cognitive Sciences, MIT, Cambridge, MA, USA
- Open Ephys Inc. Atlanta, GA, USA
- McGovern Institute for Brain Research, MIT, Cambridge, MA, USA
- HHMI Janelia Research Campus, Ashburn, VA, USA
| |
Collapse
|
21
|
Parra-Barrero E, Vijayabaskaran S, Seabrook E, Wiskott L, Cheng S. A map of spatial navigation for neuroscience. Neurosci Biobehav Rev 2023; 152:105200. [PMID: 37178943 DOI: 10.1016/j.neubiorev.2023.105200] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2023] [Revised: 04/13/2023] [Accepted: 04/24/2023] [Indexed: 05/15/2023]
Abstract
Spatial navigation has received much attention from neuroscientists, leading to the identification of key brain areas and the discovery of numerous spatially selective cells. Despite this progress, our understanding of how the pieces fit together to drive behavior is generally lacking. We argue that this is partly caused by insufficient communication between behavioral and neuroscientific researchers. This has led the latter to under-appreciate the relevance and complexity of spatial behavior, and to focus too narrowly on characterizing neural representations of space-disconnected from the computations these representations are meant to enable. We therefore propose a taxonomy of navigation processes in mammals that can serve as a common framework for structuring and facilitating interdisciplinary research in the field. Using the taxonomy as a guide, we review behavioral and neural studies of spatial navigation. In doing so, we validate the taxonomy and showcase its usefulness in identifying potential issues with common experimental approaches, designing experiments that adequately target particular behaviors, correctly interpreting neural activity, and pointing to new avenues of research.
Collapse
Affiliation(s)
- Eloy Parra-Barrero
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany; International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| | - Sandhiya Vijayabaskaran
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany
| | - Eddie Seabrook
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany
| | - Laurenz Wiskott
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany; International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| | - Sen Cheng
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany; International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany.
| |
Collapse
|
22
|
Chettih SN, Mackevicius EL, Hale S, Aronov D. Barcoding of episodic memories in the hippocampus of a food-caching bird. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.05.27.542597. [PMID: 37461442 PMCID: PMC10349996 DOI: 10.1101/2023.05.27.542597] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Indexed: 07/24/2023]
Abstract
Episodic memory, or memory of experienced events, is a critical function of the hippocampus1-3. It is therefore important to understand how hippocampal activity represents specific events in an animal's life. We addressed this question in chickadees - specialist food-caching birds that hide food at scattered locations and use memory to find their caches later in time4,5. We performed high-density neural recordings in the hippocampus of chickadees as they cached and retrieved seeds in a laboratory arena. We found that each caching event was represented by a burst of firing in a unique set of hippocampal neurons. These 'barcode-like' patterns of activity were sparse (<10% of neurons active), uncorrelated even for immediately adjacent caches, and different even for separate caches at the same location. The barcode representing a specific caching event was transiently reactivated whenever a bird later interacted with the same cache - for example, to retrieve food. Barcodes co-occurred with conventional place cell activity6,7, as well as location-independent responses to cached seeds. We propose that barcodes are signatures of episodic memories evoked during memory recall. These patterns assign a unique identifier to each event and may be a mechanism for rapid formation and storage of many non-interfering memories.
Collapse
Affiliation(s)
| | | | - Stephanie Hale
- Zuckerman Mind Brain Behavior Institute, Columbia University
| | - Dmitriy Aronov
- Zuckerman Mind Brain Behavior Institute, Columbia University
| |
Collapse
|
23
|
Ambrogioni L, Ólafsdóttir HF. Rethinking the hippocampal cognitive map as a meta-learning computational module. Trends Cogn Sci 2023:S1364-6613(23)00128-6. [PMID: 37357064 DOI: 10.1016/j.tics.2023.05.011] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 04/26/2023] [Accepted: 05/24/2023] [Indexed: 06/27/2023]
Abstract
A hallmark of biological intelligence is the ability to adaptively draw on past experience to guide behaviour under novel situations. Yet, the neurobiological principles that underlie this form of meta-learning remain relatively unexplored. In this Opinion, we review the existing literature on hippocampal spatial representations and reinforcement learning theory and describe a novel theoretical framework that aims to account for biological meta-learning. We conjecture that so-called hippocampal cognitive maps of familiar environments are part of a larger meta-representation (meta-map) that encodes information states and sources, which support exploration and provides a foundation for learning. We also introduce concrete hypotheses on how these generic states can be encoded using a principle of superposition.
Collapse
Affiliation(s)
- Luca Ambrogioni
- Donders Institute for Brain, Cognition & Behaviour, Radboud Universiteit, Nijmegen, The Netherlands.
| | - H Freyja Ólafsdóttir
- Donders Institute for Brain, Cognition & Behaviour, Radboud Universiteit, Nijmegen, The Netherlands.
| |
Collapse
|
24
|
Duszkiewicz AJ, Rossato JI, Moreno A, Takeuchi T, Yamasaki M, Genzel L, Spooner P, Canals S, Morris RGM. Execution of new trajectories toward a stable goal without a functional hippocampus. Hippocampus 2023; 33:769-786. [PMID: 36798045 PMCID: PMC10946713 DOI: 10.1002/hipo.23497] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2022] [Revised: 12/06/2022] [Accepted: 12/14/2022] [Indexed: 02/18/2023]
Abstract
The hippocampus is a critical component of a mammalian spatial navigation system, with the firing sequences of hippocampal place cells during sleep or immobility constituting a "replay" of an animal's past trajectories. A novel spatial navigation task recently revealed that such "replay" sequences of place fields can also prospectively map onto imminent new paths to a goal that occupies a stable location during each session. It was hypothesized that such "prospective replay" sequences may play a causal role in goal-directed navigation. In the present study, we query this putative causal role in finding only minimal effects of muscimol-induced inactivation of the dorsal and intermediate hippocampus on the same spatial navigation task. The concentration of muscimol used demonstrably inhibited hippocampal cell firing in vivo and caused a severe deficit in a hippocampal-dependent "episodic-like" spatial memory task in a watermaze. These findings call into question whether "prospective replay" of an imminent and direct path is actually necessary for its execution in certain navigational tasks.
Collapse
Affiliation(s)
- Adrian J. Duszkiewicz
- Centre for Discovery Brain Sciences, Edinburgh NeuroscienceUniversity of EdinburghEdinburghUK
- Department of PsychologyUniversity of StirlingStirlingScotlandUK
| | - Janine I. Rossato
- Centre for Discovery Brain Sciences, Edinburgh NeuroscienceUniversity of EdinburghEdinburghUK
- Department of PhysiologyUniversidade Federal do Rio Grande do NorteRio Grande do NorteBrazil
| | - Andrea Moreno
- Centre for Discovery Brain Sciences, Edinburgh NeuroscienceUniversity of EdinburghEdinburghUK
- Instituto de Neurociencias, CSIC‐UMHSan Juan de AlicanteSpain
- Department of Biomedicine, Danish Research Institute of Translational Neuroscience (DANDRITE)Aarhus UniversityAarhus CDenmark
| | - Tomonori Takeuchi
- Centre for Discovery Brain Sciences, Edinburgh NeuroscienceUniversity of EdinburghEdinburghUK
- Department of Biomedicine, Danish Research Institute of Translational Neuroscience (DANDRITE)Aarhus UniversityAarhus CDenmark
| | - Miwako Yamasaki
- Department of Anatomy, Graduate School of MedicineHokkaido UniversitySapporoJapan
| | - Lisa Genzel
- Centre for Discovery Brain Sciences, Edinburgh NeuroscienceUniversity of EdinburghEdinburghUK
- Donders Institute for Brain, Cognition, and BehaviourRadboud University and RadboudumcNijmegenThe Netherlands
| | - Patrick Spooner
- Centre for Discovery Brain Sciences, Edinburgh NeuroscienceUniversity of EdinburghEdinburghUK
| | - Santiago Canals
- Instituto de Neurociencias, CSIC‐UMHSan Juan de AlicanteSpain
| | - Richard G. M. Morris
- Centre for Discovery Brain Sciences, Edinburgh NeuroscienceUniversity of EdinburghEdinburghUK
- Instituto de Neurociencias, CSIC‐UMHSan Juan de AlicanteSpain
| |
Collapse
|
25
|
Hanganu-Opatz IL, Klausberger T, Sigurdsson T, Nieder A, Jacob SN, Bartos M, Sauer JF, Durstewitz D, Leibold C, Diester I. Resolving the prefrontal mechanisms of adaptive cognitive behaviors: A cross-species perspective. Neuron 2023; 111:1020-1036. [PMID: 37023708 DOI: 10.1016/j.neuron.2023.03.017] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2022] [Revised: 02/15/2023] [Accepted: 03/10/2023] [Indexed: 04/08/2023]
Abstract
The prefrontal cortex (PFC) enables a staggering variety of complex behaviors, such as planning actions, solving problems, and adapting to new situations according to external information and internal states. These higher-order abilities, collectively defined as adaptive cognitive behavior, require cellular ensembles that coordinate the tradeoff between the stability and flexibility of neural representations. While the mechanisms underlying the function of cellular ensembles are still unclear, recent experimental and theoretical studies suggest that temporal coordination dynamically binds prefrontal neurons into functional ensembles. A so far largely separate stream of research has investigated the prefrontal efferent and afferent connectivity. These two research streams have recently converged on the hypothesis that prefrontal connectivity patterns influence ensemble formation and the function of neurons within ensembles. Here, we propose a unitary concept that, leveraging a cross-species definition of prefrontal regions, explains how prefrontal ensembles adaptively regulate and efficiently coordinate multiple processes in distinct cognitive behaviors.
Collapse
Affiliation(s)
- Ileana L Hanganu-Opatz
- Institute of Developmental Neurophysiology, Center for Molecular Neurobiology, Hamburg Center of Neuroscience, University Medical Center Hamburg-Eppendorf, Hamburg, Germany.
| | - Thomas Klausberger
- Center for Brain Research, Division of Cognitive Neurobiology, Medical University of Vienna, Vienna, Austria
| | - Torfi Sigurdsson
- Institute of Neurophysiology, Goethe University, Frankfurt, Germany
| | - Andreas Nieder
- Animal Physiology Unit, Institute of Neurobiology, University of Tübingen, 72076 Tübingen, Germany
| | - Simon N Jacob
- Translational Neurotechnology Laboratory, Department of Neurosurgery, Klinikum rechts der Isar, Technical University of Munich, Munich, Germany
| | - Marlene Bartos
- Institute for Physiology I, Medical Faculty, University of Freiburg, Freiburg im Breisgau, Germany
| | - Jonas-Frederic Sauer
- Institute for Physiology I, Medical Faculty, University of Freiburg, Freiburg im Breisgau, Germany
| | - Daniel Durstewitz
- Department of Theoretical Neuroscience, Central Institute of Mental Health & Faculty of Physics and Astronomy, Heidelberg University, Heidelberg, Germany
| | - Christian Leibold
- Faculty of Biology, Bernstein Center Freiburg, BrainLinks-BrainTools, University of Freiburg, Freiburg im Breisgau, Germany
| | - Ilka Diester
- Optophysiology - Optogenetics and Neurophysiology, IMBIT // BrainLinks-BrainTools, University of Freiburg, Freiburg im Breisgau, Germany.
| |
Collapse
|
26
|
Ormond J, Serka SA, Johansen JP. Enhanced Reactivation of Remapping Place Cells during Aversive Learning. J Neurosci 2023; 43:2153-2167. [PMID: 36596695 PMCID: PMC10039748 DOI: 10.1523/jneurosci.1450-22.2022] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2022] [Revised: 11/23/2022] [Accepted: 12/01/2022] [Indexed: 01/05/2023] Open
Abstract
Study of the hippocampal place cell system has greatly enhanced our understanding of memory encoding for distinct places, but how episodic memories for distinct experiences occurring within familiar environments are encoded is less clear. We developed a spatial decision-making task in which male rats learned to navigate a multiarm maze to a goal location for food reward while avoiding maze arms in which aversive stimuli were delivered. Task learning induced partial remapping in CA1 place cells, allowing us to identify both remapping and stable cell populations. Remapping cells were recruited into sharp-wave ripples and associated replay events to a greater extent than stable cells, despite having similar firing rates during navigation of the maze. Our results suggest that recruitment into replay events may be a mechanism to incorporate new contextual information into a previously formed and stabilized spatial representation.SIGNIFICANCE STATEMENT Hippocampal place cells provide a map of space that animals use to navigate. This map can change to reflect changes in the physical properties of the environment in which the animal finds itself, and also in response to nonphysical contextual changes, such as changes in the valence of specific locations within that environment. We show here that cells which change their spatial tuning after a change in context are preferentially recruited into sharp-wave ripple-associated replay events compared with stable nonremapping cells. Thus, our data lend strong support to the hypothesis that replay is a mechanism for the storage of new spatial maps.
Collapse
Affiliation(s)
- Jake Ormond
- Laboratory for Neural Circuitry of Memory, RIKEN Center for Brain Science, Saitama 351-0198, Japan
| | - Simon A Serka
- Laboratory for Neural Circuitry of Memory, RIKEN Center for Brain Science, Saitama 351-0198, Japan
| | - Joshua P Johansen
- Laboratory for Neural Circuitry of Memory, RIKEN Center for Brain Science, Saitama 351-0198, Japan
| |
Collapse
|
27
|
Fang C, Aronov D, Abbott LF, Mackevicius EL. Neural learning rules for generating flexible predictions and computing the successor representation. eLife 2023; 12:e80680. [PMID: 36928104 PMCID: PMC10019889 DOI: 10.7554/elife.80680] [Citation(s) in RCA: 10] [Impact Index Per Article: 10.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 10/26/2022] [Indexed: 03/18/2023] Open
Abstract
The predictive nature of the hippocampus is thought to be useful for memory-guided cognitive behaviors. Inspired by the reinforcement learning literature, this notion has been formalized as a predictive map called the successor representation (SR). The SR captures a number of observations about hippocampal activity. However, the algorithm does not provide a neural mechanism for how such representations arise. Here, we show the dynamics of a recurrent neural network naturally calculate the SR when the synaptic weights match the transition probability matrix. Interestingly, the predictive horizon can be flexibly modulated simply by changing the network gain. We derive simple, biologically plausible learning rules to learn the SR in a recurrent network. We test our model with realistic inputs and match hippocampal data recorded during random foraging. Taken together, our results suggest that the SR is more accessible in neural circuits than previously thought and can support a broad range of cognitive functions.
Collapse
Affiliation(s)
- Ching Fang
- Zuckerman Institute, Department of Neuroscience, Columbia UniversityNew YorkUnited States
| | - Dmitriy Aronov
- Zuckerman Institute, Department of Neuroscience, Columbia UniversityNew YorkUnited States
| | - LF Abbott
- Zuckerman Institute, Department of Neuroscience, Columbia UniversityNew YorkUnited States
| | - Emily L Mackevicius
- Zuckerman Institute, Department of Neuroscience, Columbia UniversityNew YorkUnited States
- Basis Research InstituteNew YorkUnited States
| |
Collapse
|
28
|
Cisler JM, Tamman AJF, Fonzo GA. Diminished prospective mental representations of reward mediate reward learning strategies among youth with internalizing symptoms. Psychol Med 2023; 53:1-11. [PMID: 36878892 PMCID: PMC10600826 DOI: 10.1017/s0033291723000478] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/21/2022] [Revised: 01/09/2023] [Accepted: 02/08/2023] [Indexed: 03/08/2023]
Abstract
BACKGROUND Adolescent internalizing symptoms and trauma exposure have been linked with altered reward learning processes and decreased ventral striatal responses to rewarding cues. Recent computational work on decision-making highlights an important role for prospective representations of the imagined outcomes of different choices. This study tested whether internalizing symptoms and trauma exposure among youth impact the generation of prospective reward representations during decision-making and potentially mediate altered behavioral strategies during reward learning. METHODS Sixty-one adolescent females with varying exposure to interpersonal violence exposure (n = 31 with histories of physical or sexual assault) and severity of internalizing symptoms completed a social reward learning task during fMRI. Multivariate pattern analyses (MVPA) were used to decode neural reward representations at the time of choice. RESULTS MVPA demonstrated that rewarding outcomes could accurately be decoded within several large-scale distributed networks (e.g. frontoparietal and striatum networks), that these reward representations were reactivated prospectively at the time of choice in proportion to the expected probability of receiving reward, and that youth with behavioral strategies that favored exploiting high reward options demonstrated greater prospective generation of reward representations. Youth internalizing symptoms, but not trauma exposure characteristics, were negatively associated with both the behavioral strategy of exploiting high reward options as well as the prospective generation of reward representations in the striatum. CONCLUSIONS These data suggest diminished prospective mental simulation of reward as a mechanism of altered reward learning strategies among youth with internalizing symptoms.
Collapse
Affiliation(s)
- Josh M. Cisler
- Department of Psychiatry and Behavioral Sciences, Dell Medical School, University of Texas at Austin, USA
- Institute for Early Life Adversity Research, Dell Medical School, University of Texas at Austin, USA
| | - Amanda J. F. Tamman
- Menninger Department of Psychiatry and Behavioral Sciences, Baylor College of Medicine, Houston, TX, USA
| | - Greg A. Fonzo
- Department of Psychiatry and Behavioral Sciences, Dell Medical School, University of Texas at Austin, USA
- Institute for Early Life Adversity Research, Dell Medical School, University of Texas at Austin, USA
- Center for Psychedelic Research and Therapy, Dell Medical School, University of Texas at Austin, USA
| |
Collapse
|
29
|
Chen ZS, Wilson MA. How our understanding of memory replay evolves. J Neurophysiol 2023; 129:552-580. [PMID: 36752404 PMCID: PMC9988534 DOI: 10.1152/jn.00454.2022] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 01/20/2023] [Accepted: 01/20/2023] [Indexed: 02/09/2023] Open
Abstract
Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.
Collapse
Affiliation(s)
- Zhe Sage Chen
- Department of Psychiatry, New York University Grossman School of Medicine, New York, New York, United States
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, New York, United States
- Neuroscience Institute, New York University Grossman School of Medicine, New York, New York, United States
- Department of Biomedical Engineering, New York University Tandon School of Engineering, Brooklyn, New York, United States
| | - Matthew A Wilson
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
| |
Collapse
|
30
|
Kurth-Nelson Z, Behrens T, Wayne G, Miller K, Luettgau L, Dolan R, Liu Y, Schwartenbeck P. Replay and compositional computation. Neuron 2023; 111:454-469. [PMID: 36640765 DOI: 10.1016/j.neuron.2022.12.028] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/11/2022] [Accepted: 12/18/2022] [Indexed: 01/15/2023]
Abstract
Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.
Collapse
Affiliation(s)
- Zeb Kurth-Nelson
- DeepMind, London, UK; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK.
| | - Timothy Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - Kevin Miller
- DeepMind, London, UK; Institute of Ophthalmology, University College London, London, UK
| | - Lennart Luettgau
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Ray Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK; Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Philipp Schwartenbeck
- Max Planck Institute for Biological Cybernetics, Tubingen, Germany; University of Tubingen, Tubingen, Germany
| |
Collapse
|
31
|
Riquelme JL, Hemberger M, Laurent G, Gjorgjieva J. Single spikes drive sequential propagation and routing of activity in a cortical network. eLife 2023; 12:e79928. [PMID: 36780217 PMCID: PMC9925052 DOI: 10.7554/elife.79928] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2022] [Accepted: 12/19/2022] [Indexed: 02/14/2023] Open
Abstract
Single spikes can trigger repeatable firing sequences in cortical networks. The mechanisms that support reliable propagation of activity from such small events and their functional consequences remain unclear. By constraining a recurrent network model with experimental statistics from turtle cortex, we generate reliable and temporally precise sequences from single spike triggers. We find that rare strong connections support sequence propagation, while dense weak connections modulate propagation reliability. We identify sections of sequences corresponding to divergent branches of strongly connected neurons which can be selectively gated. Applying external inputs to specific neurons in the sparse backbone of strong connections can effectively control propagation and route activity within the network. Finally, we demonstrate that concurrent sequences interact reliably, generating a highly combinatorial space of sequence activations. Our results reveal the impact of individual spikes in cortical circuits, detailing how repeatable sequences of activity can be triggered, sustained, and controlled during cortical computations.
Collapse
Affiliation(s)
- Juan Luis Riquelme
- Max Planck Institute for Brain ResearchFrankfurt am MainGermany
- School of Life Sciences, Technical University of MunichFreisingGermany
| | - Mike Hemberger
- Max Planck Institute for Brain ResearchFrankfurt am MainGermany
| | - Gilles Laurent
- Max Planck Institute for Brain ResearchFrankfurt am MainGermany
| | - Julijana Gjorgjieva
- Max Planck Institute for Brain ResearchFrankfurt am MainGermany
- School of Life Sciences, Technical University of MunichFreisingGermany
| |
Collapse
|
32
|
Gao Y. A computational model of learning flexible navigation in a maze by layout-conforming replay of place cells. Front Comput Neurosci 2023; 17:1053097. [PMID: 36846726 PMCID: PMC9947252 DOI: 10.3389/fncom.2023.1053097] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/25/2022] [Accepted: 01/16/2023] [Indexed: 02/11/2023] Open
Abstract
Recent experimental observations have shown that the reactivation of hippocampal place cells (PC) during sleep or wakeful immobility depicts trajectories that can go around barriers and can flexibly adapt to a changing maze layout. However, existing computational models of replay fall short of generating such layout-conforming replay, restricting their usage to simple environments, like linear tracks or open fields. In this paper, we propose a computational model that generates layout-conforming replay and explains how such replay drives the learning of flexible navigation in a maze. First, we propose a Hebbian-like rule to learn the inter-PC synaptic strength during exploration. Then we use a continuous attractor network (CAN) with feedback inhibition to model the interaction among place cells and hippocampal interneurons. The activity bump of place cells drifts along paths in the maze, which models layout-conforming replay. During replay in sleep, the synaptic strengths from place cells to striatal medium spiny neurons (MSN) are learned by a novel dopamine-modulated three-factor rule to store place-reward associations. During goal-directed navigation, the CAN periodically generates replay trajectories from the animal's location for path planning, and the trajectory leading to a maximal MSN activity is followed by the animal. We have implemented our model into a high-fidelity virtual rat in the MuJoCo physics simulator. Extensive experiments have demonstrated that its superior flexibility during navigation in a maze is due to a continuous re-learning of inter-PC and PC-MSN synaptic strength.
Collapse
Affiliation(s)
- Yuanxiang Gao
- School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu, China,CAS Key Laboratory of Theoretical Physics, Institute of Theoretical Physics, Chinese Academy of Sciences, Beijing, China,*Correspondence: Yuanxiang Gao ✉
| |
Collapse
|
33
|
Aery Jones EA, Giocomo LM. Neural ensembles in navigation: From single cells to population codes. Curr Opin Neurobiol 2023; 78:102665. [PMID: 36542882 PMCID: PMC9845194 DOI: 10.1016/j.conb.2022.102665] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/23/2022] [Revised: 10/27/2022] [Accepted: 11/21/2022] [Indexed: 12/23/2022]
Abstract
The brain can represent behaviorally relevant information through the firing of individual neurons as well as the coordinated firing of ensembles of neurons. Neurons in the hippocampus and associated cortical regions participate in a variety of types of ensembles to support navigation. These ensemble types include single cell codes, population codes, time-compressed sequences, behavioral sequences, and engrams. We present the physiological basis and behavioral relevance of ensemble firing. We discuss how these traditional definitions of ensembles can constrain or expand potential analyses due to the underlying assumptions and abstractions made. We highlight how coding can change at the ensemble level while underlying single cell codes remain intact. Finally, we present how ensemble definitions could be broadened to better understand the full complexity of the brain.
Collapse
Affiliation(s)
- Emily A Aery Jones
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA, 94305, USA.
| | - Lisa M Giocomo
- Department of Neurobiology, Stanford University School of Medicine, Stanford, CA, 94305, USA.
| |
Collapse
|
34
|
Doner S, Zheng J, McAvan AS, Starrett MJ, Campbell H, Sanders D, Ekstrom A. Evidence for flexible navigation strategies during spatial learning involving path choices. SPATIAL COGNITION AND COMPUTATION 2022. [DOI: 10.1080/13875868.2022.2158090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/23/2022]
Affiliation(s)
- Stephanie Doner
- Department of Psychology, University of Arizona, 1503 E. University Blvd, 85719, Tucson, AZ, USA
| | - Jingyi Zheng
- Department of Mathematics and Statistics, Auburn University, Auburn, AL, USA
| | - Andrew S. McAvan
- Department of Psychology, University of Arizona, 1503 E. University Blvd, 85719, Tucson, AZ, USA
| | - Michael J. Starrett
- Department of Psychology, University of Arizona, 1503 E. University Blvd, 85719, Tucson, AZ, USA
| | - Hannah Campbell
- Department of Psychology, University of Arizona, 1503 E. University Blvd, 85719, Tucson, AZ, USA
| | - Delaney Sanders
- Department of Psychology, University of Arizona, 1503 E. University Blvd, 85719, Tucson, AZ, USA
| | - Arne Ekstrom
- Department of Psychology, University of Arizona, 1503 E. University Blvd, 85719, Tucson, AZ, USA
- Evelyn McKnight Brain Institute, University of Arizona, 1503 E. University Blvd, 85719, Tucson, AZ, USA
| |
Collapse
|
35
|
De Corte BJ, Akdoğan B, Balsam PD. Temporal scaling and computing time in neural circuits: Should we stop watching the clock and look for its gears? Front Behav Neurosci 2022; 16:1022713. [PMID: 36570701 PMCID: PMC9773401 DOI: 10.3389/fnbeh.2022.1022713] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2022] [Accepted: 10/31/2022] [Indexed: 12/13/2022] Open
Abstract
Timing underlies a variety of functions, from walking to perceiving causality. Neural timing models typically fall into one of two categories-"ramping" and "population-clock" theories. According to ramping models, individual neurons track time by gradually increasing or decreasing their activity as an event approaches. To time different intervals, ramping neurons adjust their slopes, ramping steeply for short intervals and vice versa. In contrast, according to "population-clock" models, multiple neurons track time as a group, and each neuron can fire nonlinearly. As each neuron changes its rate at each point in time, a distinct pattern of activity emerges across the population. To time different intervals, the brain learns the population patterns that coincide with key events. Both model categories have empirical support. However, they often differ in plausibility when applied to certain behavioral effects. Specifically, behavioral data indicate that the timing system has a rich computational capacity, allowing observers to spontaneously compute novel intervals from previously learned ones. In population-clock theories, population patterns map to time arbitrarily, making it difficult to explain how different patterns can be computationally combined. Ramping models are viewed as more plausible, assuming upstream circuits can set the slope of ramping neurons according to a given computation. Critically, recent studies suggest that neurons with nonlinear firing profiles often scale to time different intervals-compressing for shorter intervals and stretching for longer ones. This "temporal scaling" effect has led to a hybrid-theory where, like a population-clock model, population patterns encode time, yet like a ramping neuron adjusting its slope, the speed of each neuron's firing adapts to different intervals. Here, we argue that these "relative" population-clock models are as computationally plausible as ramping theories, viewing population-speed and ramp-slope adjustments as equivalent. Therefore, we view identifying these "speed-control" circuits as a key direction for evaluating how the timing system performs computations. Furthermore, temporal scaling highlights that a key distinction between different neural models is whether they propose an absolute or relative time-representation. However, we note that several behavioral studies suggest the brain processes both scales, cautioning against a dichotomy.
Collapse
Affiliation(s)
- Benjamin J. De Corte
- Department of Psychology, Columbia University, New York, NY, United States
- Division of Developmental Neuroscience, New York State Psychiatric Institute, New York, NY, United States
| | - Başak Akdoğan
- Department of Psychology, Columbia University, New York, NY, United States
- Division of Developmental Neuroscience, New York State Psychiatric Institute, New York, NY, United States
| | - Peter D. Balsam
- Department of Psychology, Columbia University, New York, NY, United States
- Division of Developmental Neuroscience, New York State Psychiatric Institute, New York, NY, United States
- Department of Neuroscience and Behavior, Barnard College, New York, NY, United States
| |
Collapse
|
36
|
Ujfalussy BB, Orbán G. Sampling motion trajectories during hippocampal theta sequences. eLife 2022; 11:e74058. [PMID: 36346218 PMCID: PMC9643003 DOI: 10.7554/elife.74058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Accepted: 09/28/2022] [Indexed: 11/06/2022] Open
Abstract
Efficient planning in complex environments requires that uncertainty associated with current inferences and possible consequences of forthcoming actions is represented. Representation of uncertainty has been established in sensory systems during simple perceptual decision making tasks but it remains unclear if complex cognitive computations such as planning and navigation are also supported by probabilistic neural representations. Here, we capitalized on gradually changing uncertainty along planned motion trajectories during hippocampal theta sequences to capture signatures of uncertainty representation in population responses. In contrast with prominent theories, we found no evidence of encoding parameters of probability distributions in the momentary population activity recorded in an open-field navigation task in rats. Instead, uncertainty was encoded sequentially by sampling motion trajectories randomly and efficiently in subsequent theta cycles from the distribution of potential trajectories. Our analysis is the first to demonstrate that the hippocampus is well equipped to contribute to optimal planning by representing uncertainty.
Collapse
Affiliation(s)
- Balazs B Ujfalussy
- Laboratory of Biological Computation, Institute of Experimental MedicineBudapestHungary
- Laboratory of Neuronal Signalling, Institute of Experimental Medicine, BudapestBudapestHungary
| | - Gergő Orbán
- Computational Systems Neuroscience Lab, Wigner Research Center for Physics, BudapestBudapestHungary
| |
Collapse
|
37
|
Gobbo F, Mitchell-Heggs R, Tse D, Al Omrani M, Spooner PA, Schultz SR, Morris RGM. Neuronal signature of spatial decision-making during navigation by freely moving rats by using calcium imaging. Proc Natl Acad Sci U S A 2022; 119:e2212152119. [PMID: 36279456 PMCID: PMC9636941 DOI: 10.1073/pnas.2212152119] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/16/2022] [Accepted: 09/20/2022] [Indexed: 11/22/2022] Open
Abstract
A challenge in spatial memory is understanding how place cell firing contributes to decision-making in navigation. A spatial recency task was created in which freely moving rats first became familiar with a spatial context over several days and thereafter were required to encode and then selectively recall one of three specific locations within it that was chosen to be rewarded that day. Calcium imaging was used to record from more than 1,000 cells in area CA1 of the hippocampus of five rats during the exploration, sample, and choice phases of the daily task. The key finding was that neural activity in the startbox rose steadily in the short period prior to entry to the arena and that this selective population cell firing was predictive of the daily changing goal on correct trials but not on trials in which the animals made errors. Single-cell and population activity measures converged on the idea that prospective coding of neural activity can be involved in navigational decision-making.
Collapse
Affiliation(s)
- Francesco Gobbo
- Centre for Discovery Brain Sciences, Edinburgh Neuroscience, The University of Edinburgh, Edinburgh, EH8 9JZ, UK
| | - Rufus Mitchell-Heggs
- Centre for Discovery Brain Sciences, Edinburgh Neuroscience, The University of Edinburgh, Edinburgh, EH8 9JZ, UK
- Department of Bioengineering and Centre for Neurotechnology, Imperial College London, London, SW7 2AZ, UK
| | - Dorothy Tse
- Centre for Discovery Brain Sciences, Edinburgh Neuroscience, The University of Edinburgh, Edinburgh, EH8 9JZ, UK
- Department of Psychology, Edge Hill University, Ormskirk, L39 4QP, UK
| | - Meera Al Omrani
- MSc Program in Integrative Neuroscience, University of Edinburgh, Edinburgh, EH8 9JZ, UK
| | - Patrick A. Spooner
- Centre for Discovery Brain Sciences, Edinburgh Neuroscience, The University of Edinburgh, Edinburgh, EH8 9JZ, UK
- Simons Initiative for the Developing Brain, University of Edinburgh, Edinburgh, EH8 9JZ, UK
| | - Simon R. Schultz
- Department of Bioengineering and Centre for Neurotechnology, Imperial College London, London, SW7 2AZ, UK
| | - Richard G. M. Morris
- Centre for Discovery Brain Sciences, Edinburgh Neuroscience, The University of Edinburgh, Edinburgh, EH8 9JZ, UK
- Simons Initiative for the Developing Brain, University of Edinburgh, Edinburgh, EH8 9JZ, UK
| |
Collapse
|
38
|
de Cothi W, Nyberg N, Griesbauer EM, Ghanamé C, Zisch F, Lefort JM, Fletcher L, Newton C, Renaudineau S, Bendor D, Grieves R, Duvelle É, Barry C, Spiers HJ. Predictive maps in rats and humans for spatial navigation. Curr Biol 2022; 32:3676-3689.e5. [PMID: 35863351 PMCID: PMC9616735 DOI: 10.1016/j.cub.2022.06.090] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2022] [Revised: 05/19/2022] [Accepted: 06/29/2022] [Indexed: 11/25/2022]
Abstract
Much of our understanding of navigation comes from the study of individual species, often with specific tasks tailored to those species. Here, we provide a novel experimental and analytic framework integrating across humans, rats, and simulated reinforcement learning (RL) agents to interrogate the dynamics of behavior during spatial navigation. We developed a novel open-field navigation task ("Tartarus maze") requiring dynamic adaptation (shortcuts and detours) to frequently changing obstructions on the path to a hidden goal. Humans and rats were remarkably similar in their trajectories. Both species showed the greatest similarity to RL agents utilizing a "successor representation," which creates a predictive map. Humans also displayed trajectory features similar to model-based RL agents, which implemented an optimal tree-search planning procedure. Our results help refine models seeking to explain mammalian navigation in dynamic environments and highlight the utility of modeling the behavior of different species to uncover the shared mechanisms that support behavior.
Collapse
Affiliation(s)
- William de Cothi
- Department of Cell and Developmental Biology, University College London, London, UK; Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK.
| | - Nils Nyberg
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Eva-Maria Griesbauer
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Carole Ghanamé
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Fiona Zisch
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK; The Bartlett School of Architecture, University College London, London, UK
| | - Julie M Lefort
- Department of Cell and Developmental Biology, University College London, London, UK
| | - Lydia Fletcher
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Coco Newton
- Department of Clinical Neurosciences, University of Cambridge, Cambridge, UK
| | - Sophie Renaudineau
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Daniel Bendor
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK
| | - Roddy Grieves
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Éléonore Duvelle
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Caswell Barry
- Department of Cell and Developmental Biology, University College London, London, UK
| | - Hugo J Spiers
- Institute of Behavioral Neuroscience, Department of Experimental Psychology, Division of Psychology and Language Sciences, University College London, London, UK.
| |
Collapse
|
39
|
Stoianov I, Maisto D, Pezzulo G. The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning. Prog Neurobiol 2022; 217:102329. [PMID: 35870678 DOI: 10.1016/j.pneurobio.2022.102329] [Citation(s) in RCA: 17] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 07/15/2022] [Accepted: 07/19/2022] [Indexed: 11/28/2022]
Abstract
We advance a novel computational theory of the hippocampal formation as a hierarchical generative model that organizes sequential experiences, such as rodent trajectories during spatial navigation, into coherent spatiotemporal contexts. We propose that the hippocampal generative model is endowed with inductive biases to identify individual items of experience (first hierarchical layer), organize them into sequences (second layer) and cluster them into maps (third layer). This theory entails a novel characterization of hippocampal reactivations as generative replay: the offline resampling of fictive sequences from the generative model, which supports the continual learning of multiple sequential experiences. We show that the model learns and efficiently retains multiple spatial navigation trajectories, by organizing them into spatial maps. Furthermore, the model reproduces flexible and prospective aspects of hippocampal dynamics that are challenging to explain within existing frameworks. This theory reconciles multiple roles of the hippocampal formation in map-based navigation, episodic memory and imagination.
Collapse
Affiliation(s)
- Ivilin Stoianov
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Domenico Maisto
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| |
Collapse
|
40
|
McNaughton BL, Saxena R. Route selection with a cognitive map. Neuron 2022; 110:1441-1442. [PMID: 35512636 DOI: 10.1016/j.neuron.2022.04.015] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
Abstract
In this issue of Neuron, Widloski and Foster (2022) show that, in a complex maze with changing barrier configurations, rat hippocampal neurons maintain their location-specific firing but learn to generate activity sequences representing possible routes to rewards, that respect the locations of barriers, and to rapidly adapt to barrier reconfiguration.
Collapse
Affiliation(s)
- Bruce L McNaughton
- Department Neurobiology and Behavior, Center for the Neurobiology of Learning and Memory, University of California, Irvine, CA, USA.
| | - Rajat Saxena
- Department Neurobiology and Behavior, Center for the Neurobiology of Learning and Memory, University of California, Irvine, CA, USA
| |
Collapse
|
41
|
Epsztein J. Mental replays enable flexible navigation. Nature 2022; 605:35-36. [DOI: 10.1038/d41586-022-01035-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|