1
|
Fenton AA. Remapping revisited: how the hippocampus represents different spaces. Nat Rev Neurosci 2024; 25:428-448. [PMID: 38714834 DOI: 10.1038/s41583-024-00817-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/04/2024] [Indexed: 05/25/2024]
Abstract
The representation of distinct spaces by hippocampal place cells has been linked to changes in their place fields (the locations in the environment where the place cells discharge strongly), a phenomenon that has been termed 'remapping'. Remapping has been assumed to be accompanied by the reorganization of subsecond cofiring relationships among the place cells, potentially maximizing hippocampal information coding capacity. However, several observations challenge this standard view. For example, place cells exhibit mixed selectivity, encode non-positional variables, can have multiple place fields and exhibit unreliable discharge in fixed environments. Furthermore, recent evidence suggests that, when measured at subsecond timescales, the moment-to-moment cofiring of a pair of cells in one environment is remarkably similar in another environment, despite remapping. Here, I propose that remapping is a misnomer for the changes in place fields across environments and suggest instead that internally organized manifold representations of hippocampal activity are actively registered to different environments to enable navigation, promote memory and organize knowledge.
Collapse
Affiliation(s)
- André A Fenton
- Center for Neural Science, New York University, New York, NY, USA.
- Neuroscience Institute at the NYU Langone Medical Center, New York, NY, USA.
| |
Collapse
|
2
|
Mocle AJ, Ramsaran AI, Jacob AD, Rashid AJ, Luchetti A, Tran LM, Richards BA, Frankland PW, Josselyn SA. Excitability mediates allocation of pre-configured ensembles to a hippocampal engram supporting contextual conditioned threat in mice. Neuron 2024; 112:1487-1497.e6. [PMID: 38447576 PMCID: PMC11065628 DOI: 10.1016/j.neuron.2024.02.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2022] [Revised: 01/08/2024] [Accepted: 02/05/2024] [Indexed: 03/08/2024]
Abstract
Little is understood about how engrams, sparse groups of neurons that store memories, are formed endogenously. Here, we combined calcium imaging, activity tagging, and optogenetics to examine the role of neuronal excitability and pre-existing functional connectivity on the allocation of mouse cornu ammonis area 1 (CA1) hippocampal neurons to an engram ensemble supporting a contextual threat memory. Engram neurons (high activity during recall or TRAP2-tagged during training) were more active than non-engram neurons 3 h (but not 24 h to 5 days) before training. Consistent with this, optogenetically inhibiting scFLARE2-tagged neurons active in homecage 3 h, but not 24 h, before conditioning disrupted memory retrieval, indicating that neurons with higher pre-training excitability were allocated to the engram. We also observed stable pre-configured functionally connected sub-ensembles of neurons whose activity cycled over days. Sub-ensembles that were more active before training were allocated to the engram, and their functional connectivity increased at training. Therefore, both neuronal excitability and pre-configured functional connectivity mediate allocation to an engram ensemble.
Collapse
Affiliation(s)
- Andrew J Mocle
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Ave., Toronto, ON M5G 1X8, Canada; Department of Physiology, University of Toronto, Toronto, ON M5G 1X8, Canada
| | - Adam I Ramsaran
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Ave., Toronto, ON M5G 1X8, Canada; Department of Psychology, University of Toronto, Toronto, ON M5G 1X8, Canada
| | - Alexander D Jacob
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Ave., Toronto, ON M5G 1X8, Canada; Department of Psychology, University of Toronto, Toronto, ON M5G 1X8, Canada
| | - Asim J Rashid
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Ave., Toronto, ON M5G 1X8, Canada
| | - Alessandro Luchetti
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Ave., Toronto, ON M5G 1X8, Canada
| | - Lina M Tran
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Ave., Toronto, ON M5G 1X8, Canada; Department of Physiology, University of Toronto, Toronto, ON M5G 1X8, Canada; Vector Institute, Toronto, ON M5G 1M1, Canada
| | | | - Paul W Frankland
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Ave., Toronto, ON M5G 1X8, Canada; Department of Physiology, University of Toronto, Toronto, ON M5G 1X8, Canada; Department of Psychology, University of Toronto, Toronto, ON M5G 1X8, Canada; Child & Brain Development Program, Canadian Institute for Advanced Research (CIFAR), Toronto, ON M5G 1M1, Canada
| | - Sheena A Josselyn
- Program in Neurosciences & Mental Health, Hospital for Sick Children, 555 University Ave., Toronto, ON M5G 1X8, Canada; Department of Physiology, University of Toronto, Toronto, ON M5G 1X8, Canada; Department of Psychology, University of Toronto, Toronto, ON M5G 1X8, Canada.
| |
Collapse
|
3
|
Attaallah B, Petitet P, Zambellas R, Toniolo S, Maio MR, Ganse-Dumrath A, Irani SR, Manohar SG, Husain M. The role of the human hippocampus in decision-making under uncertainty. Nat Hum Behav 2024:10.1038/s41562-024-01855-2. [PMID: 38684870 DOI: 10.1038/s41562-024-01855-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Accepted: 02/29/2024] [Indexed: 05/02/2024]
Abstract
The role of the hippocampus in decision-making is beginning to be more understood. Because of its prospective and inferential functions, we hypothesized that it might be required specifically when decisions involve the evaluation of uncertain values. A group of individuals with autoimmune limbic encephalitis-a condition known to focally affect the hippocampus-were tested on how they evaluate reward against uncertainty compared to reward against another key attribute: physical effort. Across four experiments requiring participants to make trade-offs between reward, uncertainty and effort, patients with acute limbic encephalitis demonstrated blunted sensitivity to reward and effort whenever uncertainty was considered, despite demonstrating intact uncertainty sensitivity. By contrast, the valuation of these two attributes (reward and effort) was intact on uncertainty-free tasks. Reduced sensitivity to changes in reward under uncertainty correlated with the severity of hippocampal damage. Together, these findings provide evidence for a context-sensitive role of the hippocampus in value-based decision-making, apparent specifically under conditions of uncertainty.
Collapse
Affiliation(s)
- Bahaaeddin Attaallah
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK.
| | - Pierre Petitet
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Rhea Zambellas
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Sofia Toniolo
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Maria Raquel Maio
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Akke Ganse-Dumrath
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Sarosh R Irani
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
| | - Sanjay G Manohar
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Masud Husain
- Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, UK
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| |
Collapse
|
4
|
Zhu M, Yasseri T, Kertész J. Individual differences in knowledge network navigation. Sci Rep 2024; 14:8331. [PMID: 38594309 DOI: 10.1038/s41598-024-58305-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/04/2023] [Accepted: 03/27/2024] [Indexed: 04/11/2024] Open
Abstract
With the rapid accumulation of online information, efficient web navigation has grown vital yet challenging. To create an easily navigable cyberspace catering to diverse demographics, understanding how people navigate differently is paramount. While previous research has unveiled individual differences in spatial navigation, such differences in knowledge space navigation remain sparse. To bridge this gap, we conducted an online experiment where participants played a navigation game on Wikipedia and completed personal information questionnaires. Our analysis shows that age negatively affects knowledge space navigation performance, while multilingualism enhances it. Under time pressure, participants' performance improves across trials and males outperform females, an effect not observed in games without time pressure. In our experiment, successful route-finding is usually not related to abilities of innovative exploration of routes. Our results underline the importance of age, multilingualism and time constraint in the knowledge space navigation.
Collapse
Affiliation(s)
- Manran Zhu
- Department of Network and Data Science, Central European University, 1100, Vienna, Austria.
- Center for Collective Learning, CIAS, Corvinus University of Budapest, Budapest, 1093, Hungary.
| | - Taha Yasseri
- School of Sociology, University College Dublin, Dublin 4, D04 V1W8, Ireland
- Geary Institute for Public Policy, University College Dublin, Dublin 4, D04 V1W8, Ireland
| | - János Kertész
- Department of Network and Data Science, Central European University, 1100, Vienna, Austria
| |
Collapse
|
5
|
Sagiv Y, Akam T, Witten IB, Daw ND. Prioritizing replay when future goals are unknown. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2024:2024.02.29.582822. [PMID: 38496674 PMCID: PMC10942393 DOI: 10.1101/2024.02.29.582822] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/19/2024]
Abstract
Although hippocampal place cells replay nonlocal trajectories, the computational function of these events remains controversial. One hypothesis, formalized in a prominent reinforcement learning account, holds that replay plans routes to current goals. However, recent puzzling data appear to contradict this perspective by showing that replayed destinations lag current goals. These results may support an alternative hypothesis that replay updates route information to build a "cognitive map." Yet no similar theory exists to formalize this view, and it is unclear how such a map is represented or what role replay plays in computing it. We address these gaps by introducing a theory of replay that learns a map of routes to candidate goals, before reward is available or when its location may change. Our work extends the planning account to capture a general map-building function for replay, reconciling it with data, and revealing an unexpected relationship between the seemingly distinct hypotheses.
Collapse
Affiliation(s)
- Yotam Sagiv
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA
| | - Thomas Akam
- Department of Experimental Psychology, Oxford University, Oxford, UK
| | - Ilana B Witten
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA
| | - Nathaniel D Daw
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey, USA
| |
Collapse
|
6
|
Grella SL, Donaldson TN. Contextual memory engrams, and the neuromodulatory influence of the locus coeruleus. Front Mol Neurosci 2024; 17:1342622. [PMID: 38375501 PMCID: PMC10875109 DOI: 10.3389/fnmol.2024.1342622] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2023] [Accepted: 01/19/2024] [Indexed: 02/21/2024] Open
Abstract
Here, we review the basis of contextual memory at a conceptual and cellular level. We begin with an overview of the philosophical foundations of traversing space, followed by theories covering the material bases of contextual representations in the hippocampus (engrams), exploring functional characteristics of the cells and subfields within. Next, we explore various methodological approaches for investigating contextual memory engrams, emphasizing plasticity mechanisms. This leads us to discuss the role of neuromodulatory inputs in governing these dynamic changes. We then outline a recent hypothesis involving noradrenergic and dopaminergic projections from the locus coeruleus (LC) to different subregions of the hippocampus, in sculpting contextual representations, giving a brief description of the neuroanatomical and physiological properties of the LC. Finally, we examine how activity in the LC influences contextual memory processes through synaptic plasticity mechanisms to alter hippocampal engrams. Overall, we find that phasic activation of the LC plays an important role in promoting new learning and altering mnemonic processes at the behavioral and cellular level through the neuromodulatory influence of NE/DA in the hippocampus. These findings may provide insight into mechanisms of hippocampal remapping and memory updating, memory processes that are potentially dysregulated in certain psychiatric and neurodegenerative disorders.
Collapse
Affiliation(s)
- Stephanie L. Grella
- MNEME Lab, Department of Psychology, Program in Neuroscience, Loyola University Chicago, Chicago, IL, United States
| | - Tia N. Donaldson
- Systems Neuroscience and Behavior Lab, Department of Psychology, The University of New Mexico, Albuquerque, NM, United States
| |
Collapse
|
7
|
Dragoi G. The generative grammar of the brain: a critique of internally generated representations. Nat Rev Neurosci 2024; 25:60-75. [PMID: 38036709 DOI: 10.1038/s41583-023-00763-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/18/2023] [Indexed: 12/02/2023]
Abstract
The past decade of progress in neurobiology has uncovered important organizational principles for network preconfiguration and neuronal selection that suggest a generative grammar exists in the brain. In this Perspective, I discuss the competence of the hippocampal neural network to generically express temporally compressed sequences of neuronal firing that represent novel experiences, which is envisioned as a form of generative neural syntax supporting a neurobiological perspective on brain function. I compare this neural competence with the hippocampal network performance that represents specific experiences with higher fidelity after new learning during replay, which is envisioned as a form of neural semantic that supports a complementary neuropsychological perspective. I also demonstrate how the syntax of network competence emerges a priori during early postnatal life and is followed by the later development of network performance that enables rapid encoding and memory consolidation. Thus, I propose that this generative grammar of the brain is essential for internally generated representations, which are crucial for the cognitive processes underlying learning and memory, prospection, and inference, which ultimately underlie our reason and representation of the world.
Collapse
Affiliation(s)
- George Dragoi
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA.
- Department of Neuroscience, Yale University School of Medicine, New Haven, CT, USA.
- Wu Tsai Institute, Yale University, New Haven, CT, USA.
| |
Collapse
|
8
|
Nicholas J, Daw ND, Shohamy D. Proactive and reactive construction of memory-based preferences. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.12.10.570977. [PMID: 38106137 PMCID: PMC10723393 DOI: 10.1101/2023.12.10.570977] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 12/19/2023]
Abstract
We are often faced with decisions we have never encountered before, requiring us to infer possible outcomes before making a choice. Computational theories suggest that one way to make these types of decisions is by accessing and linking related experiences stored in memory. Past work has shown that such memory-based preference construction can occur at a number of different timepoints relative to the moment a decision is made. Some studies have found that memories are integrated at the time a decision is faced (reactively) while others found that memory integration happens earlier, when memories are encoded (proactively). Here we offer a resolution to this inconsistency. We demonstrate behavioral and neural evidence for both strategies and for how they tradeoff rationally depending on the associative structure of memory. Using fMRI to decode patterns of brain responses unique to categories of images in memory, we found that proactive memory access is more common and allows more efficient inference. However, participants also use reactive access when choice options are linked to more numerous memory associations. Together, these results indicate that the brain judiciously conducts proactive inference by accessing memories ahead of time in conditions when this strategy is most favorable.
Collapse
Affiliation(s)
- Jonathan Nicholas
- Department of Psychology, Columbia University, New York, NY, USA
- Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, New York, NY, USA
- Department of Psychology, New York University, New York, NY, USA
| | - Nathaniel D Daw
- Department of Psychology, Princeton University, Princeton, NJ, USA
- Princeton Neuroscience Institute, Princeton University, Princeton, NJ, USA
| | - Daphna Shohamy
- Department of Psychology, Columbia University, New York, NY, USA
- Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia University, New York, NY, USA
- The Kavli Institute for Brain Science, Columbia University, New York, NY, USA
| |
Collapse
|
9
|
Schwartenbeck P, Baram A, Liu Y, Mark S, Muller T, Dolan R, Botvinick M, Kurth-Nelson Z, Behrens T. Generative replay underlies compositional inference in the hippocampal-prefrontal circuit. Cell 2023; 186:4885-4897.e14. [PMID: 37804832 PMCID: PMC10914680 DOI: 10.1016/j.cell.2023.09.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 01/23/2023] [Accepted: 09/06/2023] [Indexed: 10/09/2023]
Abstract
Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.
Collapse
Affiliation(s)
- Philipp Schwartenbeck
- University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Baden-Württemberg, Germany; Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK.
| | - Alon Baram
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Shirley Mark
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK
| | - Timothy Muller
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Institute of Neurology, University College London, London WC1N 3BG, UK
| | - Raymond Dolan
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Department of Psychiatry, Universitätsmedizin Berlin (Campus Charité Mitte), Berlin, Germany
| | - Matthew Botvinick
- Google DeepMind, London, UK; Gatsby Computational Neuroscience Unit, University College London, London, UK
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Google DeepMind, London, UK
| | - Timothy Behrens
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Sainsbury Wellcome Centre for Neural Circuits and Behaviour, UCL, London W1T 4JG, UK
| |
Collapse
|
10
|
Liu C, Todorova R, Tang W, Oliva A, Fernandez-Ruiz A. Associative and predictive hippocampal codes support memory-guided behaviors. Science 2023; 382:eadi8237. [PMID: 37856604 PMCID: PMC10894649 DOI: 10.1126/science.adi8237] [Citation(s) in RCA: 6] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/19/2023] [Accepted: 08/21/2023] [Indexed: 10/21/2023]
Abstract
Episodic memory involves learning and recalling associations between items and their spatiotemporal context. Those memories can be further used to generate internal models of the world that enable predictions to be made. The mechanisms that support these associative and predictive aspects of memory are not yet understood. In this study, we used an optogenetic manipulation to perturb the sequential structure, but not global network dynamics, of place cells as rats traversed specific spatial trajectories. This perturbation abolished replay of those trajectories and the development of predictive representations, leading to impaired learning of new optimal trajectories during memory-guided navigation. However, place cell assembly reactivation and reward-context associative learning were unaffected. Our results show a mechanistic dissociation between two complementary hippocampal codes: an associative code (through coactivity) and a predictive code (through sequences).
Collapse
Affiliation(s)
| | | | - Wenbo Tang
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, USA
| | - Azahara Oliva
- Department of Neurobiology and Behavior, Cornell University, Ithaca, NY, USA
| | | |
Collapse
|
11
|
Yang M, Singh A, McDougle M, Décarie-Spain L, Kanoski S, de Lartigue G. Separate orexigenic hippocampal ensembles shape dietary choice by enhancing contextual memory and motivation. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.09.561580. [PMID: 37873148 PMCID: PMC10592764 DOI: 10.1101/2023.10.09.561580] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
The hippocampus (HPC), traditionally known for its role in learning and memory, has emerged as a controller of food intake. While prior studies primarily associated the HPC with food intake inhibition, recent research suggests a critical role in appetitive processes. We hypothesized that orexigenic HPC neurons differentially respond to fats and/or sugars, potent natural reinforcers that contribute to obesity development. Results uncover previously-unrecognized, spatially-distinct neuronal ensembles within the dorsal HPC (dHPC) that are responsive to separate nutrient signals originating from the gut. Using activity-dependent genetic capture of nutrient-responsive HPC neurons, we demonstrate a causal role of both populations in promoting nutrient-specific preference through different mechanisms. Sugar-responsive neurons encode an appetitive spatial memory engram for meal location, whereas fat-responsive neurons selectively enhance the preference and motivation for fat intake. Collectively, these findings uncover a neural basis for the exquisite specificity in processing macronutrient signals from a meal that shape dietary choices.
Collapse
|
12
|
Mehrotra D, Dubé L. Accounting for multiscale processing in adaptive real-world decision-making via the hippocampus. Front Neurosci 2023; 17:1200842. [PMID: 37732307 PMCID: PMC10508350 DOI: 10.3389/fnins.2023.1200842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 08/25/2023] [Indexed: 09/22/2023] Open
Abstract
For adaptive real-time behavior in real-world contexts, the brain needs to allow past information over multiple timescales to influence current processing for making choices that create the best outcome as a person goes about making choices in their everyday life. The neuroeconomics literature on value-based decision-making has formalized such choice through reinforcement learning models for two extreme strategies. These strategies are model-free (MF), which is an automatic, stimulus-response type of action, and model-based (MB), which bases choice on cognitive representations of the world and causal inference on environment-behavior structure. The emphasis of examining the neural substrates of value-based decision making has been on the striatum and prefrontal regions, especially with regards to the "here and now" decision-making. Yet, such a dichotomy does not embrace all the dynamic complexity involved. In addition, despite robust research on the role of the hippocampus in memory and spatial learning, its contribution to value-based decision making is just starting to be explored. This paper aims to better appreciate the role of the hippocampus in decision-making and advance the successor representation (SR) as a candidate mechanism for encoding state representations in the hippocampus, separate from reward representations. To this end, we review research that relates hippocampal sequences to SR models showing that the implementation of such sequences in reinforcement learning agents improves their performance. This also enables the agents to perform multiscale temporal processing in a biologically plausible manner. Altogether, we articulate a framework to advance current striatal and prefrontal-focused decision making to better account for multiscale mechanisms underlying various real-world time-related concepts such as the self that cumulates over a person's life course.
Collapse
Affiliation(s)
- Dhruv Mehrotra
- Integrated Program in Neuroscience, McGill University, Montréal, QC, Canada
- Montréal Neurological Institute, McGill University, Montréal, QC, Canada
| | - Laurette Dubé
- Desautels Faculty of Management, McGill University, Montréal, QC, Canada
- McGill Center for the Convergence of Health and Economics, McGill University, Montréal, QC, Canada
| |
Collapse
|
13
|
Etter G, Carmichael JE, Williams S. Linking temporal coordination of hippocampal activity to memory function. Front Cell Neurosci 2023; 17:1233849. [PMID: 37720546 PMCID: PMC10501408 DOI: 10.3389/fncel.2023.1233849] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2023] [Accepted: 08/01/2023] [Indexed: 09/19/2023] Open
Abstract
Oscillations in neural activity are widespread throughout the brain and can be observed at the population level through the local field potential. These rhythmic patterns are associated with cycles of excitability and are thought to coordinate networks of neurons, in turn facilitating effective communication both within local circuits and across brain regions. In the hippocampus, theta rhythms (4-12 Hz) could contribute to several key physiological mechanisms including long-range synchrony, plasticity, and at the behavioral scale, support memory encoding and retrieval. While neurons in the hippocampus appear to be temporally coordinated by theta oscillations, they also tend to fire in sequences that are developmentally preconfigured. Although loss of theta rhythmicity impairs memory, these sequences of spatiotemporal representations persist in conditions of altered hippocampal oscillations. The focus of this review is to disentangle the relative contribution of hippocampal oscillations from single-neuron activity in learning and memory. We first review cellular, anatomical, and physiological mechanisms underlying the generation and maintenance of hippocampal rhythms and how they contribute to memory function. We propose candidate hypotheses for how septohippocampal oscillations could support memory function while not contributing directly to hippocampal sequences. In particular, we explore how theta rhythms could coordinate the integration of upstream signals in the hippocampus to form future decisions, the relevance of such integration to downstream regions, as well as setting the stage for behavioral timescale synaptic plasticity. Finally, we leverage stimulation-based treatment in Alzheimer's disease conditions as an opportunity to assess the sufficiency of hippocampal oscillations for memory function.
Collapse
Affiliation(s)
| | | | - Sylvain Williams
- Department of Psychiatry, Douglas Mental Health Research Institute, McGill University, Montreal, QC, Canada
| |
Collapse
|
14
|
Yagi S, Igata H, Ikegaya Y, Sasaki T. Awake hippocampal synchronous events are incorporated into offline neuronal reactivation. Cell Rep 2023; 42:112871. [PMID: 37494183 DOI: 10.1016/j.celrep.2023.112871] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2023] [Revised: 05/30/2023] [Accepted: 07/11/2023] [Indexed: 07/28/2023] Open
Abstract
Learning novel experiences reorganizes hippocampal neuronal circuits, represented as coordinated reactivation patterns in post-experience offline states for memory consolidation. This study examines how awake synchronous events during a novel run are related to post-run reactivation patterns. The disruption of awake sharp-wave ripples inhibited experience-induced increases in the contributions of neurons to post-experience synchronous events. Hippocampal place cells that participate more in awake synchronous events are more strongly reactivated during post-experience synchronous events. Awake synchronous neuronal patterns, in cooperation with place-selective firing patterns, determine cell ensembles that undergo pronounced increases and decreases in their correlated spikes. Taken together, awake synchronous events are fundamental for identifying hippocampal neuronal ensembles to be incorporated into synchronous reactivation during subsequent offline states, thereby facilitating memory consolidation.
Collapse
Affiliation(s)
- Saichiro Yagi
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, Tokyo 113-0033, Japan
| | - Hideyoshi Igata
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, Tokyo 113-0033, Japan
| | - Yuji Ikegaya
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, Tokyo 113-0033, Japan; Institute for AI and Beyond, The University of Tokyo, Tokyo 113-0033, Japan; Center for Information and Neural Networks, National Institute of Information and Communications Technology, Suita City, Osaka 565-0871, Japan
| | - Takuya Sasaki
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, Tokyo 113-0033, Japan; Department of Pharmacology, Graduate School of Pharmaceutical Sciences, Tohoku University, 6-3 Aramaki-Aoba, Aoba-Ku, Sendai 980-8578, Japan.
| |
Collapse
|
15
|
Milstein AD, Tran S, Ng G, Soltesz I. Offline memory replay in recurrent neuronal networks emerges from constraints on online dynamics. J Physiol 2023; 601:3241-3264. [PMID: 35907087 PMCID: PMC9885000 DOI: 10.1113/jp283216] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Accepted: 07/22/2022] [Indexed: 02/01/2023] Open
Abstract
During spatial exploration, neural circuits in the hippocampus store memories of sequences of sensory events encountered in the environment. When sensory information is absent during 'offline' resting periods, brief neuronal population bursts can 'replay' sequences of activity that resemble bouts of sensory experience. These sequences can occur in either forward or reverse order, and can even include spatial trajectories that have not been experienced, but are consistent with the topology of the environment. The neural circuit mechanisms underlying this variable and flexible sequence generation are unknown. Here we demonstrate in a recurrent spiking network model of hippocampal area CA3 that experimental constraints on network dynamics such as population sparsity, stimulus selectivity, rhythmicity and spike rate adaptation, as well as associative synaptic connectivity, enable additional emergent properties, including variable offline memory replay. In an online stimulus-driven state, we observed the emergence of neuronal sequences that swept from representations of past to future stimuli on the timescale of the theta rhythm. In an offline state driven only by noise, the network generated both forward and reverse neuronal sequences, and recapitulated the experimental observation that offline memory replay events tend to include salient locations like the site of a reward. These results demonstrate that biological constraints on the dynamics of recurrent neural circuits are sufficient to enable memories of sensory events stored in the strengths of synaptic connections to be flexibly read out during rest and sleep, which is thought to be important for memory consolidation and planning of future behaviour. KEY POINTS: A recurrent spiking network model of hippocampal area CA3 was optimized to recapitulate experimentally observed network dynamics during simulated spatial exploration. During simulated offline rest, the network exhibited the emergent property of generating flexible forward, reverse and mixed direction memory replay events. Network perturbations and analysis of model diversity and degeneracy identified associative synaptic connectivity and key features of network dynamics as important for offline sequence generation. Network simulations demonstrate that population over-representation of salient positions like the site of reward results in biased memory replay.
Collapse
Affiliation(s)
- Aaron D. Milstein
- Department of Neurosurgery, Stanford University School of Medicine, Stanford CA
- Department of Neuroscience and Cell Biology, Robert Wood Johnson Medical School and Center for Advanced Biotechnology and Medicine, Rutgers University, Piscataway, NJ
| | - Sarah Tran
- Department of Neurosurgery, Stanford University School of Medicine, Stanford CA
| | - Grace Ng
- Department of Neurosurgery, Stanford University School of Medicine, Stanford CA
| | - Ivan Soltesz
- Department of Neurosurgery, Stanford University School of Medicine, Stanford CA
| |
Collapse
|
16
|
Montagrin A, Croote DE, Preti MG, Lerman L, Baxter MG, Schiller D. Hippocampal timestamp for goals. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.07.27.550892. [PMID: 37546946 PMCID: PMC10402162 DOI: 10.1101/2023.07.27.550892] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/08/2023]
Abstract
Our brain must manage multiple goals that differ in their temporal proximity. Some goals require immediate attention, while others have already been accomplished, or will be relevant later in time. Here, we examined how the hippocampus represents the temporal distance to different goals using a novel space-themed paradigm during 7T functional MRI (n=31). The hippocampus has an established role in mental time travel and a system in place to stratify information along its longitudinal axis on the basis of representational granularity. Previous work has documented a functional transformation from fine-grained, detail rich representations in the posterior hippocampus to coarse, gist-like representations in the anterior hippocampus. We tested whether the hippocampus uses this long axis system to dissociate goals based upon their temporal distance from the present. We hypothesized that the hippocampus would distinguish goals relevant for ones' current needs from those that are removed in time along the long axis, with temporally removed past and future goals eliciting increasingly anterior activation. We sent participants on a mission to Mars where they had to track goals that differed in when they needed to be accomplished. We observed a long-axis dissociation, where temporally removed past and future goals activated the left anterior hippocampus and current goals activated the left posterior hippocampus. Altogether, this study demonstrates that the timestamp attached to a goal is a key driver in where the goal is represented in the hippocampus. This work extends the scope of the hippocampus' long axis system to the goal-mapping domain.
Collapse
|
17
|
Ambrogioni L, Ólafsdóttir HF. Rethinking the hippocampal cognitive map as a meta-learning computational module. Trends Cogn Sci 2023:S1364-6613(23)00128-6. [PMID: 37357064 DOI: 10.1016/j.tics.2023.05.011] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 04/26/2023] [Accepted: 05/24/2023] [Indexed: 06/27/2023]
Abstract
A hallmark of biological intelligence is the ability to adaptively draw on past experience to guide behaviour under novel situations. Yet, the neurobiological principles that underlie this form of meta-learning remain relatively unexplored. In this Opinion, we review the existing literature on hippocampal spatial representations and reinforcement learning theory and describe a novel theoretical framework that aims to account for biological meta-learning. We conjecture that so-called hippocampal cognitive maps of familiar environments are part of a larger meta-representation (meta-map) that encodes information states and sources, which support exploration and provides a foundation for learning. We also introduce concrete hypotheses on how these generic states can be encoded using a principle of superposition.
Collapse
Affiliation(s)
- Luca Ambrogioni
- Donders Institute for Brain, Cognition & Behaviour, Radboud Universiteit, Nijmegen, The Netherlands.
| | - H Freyja Ólafsdóttir
- Donders Institute for Brain, Cognition & Behaviour, Radboud Universiteit, Nijmegen, The Netherlands.
| |
Collapse
|
18
|
Parra-Barrero E, Cheng S. Learning to predict future locations with internally generated theta sequences. PLoS Comput Biol 2023; 19:e1011101. [PMID: 37172053 DOI: 10.1371/journal.pcbi.1011101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 05/24/2023] [Accepted: 04/13/2023] [Indexed: 05/14/2023] Open
Abstract
Representing past, present and future locations is key for spatial navigation. Indeed, within each cycle of the theta oscillation, the population of hippocampal place cells appears to represent trajectories starting behind the current position of the animal and sweeping ahead of it. In particular, we reported recently that the position represented by CA1 place cells at a given theta phase corresponds to the location where animals were or will be located at a fixed time interval into the past or future assuming the animal ran at its typical, not the current, speed through that part of the environment. This coding scheme leads to longer theta trajectories, larger place fields and shallower phase precession in areas where animals typically run faster. Here we present a mechanistic computational model that accounts for these experimental observations. The model consists of a continuous attractor network with short-term synaptic facilitation and depression that internally generates theta sequences that advance at a fixed pace. Spatial locations are then mapped onto the active units via modified Hebbian plasticity. As a result, neighboring units become associated with spatial locations further apart where animals run faster, reproducing our earlier experimental results. The model also accounts for the higher density of place fields generally observed where animals slow down, such as around rewards. Furthermore, our modeling results reveal that an artifact of the decoding analysis might be partly responsible for the observation that theta trajectories start behind the animal's current position. Overall, our results shed light on how the hippocampal code might arise from the interplay between behavior, sensory input and predefined network dynamics.
Collapse
Affiliation(s)
- Eloy Parra-Barrero
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany
- International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| | - Sen Cheng
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany
- International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| |
Collapse
|
19
|
Brodt S, Inostroza M, Niethard N, Born J. Sleep-A brain-state serving systems memory consolidation. Neuron 2023; 111:1050-1075. [PMID: 37023710 DOI: 10.1016/j.neuron.2023.03.005] [Citation(s) in RCA: 29] [Impact Index Per Article: 29.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Revised: 02/23/2023] [Accepted: 03/06/2023] [Indexed: 04/08/2023]
Abstract
Although long-term memory consolidation is supported by sleep, it is unclear how it differs from that during wakefulness. Our review, focusing on recent advances in the field, identifies the repeated replay of neuronal firing patterns as a basic mechanism triggering consolidation during sleep and wakefulness. During sleep, memory replay occurs during slow-wave sleep (SWS) in hippocampal assemblies together with ripples, thalamic spindles, neocortical slow oscillations, and noradrenergic activity. Here, hippocampal replay likely favors the transformation of hippocampus-dependent episodic memory into schema-like neocortical memory. REM sleep following SWS might balance local synaptic rescaling accompanying memory transformation with a sleep-dependent homeostatic process of global synaptic renormalization. Sleep-dependent memory transformation is intensified during early development despite the immaturity of the hippocampus. Overall, beyond its greater efficacy, sleep consolidation differs from wake consolidation mainly in that it is supported, rather than impaired, by spontaneous hippocampal replay activity possibly gating memory formation in neocortex.
Collapse
Affiliation(s)
- Svenja Brodt
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany; Max-Planck-Institute for Biological Cybernetics, Tübingen, Germany
| | - Marion Inostroza
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Niels Niethard
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Jan Born
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany; Werner Reichert Center for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.
| |
Collapse
|
20
|
McFadyen J, Liu Y, Dolan RJ. Differential replay of reward and punishment paths predicts approach and avoidance. Nat Neurosci 2023; 26:627-637. [PMID: 37020116 DOI: 10.1038/s41593-023-01287-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 02/16/2023] [Indexed: 04/07/2023]
Abstract
Neural replay is implicated in planning, where states relevant to a task goal are rapidly reactivated in sequence. It remains unclear whether, during planning, replay relates to an actual prospective choice. Here, using magnetoencephalography (MEG), we studied replay in human participants while they planned to either approach or avoid an uncertain environment containing paths leading to reward or punishment. We find evidence for forward sequential replay during planning, with rapid state-to-state transitions from 20 to 90 ms. Replay of rewarding paths was boosted, relative to aversive paths, before a decision to avoid and attenuated before a decision to approach. A trial-by-trial bias toward replaying prospective punishing paths predicted irrational decisions to approach riskier environments, an effect more pronounced in participants with higher trait anxiety. The findings indicate a coupling of replay with planned behavior, where replay prioritizes an online representation of a worst-case scenario for approaching or avoiding.
Collapse
Affiliation(s)
- Jessica McFadyen
- The UCL Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.
- Wellcome Centre for Human Neuroimaging, University College London, London, UK.
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Chinese Institute for Brain Research, Beijing, China
| | - Raymond J Dolan
- The UCL Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| |
Collapse
|
21
|
Bono J, Zannone S, Pedrosa V, Clopath C. Learning predictive cognitive maps with spiking neurons during behavior and replays. eLife 2023; 12:e80671. [PMID: 36927625 PMCID: PMC10019888 DOI: 10.7554/elife.80671] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/30/2022] [Accepted: 01/12/2023] [Indexed: 03/18/2023] Open
Abstract
The hippocampus has been proposed to encode environments using a representation that contains predictive information about likely future states, called the successor representation. However, it is not clear how such a representation could be learned in the hippocampal circuit. Here, we propose a plasticity rule that can learn this predictive map of the environment using a spiking neural network. We connect this biologically plausible plasticity rule to reinforcement learning, mathematically and numerically showing that it implements the TD-lambda algorithm. By spanning these different levels, we show how our framework naturally encompasses behavioral activity and replays, smoothly moving from rate to temporal coding, and allows learning over behavioral timescales with a plasticity rule acting on a timescale of milliseconds. We discuss how biological parameters such as dwelling times at states, neuronal firing rates and neuromodulation relate to the delay discounting parameter of the TD algorithm, and how they influence the learned representation. We also find that, in agreement with psychological studies and contrary to reinforcement learning theory, the discount factor decreases hyperbolically with time. Finally, our framework suggests a role for replays, in both aiding learning in novel environments and finding shortcut trajectories that were not experienced during behavior, in agreement with experimental data.
Collapse
Affiliation(s)
- Jacopo Bono
- Department of Bioengineering, Imperial College LondonLondonUnited Kingdom
| | - Sara Zannone
- Department of Bioengineering, Imperial College LondonLondonUnited Kingdom
| | - Victor Pedrosa
- Department of Bioengineering, Imperial College LondonLondonUnited Kingdom
| | - Claudia Clopath
- Department of Bioengineering, Imperial College LondonLondonUnited Kingdom
| |
Collapse
|
22
|
Kurth-Nelson Z, Behrens T, Wayne G, Miller K, Luettgau L, Dolan R, Liu Y, Schwartenbeck P. Replay and compositional computation. Neuron 2023; 111:454-469. [PMID: 36640765 DOI: 10.1016/j.neuron.2022.12.028] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/11/2022] [Accepted: 12/18/2022] [Indexed: 01/15/2023]
Abstract
Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.
Collapse
Affiliation(s)
- Zeb Kurth-Nelson
- DeepMind, London, UK; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK.
| | - Timothy Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - Kevin Miller
- DeepMind, London, UK; Institute of Ophthalmology, University College London, London, UK
| | - Lennart Luettgau
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Ray Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK; Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Philipp Schwartenbeck
- Max Planck Institute for Biological Cybernetics, Tubingen, Germany; University of Tubingen, Tubingen, Germany
| |
Collapse
|
23
|
Abstract
Research on concepts has focused on categorization. Categorization starts with a stimulus. Equally important are episodes that start with a thought. We engage in thinking to draw out new consequences from stored information, or to work out how to act. Each of the concepts out of which thought is constructed provides access to a large body of stored information. Access is not always just a matter of retrieving a stored belief (semantic memory). Often it depends on running a simulation. Simulation allows conceptual thought to draw on information in special-purpose systems, information stored in special-purpose computational dispositions and special-purpose representational structures. While the utility of simulation, prospection or imagination is widely appreciated, the role of concepts in the process is not well understood. This paper turns to cognitive and computational neuroscience for a model of how simulations enable thinkers to reach novel conclusions. Carried over to conceptual thought, the model suggests that concepts are 'plug & play' devices. The distinctive power of thought-driven simulation derives from the ability of concepts to plug into two kinds of structure at once: the combinatorial structure of a thought at one end and special-purpose structural representations at the other. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.
Collapse
Affiliation(s)
- Nicholas Shea
- Faculty of Philosophy, University of Oxford, Radcliffe Humanities, Woodstock Road, Oxford OX2 6GG, UK,Institute of Philosophy, University of London School of Advanced Study, Senate House, Malet Street, London WC1E 7HU, UK
| |
Collapse
|
24
|
Wimmer GE, Liu Y, McNamee DC, Dolan RJ. Distinct replay signatures for prospective decision-making and memory preservation. Proc Natl Acad Sci U S A 2023; 120:e2205211120. [PMID: 36719914 PMCID: PMC9963918 DOI: 10.1073/pnas.2205211120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 12/05/2022] [Indexed: 02/01/2023] Open
Abstract
Theories of neural replay propose that it supports a range of functions, most prominently planning and memory consolidation. Here, we test the hypothesis that distinct signatures of replay in the same task are related to model-based decision-making ("planning") and memory preservation. We designed a reward learning task wherein participants utilized structure knowledge for model-based evaluation, while at the same time had to maintain knowledge of two independent and randomly alternating task environments. Using magnetoencephalography and multivariate analysis, we first identified temporally compressed sequential reactivation, or replay, both prior to choice and following reward feedback. Before choice, prospective replay strength was enhanced for the current task-relevant environment when a model-based planning strategy was beneficial. Following reward receipt, and consistent with a memory preservation role, replay for the alternative distal task environment was enhanced as a function of decreasing recency of experience with that environment. Critically, these planning and memory preservation relationships were selective to pre-choice and post-feedback periods, respectively. Our results provide support for key theoretical proposals regarding the functional role of replay and demonstrate that the relative strength of planning and memory-related signals are modulated by ongoing computational and task demands.
Collapse
Affiliation(s)
- G. Elliott Wimmer
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Chinese Institute for Brain Research, Beijing100875, China
| | - Daniel C. McNamee
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
- Neuroscience Programme, Champalimaud Research, Lisbon1400-038, Portugal
| | - Raymond J. Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| |
Collapse
|
25
|
Zeng X, Diekmann N, Wiskott L, Cheng S. Modeling the function of episodic memory in spatial learning. Front Psychol 2023; 14:1160648. [PMID: 37138984 PMCID: PMC10149844 DOI: 10.3389/fpsyg.2023.1160648] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/07/2023] [Accepted: 03/31/2023] [Indexed: 05/05/2023] Open
Abstract
Episodic memory has been studied extensively in the past few decades, but so far little is understood about how it drives future behavior. Here we propose that episodic memory can facilitate learning in two fundamentally different modes: retrieval and replay, which is the reinstatement of hippocampal activity patterns during later sleep or awake quiescence. We study their properties by comparing three learning paradigms using computational modeling based on visually-driven reinforcement learning. Firstly, episodic memories are retrieved to learn from single experiences (one-shot learning); secondly, episodic memories are replayed to facilitate learning of statistical regularities (replay learning); and, thirdly, learning occurs online as experiences arise with no access to memories of past experiences (online learning). We found that episodic memory benefits spatial learning in a broad range of conditions, but the performance difference is meaningful only when the task is sufficiently complex and the number of learning trials is limited. Furthermore, the two modes of accessing episodic memory affect spatial learning differently. One-shot learning is typically faster than replay learning, but the latter may reach a better asymptotic performance. In the end, we also investigated the benefits of sequential replay and found that replaying stochastic sequences results in faster learning as compared to random replay when the number of replays is limited. Understanding how episodic memory drives future behavior is an important step toward elucidating the nature of episodic memory.
Collapse
Affiliation(s)
- Xiangshuai Zeng
- Department of Computer Science, Institute for Neural Computation, Ruhr University Bochum, Bochum, Germany
- International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| | - Nicolas Diekmann
- Department of Computer Science, Institute for Neural Computation, Ruhr University Bochum, Bochum, Germany
- International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| | - Laurenz Wiskott
- Department of Computer Science, Institute for Neural Computation, Ruhr University Bochum, Bochum, Germany
- International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| | - Sen Cheng
- Department of Computer Science, Institute for Neural Computation, Ruhr University Bochum, Bochum, Germany
- International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
- *Correspondence: Sen Cheng
| |
Collapse
|
26
|
Gateway identity and spatial remapping in a combined grid and place cell attractor. Neural Netw 2023; 157:226-239. [DOI: 10.1016/j.neunet.2022.10.019] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/11/2022] [Revised: 08/04/2022] [Accepted: 10/18/2022] [Indexed: 11/09/2022]
|
27
|
Fukai T. Computational models of Idling brain activity for memory processing. Neurosci Res 2022; 189:75-82. [PMID: 36592825 DOI: 10.1016/j.neures.2022.12.024] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2022] [Accepted: 12/29/2022] [Indexed: 01/01/2023]
Abstract
Studying the underlying neural mechanisms of cognitive functions of the brain is one of the central questions in modern biology. Moreover, it has significantly impacted the development of novel technologies in artificial intelligence. Spontaneous activity is a unique feature of the brain and is currently lacking in many artificially constructed intelligent machines. Spontaneous activity may represent the brain's idling states, which are internally driven by neuronal networks and possibly participate in offline processing during awake, sleep, and resting states. Evidence is accumulating that the brain's spontaneous activity is not mere noise but part of the mechanisms to process information about previous experiences. A bunch of literature has shown how previous sensory and behavioral experiences influence the subsequent patterns of brain activity with various methods in various animals. It seems, however, that the patterns of neural activity and their computational roles differ significantly from area to area and from function to function. In this article, I review the various forms of the brain's spontaneous activity, especially those observed during memory processing, and some attempts to model the generation mechanisms and computational roles of such activities.
Collapse
Affiliation(s)
- Tomoki Fukai
- Okinawa Institute of Science and Technology, Tancha 1919-1, Onna-son, Okinawa 904-0495, Japan.
| |
Collapse
|
28
|
McNamee DC, Stachenfeld KL, Botvinick MM, Gershman SJ. Compositional Sequence Generation in the Entorhinal-Hippocampal System. ENTROPY (BASEL, SWITZERLAND) 2022; 24:1791. [PMID: 36554196 PMCID: PMC9778317 DOI: 10.3390/e24121791] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/08/2022] [Revised: 11/01/2022] [Accepted: 11/29/2022] [Indexed: 06/17/2023]
Abstract
Neurons in the medial entorhinal cortex exhibit multiple, periodically organized, firing fields which collectively appear to form an internal representation of space. Neuroimaging data suggest that this grid coding is also present in other cortical areas such as the prefrontal cortex, indicating that it may be a general principle of neural functionality in the brain. In a recent analysis through the lens of dynamical systems theory, we showed how grid coding can lead to the generation of a diversity of empirically observed sequential reactivations of hippocampal place cells corresponding to traversals of cognitive maps. Here, we extend this sequence generation model by describing how the synthesis of multiple dynamical systems can support compositional cognitive computations. To empirically validate the model, we simulate two experiments demonstrating compositionality in space or in time during sequence generation. Finally, we describe several neural network architectures supporting various types of compositionality based on grid coding and highlight connections to recent work in machine learning leveraging analogous techniques.
Collapse
Affiliation(s)
- Daniel C. McNamee
- Neuroscience Programme, Champalimaud Research, 1400-038 Lisbon, Portugal
| | | | - Matthew M. Botvinick
- Google DeepMind, London N1C 4DN, UK
- Gatsby Computational Neuroscience Unit, University College London, London W1T 4JG, UK
| | - Samuel J. Gershman
- Department of Psychology and Center for Brain Science, Harvard University, Cambridge, MA 02138, USA
- Center for Brains, Minds and Machines, MIT, Cambridge, MA 02139, USA
| |
Collapse
|
29
|
Wirtshafter HS, Wilson MA. Artificial intelligence insights into hippocampal processing. Front Comput Neurosci 2022; 16:1044659. [DOI: 10.3389/fncom.2022.1044659] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/14/2022] [Accepted: 10/21/2022] [Indexed: 11/09/2022] Open
Abstract
Advances in artificial intelligence, machine learning, and deep neural networks have led to new discoveries in human and animal learning and intelligence. A recent artificial intelligence agent in the DeepMind family, muZero, can complete a variety of tasks with limited information about the world in which it is operating and with high uncertainty about features of current and future space. To perform, muZero uses only three functions that are general yet specific enough to allow learning across a variety of tasks without overgeneralization across different contexts. Similarly, humans and animals are able to learn and improve in complex environments while transferring learning from other contexts and without overgeneralizing. In particular, the mammalian extrahippocampal system (eHPCS) can guide spatial decision making while simultaneously encoding and processing spatial and contextual information. Like muZero, the eHPCS is also able to adjust contextual representations depending on the degree and significance of environmental changes and environmental cues. In this opinion, we will argue that the muZero functions parallel those of the hippocampal system. We will show that the different components of the muZero model provide a framework for thinking about generalizable learning in the eHPCS, and that the evaluation of how transitions in cell representations occur between similar and distinct contexts can be informed by advances in artificial intelligence agents such as muZero. We additionally explain how advances in AI agents will provide frameworks and predictions by which to investigate the expected link between state changes and neuronal firing. Specifically, we will discuss testable predictions about the eHPCS, including the functions of replay and remapping, informed by the mechanisms behind muZero learning. We conclude with additional ways in which agents such as muZero can aid in illuminating prospective questions about neural functioning, as well as how these agents may shed light on potential expected answers.
Collapse
|
30
|
Zhang J, Song C, Dai J, Li L, Yang X, Chen Z. Mechanism of opioid addiction and its intervention therapy: Focusing on the reward circuitry and mu‐opioid receptor. MedComm (Beijing) 2022; 3:e148. [PMID: 35774845 PMCID: PMC9218544 DOI: 10.1002/mco2.148] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/25/2021] [Revised: 05/03/2022] [Accepted: 05/07/2022] [Indexed: 11/09/2022] Open
Affiliation(s)
- Jia‐Jia Zhang
- National Translational Science Center for Molecular Medicine & Department of Cell Biology The Fourth Military Medical University Xi'an China
| | - Chang‐Geng Song
- Department of Neurology Xijing Hospital The Fourth Military Medical University Xi'an China
| | - Ji‐Min Dai
- Department of Hepatobiliary Surgery Xijing Hospital The Fourth Military Medical University Xi'an China
| | - Ling Li
- National Translational Science Center for Molecular Medicine & Department of Cell Biology The Fourth Military Medical University Xi'an China
| | - Xiang‐Min Yang
- National Translational Science Center for Molecular Medicine & Department of Cell Biology The Fourth Military Medical University Xi'an China
| | - Zhi‐Nan Chen
- National Translational Science Center for Molecular Medicine & Department of Cell Biology The Fourth Military Medical University Xi'an China
| |
Collapse
|
31
|
A Brain-Inspired Model of Hippocampal Spatial Cognition Based on a Memory-Replay Mechanism. Brain Sci 2022; 12:brainsci12091176. [PMID: 36138911 PMCID: PMC9496859 DOI: 10.3390/brainsci12091176] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2022] [Revised: 08/13/2022] [Accepted: 08/19/2022] [Indexed: 11/17/2022] Open
Abstract
Since the hippocampus plays an important role in memory and spatial cognition, the study of spatial computation models inspired by the hippocampus has attracted much attention. This study relies mainly on reward signals for learning environments and planning paths. As reward signals in a complex or large-scale environment attenuate sharply, the spatial cognition and path planning performance of such models will decrease clearly as a result. Aiming to solve this problem, we present a brain-inspired mechanism, a Memory-Replay Mechanism, that is inspired by the reactivation function of place cells in the hippocampus. We classify the path memory according to the reward information and find the overlapping place cells in different categories of path memory to segment and reconstruct the memory to form a “virtual path”, replaying the memory by associating the reward information. We conducted a series of navigation experiments in a simple environment called a Morris water maze (MWM) and in a complex environment, and we compared our model with a reinforcement learning model and other brain-inspired models. The experimental results show that under the same conditions, our model has a higher rate of environmental exploration and more stable signal transmission, and the average reward obtained under stable conditions was 14.12% higher than RL with random-experience replay. Our model also shows good performance in complex maze environments where signals are easily attenuated. Moreover, the performance of our model at bifurcations is consistent with neurophysiological studies.
Collapse
|
32
|
Replay, the default mode network and the cascaded memory systems model. Nat Rev Neurosci 2022; 23:628-640. [PMID: 35970912 DOI: 10.1038/s41583-022-00620-6] [Citation(s) in RCA: 26] [Impact Index Per Article: 13.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/07/2022] [Indexed: 12/25/2022]
Abstract
The spontaneous replay of patterns of activity related to past experiences and memories is a striking feature of brain activity, as is the coherent activation of sets of brain areas - particularly those comprising the default mode network (DMN) - during rest. We propose that these two phenomena are strongly intertwined and that their potential functions overlap. In the 'cascaded memory systems model' that we outline here, we hypothesize that the DMN forms the backbone for the propagation of replay, mediating interactions between the hippocampus and the neocortex that enable the consolidation of new memories. The DMN may also independently ignite replay cascades, which support reactivation of older memories or high-level semantic representations. We suggest that transient cortical activations, inducing long-range correlations across the neocortex, are a key mechanism supporting a hierarchy of representations that progresses from simple percepts to semantic representations of causes and, finally, to whole episodes.
Collapse
|
33
|
Abstract
Passive priming of prior knowledge to assimilate ongoing experiences underlies advanced cognitive processing. However, the necessary neural dynamics of memory assimilation remains elusive. Uninstructed brain could also show boosted creativity, particularly after idling states, yet it remains unclear whether the idling brain can spontaneously spark relevant knowledge assimilations. We established a paradigm that links/separates context-dependent memories according to geometrical similarities. Mice exploring one of four contexts 1 d before undergoing contextual fear conditioning in a square context showed a gradual fear transfer to preexposed geometrically relevant contexts the next day, but not after 15 min. Anterior cingulate cortex neurons representing relevant, rather than distinct, memories were significantly coreactivated during postconditioning sleep only, before their selective integration the next day during testing. Disrupting sleep coreactivations prevented assimilation while preserving recent memory consolidation. Thus, assimilating pertinent memories during sleep through coreactivation of their respective engrams represents the neural underpinnings of sleep-triggered implicit cortical learning.
Collapse
|
34
|
Predicting real world spatial disorientation in Alzheimer's disease patients using virtual reality navigation tests. Sci Rep 2022; 12:13397. [PMID: 35927285 PMCID: PMC9352716 DOI: 10.1038/s41598-022-17634-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/02/2022] [Accepted: 07/28/2022] [Indexed: 11/08/2022] Open
Abstract
Spatial navigation impairments in Alzheimer's disease (AD) have been suggested to underlie patients experiencing spatial disorientation. Though many studies have highlighted navigation impairments for AD patients in virtual reality (VR) environments, the extent to which these impairments predict a patient's risk for spatial disorientation in the real world is still poorly understood. The aims of this study were to (a) investigate the spatial navigation abilities of AD patients in VR environments as well as in a real world community setting and (b) explore whether we could predict patients at a high risk for spatial disorientation in the community based on their VR navigation. Sixteen community-dwelling AD patients and 21 age/gender matched controls were assessed on their egocentric and allocentric navigation abilities in VR environments using the Virtual Supermarket Test (VST) and Sea Hero Quest (SHQ) as well as in the community using the Detour Navigation Test (DNT). When compared to controls, AD patients exhibited impairments on the VST, SHQ, and DNT. For patients, only SHQ wayfinding distance and wayfinding duration significantly predicted composite disorientation score on the DNT (β = 0.422, p = 0.034, R2 = 0.299 and β = 0.357, p = 0.046, R2 = 0.27 respectively). However, these same VR measures could not reliably predict which patients were at highest risk of spatial disorientation in the community (p > 0.1). Future studies should focus on developing VR-based tests which can predict AD patients at high risk of getting spatially disorientated in the real world.
Collapse
|
35
|
Stoianov I, Maisto D, Pezzulo G. The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning. Prog Neurobiol 2022; 217:102329. [PMID: 35870678 DOI: 10.1016/j.pneurobio.2022.102329] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 07/15/2022] [Accepted: 07/19/2022] [Indexed: 11/28/2022]
Abstract
We advance a novel computational theory of the hippocampal formation as a hierarchical generative model that organizes sequential experiences, such as rodent trajectories during spatial navigation, into coherent spatiotemporal contexts. We propose that the hippocampal generative model is endowed with inductive biases to identify individual items of experience (first hierarchical layer), organize them into sequences (second layer) and cluster them into maps (third layer). This theory entails a novel characterization of hippocampal reactivations as generative replay: the offline resampling of fictive sequences from the generative model, which supports the continual learning of multiple sequential experiences. We show that the model learns and efficiently retains multiple spatial navigation trajectories, by organizing them into spatial maps. Furthermore, the model reproduces flexible and prospective aspects of hippocampal dynamics that are challenging to explain within existing frameworks. This theory reconciles multiple roles of the hippocampal formation in map-based navigation, episodic memory and imagination.
Collapse
Affiliation(s)
- Ivilin Stoianov
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Domenico Maisto
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| |
Collapse
|
36
|
Doublet T, Nosrati M, Kentros CG. Social Learning of a Spatial Task by Observation Alone. Front Behav Neurosci 2022; 16:902675. [PMID: 35910679 PMCID: PMC9325960 DOI: 10.3389/fnbeh.2022.902675] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/23/2022] [Accepted: 06/10/2022] [Indexed: 11/16/2022] Open
Abstract
Interactions between conspecifics are central to the acquisition of useful memories in the real world. Observational learning, i.e., learning a task by observing the success or failure of others, has been reported in many species, including rodents. However, previous work in rats with NMDA-receptor blockade has shown that even extensive observation of an unexplored space through a clear barrier is not sufficient to generate a stable hippocampal representation of that space. This raises the question of whether rats can learn a spatial task in a purely observed space from watching a conspecific, and if so, does this somehow stabilize their hippocampal representation? To address these questions, we designed an observational spatial task in a two-part environment that is nearly identical to that of the aforementioned electrophysiological study, in which an observer rat watches a demonstrator animal to learn the location of a hidden reward. Our results demonstrate that rats do not need to physically explore an environment to learn a reward location, provided a conspecific demonstrates where it is. We also show that the behavioral memory is not affected by NMDA receptor blockade, suggesting that the spatial representation underlying the behavior has been consolidated by observation alone.
Collapse
|
37
|
The shallow cognitive map hypothesis: A hippocampal framework for thought disorder in schizophrenia. SCHIZOPHRENIA (HEIDELBERG, GERMANY) 2022; 8:34. [PMID: 35853896 PMCID: PMC9261089 DOI: 10.1038/s41537-022-00247-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Accepted: 03/11/2022] [Indexed: 12/31/2022]
Abstract
Memories are not formed in isolation. They are associated and organized into relational knowledge structures that allow coherent thought. Failure to express such coherent thought is a key hallmark of Schizophrenia. Here we explore the hypothesis that thought disorder arises from disorganized Hippocampal cognitive maps. In doing so, we combine insights from two key lines of investigation, one concerning the neural signatures of cognitive mapping, and another that seeks to understand lower-level cellular mechanisms of cognition within a dynamical systems framework. Specifically, we propose that multiple distinct pathological pathways converge on the shallowing of Hippocampal attractors, giving rise to disorganized Hippocampal cognitive maps and driving conceptual disorganization. We discuss the available evidence at the computational, behavioural, network, and cellular levels. We also outline testable predictions from this framework, including how it could unify major chemical and psychological theories of schizophrenia and how it can provide a rationale for understanding the aetiology and treatment of the disease.
Collapse
|
38
|
Abstract
In human neuroscience, studies of cognition are rarely grounded in non-task-evoked, 'spontaneous' neural activity. Indeed, studies of spontaneous activity tend to focus predominantly on intrinsic neural patterns (for example, resting-state networks). Taking a 'representation-rich' approach bridges the gap between cognition and resting-state communities: this approach relies on decoding task-related representations from spontaneous neural activity, allowing quantification of the representational content and rich dynamics of such activity. For example, if we know the neural representation of an episodic memory, we can decode its subsequent replay during rest. We argue that such an approach advances cognitive research beyond a focus on immediate task demand and provides insight into the functional relevance of the intrinsic neural pattern (for example, the default mode network). This in turn enables a greater integration between human and animal neuroscience, facilitating experimental testing of theoretical accounts of intrinsic activity, and opening new avenues of research in psychiatry.
Collapse
|
39
|
Widloski J, Foster DJ. Flexible rerouting of hippocampal replay sequences around changing barriers in the absence of global place field remapping. Neuron 2022; 110:1547-1558.e8. [PMID: 35180390 PMCID: PMC9473153 DOI: 10.1016/j.neuron.2022.02.002] [Citation(s) in RCA: 33] [Impact Index Per Article: 16.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/28/2021] [Revised: 11/30/2021] [Accepted: 02/01/2022] [Indexed: 01/12/2023]
Abstract
Flexibility is a hallmark of memories that depend on the hippocampus. For navigating animals, flexibility is necessitated by environmental changes such as blocked paths and extinguished food sources. To better understand the neural basis of this flexibility, we recorded hippocampal replays in a spatial memory task where barriers as well as goals were moved between sessions to see whether replays could adapt to new spatial and reward contingencies. Strikingly, replays consistently depicted new goal-directed trajectories around each new barrier configuration and largely avoided barrier violations. Barrier-respecting replays were learned rapidly and did not rely on place cell remapping. These data distinguish sharply between place field responses, which were largely stable and remained tied to sensory cues, and replays, which changed flexibly to reflect the learned contingencies in the environment and suggest sequenced activations such as replay to be an important link between the hippocampus and flexible memory.
Collapse
Affiliation(s)
- John Widloski
- Helen Wills Neuroscience Institute and Department of Psychology, University of California, Berkeley, CA 94720, USA
| | - David J Foster
- Helen Wills Neuroscience Institute and Department of Psychology, University of California, Berkeley, CA 94720, USA.
| |
Collapse
|
40
|
Abstract
Recent breakthroughs in artificial intelligence (AI) have enabled machines to plan in tasks previously thought to be uniquely human. Meanwhile, the planning algorithms implemented by the brain itself remain largely unknown. Here, we review neural and behavioral data in sequential decision-making tasks that elucidate the ways in which the brain does-and does not-plan. To systematically review available biological data, we create a taxonomy of planning algorithms by summarizing the relevant design choices for such algorithms in AI. Across species, recording techniques, and task paradigms, we find converging evidence that the brain represents future states consistent with a class of planning algorithms within our taxonomy-focused, depth-limited, and serial. However, we argue that current data are insufficient for addressing more detailed algorithmic questions. We propose a new approach leveraging AI advances to drive experiments that can adjudicate between competing candidate algorithms.
Collapse
|
41
|
Optimism and pessimism in optimised replay. PLoS Comput Biol 2022; 18:e1009634. [PMID: 35020718 PMCID: PMC8809607 DOI: 10.1371/journal.pcbi.1009634] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2021] [Revised: 02/02/2022] [Accepted: 11/12/2021] [Indexed: 11/24/2022] Open
Abstract
The replay of task-relevant trajectories is known to contribute to memory consolidation and improved task performance. A wide variety of experimental data show that the content of replayed sequences is highly specific and can be modulated by reward as well as other prominent task variables. However, the rules governing the choice of sequences to be replayed still remain poorly understood. One recent theoretical suggestion is that the prioritization of replay experiences in decision-making problems is based on their effect on the choice of action. We show that this implies that subjects should replay sub-optimal actions that they dysfunctionally choose rather than optimal ones, when, by being forgetful, they experience large amounts of uncertainty in their internal models of the world. We use this to account for recent experimental data demonstrating exactly pessimal replay, fitting model parameters to the individual subjects’ choices. When animals are asleep or restfully awake, populations of neurons in their brains recapitulate activity associated with extended behaviourally-relevant experiences. This process is called replay, and it has been established for a long time in rodents, and very recently in humans, to be important for good performance in decision-making tasks. The specific experiences which are replayed during those epochs follow highly ordered patterns, but the mechanisms which establish their priority are still not fully understood. One promising theoretical suggestion is that each replay experience is chosen in such a way that the learning that ensues is most helpful for the subsequent performance of the animal. A very recent study reported a surprising result that humans who achieved high performance in a planning task tended to replay actions they found to be sub-optimal, and that this was associated with a useful deprecation of those actions in subsequent performance. In this study, we examine the nature of this pessimized form of replay and show that it is exactly appropriate for forgetful agents. We analyse the role of forgetting for replay choices of our model, and verify our predictions using human subject data.
Collapse
|
42
|
Nyberg N, Duvelle É, Barry C, Spiers HJ. Spatial goal coding in the hippocampal formation. Neuron 2022; 110:394-422. [PMID: 35032426 DOI: 10.1016/j.neuron.2021.12.012] [Citation(s) in RCA: 39] [Impact Index Per Article: 19.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/02/2021] [Revised: 10/18/2021] [Accepted: 12/08/2021] [Indexed: 12/22/2022]
Abstract
The mammalian hippocampal formation contains several distinct populations of neurons involved in representing self-position and orientation. These neurons, which include place, grid, head direction, and boundary-vector cells, are thought to collectively instantiate cognitive maps supporting flexible navigation. However, to flexibly navigate, it is necessary to also maintain internal representations of goal locations, such that goal-directed routes can be planned and executed. Although it has remained unclear how the mammalian brain represents goal locations, multiple neural candidates have recently been uncovered during different phases of navigation. For example, during planning, sequential activation of spatial cells may enable simulation of future routes toward the goal. During travel, modulation of spatial cells by the prospective route, or by distance and direction to the goal, may allow maintenance of route and goal-location information, supporting navigation on an ongoing basis. As the goal is approached, an increased activation of spatial cells may enable the goal location to become distinctly represented within cognitive maps, aiding goal localization. Lastly, after arrival at the goal, sequential activation of spatial cells may represent the just-taken route, enabling route learning and evaluation. Here, we review and synthesize these and other evidence for goal coding in mammalian brains, relate the experimental findings to predictions from computational models, and discuss outstanding questions and future challenges.
Collapse
Affiliation(s)
- Nils Nyberg
- Institute of Behavioural Neuroscience, Department of Experimental Psychology, University College London, London, UK.
| | - Éléonore Duvelle
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Caswell Barry
- Department of Cell and Developmental Biology, University College London, London, UK
| | - Hugo J Spiers
- Institute of Behavioural Neuroscience, Department of Experimental Psychology, University College London, London, UK.
| |
Collapse
|
43
|
Zajner C, Spreng RN, Bzdok D. Loneliness is linked to specific subregional alterations in hippocampus-default network covariation. J Neurophysiol 2021; 126:2138-2157. [PMID: 34817294 PMCID: PMC8715056 DOI: 10.1152/jn.00339.2021] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/22/2022] Open
Abstract
Social interaction complexity makes humans unique. But in times of social deprivation, this strength risks exposure of important vulnerabilities. Human social neuroscience studies have placed a premium on the default network (DN). In contrast, hippocampus (HC) subfields have been intensely studied in rodents and monkeys. To bridge these two literatures, we here quantified how DN subregions systematically covary with specific HC subfields in the context of subjective social isolation (i.e., loneliness). By codecomposition using structural brain scans of ∼40,000 UK Biobank participants, loneliness was specially linked to midline subregions in the uncovered DN patterns. These association cortex patterns coincided with concomitant HC patterns implicating especially CA1 and molecular layer. These patterns also showed a strong affiliation with the fornix white matter tract and the nucleus accumbens. In addition, separable signatures of structural HC-DN covariation had distinct associations with the genetic predisposition for loneliness at the population level. NEW & NOTEWORTHY The hippocampus and default network have been implicated in rich social interaction. Yet, these allocortical and neocortical neural systems have been interrogated in mostly separate literatures. Here, we conjointly investigate the hippocampus and default network at a subregion level, by capitalizing structural brain scans from ∼40,000 participants. We thus reveal unique insights on the nature of the “lonely brain” by estimating the regimes of covariation between the hippocampus and default network at population scale.
Collapse
Affiliation(s)
- Chris Zajner
- McConnell Brain Imaging Centre (BIC), Montreal Neurological Institute (MNI), Faculty of Medicine, McGill University, Montreal, Quebec, Canada
| | - R Nathan Spreng
- McConnell Brain Imaging Centre (BIC), Montreal Neurological Institute (MNI), Faculty of Medicine, McGill University, Montreal, Quebec, Canada.,Department of Neurology and Neurosurgery, McGill University, Montreal, Quebec, Canada.,Departments of Psychiatry and Psychology, McGill University, Montreal, QC, Canada.,Douglas Mental Health University Institute, Verdun, Quebec, Canada
| | - Danilo Bzdok
- McConnell Brain Imaging Centre (BIC), Montreal Neurological Institute (MNI), Faculty of Medicine, McGill University, Montreal, Quebec, Canada.,Department of Biomedical Engineering, Faculty of Medicine, McGill University, Montreal, Quebec, Canada.,Mila-Quebec Artificial Intelligence Institute, Montreal, Quebec, Canada
| |
Collapse
|
44
|
Ross TW, Easton A. The Hippocampal Horizon: Constructing and Segmenting Experience for Episodic Memory. Neurosci Biobehav Rev 2021; 132:181-196. [PMID: 34826509 DOI: 10.1016/j.neubiorev.2021.11.038] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/04/2021] [Revised: 11/19/2021] [Accepted: 11/22/2021] [Indexed: 12/29/2022]
Abstract
How do we recollect specific events that have occurred during continuous ongoing experience? There is converging evidence from non-human animals that spatially modulated cellular activity of the hippocampal formation supports the construction of ongoing events. On the other hand, recent human oriented event cognition models have outlined that our experience is segmented into discrete units, and that such segmentation can operate on shorter or longer timescales. Here, we describe a unification of how these dynamic physiological mechanisms of the hippocampus relate to ongoing externally and internally driven event segmentation, facilitating the demarcation of specific moments during experience. Our cross-species interdisciplinary approach offers a novel perspective in the way we construct and remember specific events, leading to the generation of many new hypotheses for future research.
Collapse
Affiliation(s)
- T W Ross
- Department of Psychology, Durham University, South Road, Durham, DH1 3LE, United Kingdom; Centre for Learning and Memory Processes, Durham University, United Kingdom.
| | - A Easton
- Department of Psychology, Durham University, South Road, Durham, DH1 3LE, United Kingdom; Centre for Learning and Memory Processes, Durham University, United Kingdom
| |
Collapse
|
45
|
Subramanian A, Chitlangia S, Baths V. Reinforcement learning and its connections with neuroscience and psychology. Neural Netw 2021; 145:271-287. [PMID: 34781215 DOI: 10.1016/j.neunet.2021.10.003] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Revised: 09/26/2021] [Accepted: 10/01/2021] [Indexed: 11/19/2022]
Abstract
Reinforcement learning methods have recently been very successful at performing complex sequential tasks like playing Atari games, Go and Poker. These algorithms have outperformed humans in several tasks by learning from scratch, using only scalar rewards obtained through interaction with their environment. While there certainly has been considerable independent innovation to produce such results, many core ideas in reinforcement learning are inspired by phenomena in animal learning, psychology and neuroscience. In this paper, we comprehensively review a large number of findings in both neuroscience and psychology that evidence reinforcement learning as a promising candidate for modeling learning and decision making in the brain. In doing so, we construct a mapping between various classes of modern RL algorithms and specific findings in both neurophysiological and behavioral literature. We then discuss the implications of this observed relationship between RL, neuroscience and psychology and its role in advancing research in both AI and brain science.
Collapse
Affiliation(s)
- Ajay Subramanian
- Department of Psychology, New York University, New York, New York, 10003, USA; Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| | - Sharad Chitlangia
- Amazon; Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| | - Veeky Baths
- Cognitive Neuroscience Lab, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India; Department of Biological Sciences, BITS Pilani K K Birla Goa Campus, NH-17B, Zuarinagar, Goa, 403726, India.
| |
Collapse
|
46
|
Reactivation predicts the consolidation of unbiased long-term cognitive maps. Nat Neurosci 2021; 24:1574-1585. [PMID: 34663956 DOI: 10.1038/s41593-021-00920-7] [Citation(s) in RCA: 23] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/24/2020] [Accepted: 08/06/2021] [Indexed: 11/08/2022]
Abstract
Spatial memories that can last a lifetime are thought to be encoded during 'online' periods of exploration and subsequently consolidated into stable cognitive maps through their 'offline' reactivation. However, the mechanisms and computational principles by which offline reactivation stabilize long-lasting spatial representations remain poorly understood. Here, we employed simultaneous fast calcium imaging and electrophysiology to track hippocampal place cells over 2 weeks of online spatial reward learning behavior and offline resting. We describe that recruitment to persistent network-level offline reactivation of spatial experiences in mice predicts the future representational stability of place cells days in advance of their online reinstatement. Moreover, while representations of reward-adjacent locations are generally more stable across days, offline-reactivation-related stability is, conversely, most prominent for reward-distal locations. Thus, while occurring on the tens of milliseconds timescale, offline reactivation is uniquely associated with the stability of multiday representations that counterbalance the overall reward-adjacency bias, thereby predicting the stabilization of cognitive maps that comprehensively reflect entire underlying spatial contexts. These findings suggest that post-learning offline-related memory consolidation plays a complimentary and computationally distinct role in learning compared to online encoding.
Collapse
|
47
|
Bush D, Ólafsdóttir HF, Barry C, Burgess N. Ripple band phase precession of place cell firing during replay. Curr Biol 2021; 32:64-73.e5. [PMID: 34731677 PMCID: PMC8751637 DOI: 10.1016/j.cub.2021.10.033] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/23/2021] [Revised: 09/06/2021] [Accepted: 10/14/2021] [Indexed: 11/17/2022]
Abstract
Neuronal “replay,” in which place cell firing during rest recapitulates recently experienced trajectories, is thought to mediate the transmission of information from hippocampus to neocortex, but the mechanism for this transmission is unknown. Here, we show that replay uses a phase code to represent spatial trajectories by the phase of firing relative to the 150- to 250-Hz “ripple” oscillations that accompany replay events. This phase code is analogous to the theta phase precession of place cell firing during navigation, in which place cells fire at progressively earlier phases of the 6- to 12-Hz theta oscillation as their place field is traversed, providing information about self-location that is additional to the rate code and a necessary precursor of replay. Thus, during replay, each ripple cycle contains a “forward sweep” of decoded locations along the recapitulated trajectory. Our results indicate a novel encoding of trajectory information during replay and implicates phase coding as a general mechanism by which the hippocampus transmits experienced and replayed sequential information to downstream targets. Place cells fire at successively earlier ripple band phases during replay Ripple band firing phase during replay encodes location within the place field This produces forward sweeps of place cell activity during each ripple cycle
Collapse
Affiliation(s)
- Daniel Bush
- UCL Institute of Cognitive Neuroscience, Queen Square, London, UK; UCL Institute of Neurology, Queen Square, London, UK.
| | - H Freyja Ólafsdóttir
- Donders Institute for Brain Cognition and Behaviour, Radboud University, Nijmegen, the Netherlands
| | - Caswell Barry
- UCL Department of Cell and Developmental Biology, Gower Street, London, UK.
| | - Neil Burgess
- UCL Institute of Cognitive Neuroscience, Queen Square, London, UK; UCL Institute of Neurology, Queen Square, London, UK
| |
Collapse
|
48
|
Hayes TL, Krishnan GP, Bazhenov M, Siegelmann HT, Sejnowski TJ, Kanan C. Replay in Deep Learning: Current Approaches and Missing Biological Elements. Neural Comput 2021; 33:2908-2950. [PMID: 34474476 PMCID: PMC9074752 DOI: 10.1162/neco_a_01433] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2021] [Accepted: 05/28/2021] [Indexed: 11/04/2022]
Abstract
Replay is the reactivation of one or more neural patterns that are similar to the activation patterns experienced during past waking experiences. Replay was first observed in biological neural networks during sleep, and it is now thought to play a critical role in memory formation, retrieval, and consolidation. Replay-like mechanisms have been incorporated in deep artificial neural networks that learn over time to avoid catastrophic forgetting of previous knowledge. Replay algorithms have been successfully used in a wide range of deep learning methods within supervised, unsupervised, and reinforcement learning paradigms. In this letter, we provide the first comprehensive comparison between replay in the mammalian brain and replay in artificial neural networks. We identify multiple aspects of biological replay that are missing in deep learning systems and hypothesize how they could be used to improve artificial neural networks.
Collapse
Affiliation(s)
- Tyler L Hayes
- Rochester Institute of Technology, Rochester, NY 14623, U.S.A.
| | - Giri P Krishnan
- University of California at San Diego, La Jolla, CA 92093, U.S.A.
| | - Maxim Bazhenov
- University of California at San Diego, La Jolla, CA 92093, U.S.A.
| | | | - Terrence J Sejnowski
- University of California at San Diego, La Jolla, CA 92093, U.S.A., and Salk Institute for Biological Studies, La Jolla, CA 92037, U.S.A.
| | - Christopher Kanan
- Rochester Institute of Technology, Rochester, NY 14623, U.S.A.; Paige, New York, NY 10036, U.S.A.; and Cornell Tech, New York, NY 10044, U.S.A.
| |
Collapse
|
49
|
Gillespie AK, Astudillo Maya DA, Denovellis EL, Liu DF, Kastner DB, Coulter ME, Roumis DK, Eden UT, Frank LM. Hippocampal replay reflects specific past experiences rather than a plan for subsequent choice. Neuron 2021; 109:3149-3163.e6. [PMID: 34450026 DOI: 10.1016/j.neuron.2021.07.029] [Citation(s) in RCA: 65] [Impact Index Per Article: 21.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2021] [Revised: 06/21/2021] [Accepted: 07/29/2021] [Indexed: 01/06/2023]
Abstract
Executing memory-guided behavior requires storage of information about experience and later recall of that information to inform choices. Awake hippocampal replay, when hippocampal neural ensembles briefly reactivate a representation related to prior experience, has been proposed to critically contribute to these memory-related processes. However, it remains unclear whether awake replay contributes to memory function by promoting the storage of past experiences, facilitating planning based on evaluation of those experiences, or both. We designed a dynamic spatial task that promotes replay before a memory-based choice and assessed how the content of replay related to past and future behavior. We found that replay content was decoupled from subsequent choice and instead was enriched for representations of previously rewarded locations and places that had not been visited recently, indicating a role in memory storage rather than in directly guiding subsequent behavior.
Collapse
Affiliation(s)
- Anna K Gillespie
- Departments of Physiology and Psychiatry, University of California, San Francisco, San Francisco, CA 94158, USA; Kavli Institute for Fundamental Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA.
| | - Daniela A Astudillo Maya
- Departments of Physiology and Psychiatry, University of California, San Francisco, San Francisco, CA 94158, USA; Kavli Institute for Fundamental Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Eric L Denovellis
- Departments of Physiology and Psychiatry, University of California, San Francisco, San Francisco, CA 94158, USA; Kavli Institute for Fundamental Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA; Howard Hughes Medical Institute, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Daniel F Liu
- Departments of Physiology and Psychiatry, University of California, San Francisco, San Francisco, CA 94158, USA; Kavli Institute for Fundamental Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA
| | - David B Kastner
- Departments of Physiology and Psychiatry, University of California, San Francisco, San Francisco, CA 94158, USA; Kavli Institute for Fundamental Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Michael E Coulter
- Departments of Physiology and Psychiatry, University of California, San Francisco, San Francisco, CA 94158, USA; Kavli Institute for Fundamental Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Demetris K Roumis
- Departments of Physiology and Psychiatry, University of California, San Francisco, San Francisco, CA 94158, USA; Kavli Institute for Fundamental Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA; Howard Hughes Medical Institute, University of California, San Francisco, San Francisco, CA 94158, USA
| | - Uri T Eden
- Department of Mathematics and Statistics, Boston University, Boston, MA 02215, USA
| | - Loren M Frank
- Departments of Physiology and Psychiatry, University of California, San Francisco, San Francisco, CA 94158, USA; Kavli Institute for Fundamental Neuroscience, University of California, San Francisco, San Francisco, CA 94158, USA; Howard Hughes Medical Institute, University of California, San Francisco, San Francisco, CA 94158, USA.
| |
Collapse
|
50
|
Roscow EL, Chua R, Costa RP, Jones MW, Lepora N. Learning offline: memory replay in biological and artificial reinforcement learning. Trends Neurosci 2021; 44:808-821. [PMID: 34481635 DOI: 10.1016/j.tins.2021.07.007] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/12/2021] [Revised: 07/13/2021] [Accepted: 07/21/2021] [Indexed: 10/20/2022]
Abstract
Learning to act in an environment to maximise rewards is among the brain's key functions. This process has often been conceptualised within the framework of reinforcement learning, which has also gained prominence in machine learning and artificial intelligence (AI) as a way to optimise decision making. A common aspect of both biological and machine reinforcement learning is the reactivation of previously experienced episodes, referred to as replay. Replay is important for memory consolidation in biological neural networks and is key to stabilising learning in deep neural networks. Here, we review recent developments concerning the functional roles of replay in the fields of neuroscience and AI. Complementary progress suggests how replay might support learning processes, including generalisation and continual learning, affording opportunities to transfer knowledge across the two fields to advance the understanding of biological and artificial learning and memory.
Collapse
Affiliation(s)
| | | | - Rui Ponte Costa
- Bristol Computational Neuroscience Unit, Intelligent Systems Lab, Department of Computer Science, University of Bristol, Bristol, UK
| | - Matt W Jones
- School of Physiology, Pharmacology and Neuroscience, University of Bristol, Bristol, UK
| | - Nathan Lepora
- Department of Engineering Mathematics and Bristol Robotics Laboratory, University of Bristol, Bristol, UK
| |
Collapse
|