1
|
Fayed MR, Ghandour K, Inokuchi K. Sleep and quiet wakefulness signify an idling brain hub for creative insights. Philos Trans R Soc Lond B Biol Sci 2024; 379:20230226. [PMID: 38853559 DOI: 10.1098/rstb.2023.0226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/30/2023] [Accepted: 04/09/2024] [Indexed: 06/11/2024] Open
Abstract
Long-term potentiation of synaptic strength is a fundamental aspect of learning and memory. Memories are believed to be stored within specific populations of neurons known as engram cells, which are subsequently reactivated during sleep, facilitating the consolidation of stored information. However, sleep and offline reactivations are associated not only with past experiences but also with anticipation of future events. During periods of offline reactivation, which occur during sleep and quiet wakefulness, the brain exhibits a capability to form novel connections. This process links various past experiences, often leading to the emergence of qualitatively new information that was not initially available. Brain activity during sleep and quiet wakefulness is referred to as the 'idling brain'. Idling brain activity is believed to play a pivotal role in abstracting essential information, comprehending underlying rules, generating creative ideas and fostering insightful thoughts. In this review, we will explore the current state of research and future directions in understanding how sleep and idling brain activity are interconnected with various cognitive functions, especially creative insights. These insights have profound implications for our daily lives, impacting our ability to process information, make decisions and navigate complex situations effectively. This article is part of a discussion meeting issue 'Long-term potentiation: 50 years on'.
Collapse
Affiliation(s)
- Mostafa R Fayed
- Research Centre for Idling Brain Science, University of Toyama , Toyama 930-0194, Japan
- Department of Biochemistry, Graduate School of Medicine and Pharmaceutical Sciences, University of Toyama , Toyama 930-0194, Japan
- Department of Pharmacology and Toxicology, Faculty of Pharmacy, Kafrelsheikh University , Kafrelsheikh 33516, Egypt
| | - Khaled Ghandour
- Research Centre for Idling Brain Science, University of Toyama , Toyama 930-0194, Japan
- Department of Biochemistry, Graduate School of Medicine and Pharmaceutical Sciences, University of Toyama , Toyama 930-0194, Japan
- Department of Biochemistry, Faculty of Pharmacy, Cairo University , Cairo 11562, Egypt
| | - Kaoru Inokuchi
- Research Centre for Idling Brain Science, University of Toyama , Toyama 930-0194, Japan
- Department of Biochemistry, Graduate School of Medicine and Pharmaceutical Sciences, University of Toyama , Toyama 930-0194, Japan
| |
Collapse
|
2
|
Jensen KT, Hennequin G, Mattar MG. A recurrent network model of planning explains hippocampal replay and human behavior. Nat Neurosci 2024:10.1038/s41593-024-01675-7. [PMID: 38849521 DOI: 10.1038/s41593-024-01675-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/22/2023] [Accepted: 05/07/2024] [Indexed: 06/09/2024]
Abstract
When faced with a novel situation, people often spend substantial periods of time contemplating possible futures. For such planning to be rational, the benefits to behavior must compensate for the time spent thinking. Here, we capture these features of behavior by developing a neural network model where planning itself is controlled by the prefrontal cortex. This model consists of a meta-reinforcement learning agent augmented with the ability to plan by sampling imagined action sequences from its own policy, which we call 'rollouts'. In a spatial navigation task, the agent learns to plan when it is beneficial, which provides a normative explanation for empirical variability in human thinking times. Additionally, the patterns of policy rollouts used by the artificial agent closely resemble patterns of rodent hippocampal replays. Our work provides a theory of how the brain could implement planning through prefrontal-hippocampal interactions, where hippocampal replays are triggered by-and adaptively affect-prefrontal dynamics.
Collapse
Affiliation(s)
- Kristopher T Jensen
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK.
- Sainsbury Wellcome Centre, University College London, London, UK.
| | - Guillaume Hennequin
- Computational and Biological Learning Lab, Department of Engineering, University of Cambridge, Cambridge, UK
| | - Marcelo G Mattar
- Department of Cognitive Science, University of California, San Diego, CA, USA
- Department of Psychology, New York University, New York, NY, USA
| |
Collapse
|
3
|
Choucry A, Nomoto M, Inokuchi K. Engram mechanisms of memory linking and identity. Nat Rev Neurosci 2024; 25:375-392. [PMID: 38664582 DOI: 10.1038/s41583-024-00814-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/25/2024] [Indexed: 05/25/2024]
Abstract
Memories are thought to be stored in neuronal ensembles referred to as engrams. Studies have suggested that when two memories occur in quick succession, a proportion of their engrams overlap and the memories become linked (in a process known as prospective linking) while maintaining their individual identities. In this Review, we summarize the key principles of memory linking through engram overlap, as revealed by experimental and modelling studies. We describe evidence of the involvement of synaptic memory substrates, spine clustering and non-linear neuronal capacities in prospective linking, and suggest a dynamic somato-synaptic model, in which memories are shared between neurons yet remain separable through distinct dendritic and synaptic allocation patterns. We also bring into focus retrospective linking, in which memories become associated after encoding via offline reactivation, and discuss key temporal and mechanistic differences between prospective and retrospective linking, as well as the potential differences in their cognitive outcomes.
Collapse
Affiliation(s)
- Ali Choucry
- Graduate School of Medicine and Pharmaceutical Sciences, University of Toyama, Toyama, Japan
- Research Center for Idling Brain Science, University of Toyama, Toyama, Japan
- Department of Pharmacology and Toxicology, Faculty of Pharmacy, Cairo University, Cairo, Egypt
| | - Masanori Nomoto
- Graduate School of Medicine and Pharmaceutical Sciences, University of Toyama, Toyama, Japan
- Research Center for Idling Brain Science, University of Toyama, Toyama, Japan
- CREST, Japan Science and Technology Agency (JST), University of Toyama, Toyama, Japan
- Japan Agency for Medical Research and Development (AMED), Tokyo, Japan
| | - Kaoru Inokuchi
- Graduate School of Medicine and Pharmaceutical Sciences, University of Toyama, Toyama, Japan.
- Research Center for Idling Brain Science, University of Toyama, Toyama, Japan.
- CREST, Japan Science and Technology Agency (JST), University of Toyama, Toyama, Japan.
| |
Collapse
|
4
|
Kern S, Nagel J, Gerchen MF, Gürsoy Ç, Meyer-Lindenberg A, Kirsch P, Dolan RJ, Gais S, Feld GB. Reactivation strength during cued recall is modulated by graph distance within cognitive maps. eLife 2024; 12:RP93357. [PMID: 38810249 PMCID: PMC11136493 DOI: 10.7554/elife.93357] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 05/31/2024] Open
Abstract
Declarative memory retrieval is thought to involve reinstatement of neuronal activity patterns elicited and encoded during a prior learning episode. Furthermore, it is suggested that two mechanisms operate during reinstatement, dependent on task demands: individual memory items can be reactivated simultaneously as a clustered occurrence or, alternatively, replayed sequentially as temporally separate instances. In the current study, participants learned associations between images that were embedded in a directed graph network and retained this information over a brief 8 min consolidation period. During a subsequent cued recall session, participants retrieved the learned information while undergoing magnetoencephalographic recording. Using a trained stimulus decoder, we found evidence for clustered reactivation of learned material. Reactivation strength of individual items during clustered reactivation decreased as a function of increasing graph distance, an ordering present solely for successful retrieval but not for retrieval failure. In line with previous research, we found evidence that sequential replay was dependent on retrieval performance and was most evident in low performers. The results provide evidence for distinct performance-dependent retrieval mechanisms, with graded clustered reactivation emerging as a plausible mechanism to search within abstract cognitive maps.
Collapse
Affiliation(s)
- Simon Kern
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Addiction Behavior and Addiction Medicine, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
| | - Juliane Nagel
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Addiction Behavior and Addiction Medicine, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
| | - Martin F Gerchen
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Department of Psychology, Ruprecht Karl University of HeidelbergHeidelbergGermany
- Bernstein Center for Computational Neuroscience Heidelberg/MannheimMannheimGermany
| | - Çağatay Gürsoy
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Addiction Behavior and Addiction Medicine, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
| | - Andreas Meyer-Lindenberg
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Bernstein Center for Computational Neuroscience Heidelberg/MannheimMannheimGermany
| | - Peter Kirsch
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Department of Psychology, Ruprecht Karl University of HeidelbergHeidelbergGermany
- Bernstein Center for Computational Neuroscience Heidelberg/MannheimMannheimGermany
| | - Raymond J Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing ResearchLondonUnited Kingdom
- Wellcome Centre for Human Neuroimaging, University College LondonLondonUnited Kingdom
| | - Steffen Gais
- Institute of Medical Psychology and Behavioral Neurobiology, Eberhard-Karls-University TübingenTübingenGermany
| | - Gordon B Feld
- Clinical Psychology, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Psychiatry and Psychotherapy, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Addiction Behavior and Addiction Medicine, Central Institute of Mental Health, Medical Faculty Mannheim, University of HeidelbergMannheimGermany
- Department of Psychology, Ruprecht Karl University of HeidelbergHeidelbergGermany
| |
Collapse
|
5
|
Dubinsky JM, Hamid AA. The neuroscience of active learning and direct instruction. Neurosci Biobehav Rev 2024; 163:105737. [PMID: 38796122 DOI: 10.1016/j.neubiorev.2024.105737] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/19/2023] [Revised: 05/13/2024] [Accepted: 05/20/2024] [Indexed: 05/28/2024]
Abstract
Throughout the educational system, students experiencing active learning pedagogy perform better and fail less than those taught through direct instruction. Can this be ascribed to differences in learning from a neuroscientific perspective? This review examines mechanistic, neuroscientific evidence that might explain differences in cognitive engagement contributing to learning outcomes between these instructional approaches. In classrooms, direct instruction comprehensively describes academic content, while active learning provides structured opportunities for learners to explore, apply, and manipulate content. Synaptic plasticity and its modulation by arousal or novelty are central to all learning and both approaches. As a form of social learning, direct instruction relies upon working memory. The reinforcement learning circuit, associated agency, curiosity, and peer-to-peer social interactions combine to enhance motivation, improve retention, and build higher-order-thinking skills in active learning environments. When working memory becomes overwhelmed, additionally engaging the reinforcement learning circuit improves retention, providing an explanation for the benefits of active learning. This analysis provides a mechanistic examination of how emerging neuroscience principles might inform pedagogical choices at all educational levels.
Collapse
Affiliation(s)
- Janet M Dubinsky
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA.
| | - Arif A Hamid
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
6
|
Schönberger DK, Bruns P, Röder B. Visual artificial grammar learning across 1 year in 7-year-olds and adults. J Exp Child Psychol 2024; 241:105864. [PMID: 38335709 DOI: 10.1016/j.jecp.2024.105864] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 11/30/2023] [Accepted: 01/08/2024] [Indexed: 02/12/2024]
Abstract
Acquiring sequential information is of utmost importance, for example, for language acquisition in children. Yet, the long-term storage of statistical learning in children is poorly understood. To address this question, 27 7-year-olds and 28 young adults completed four sessions of visual sequence learning (Year 1). From this sample, 16 7-year-olds and 20 young adults participated in another four equivalent sessions after a 12-month-delay (Year 2). The first three sessions of each year used Stimulus Set 1, and the last session used Stimulus Set 2 to investigate transfer effects. Each session consisted of alternating learning and test phases in a modified artificial grammar learning task. In Year 1, 7-year-olds and adults learned the regularities and showed transfer to Stimulus Set 2. Both groups retained their final performance level over the 1-year period. In Year 2, children and adults continued to improve with Stimulus Set 1 but did not show additional transfer gains. Adults overall outperformed children, but transfer effects were indistinguishable between both groups. The current results suggest that long-term memory traces are formed from repeated sequence learning that can be used to generalize sequence rules to new visual input. However, the current study did not provide evidence for a childhood advantage in learning and remembering sequence rules.
Collapse
Affiliation(s)
- Daniela K Schönberger
- Biological Psychology and Neuropsychology, University of Hamburg, D-20146 Hamburg, Germany.
| | - Patrick Bruns
- Biological Psychology and Neuropsychology, University of Hamburg, D-20146 Hamburg, Germany
| | - Brigitte Röder
- Biological Psychology and Neuropsychology, University of Hamburg, D-20146 Hamburg, Germany; LV Prasad Eye Institute, Hyderabad 500 034, India
| |
Collapse
|
7
|
Heinen R, Bierbrauer A, Wolf OT, Axmacher N. Representational formats of human memory traces. Brain Struct Funct 2024; 229:513-529. [PMID: 37022435 PMCID: PMC10978732 DOI: 10.1007/s00429-023-02636-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/06/2022] [Accepted: 03/28/2023] [Indexed: 04/07/2023]
Abstract
Neural representations are internal brain states that constitute the brain's model of the external world or some of its features. In the presence of sensory input, a representation may reflect various properties of this input. When perceptual information is no longer available, the brain can still activate representations of previously experienced episodes due to the formation of memory traces. In this review, we aim at characterizing the nature of neural memory representations and how they can be assessed with cognitive neuroscience methods, mainly focusing on neuroimaging. We discuss how multivariate analysis techniques such as representational similarity analysis (RSA) and deep neural networks (DNNs) can be leveraged to gain insights into the structure of neural representations and their different representational formats. We provide several examples of recent studies which demonstrate that we are able to not only measure memory representations using RSA but are also able to investigate their multiple formats using DNNs. We demonstrate that in addition to slow generalization during consolidation, memory representations are subject to semantization already during short-term memory, by revealing a shift from visual to semantic format. In addition to perceptual and conceptual formats, we describe the impact of affective evaluations as an additional dimension of episodic memories. Overall, these studies illustrate how the analysis of neural representations may help us gain a deeper understanding of the nature of human memory.
Collapse
Affiliation(s)
- Rebekka Heinen
- Department of Neuropsychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Universitätsstraße 150, 44801, Bochum, Germany.
| | - Anne Bierbrauer
- Department of Neuropsychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Universitätsstraße 150, 44801, Bochum, Germany
- Institute for Systems Neuroscience, Medical Center Hamburg-Eppendorf, Martinistraße 52, 20251, Hamburg, Germany
| | - Oliver T Wolf
- Department of Cognitive Psychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Universitätsstraße 150, 44801, Bochum, Germany
| | - Nikolai Axmacher
- Department of Neuropsychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, Universitätsstraße 150, 44801, Bochum, Germany
| |
Collapse
|
8
|
Pezzulo G, D'Amato L, Mannella F, Priorelli M, Van de Maele T, Stoianov IP, Friston K. Neural representation in active inference: Using generative models to interact with-and understand-the lived world. Ann N Y Acad Sci 2024; 1534:45-68. [PMID: 38528782 DOI: 10.1111/nyas.15118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/27/2024]
Abstract
This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from explicit models that entail variables for predicting concurrent sensations, like objects, faces, or people-to action-oriented models that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities-and the gradual transition from pragmatic to detached neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Leo D'Amato
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
- Polytechnic University of Turin, Turin, Italy
| | - Francesco Mannella
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Matteo Priorelli
- Institute of Cognitive Sciences and Technologies, National Research Council, Padua, Italy
| | - Toon Van de Maele
- IDLab, Department of Information Technology, Ghent University - imec, Ghent, Belgium
| | - Ivilin Peev Stoianov
- Institute of Cognitive Sciences and Technologies, National Research Council, Padua, Italy
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, UK
- VERSES Research Lab, Los Angeles, California, USA
| |
Collapse
|
9
|
Deceuninck L, Kloosterman F. Disruption of awake sharp-wave ripples does not affect memorization of locations in repeated-acquisition spatial memory tasks. eLife 2024; 13:e84004. [PMID: 38530125 PMCID: PMC11018343 DOI: 10.7554/elife.84004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2022] [Accepted: 03/25/2024] [Indexed: 03/27/2024] Open
Abstract
Storing and accessing memories is required to successfully perform day-to-day tasks, for example for engaging in a meaningful conversation. Previous studies in both rodents and primates have correlated hippocampal cellular activity with behavioral expression of memory. A key role has been attributed to awake hippocampal replay - a sequential reactivation of neurons representing a trajectory through space. However, it is unclear if awake replay impacts immediate future behavior, gradually creates and stabilizes long-term memories over a long period of time (hours and longer), or enables the temporary memorization of relevant events at an intermediate time scale (seconds to minutes). In this study, we aimed to address the uncertainty around the timeframe of impact of awake replay by collecting causal evidence from behaving rats. We detected and disrupted sharp wave ripples (SWRs) - signatures of putative replay events - using electrical stimulation of the ventral hippocampal commissure in rats that were trained on three different spatial memory tasks. In each task, rats were required to memorize a new set of locations in each trial or each daily session. Interestingly, the rats performed equally well with or without SWR disruptions. These data suggest that awake SWRs - and potentially replay - does not affect the immediate behavior nor the temporary memorization of relevant events at a short timescale that are required to successfully perform the spatial tasks. Based on these results, we hypothesize that the impact of awake replay on memory and behavior is long-term and cumulative over time.
Collapse
Affiliation(s)
- Lies Deceuninck
- KU Leuven, Department of Physics and Astronomy, Soft Matter and BiophysicsHeverleeBelgium
- NERF-NeuroElectronics Research Flanders, Kloosterman LabHeverleeBelgium
| | - Fabian Kloosterman
- NERF-NeuroElectronics Research Flanders, Kloosterman LabHeverleeBelgium
- KU Leuven, Faculty of Psychology & Educational SciencesLeuvenBelgium
| |
Collapse
|
10
|
Desbordes T, King JR, Dehaene S. Tracking the neural codes for words and phrases during semantic composition, working-memory storage, and retrieval. Cell Rep 2024; 43:113847. [PMID: 38412098 DOI: 10.1016/j.celrep.2024.113847] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/20/2023] [Revised: 11/02/2023] [Accepted: 02/07/2024] [Indexed: 02/29/2024] Open
Abstract
The ability to compose successive words into a meaningful phrase is a characteristic feature of human cognition, yet its neural mechanisms remain incompletely understood. Here, we analyze the cortical mechanisms of semantic composition using magnetoencephalography (MEG) while participants read one-word, two-word, and five-word noun phrases and compared them with a subsequent image. Decoding of MEG signals revealed three processing stages. During phrase comprehension, the representation of individual words was sustained for a variable duration depending on phrasal context. During the delay period, the word code was replaced by a working-memory code whose activation increased with semantic complexity. Finally, the speed and accuracy of retrieval depended on semantic complexity and was faster for surface than for deep semantic properties. In conclusion, we propose that the brain initially encodes phrases using factorized dimensions for successive words but later compresses them in working memory and requires a period of decompression to access them.
Collapse
Affiliation(s)
- Théo Desbordes
- Meta AI, Paris, France; Cognitive Neuroimaging Unit, NeuroSpin Center, 91191 Gif-sur-Yvette, France.
| | - Jean-Rémi King
- Meta AI, Paris, France; École Normale Supérieure, PSL University, Paris, France
| | - Stanislas Dehaene
- Université Paris Saclay, INSERM, CEA, Cognitive Neuroimaging Unit, NeuroSpin Center, 91191 Gif-sur-Yvette, France; Collège de France, PSL University, Paris, France
| |
Collapse
|
11
|
Wise T, Emery K, Radulescu A. Naturalistic reinforcement learning. Trends Cogn Sci 2024; 28:144-158. [PMID: 37777463 PMCID: PMC10878983 DOI: 10.1016/j.tics.2023.08.016] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 08/23/2023] [Accepted: 08/24/2023] [Indexed: 10/02/2023]
Abstract
Humans possess a remarkable ability to make decisions within real-world environments that are expansive, complex, and multidimensional. Human cognitive computational neuroscience has sought to exploit reinforcement learning (RL) as a framework within which to explain human decision-making, often focusing on constrained, artificial experimental tasks. In this article, we review recent efforts that use naturalistic approaches to determine how humans make decisions in complex environments that better approximate the real world, providing a clearer picture of how humans navigate the challenges posed by real-world decisions. These studies purposely embed elements of naturalistic complexity within experimental paradigms, rather than focusing on simplification, generating insights into the processes that likely underpin humans' ability to navigate complex, multidimensional real-world environments so successfully.
Collapse
Affiliation(s)
- Toby Wise
- Department of Neuroimaging, King's College London, London, UK.
| | - Kara Emery
- Center for Data Science, New York University, New York, NY, USA
| | - Angela Radulescu
- Center for Computational Psychiatry, Icahn School of Medicine at Mt. Sinai, New York, NY, USA
| |
Collapse
|
12
|
Chen D, Axmacher N, Wang L. Grid codes underlie multiple cognitive maps in the human brain. Prog Neurobiol 2024; 233:102569. [PMID: 38232782 DOI: 10.1016/j.pneurobio.2024.102569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 01/07/2024] [Accepted: 01/10/2024] [Indexed: 01/19/2024]
Abstract
Grid cells fire at multiple positions that organize the vertices of equilateral triangles tiling a 2D space and are well studied in rodents. The last decade witnessed rapid progress in two other research lines on grid codes-empirical studies on distributed human grid-like representations in physical and multiple non-physical spaces, and cognitive computational models addressing the function of grid cells based on principles of efficient and predictive coding. Here, we review the progress in these fields and integrate these lines into a systematic organization. We also discuss the coordinate mechanisms of grid codes in the human entorhinal cortex and medial prefrontal cortex and their role in neurological and psychiatric diseases.
Collapse
Affiliation(s)
- Dong Chen
- CAS Key Laboratory of Mental Health, Institute of Psychology, 100101, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, 100101, Beijing, China
| | - Nikolai Axmacher
- Department of Neuropsychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, 44801, Bochum, Germany
| | - Liang Wang
- CAS Key Laboratory of Mental Health, Institute of Psychology, 100101, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, 100101, Beijing, China.
| |
Collapse
|
13
|
Peters B, DiCarlo JJ, Gureckis T, Haefner R, Isik L, Tenenbaum J, Konkle T, Naselaris T, Stachenfeld K, Tavares Z, Tsao D, Yildirim I, Kriegeskorte N. How does the primate brain combine generative and discriminative computations in vision? ARXIV 2024:arXiv:2401.06005v1. [PMID: 38259351 PMCID: PMC10802669] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Subscribe] [Scholar Register] [Indexed: 01/24/2024]
Abstract
Vision is widely understood as an inference problem. However, two contrasting conceptions of the inference process have each been influential in research on biological vision as well as the engineering of machine vision. The first emphasizes bottom-up signal flow, describing vision as a largely feedforward, discriminative inference process that filters and transforms the visual information to remove irrelevant variation and represent behaviorally relevant information in a format suitable for downstream functions of cognition and behavioral control. In this conception, vision is driven by the sensory data, and perception is direct because the processing proceeds from the data to the latent variables of interest. The notion of "inference" in this conception is that of the engineering literature on neural networks, where feedforward convolutional neural networks processing images are said to perform inference. The alternative conception is that of vision as an inference process in Helmholtz's sense, where the sensory evidence is evaluated in the context of a generative model of the causal processes that give rise to it. In this conception, vision inverts a generative model through an interrogation of the sensory evidence in a process often thought to involve top-down predictions of sensory data to evaluate the likelihood of alternative hypotheses. The authors include scientists rooted in roughly equal numbers in each of the conceptions and motivated to overcome what might be a false dichotomy between them and engage the other perspective in the realm of theory and experiment. The primate brain employs an unknown algorithm that may combine the advantages of both conceptions. We explain and clarify the terminology, review the key empirical evidence, and propose an empirical research program that transcends the dichotomy and sets the stage for revealing the mysterious hybrid algorithm of primate vision.
Collapse
Affiliation(s)
- Benjamin Peters
- Zuckerman Mind Brain Behavior Institute, Columbia University
- School of Psychology & Neuroscience, University of Glasgow
| | - James J DiCarlo
- Department of Brain and Cognitive Sciences, MIT
- McGovern Institute for Brain Research, MIT
- NSF Center for Brains, Minds and Machines, MIT
- Quest for Intelligence, Schwarzman College of Computing, MIT
| | | | - Ralf Haefner
- Brain and Cognitive Sciences, University of Rochester
- Center for Visual Science, University of Rochester
| | - Leyla Isik
- Department of Cognitive Science, Johns Hopkins University
| | - Joshua Tenenbaum
- Department of Brain and Cognitive Sciences, MIT
- NSF Center for Brains, Minds and Machines, MIT
- Computer Science and Artificial Intelligence Laboratory, MIT
| | - Talia Konkle
- Department of Psychology, Harvard University
- Center for Brain Science, Harvard University
- Kempner Institute for Natural and Artificial Intelligence, Harvard University
| | | | | | - Zenna Tavares
- Zuckerman Mind Brain Behavior Institute, Columbia University
- Data Science Institute, Columbia University
| | - Doris Tsao
- Dept of Molecular & Cell Biology, University of California Berkeley
- Howard Hughes Medical Institute
| | - Ilker Yildirim
- Department of Psychology, Yale University
- Department of Statistics and Data Science, Yale University
| | - Nikolaus Kriegeskorte
- Zuckerman Mind Brain Behavior Institute, Columbia University
- Department of Psychology, Columbia University
- Department of Neuroscience, Columbia University
- Department of Electrical Engineering, Columbia University
| |
Collapse
|
14
|
Dragoi G. The generative grammar of the brain: a critique of internally generated representations. Nat Rev Neurosci 2024; 25:60-75. [PMID: 38036709 DOI: 10.1038/s41583-023-00763-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/18/2023] [Indexed: 12/02/2023]
Abstract
The past decade of progress in neurobiology has uncovered important organizational principles for network preconfiguration and neuronal selection that suggest a generative grammar exists in the brain. In this Perspective, I discuss the competence of the hippocampal neural network to generically express temporally compressed sequences of neuronal firing that represent novel experiences, which is envisioned as a form of generative neural syntax supporting a neurobiological perspective on brain function. I compare this neural competence with the hippocampal network performance that represents specific experiences with higher fidelity after new learning during replay, which is envisioned as a form of neural semantic that supports a complementary neuropsychological perspective. I also demonstrate how the syntax of network competence emerges a priori during early postnatal life and is followed by the later development of network performance that enables rapid encoding and memory consolidation. Thus, I propose that this generative grammar of the brain is essential for internally generated representations, which are crucial for the cognitive processes underlying learning and memory, prospection, and inference, which ultimately underlie our reason and representation of the world.
Collapse
Affiliation(s)
- George Dragoi
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA.
- Department of Neuroscience, Yale University School of Medicine, New Haven, CT, USA.
- Wu Tsai Institute, Yale University, New Haven, CT, USA.
| |
Collapse
|
15
|
Jarovi J, Pilkiw M, Takehara-Nishiuchi K. Prefrontal neuronal ensembles link prior knowledge with novel actions during flexible action selection. Cell Rep 2023; 42:113492. [PMID: 37999978 DOI: 10.1016/j.celrep.2023.113492] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2023] [Revised: 10/23/2023] [Accepted: 11/10/2023] [Indexed: 11/26/2023] Open
Abstract
We make decisions based on currently perceivable information or an internal model of the environment. The medial prefrontal cortex (mPFC) and its interaction with the hippocampus have been implicated in the latter, model-based decision-making; however, the underlying computational properties remain incompletely understood. We have examined mPFC spiking and hippocampal oscillatory activity while rats flexibly select new actions using a known associative structure of environmental cues and outcomes. During action selection, the mPFC reinstates representations of the associative structure. These awake reactivation events are accompanied by synchronous firings among neurons coding the associative structure and those coding actions. Moreover, their functional coupling is strengthened upon the reactivation events leading to adaptive actions. In contrast, only cue-coding neurons improve functional coupling during hippocampal sharp wave ripples. Thus, the lack of direct experience disconnects the mPFC from the hippocampus to independently form self-organized neuronal ensemble dynamics linking prior knowledge with novel actions.
Collapse
Affiliation(s)
- Justin Jarovi
- Department of Cell and Systems Biology, University of Toronto, Toronto, ON M5S 3G5, Canada
| | - Maryna Pilkiw
- Department of Cell and Systems Biology, University of Toronto, Toronto, ON M5S 3G5, Canada
| | - Kaori Takehara-Nishiuchi
- Department of Cell and Systems Biology, University of Toronto, Toronto, ON M5S 3G5, Canada; Department of Psychology, University of Toronto, Toronto, ON M5S 3G3, Canada; Collaborative Program in Neuroscience, University of Toronto, Toronto, ON M5S 1A8, Canada.
| |
Collapse
|
16
|
Veselic S, Muller TH, Gutierrez E, Behrens TEJ, Hunt LT, Butler JL, Kennerley SW. A cognitive map for value-guided choice in ventromedial prefrontal cortex. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.12.15.571895. [PMID: 38168410 PMCID: PMC10760117 DOI: 10.1101/2023.12.15.571895] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 01/05/2024]
Abstract
The prefrontal cortex is crucial for economic decision-making and representing the value of options. However, how such representations facilitate flexible decisions remains unknown. We reframe economic decision-making in prefrontal cortex in line with representations of structure within the medial temporal lobe because such cognitive map representations are known to facilitate flexible behaviour. Specifically, we framed choice between different options as a navigation process in value space. Here we show that choices in a 2D value space defined by reward magnitude and probability were represented with a grid-like code, analogous to that found in spatial navigation. The grid-like code was present in ventromedial prefrontal cortex (vmPFC) local field potential theta frequency and the result replicated in an independent dataset. Neurons in vmPFC similarly contained a grid-like code, in addition to encoding the linear value of the chosen option. Importantly, both signals were modulated by theta frequency - occurring at theta troughs but on separate theta cycles. Furthermore, we found sharp-wave ripples - a key neural signature of planning and flexible behaviour - in vmPFC, which were modulated by accuracy and reward. These results demonstrate that multiple cognitive map-like computations are deployed in vmPFC during economic decision-making, suggesting a new framework for the implementation of choice in prefrontal cortex.
Collapse
Affiliation(s)
- Sebastijan Veselic
- Department of Experimental Psychology, University of Oxford, UK
- Clinical and Movement Neurosciences, Department of Motor Neuroscience, University College London, London, UK
| | - Timothy H Muller
- Department of Experimental Psychology, University of Oxford, UK
- Clinical and Movement Neurosciences, Department of Motor Neuroscience, University College London, London, UK
| | - Elena Gutierrez
- Department of Experimental Psychology, University of Oxford, UK
- Clinical and Movement Neurosciences, Department of Motor Neuroscience, University College London, London, UK
| | - Timothy E J Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, FMRIB, John Radcliffe Hospital, Oxford, UK
- Sainsbury Wellcome Centre for Neural Circuits and Behaviour College, University College London, London, UK
| | - Laurence T Hunt
- Department of Experimental Psychology, University of Oxford, UK
- Department of Psychiatry, University of Oxford, Oxford, UK
| | - James L Butler
- Department of Experimental Psychology, University of Oxford, UK
| | - Steven W Kennerley
- Department of Experimental Psychology, University of Oxford, UK
- Clinical and Movement Neurosciences, Department of Motor Neuroscience, University College London, London, UK
| |
Collapse
|
17
|
Walsh CR, Rissman J. Behavioral representational similarity analysis reveals how episodic learning is influenced by and reshapes semantic memory. Nat Commun 2023; 14:7548. [PMID: 37985774 PMCID: PMC10662157 DOI: 10.1038/s41467-023-42770-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Accepted: 10/20/2023] [Indexed: 11/22/2023] Open
Abstract
While semantic and episodic memory have been shown to influence each other, uncertainty remains as to how this interplay occurs. We introduce a behavioral representational similarity analysis approach to assess whether semantic space can be subtly re-sculpted by episodic learning. Eighty participants learned word pairs that varied in semantic relatedness, and learning was bolstered via either testing or restudying. Next-day recall is superior for semantically related pairs, but there is a larger benefit of testing for unrelated pairs. Analyses of representational change reveal that successful recall is accompanied by a pulling together of paired associates, with cue words in semantically related (but not unrelated) pairs changing more across learning than target words. Our findings show that episodic learning is associated with systematic and asymmetrical distortions of semantic space which improve later recall by making cues more predictive of targets, reducing interference from potential lures, and establishing novel connections within pairs.
Collapse
Affiliation(s)
- Catherine R Walsh
- Department of Psychology, University of California, Los Angeles, CA, USA.
| | - Jesse Rissman
- Department of Psychology, University of California, Los Angeles, CA, USA
- Department of Psychiatry & Biobehavioral Sciences, University of California, Los Angeles, CA, USA
- Brain Research Institute, University of California, Los Angeles, CA, USA
- Integrative Center for Learning and Memory, University of California, Los Angeles, CA, USA
| |
Collapse
|
18
|
Csaky R, van Es MWJ, Jones OP, Woolrich M. Interpretable many-class decoding for MEG. Neuroimage 2023; 282:120396. [PMID: 37805019 PMCID: PMC10938061 DOI: 10.1016/j.neuroimage.2023.120396] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/12/2023] [Revised: 09/11/2023] [Accepted: 09/27/2023] [Indexed: 10/09/2023] Open
Abstract
Multivariate pattern analysis (MVPA) of Magnetoencephalography (MEG) and Electroencephalography (EEG) data is a valuable tool for understanding how the brain represents and discriminates between different stimuli. Identifying the spatial and temporal signatures of stimuli is typically a crucial output of these analyses. Such analyses are mainly performed using linear, pairwise, sliding window decoding models. These allow for relative ease of interpretation, e.g. by estimating a time-course of decoding accuracy, but have limited decoding performance. On the other hand, full epoch multiclass decoding models, commonly used for brain-computer interface (BCI) applications, can provide better decoding performance. However interpretation methods for such models have been designed with a low number of classes in mind. In this paper, we propose an approach that combines a multiclass, full epoch decoding model with supervised dimensionality reduction, while still being able to reveal the contributions of spatiotemporal and spectral features using permutation feature importance. Crucially, we introduce a way of doing supervised dimensionality reduction of input features within a neural network optimised for the classification task, improving performance substantially. We demonstrate the approach on 3 different many-class task-MEG datasets using image presentations. Our results demonstrate that this approach consistently achieves higher accuracy than the peak accuracy of a sliding window decoder while estimating the relevant spatiotemporal features in the MEG signal.
Collapse
Affiliation(s)
- Richard Csaky
- Oxford Centre for Human Brain Activity, Department of Psychiatry, University of Oxford, OX3 7JX, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, OX3 9DU, Oxford, UK; Christ Church, OX1 1DP, Oxford, UK.
| | - Mats W J van Es
- Oxford Centre for Human Brain Activity, Department of Psychiatry, University of Oxford, OX3 7JX, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, OX3 9DU, Oxford, UK.
| | - Oiwi Parker Jones
- Wellcome Centre for Integrative Neuroimaging, OX3 9DU, Oxford, UK; Department of Engineering Science, University of Oxford, OX1 3PJ, Oxford, UK; Jesus College, OX1 3DW, Oxford, UK.
| | - Mark Woolrich
- Oxford Centre for Human Brain Activity, Department of Psychiatry, University of Oxford, OX3 7JX, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, OX3 9DU, Oxford, UK.
| |
Collapse
|
19
|
Eppinger B, Ruel A, Bolenz F. Diminished State Space Theory of Human Aging. PERSPECTIVES ON PSYCHOLOGICAL SCIENCE 2023:17456916231204811. [PMID: 37931229 DOI: 10.1177/17456916231204811] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2023]
Abstract
Many new technologies, such as smartphones, computers, or public-access systems (like ticket-vending machines), are a challenge for older adults. One feature that these technologies have in common is that they involve underlying, partially observable, structures (state spaces) that determine the actions that are necessary to reach a certain goal (e.g., to move from one menu to another, to change a function, or to activate a new service). In this work we provide a theoretical, neurocomputational account to explain these behavioral difficulties in older adults. Based on recent findings from age-comparative computational- and cognitive-neuroscience studies, we propose that age-related impairments in complex goal-directed behavior result from an underlying deficit in the representation of state spaces of cognitive tasks. Furthermore, we suggest that these age-related deficits in adaptive decision-making are due to impoverished neural representations in the orbitofrontal cortex and hippocampus.
Collapse
Affiliation(s)
- Ben Eppinger
- Institute of Psychology, University of Greifswald
- Department of Psychology, Concordia University
- PERFORM Centre, Concordia University
- Faculty of Psychology, Technische Universität Dresden
| | - Alexa Ruel
- Department of Psychology, Concordia University
- PERFORM Centre, Concordia University
- Institute of Psychology, University of Hamburg
| | - Florian Bolenz
- Center for Adaptive Rationality, Max Planck Institute for Human Development, Berlin, Germany
- Science of Intelligence/Cluster of Excellence, Technical University of Berlin
| |
Collapse
|
20
|
Kucyi A, Kam JWY, Andrews-Hanna JR, Christoff K, Whitfield-Gabrieli S. Recent advances in the neuroscience of spontaneous and off-task thought: implications for mental health. NATURE MENTAL HEALTH 2023; 1:827-840. [PMID: 37974566 PMCID: PMC10653280 DOI: 10.1038/s44220-023-00133-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/27/2022] [Accepted: 08/25/2023] [Indexed: 11/19/2023]
Abstract
People spend a remarkable 30-50% of awake life thinking about something other than what they are currently doing. These experiences of being "off-task" can be described as spontaneous thought when mental dynamics are relatively flexible. Here we review recent neuroscience developments in this area and consider implications for mental wellbeing and illness. We provide updated overviews of the roles of the default mode network and large-scale network dynamics, and we discuss emerging candidate mechanisms involving hippocampal memory (sharp-wave ripples, replay) and neuromodulatory (noradrenergic and serotonergic) systems. We explore how distinct brain states can be associated with or give rise to adaptive and maladaptive forms of thought linked to distinguishable mental health outcomes. We conclude by outlining new directions in the neuroscience of spontaneous and off-task thought that may clarify mechanisms, lead to personalized biomarkers, and facilitate therapy developments toward the goals of better understanding and improving mental health.
Collapse
Affiliation(s)
- Aaron Kucyi
- Department of Psychological and Brain Sciences, Drexel University
| | - Julia W. Y. Kam
- Department of Psychology and Hotchkiss Brain Institute, University of Calgary
| | | | | | | |
Collapse
|
21
|
Schwartenbeck P, Baram A, Liu Y, Mark S, Muller T, Dolan R, Botvinick M, Kurth-Nelson Z, Behrens T. Generative replay underlies compositional inference in the hippocampal-prefrontal circuit. Cell 2023; 186:4885-4897.e14. [PMID: 37804832 PMCID: PMC10914680 DOI: 10.1016/j.cell.2023.09.004] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 01/23/2023] [Accepted: 09/06/2023] [Indexed: 10/09/2023]
Abstract
Human reasoning depends on reusing pieces of information by putting them together in new ways. However, very little is known about how compositional computation is implemented in the brain. Here, we ask participants to solve a series of problems that each require constructing a whole from a set of elements. With fMRI, we find that representations of novel constructed objects in the frontal cortex and hippocampus are relational and compositional. With MEG, we find that replay assembles elements into compounds, with each replay sequence constituting a hypothesis about a possible configuration of elements. The content of sequences evolves as participants solve each puzzle, progressing from predictable to uncertain elements and gradually converging on the correct configuration. Together, these results suggest a computational bridge between apparently distinct functions of hippocampal-prefrontal circuitry and a role for generative replay in compositional inference and hypothesis testing.
Collapse
Affiliation(s)
- Philipp Schwartenbeck
- University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Baden-Württemberg, Germany; Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK.
| | - Alon Baram
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Shirley Mark
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK
| | - Timothy Muller
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Institute of Neurology, University College London, London WC1N 3BG, UK
| | - Raymond Dolan
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Department of Psychiatry, Universitätsmedizin Berlin (Campus Charité Mitte), Berlin, Germany
| | - Matthew Botvinick
- Google DeepMind, London, UK; Gatsby Computational Neuroscience Unit, University College London, London, UK
| | - Zeb Kurth-Nelson
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Google DeepMind, London, UK
| | - Timothy Behrens
- Wellcome Trust Centre for Neuroimaging, University College London, London WC1N 3AR, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford OX3 9DU, UK; Sainsbury Wellcome Centre for Neural Circuits and Behaviour, UCL, London W1T 4JG, UK
| |
Collapse
|
22
|
Nour MM, McNamee DC, Liu Y, Dolan RJ. Trajectories through semantic spaces in schizophrenia and the relationship to ripple bursts. Proc Natl Acad Sci U S A 2023; 120:e2305290120. [PMID: 37816054 PMCID: PMC10589662 DOI: 10.1073/pnas.2305290120] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2023] [Accepted: 07/31/2023] [Indexed: 10/12/2023] Open
Abstract
Human cognition is underpinned by structured internal representations that encode relationships between entities in the world (cognitive maps). Clinical features of schizophrenia-from thought disorder to delusions-are proposed to reflect disorganization in such conceptual representations. Schizophrenia is also linked to abnormalities in neural processes that support cognitive map representations, including hippocampal replay and high-frequency ripple oscillations. Here, we report a computational assay of semantically guided conceptual sampling and exploit this to test a hypothesis that people with schizophrenia (PScz) exhibit abnormalities in semantically guided cognition that relate to hippocampal replay and ripples. Fifty-two participants [26 PScz (13 unmedicated) and 26 age-, gender-, and intelligence quotient (IQ)-matched nonclinical controls] completed a category- and letter-verbal fluency task, followed by a magnetoencephalography (MEG) scan involving a separate sequence-learning task. We used a pretrained word embedding model of semantic similarity, coupled to a computational model of word selection, to quantify the degree to which each participant's verbal behavior was guided by semantic similarity. Using MEG, we indexed neural replay and ripple power in a post-task rest session. Across all participants, word selection was strongly influenced by semantic similarity. The strength of this influence showed sensitivity to task demands (category > letter fluency) and predicted performance. In line with our hypothesis, the influence of semantic similarity on behavior was reduced in schizophrenia relative to controls, predicted negative psychotic symptoms, and correlated with an MEG signature of hippocampal ripple power (but not replay). The findings bridge a gap between phenomenological and neurocomputational accounts of schizophrenia.
Collapse
Affiliation(s)
- Matthew M. Nour
- Department of Psychiatry, University of Oxford, OxfordOX3 7JX, United Kingdom
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, LondonWC1B 5EH, United Kingdom
| | - Daniel C. McNamee
- Champalimaud Research, Centre for the Unknown, 1400-038Lisbon, Portugal
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Chinese Institute for Brain Research, Beijing102206, China
| | - Raymond J. Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, LondonWC1B 5EH, United Kingdom
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3AR, United Kingdom
| |
Collapse
|
23
|
Griffiths BJ, Jensen O. Gamma oscillations and episodic memory. Trends Neurosci 2023; 46:832-846. [PMID: 37550159 DOI: 10.1016/j.tins.2023.07.003] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/09/2023] [Revised: 06/20/2023] [Accepted: 07/16/2023] [Indexed: 08/09/2023]
Abstract
Enhanced gamma oscillatory activity (30-80 Hz) accompanies the successful formation and retrieval of episodic memories. While this co-occurrence is well documented, the mechanistic contributions of gamma oscillatory activity to episodic memory remain unclear. Here, we review how gamma oscillatory activity may facilitate spike timing-dependent plasticity, neural communication, and sequence encoding/retrieval, thereby ensuring the successful formation and/or retrieval of an episodic memory. Based on the evidence reviewed, we propose that multiple, distinct forms of gamma oscillation can be found within the canonical gamma band, each of which has a complementary role in the neural processes listed above. Further exploration of these theories using causal manipulations may be key to elucidating the relevance of gamma oscillatory activity to episodic memory.
Collapse
Affiliation(s)
| | - Ole Jensen
- Centre for Human Brain Health, University of Birmingham, Birmingham, UK
| |
Collapse
|
24
|
Mehrotra D, Dubé L. Accounting for multiscale processing in adaptive real-world decision-making via the hippocampus. Front Neurosci 2023; 17:1200842. [PMID: 37732307 PMCID: PMC10508350 DOI: 10.3389/fnins.2023.1200842] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/05/2023] [Accepted: 08/25/2023] [Indexed: 09/22/2023] Open
Abstract
For adaptive real-time behavior in real-world contexts, the brain needs to allow past information over multiple timescales to influence current processing for making choices that create the best outcome as a person goes about making choices in their everyday life. The neuroeconomics literature on value-based decision-making has formalized such choice through reinforcement learning models for two extreme strategies. These strategies are model-free (MF), which is an automatic, stimulus-response type of action, and model-based (MB), which bases choice on cognitive representations of the world and causal inference on environment-behavior structure. The emphasis of examining the neural substrates of value-based decision making has been on the striatum and prefrontal regions, especially with regards to the "here and now" decision-making. Yet, such a dichotomy does not embrace all the dynamic complexity involved. In addition, despite robust research on the role of the hippocampus in memory and spatial learning, its contribution to value-based decision making is just starting to be explored. This paper aims to better appreciate the role of the hippocampus in decision-making and advance the successor representation (SR) as a candidate mechanism for encoding state representations in the hippocampus, separate from reward representations. To this end, we review research that relates hippocampal sequences to SR models showing that the implementation of such sequences in reinforcement learning agents improves their performance. This also enables the agents to perform multiscale temporal processing in a biologically plausible manner. Altogether, we articulate a framework to advance current striatal and prefrontal-focused decision making to better account for multiscale mechanisms underlying various real-world time-related concepts such as the self that cumulates over a person's life course.
Collapse
Affiliation(s)
- Dhruv Mehrotra
- Integrated Program in Neuroscience, McGill University, Montréal, QC, Canada
- Montréal Neurological Institute, McGill University, Montréal, QC, Canada
| | - Laurette Dubé
- Desautels Faculty of Management, McGill University, Montréal, QC, Canada
- McGill Center for the Convergence of Health and Economics, McGill University, Montréal, QC, Canada
| |
Collapse
|
25
|
Li Z, Athwal D, Lee HL, Sah P, Opazo P, Chuang KH. Locating causal hubs of memory consolidation in spontaneous brain network in male mice. Nat Commun 2023; 14:5399. [PMID: 37669938 PMCID: PMC10480429 DOI: 10.1038/s41467-023-41024-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Accepted: 08/17/2023] [Indexed: 09/07/2023] Open
Abstract
Memory consolidation after learning involves spontaneous, brain-wide network reorganization during rest and sleep, but how this is achieved is still poorly understood. Current theory suggests that the hippocampus is pivotal for this reshaping of connectivity. Using fMRI in male mice, we identify that a different set of spontaneous networks and their hubs are instrumental in consolidating memory during post-learning rest. We found that two types of spatial memory training invoke distinct functional connections, but that a network of the sensory cortex and subcortical areas is common for both tasks. Furthermore, learning increased brain-wide network integration, with the prefrontal, striatal and thalamic areas being influential for this network-level reconfiguration. Chemogenetic suppression of each hub identified after learning resulted in retrograde amnesia, confirming the behavioral significance. These results demonstrate the causal and functional roles of resting-state network hubs in memory consolidation and suggest that a distributed network beyond the hippocampus subserves this process.
Collapse
Affiliation(s)
- Zengmin Li
- Queensland Brain Institute, The University of Queensland, Brisbane, QLD, Australia
| | - Dilsher Athwal
- Queensland Brain Institute, The University of Queensland, Brisbane, QLD, Australia
| | - Hsu-Lei Lee
- Queensland Brain Institute, The University of Queensland, Brisbane, QLD, Australia
| | - Pankaj Sah
- Queensland Brain Institute, The University of Queensland, Brisbane, QLD, Australia
- Joint Center for Neuroscience and Neural Engineering, and Department of Biology, Southern University of Science and Technology, Shenzhen, Guangdong, PR China
| | - Patricio Opazo
- Queensland Brain Institute, The University of Queensland, Brisbane, QLD, Australia
- Clem Jones Centre for Ageing Dementia Research, The University of Queensland, Brisbane, QLD, Australia
- UK Dementia Research Institute, Centre for Discovery Brain Sciences, The University of Edinburgh, Edinburgh, UK
| | - Kai-Hsiang Chuang
- Queensland Brain Institute, The University of Queensland, Brisbane, QLD, Australia.
- Centre of Advanced Imaging, The University of Queensland, Brisbane, QLD, Australia.
- Australian Research Council Training Centre for Innovation in Biomedical Imaging Technology, Brisbane, QLD, Australia.
| |
Collapse
|
26
|
Foldes T, Santamaria L, Lewis P. Sleep-related benefits to transitive inference are modulated by encoding strength and joint rank. Learn Mem 2023; 30:201-211. [PMID: 37726142 PMCID: PMC10547378 DOI: 10.1101/lm.053787.123] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2023] [Accepted: 07/11/2023] [Indexed: 09/21/2023]
Abstract
Transitive inference is a measure of relational learning that has been shown to improve across sleep. Here, we examine this phenomenon further by studying the impact of encoding strength and joint rank. In experiment 1, participants learned adjacent premise pairs and were then tested on inferential problems derived from those pairs. In line with prior work, we found improved transitive inference performance after retention across a night of sleep compared with wake alone. Experiment 2 extended these findings using a within-subject design and found superior transitive inference performance on a hierarchy, consolidated across 27 h including sleep compared with just 3 h of wake. In both experiments, consolidation-related improvement was enhanced when presleep learning (i.e., encoding strength) was stronger. We also explored the interaction of these effects with the joint rank effect, in which items were scored according to their rank in the hierarchy, with more dominant item pairs having the lowest scores. Interestingly, the consolidation-related benefit was greatest for more dominant inference pairs (i.e., those with low joint rank scores). Overall, our findings provide further support for the improvement of transitive inference across a consolidation period that includes sleep. We additionally show that encoding strength and joint rank strongly modulate this effect.
Collapse
Affiliation(s)
- Tamas Foldes
- Cardiff University Brain Research Imaging Centre (CUBRIC), Cardiff University, Cardiff, Wales CF24 4HQ, United Kingdom
| | - Lorena Santamaria
- Cardiff University Brain Research Imaging Centre (CUBRIC), Cardiff University, Cardiff, Wales CF24 4HQ, United Kingdom
| | - Penny Lewis
- Cardiff University Brain Research Imaging Centre (CUBRIC), Cardiff University, Cardiff, Wales CF24 4HQ, United Kingdom
| |
Collapse
|
27
|
Chen YY, Areti A, Yoshor D, Foster BL. Individual-specific memory reinstatement patterns within human face-selective cortex. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.08.06.552130. [PMID: 37609262 PMCID: PMC10441346 DOI: 10.1101/2023.08.06.552130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 08/24/2023]
Abstract
Humans have the remarkable ability to vividly retrieve sensory details of past events. According to the theory of sensory reinstatement, during remembering, brain regions involved in the sensory processing of prior events are reactivated to support this perception of the past. Recently, several studies have emphasized potential transformations in the spatial organization of reinstated activity patterns. In particular, studies of scene stimuli suggest a clear anterior shift in the location of retrieval activations compared with those during perception. However, it is not clear that such transformations occur universally, with evidence lacking for other important stimulus categories, particularly faces. Critical to addressing these questions, and to studies of reinstatement more broadly, is the growing importance of considering meaningful variations in the organization of sensory systems across individuals. Therefore, we conducted a multi-session neuroimaging study to first carefully map individual participants face-selective regions within ventral temporal cortex (VTC), followed by a second session to examine the correspondence of activity patterns during face memory encoding and retrieval. Our results showed distinct configurations of face-selective regions within the VTC across individuals. While a significant degree of overlap was observed between face perception and memory encoding, memory retrieval engagement exhibited a more selective and constricted reinstatement pattern within these regions. Importantly, these activity patterns were consistently tied to individual-specific neural substrates, but did not show any consistent direction of spatial transformation (e.g., anteriorization). To provide further insight to these findings, we also report on unique human intracranial recordings from VTC under the same experimental conditions. Our findings highlight the importance of considering individual variations in functional neuroanatomy in the context of assessing the nature of cortical reinstatement. Consideration of such factors will be important for establishing general principles shaping the neural transformations that occur from perception to memory.
Collapse
Affiliation(s)
- Yvonne Y Chen
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA
| | | | - Daniel Yoshor
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA
| | - Brett L Foster
- Department of Neurosurgery, Perelman School of Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, 19104, USA
| |
Collapse
|
28
|
Desbordes T, Lakretz Y, Chanoine V, Oquab M, Badier JM, Trébuchon A, Carron R, Bénar CG, Dehaene S, King JR. Dimensionality and Ramping: Signatures of Sentence Integration in the Dynamics of Brains and Deep Language Models. J Neurosci 2023; 43:5350-5364. [PMID: 37217308 PMCID: PMC10359032 DOI: 10.1523/jneurosci.1163-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/14/2022] [Revised: 02/07/2023] [Accepted: 02/19/2023] [Indexed: 05/24/2023] Open
Abstract
A sentence is more than the sum of its words: its meaning depends on how they combine with one another. The brain mechanisms underlying such semantic composition remain poorly understood. To shed light on the neural vector code underlying semantic composition, we introduce two hypotheses: (1) the intrinsic dimensionality of the space of neural representations should increase as a sentence unfolds, paralleling the growing complexity of its semantic representation; and (2) this progressive integration should be reflected in ramping and sentence-final signals. To test these predictions, we designed a dataset of closely matched normal and jabberwocky sentences (composed of meaningless pseudo words) and displayed them to deep language models and to 11 human participants (5 men and 6 women) monitored with simultaneous MEG and intracranial EEG. In both deep language models and electrophysiological data, we found that representational dimensionality was higher for meaningful sentences than jabberwocky. Furthermore, multivariate decoding of normal versus jabberwocky confirmed three dynamic patterns: (1) a phasic pattern following each word, peaking in temporal and parietal areas; (2) a ramping pattern, characteristic of bilateral inferior and middle frontal gyri; and (3) a sentence-final pattern in left superior frontal gyrus and right orbitofrontal cortex. These results provide a first glimpse into the neural geometry of semantic integration and constrain the search for a neural code of linguistic composition.SIGNIFICANCE STATEMENT Starting from general linguistic concepts, we make two sets of predictions in neural signals evoked by reading multiword sentences. First, the intrinsic dimensionality of the representation should grow with additional meaningful words. Second, the neural dynamics should exhibit signatures of encoding, maintaining, and resolving semantic composition. We successfully validated these hypotheses in deep neural language models, artificial neural networks trained on text and performing very well on many natural language processing tasks. Then, using a unique combination of MEG and intracranial electrodes, we recorded high-resolution brain data from human participants while they read a controlled set of sentences. Time-resolved dimensionality analysis showed increasing dimensionality with meaning, and multivariate decoding allowed us to isolate the three dynamical patterns we had hypothesized.
Collapse
Affiliation(s)
- Théo Desbordes
- Meta AI Research, Paris 75002, France; and Cognitive Neuroimaging Unit NeuroSpin center, 91191, Gif-sur-Yvette, France
| | - Yair Lakretz
- Cognitive Neuroimaging Unit NeuroSpin center, Gif-sur-Yvette, 91191, France
| | - Valérie Chanoine
- Institute of Language, Communication and the Brain, Aix-en-Provence, 13100, France; and Aix-Marseille Université, Centre National de la Recherche Scientifique, LPL, Aix-en-Provence, 13100, France
| | | | - Jean-Michel Badier
- Aix Marseille Université, Institut National de la Santé et de la Recherche Médicale, CNRS, LPL, Aix-en-Provence 13100; and Inst Neurosci Syst, Marseille, 13005, France
| | - Agnès Trébuchon
- Aix Marseille Université, Institut National de la Santé et de la Recherche Médicale, CNRS, LPL, Aix-en-Provence 13100, France; and Inst Neurosci Syst, Marseille, 13005, France; and Assistance Publique Hopitaux de Marseille, Timone hospital, Epileptology and Cerebral Rythmology, Marseille, 13385, France
| | - Romain Carron
- Aix Marseille Université, Institut National de la Santé et de la Recherche Médicale, CNRS, LPL, Aix-en-Provence 13100, France; and Inst Neurosci Syst, Marseille, 13005, France; and Assistance Publique Hopitaux de Marseille, Timone hospital, Functional and Stereotactic Neurosurgery, Marseille, 13385, France
| | - Christian-G Bénar
- Aix Marseille Université, Institut National de la Santé et de la Recherche Médicale, CNRS, LPL, Aix-en-Provence 13100, France; and Inst Neurosci Syst, Marseille, 13005, France
| | - Stanislas Dehaene
- Université Paris Saclay, Institut National de la Santé et de la Recherche Médicale, Commissariat à l'Energie Atomique, Cognitive Neuroimaging Unit, NeuroSpin center, Saclay, 91191, France; and Collège de France, PSL University, Paris, 75231, France
| | - Jean-Rémi King
- Meta AI Research, Paris 75002, France; and Cognitive Neuroimaging Unit NeuroSpin center, 91191, Gif-sur-Yvette, France
- LSP, École normale supérieure, PSL (Paris Sciences & Lettres) University, CNRS, 75005 Paris, France
| |
Collapse
|
29
|
Hahamy A, Dubossarsky H, Behrens TEJ. The human brain reactivates context-specific past information at event boundaries of naturalistic experiences. Nat Neurosci 2023:10.1038/s41593-023-01331-6. [PMID: 37248340 DOI: 10.1038/s41593-023-01331-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/13/2022] [Accepted: 04/13/2023] [Indexed: 05/31/2023]
Abstract
Although we perceive the world in a continuous manner, our experience is partitioned into discrete events. However, to make sense of these events, they must be stitched together into an overarching narrative-a model of unfolding events. It has been proposed that such a stitching process happens in offline neural reactivations when rodents build models of spatial environments. Here we show that, while understanding a natural narrative, humans reactivate neural representations of past events. Similar to offline replay, these reactivations occur in the hippocampus and default mode network, where reactivations are selective to relevant past events. However, these reactivations occur, not during prolonged offline periods, but at the boundaries between ongoing narrative events. These results, replicated across two datasets, suggest reactivations as a candidate mechanism for binding temporally distant information into a coherent understanding of ongoing experience.
Collapse
Affiliation(s)
- Avital Hahamy
- Wellcome Trust Centre for Neuroimaging, UCL Queen Square Institute of Neurology, University College London, London, UK.
| | - Haim Dubossarsky
- School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK
- Language Technology Laboratory, University of Cambridge, Cambridge, UK
| | - Timothy E J Behrens
- Wellcome Trust Centre for Neuroimaging, UCL Queen Square Institute of Neurology, University College London, London, UK
- Wellcome Centre for Integrative Neuroimaging, University of Oxford, John Radcliffe Hospital, Oxford, UK
| |
Collapse
|
30
|
Parra-Barrero E, Cheng S. Learning to predict future locations with internally generated theta sequences. PLoS Comput Biol 2023; 19:e1011101. [PMID: 37172053 DOI: 10.1371/journal.pcbi.1011101] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/16/2022] [Revised: 05/24/2023] [Accepted: 04/13/2023] [Indexed: 05/14/2023] Open
Abstract
Representing past, present and future locations is key for spatial navigation. Indeed, within each cycle of the theta oscillation, the population of hippocampal place cells appears to represent trajectories starting behind the current position of the animal and sweeping ahead of it. In particular, we reported recently that the position represented by CA1 place cells at a given theta phase corresponds to the location where animals were or will be located at a fixed time interval into the past or future assuming the animal ran at its typical, not the current, speed through that part of the environment. This coding scheme leads to longer theta trajectories, larger place fields and shallower phase precession in areas where animals typically run faster. Here we present a mechanistic computational model that accounts for these experimental observations. The model consists of a continuous attractor network with short-term synaptic facilitation and depression that internally generates theta sequences that advance at a fixed pace. Spatial locations are then mapped onto the active units via modified Hebbian plasticity. As a result, neighboring units become associated with spatial locations further apart where animals run faster, reproducing our earlier experimental results. The model also accounts for the higher density of place fields generally observed where animals slow down, such as around rewards. Furthermore, our modeling results reveal that an artifact of the decoding analysis might be partly responsible for the observation that theta trajectories start behind the animal's current position. Overall, our results shed light on how the hippocampal code might arise from the interplay between behavior, sensory input and predefined network dynamics.
Collapse
Affiliation(s)
- Eloy Parra-Barrero
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany
- International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| | - Sen Cheng
- Institute for Neural Computation, Faculty of Computer Science, Ruhr University Bochum, Bochum, Germany
- International Graduate School of Neuroscience, Ruhr University Bochum, Bochum, Germany
| |
Collapse
|
31
|
Nelli S, Braun L, Dumbalska T, Saxe A, Summerfield C. Neural knowledge assembly in humans and neural networks. Neuron 2023; 111:1504-1516.e9. [PMID: 36898375 PMCID: PMC10618408 DOI: 10.1016/j.neuron.2023.02.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Revised: 12/21/2022] [Accepted: 02/09/2023] [Indexed: 03/11/2023]
Abstract
Human understanding of the world can change rapidly when new information comes to light, such as when a plot twist occurs in a work of fiction. This flexible "knowledge assembly" requires few-shot reorganization of neural codes for relations among objects and events. However, existing computational theories are largely silent about how this could occur. Here, participants learned a transitive ordering among novel objects within two distinct contexts before exposure to new knowledge that revealed how they were linked. Blood-oxygen-level-dependent (BOLD) signals in dorsal frontoparietal cortical areas revealed that objects were rapidly and dramatically rearranged on the neural manifold after minimal exposure to linking information. We then adapt online stochastic gradient descent to permit similar rapid knowledge assembly in a neural network model.
Collapse
Affiliation(s)
- Stephanie Nelli
- Department of Cognitive Science, Occidental College, Los Angeles, CA 90041, USA; Department of Experimental Psychology, University of Oxford, Oxford OX2 6GC, UK.
| | - Lukas Braun
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GC, UK
| | | | - Andrew Saxe
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GC, UK; Gatsby Unit & Sainsbury Wellcome Centre, University College London, London W1T 4JG, UK; CIFAR Azrieli Global Scholars Program, CIFAR, Toronto, ON M5G 1M1, Canada
| | | |
Collapse
|
32
|
Zhu SL, Lakshminarasimhan KJ, Angelaki DE. Computational cross-species views of the hippocampal formation. Hippocampus 2023; 33:586-599. [PMID: 37038890 PMCID: PMC10947336 DOI: 10.1002/hipo.23535] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/17/2023] [Accepted: 03/21/2023] [Indexed: 04/12/2023]
Abstract
The discovery of place cells and head direction cells in the hippocampal formation of freely foraging rodents has led to an emphasis of its role in encoding allocentric spatial relationships. In contrast, studies in head-fixed primates have additionally found representations of spatial views. We review recent experiments in freely moving monkeys that expand upon these findings and show that postural variables such as eye/head movements strongly influence neural activity in the hippocampal formation, suggesting that the function of the hippocampus depends on where the animal looks. We interpret these results in the light of recent studies in humans performing challenging navigation tasks which suggest that depending on the context, eye/head movements serve one of two roles-gathering information about the structure of the environment (active sensing) or externalizing the contents of internal beliefs/deliberation (embodied cognition). These findings prompt future experimental investigations into the information carried by signals flowing between the hippocampal formation and the brain regions controlling postural variables, and constitute a basis for updating computational theories of the hippocampal system to accommodate the influence of eye/head movements.
Collapse
Affiliation(s)
- Seren L Zhu
- Center for Neural Science, New York University, New York, New York, USA
| | - Kaushik J Lakshminarasimhan
- Center for Theoretical Neuroscience, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, USA
| | - Dora E Angelaki
- Center for Neural Science, New York University, New York, New York, USA
- Mechanical and Aerospace Engineering, Tandon School of Engineering, New York University, New York, New York, USA
| |
Collapse
|
33
|
Kuga N, Nakayama R, Morikawa S, Yagishita H, Konno D, Shiozaki H, Honjoya N, Ikegaya Y, Sasaki T. Hippocampal sharp wave ripples underlie stress susceptibility in male mice. Nat Commun 2023; 14:2105. [PMID: 37080967 PMCID: PMC10119298 DOI: 10.1038/s41467-023-37736-x] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2022] [Accepted: 03/28/2023] [Indexed: 04/22/2023] Open
Abstract
The ventral hippocampus (vHC) is a core brain region for emotional memory. Here, we examined how the vHC regulates stress susceptibility from the level of gene expression to neuronal population dynamics in male mice. Transcriptome analysis of samples from stress-naïve mice revealed that intrinsic calbindin (Calb1) expression in the vHC is associated with susceptibility to social defeat stress. Mice with Calb1 gene knockdown in the vHC exhibited increased stress resilience and failed to show the increase in the poststress ventral hippocampal sharp wave ripple (SWR) rate. Poststress vHC SWRs triggered synchronous reactivation of stress memory-encoding neuronal ensembles and facilitated information transfer to the amygdala. Suppression of poststress vHC SWRs by real-time feedback stimulation or walking prevented social behavior deficits. Taken together, our results demonstrate that internal reactivation of memories of negative stressful episodes supported by ventral hippocampal SWRs serves as a crucial neurophysiological substrate for determining stress susceptibility.
Collapse
Affiliation(s)
- Nahoko Kuga
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
- Department of Pharmacology, Graduate School of Pharmaceutical Sciences, Tohoku University, 6-3 Aramaki-Aoba, Aoba-Ku, Sendai, 980-8578, Japan
| | - Ryota Nakayama
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Shota Morikawa
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Haruya Yagishita
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
- Department of Pharmacology, Graduate School of Pharmaceutical Sciences, Tohoku University, 6-3 Aramaki-Aoba, Aoba-Ku, Sendai, 980-8578, Japan
| | - Daichi Konno
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
- Laboratory of Geriatric Medicine, Graduate School of Medicine, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Hiromi Shiozaki
- Department of Pharmacology, Graduate School of Pharmaceutical Sciences, Tohoku University, 6-3 Aramaki-Aoba, Aoba-Ku, Sendai, 980-8578, Japan
| | - Natsumi Honjoya
- Department of Pharmacology, Graduate School of Pharmaceutical Sciences, Tohoku University, 6-3 Aramaki-Aoba, Aoba-Ku, Sendai, 980-8578, Japan
| | - Yuji Ikegaya
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
- Center for Information and Neural Networks, 1-4 Yamadaoka, Suita City, Osaka, 565-0871, Japan
- Institute for AI and Beyond, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan
| | - Takuya Sasaki
- Laboratory of Chemical Pharmacology, Graduate School of Pharmaceutical Sciences, The University of Tokyo, 7-3-1 Hongo, Bunkyo-ku, Tokyo, 113-0033, Japan.
- Department of Pharmacology, Graduate School of Pharmaceutical Sciences, Tohoku University, 6-3 Aramaki-Aoba, Aoba-Ku, Sendai, 980-8578, Japan.
| |
Collapse
|
34
|
McFadyen J, Dolan RJ. Spatiotemporal Precision of Neuroimaging in Psychiatry. Biol Psychiatry 2023; 93:671-680. [PMID: 36376110 DOI: 10.1016/j.biopsych.2022.08.016] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/09/2022] [Revised: 07/20/2022] [Accepted: 08/12/2022] [Indexed: 12/23/2022]
Abstract
Aberrant patterns of cognition, perception, and behavior seen in psychiatric disorders are thought to be driven by a complex interplay of neural processes that evolve at a rapid temporal scale. Understanding these dynamic processes in vivo in humans has been hampered by a trade-off between spatial and temporal resolutions inherent to current neuroimaging technology. A recent trend in psychiatric research has been the use of high temporal resolution imaging, particularly magnetoencephalography, often in conjunction with sophisticated machine learning decoding techniques. Developments here promise novel insights into the spatiotemporal dynamics of cognitive phenomena, including domains relevant to psychiatric illnesses such as reward and avoidance learning, memory, and planning. This review considers recent advances afforded by exploiting this increased spatiotemporal precision, with specific reference to applications that seek to drive a mechanistic understanding of psychopathology and the realization of preclinical translation.
Collapse
Affiliation(s)
- Jessica McFadyen
- UCL Max Planck Centre for Computational Psychiatry and Ageing Research and Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom; State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China.
| | - Raymond J Dolan
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
| |
Collapse
|
35
|
Brodt S, Inostroza M, Niethard N, Born J. Sleep-A brain-state serving systems memory consolidation. Neuron 2023; 111:1050-1075. [PMID: 37023710 DOI: 10.1016/j.neuron.2023.03.005] [Citation(s) in RCA: 29] [Impact Index Per Article: 29.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/12/2023] [Revised: 02/23/2023] [Accepted: 03/06/2023] [Indexed: 04/08/2023]
Abstract
Although long-term memory consolidation is supported by sleep, it is unclear how it differs from that during wakefulness. Our review, focusing on recent advances in the field, identifies the repeated replay of neuronal firing patterns as a basic mechanism triggering consolidation during sleep and wakefulness. During sleep, memory replay occurs during slow-wave sleep (SWS) in hippocampal assemblies together with ripples, thalamic spindles, neocortical slow oscillations, and noradrenergic activity. Here, hippocampal replay likely favors the transformation of hippocampus-dependent episodic memory into schema-like neocortical memory. REM sleep following SWS might balance local synaptic rescaling accompanying memory transformation with a sleep-dependent homeostatic process of global synaptic renormalization. Sleep-dependent memory transformation is intensified during early development despite the immaturity of the hippocampus. Overall, beyond its greater efficacy, sleep consolidation differs from wake consolidation mainly in that it is supported, rather than impaired, by spontaneous hippocampal replay activity possibly gating memory formation in neocortex.
Collapse
Affiliation(s)
- Svenja Brodt
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany; Max-Planck-Institute for Biological Cybernetics, Tübingen, Germany
| | - Marion Inostroza
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Niels Niethard
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany
| | - Jan Born
- Institute of Medical Psychology and Behavioral Neurobiology, University of Tübingen, Tübingen, Germany; Werner Reichert Center for Integrative Neuroscience, University of Tübingen, Tübingen, Germany.
| |
Collapse
|
36
|
McFadyen J, Liu Y, Dolan RJ. Differential replay of reward and punishment paths predicts approach and avoidance. Nat Neurosci 2023; 26:627-637. [PMID: 37020116 DOI: 10.1038/s41593-023-01287-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/17/2021] [Accepted: 02/16/2023] [Indexed: 04/07/2023]
Abstract
Neural replay is implicated in planning, where states relevant to a task goal are rapidly reactivated in sequence. It remains unclear whether, during planning, replay relates to an actual prospective choice. Here, using magnetoencephalography (MEG), we studied replay in human participants while they planned to either approach or avoid an uncertain environment containing paths leading to reward or punishment. We find evidence for forward sequential replay during planning, with rapid state-to-state transitions from 20 to 90 ms. Replay of rewarding paths was boosted, relative to aversive paths, before a decision to avoid and attenuated before a decision to approach. A trial-by-trial bias toward replaying prospective punishing paths predicted irrational decisions to approach riskier environments, an effect more pronounced in participants with higher trait anxiety. The findings indicate a coupling of replay with planned behavior, where replay prioritizes an online representation of a worst-case scenario for approaching or avoiding.
Collapse
Affiliation(s)
- Jessica McFadyen
- The UCL Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London, UK.
- Wellcome Centre for Human Neuroimaging, University College London, London, UK.
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China
- Chinese Institute for Brain Research, Beijing, China
| | - Raymond J Dolan
- The UCL Max Planck Centre for Computational Psychiatry and Ageing Research, University College London, London, UK
- Wellcome Centre for Human Neuroimaging, University College London, London, UK
| |
Collapse
|
37
|
Baczkowski BM, Haaker J, Schwabe L. Inferring danger with minimal aversive experience. Trends Cogn Sci 2023; 27:456-467. [PMID: 36941184 DOI: 10.1016/j.tics.2023.02.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/14/2022] [Revised: 01/11/2023] [Accepted: 02/23/2023] [Indexed: 03/22/2023]
Abstract
Learning about threats is crucial for survival and fundamentally rests upon Pavlovian conditioning. However, Pavlovian threat learning is largely limited to detecting known (or similar) threats and involves first-hand exposure to danger, which inevitably poses a risk of harm. We discuss how individuals leverage a rich repertoire of mnemonic processes that operate largely in safety and significantly expand our ability to recognize danger beyond Pavlovian threat associations. These processes result in complementary memories - acquired individually or through social interactions - that represent potential threats and the relational structure of our environment. The interplay between these memories allows danger to be inferred rather than directly learned, thereby flexibly protecting us from potential harm in novel situations despite minimal prior aversive experience.
Collapse
Affiliation(s)
- Blazej M Baczkowski
- Department of Cognitive Psychology, Universität Hamburg, Von-Melle-Park 5, 20146 Hamburg, Germany
| | - Jan Haaker
- Department of Systems Neuroscience, University Medical Center Hamburg-Eppendorf, 20246 Hamburg, Germany
| | - Lars Schwabe
- Department of Cognitive Psychology, Universität Hamburg, Von-Melle-Park 5, 20146 Hamburg, Germany.
| |
Collapse
|
38
|
Zhang L, Pini L, Kim D, Shulman GL, Corbetta M. Spontaneous Activity Patterns in Human Attention Networks Code for Hand Movements. J Neurosci 2023; 43:1976-1986. [PMID: 36788030 PMCID: PMC10027113 DOI: 10.1523/jneurosci.1601-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/17/2022] [Revised: 01/18/2023] [Accepted: 02/03/2023] [Indexed: 02/16/2023] Open
Abstract
Recent evidence suggests that, in the absence of any task, spontaneous brain activity patterns and connectivity in the visual and motor cortex code for natural stimuli and actions, respectively. These "resting-state" activity patterns may underlie the maintenance and consolidation (replay) of information states coding for ecological stimuli and behaviors. In this study, we examine whether replay patterns occur in resting-state activity in association cortex grouped into high-order cognitive networks not directly processing sensory inputs or motor outputs. Fifteen participants (7 females) performed four hand movements during an fMRI study. Three movements were ecological. The fourth movement as control was less ecological. Before and after the task scans, we acquired resting-state fMRI scans. The analysis examined whether multivertex task activation patterns for the four movements computed at the cortical surface in different brain networks resembled spontaneous activity patterns measured at rest. For each movement, we computed a vector of r values indicating the strength of the similarity between the mean task activation pattern and frame-by-frame resting-state patterns. We computed a cumulative distribution function of r 2 values and used the 90th percentile cutoff value for comparison. In the dorsal attention network, resting-state patterns were more likely to match task patterns for the ecological movements than the control movement. In contrast, rest-task pattern correlation was more likely for less ecological movement in the ventral attention network. These findings show that spontaneous activity patterns in human attention networks code for hand movements.SIGNIFICANCE STATEMENT fMRI indirectly measures neural activity noninvasively. Resting-state (spontaneous) fMRI signals measured in the absence of any task resemble signals evoked by task performance both in topography and inter-regional (functional) connectivity. However, the function of spontaneous brain activity is unknown. We recently showed that spatial activity patterns evoked by visual and motor tasks in visual and motor cortex, respectively, occur at rest in the absence of any stimulus or response. Here we show that activity patterns related to hand movements replay at rest in frontoparietal regions of the human attention system. These findings show that spontaneous activity in the human cortex may mediate the maintenance and consolidation of information states coding for ecological stimuli and behaviors.
Collapse
Affiliation(s)
- Lu Zhang
- Padova Neuroscience Center, University of Padova, Padova, 35131, Italy
| | - Lorenzo Pini
- Padova Neuroscience Center, University of Padova, Padova, 35131, Italy
| | - DoHyun Kim
- Departments of Neurology and Radiology, Washington University-St Louis, St Louis, Missouri 63110
| | - Gordon L Shulman
- Departments of Neurology and Radiology, Washington University-St Louis, St Louis, Missouri 63110
| | - Maurizio Corbetta
- Padova Neuroscience Center, University of Padova, Padova, 35131, Italy
- Departments of Neurology and Radiology, Washington University-St Louis, St Louis, Missouri 63110
- Department of Neuroscience, University of Padova, Padova, 35131, Italy
- Venetian Institute of Molecular Medicine, Padova, 35129, Italy
| |
Collapse
|
39
|
Murphy E. ROSE: A Neurocomputational Architecture for Syntax. ARXIV 2023:arXiv:2303.08877v1. [PMID: 36994166 PMCID: PMC10055479] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 03/31/2023]
Abstract
A comprehensive model of natural language processing in the brain must accommodate four components: representations, operations, structures and encoding. It further requires a principled account of how these different components mechanistically, and causally, relate to each another. While previous models have isolated regions of interest for structure-building and lexical access, and have utilized specific neural recording measures to expose possible signatures of syntax, many gaps remain with respect to bridging distinct scales of analysis that map onto these four components. By expanding existing accounts of how neural oscillations can index various linguistic processes, this article proposes a neurocomputational architecture for syntax, termed the ROSE model (Representation, Operation, Structure, Encoding). Under ROSE, the basic data structures of syntax are atomic features, types of mental representations (R), and are coded at the single-unit and ensemble level. Elementary computations (O) that transform these units into manipulable objects accessible to subsequent structure-building levels are coded via high frequency broadband γ activity. Low frequency synchronization and cross-frequency coupling code for recursive categorial inferences (S). Distinct forms of low frequency coupling and phase-amplitude coupling (δ-θ coupling via pSTS-IFG; θ-γ coupling via IFG to conceptual hubs in lateral and ventral temporal cortex) then encode these structures onto distinct workspaces (E). Causally connecting R to O is spike-phase/LFP coupling; connecting O to S is phase-amplitude coupling; connecting S to E is a system of frontotemporal traveling oscillations; connecting E back to lower levels is low-frequency phase resetting of spike-LFP coupling. This compositional neural code has important implications for algorithmic accounts, since it makes concrete predictions for the appropriate level of study for psycholinguistic parsing models. ROSE is reliant on neurophysiologically plausible mechanisms, is supported at all four levels by a range of recent empirical research, and provides an anatomically precise and falsifiable grounding for the basic property of natural language syntax: hierarchical, recursive structure-building.
Collapse
Affiliation(s)
- Elliot Murphy
- Vivian L. Smith Department of Neurosurgery, McGovern Medical School, UTHealth, Houston, TX, USA
- Texas Institute for Restorative Neurotechnologies, UTHealth, Houston, TX, USA
| |
Collapse
|
40
|
Chen ZS, Wilson MA. How our understanding of memory replay evolves. J Neurophysiol 2023; 129:552-580. [PMID: 36752404 PMCID: PMC9988534 DOI: 10.1152/jn.00454.2022] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 01/20/2023] [Accepted: 01/20/2023] [Indexed: 02/09/2023] Open
Abstract
Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.
Collapse
Affiliation(s)
- Zhe Sage Chen
- Department of Psychiatry, New York University Grossman School of Medicine, New York, New York, United States
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, New York, United States
- Neuroscience Institute, New York University Grossman School of Medicine, New York, New York, United States
- Department of Biomedical Engineering, New York University Tandon School of Engineering, Brooklyn, New York, United States
| | - Matthew A Wilson
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
| |
Collapse
|
41
|
Flesch T, Saxe A, Summerfield C. Continual task learning in natural and artificial agents. Trends Neurosci 2023; 46:199-210. [PMID: 36682991 PMCID: PMC10914671 DOI: 10.1016/j.tins.2022.12.006] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/10/2022] [Revised: 12/07/2022] [Accepted: 12/15/2022] [Indexed: 01/21/2023]
Abstract
How do humans and other animals learn new tasks? A wave of brain recording studies has investigated how neural representations change during task learning, with a focus on how tasks can be acquired and coded in ways that minimise mutual interference. We review recent work that has explored the geometry and dimensionality of neural task representations in neocortex, and computational models that have exploited these findings to understand how the brain may partition knowledge between tasks. We discuss how ideas from machine learning, including those that combine supervised and unsupervised learning, are helping neuroscientists understand how natural tasks are learned and coded in biological brains.
Collapse
Affiliation(s)
- Timo Flesch
- Department of Experimental Psychology, University of Oxford, Oxford, UK
| | - Andrew Saxe
- Gatsby Computational Neuroscience Unit & Sainsbury Wellcome Centre, UCL, London, UK.
| | | |
Collapse
|
42
|
Michelmann S, Hasson U, Norman KA. Evidence That Event Boundaries Are Access Points for Memory Retrieval. Psychol Sci 2023; 34:326-344. [PMID: 36595492 PMCID: PMC10152118 DOI: 10.1177/09567976221128206] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/19/2022] [Accepted: 08/30/2022] [Indexed: 01/04/2023] Open
Abstract
When recalling memories, we often scan information-rich continuous episodes, for example, to find our keys. How does our brain access and search through those memories? We suggest that high-level structure, marked by event boundaries, guides us through this process: In our computational model, memory scanning is sped up by skipping ahead to the next event boundary upon reaching a decision threshold. In adult Mechanical Turk workers from the United States, we used a movie (normed for event boundaries; Study 1, N = 203) to prompt memory scanning of movie segments for answers (Study 2, N = 298) and mental simulation (Study 3, N = 100) of these segments. Confirming model predictions, we found that memory-scanning times varied as a function of the number of event boundaries within a segment and the distance of the search target to the previous boundary (the key diagnostic parameter). Mental simulation times were also described by a skipping process with a higher skipping threshold than memory scanning. These findings identify event boundaries as access points to memory.
Collapse
Affiliation(s)
| | - Uri Hasson
- Princeton Neuroscience Institute,
Princeton University
- Department of Psychology, Princeton
University
| | - Kenneth A. Norman
- Princeton Neuroscience Institute,
Princeton University
- Department of Psychology, Princeton
University
| |
Collapse
|
43
|
Kurth-Nelson Z, Behrens T, Wayne G, Miller K, Luettgau L, Dolan R, Liu Y, Schwartenbeck P. Replay and compositional computation. Neuron 2023; 111:454-469. [PMID: 36640765 DOI: 10.1016/j.neuron.2022.12.028] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/11/2022] [Accepted: 12/18/2022] [Indexed: 01/15/2023]
Abstract
Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.
Collapse
Affiliation(s)
- Zeb Kurth-Nelson
- DeepMind, London, UK; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK.
| | - Timothy Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - Kevin Miller
- DeepMind, London, UK; Institute of Ophthalmology, University College London, London, UK
| | - Lennart Luettgau
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Ray Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK; Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Philipp Schwartenbeck
- Max Planck Institute for Biological Cybernetics, Tubingen, Germany; University of Tubingen, Tubingen, Germany
| |
Collapse
|
44
|
Abstract
Research on concepts has focused on categorization. Categorization starts with a stimulus. Equally important are episodes that start with a thought. We engage in thinking to draw out new consequences from stored information, or to work out how to act. Each of the concepts out of which thought is constructed provides access to a large body of stored information. Access is not always just a matter of retrieving a stored belief (semantic memory). Often it depends on running a simulation. Simulation allows conceptual thought to draw on information in special-purpose systems, information stored in special-purpose computational dispositions and special-purpose representational structures. While the utility of simulation, prospection or imagination is widely appreciated, the role of concepts in the process is not well understood. This paper turns to cognitive and computational neuroscience for a model of how simulations enable thinkers to reach novel conclusions. Carried over to conceptual thought, the model suggests that concepts are 'plug & play' devices. The distinctive power of thought-driven simulation derives from the ability of concepts to plug into two kinds of structure at once: the combinatorial structure of a thought at one end and special-purpose structural representations at the other. This article is part of the theme issue 'Concepts in interaction: social engagement and inner experiences'.
Collapse
Affiliation(s)
- Nicholas Shea
- Faculty of Philosophy, University of Oxford, Radcliffe Humanities, Woodstock Road, Oxford OX2 6GG, UK,Institute of Philosophy, University of London School of Advanced Study, Senate House, Malet Street, London WC1E 7HU, UK
| |
Collapse
|
45
|
Wimmer GE, Liu Y, McNamee DC, Dolan RJ. Distinct replay signatures for prospective decision-making and memory preservation. Proc Natl Acad Sci U S A 2023; 120:e2205211120. [PMID: 36719914 PMCID: PMC9963918 DOI: 10.1073/pnas.2205211120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 12/05/2022] [Indexed: 02/01/2023] Open
Abstract
Theories of neural replay propose that it supports a range of functions, most prominently planning and memory consolidation. Here, we test the hypothesis that distinct signatures of replay in the same task are related to model-based decision-making ("planning") and memory preservation. We designed a reward learning task wherein participants utilized structure knowledge for model-based evaluation, while at the same time had to maintain knowledge of two independent and randomly alternating task environments. Using magnetoencephalography and multivariate analysis, we first identified temporally compressed sequential reactivation, or replay, both prior to choice and following reward feedback. Before choice, prospective replay strength was enhanced for the current task-relevant environment when a model-based planning strategy was beneficial. Following reward receipt, and consistent with a memory preservation role, replay for the alternative distal task environment was enhanced as a function of decreasing recency of experience with that environment. Critically, these planning and memory preservation relationships were selective to pre-choice and post-feedback periods, respectively. Our results provide support for key theoretical proposals regarding the functional role of replay and demonstrate that the relative strength of planning and memory-related signals are modulated by ongoing computational and task demands.
Collapse
Affiliation(s)
- G. Elliott Wimmer
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
- Chinese Institute for Brain Research, Beijing100875, China
| | - Daniel C. McNamee
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
- Neuroscience Programme, Champalimaud Research, Lisbon1400-038, Portugal
| | - Raymond J. Dolan
- Max Planck University College London Centre for Computational Psychiatry and Ageing Research, University College London, LondonWC1B 5EH, UK
- Wellcome Centre for Human Neuroimaging, University College London, LondonWC1N 3BG, UK
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing100875, China
| |
Collapse
|
46
|
Aru J, Drüke M, Pikamäe J, Larkum ME. Mental navigation and the neural mechanisms of insight. Trends Neurosci 2023; 46:100-109. [PMID: 36462993 DOI: 10.1016/j.tins.2022.11.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2022] [Revised: 11/01/2022] [Accepted: 11/07/2022] [Indexed: 12/03/2022]
Abstract
How do new ideas come about? The central hypothesis presented here states that insights might happen during mental navigation and correspond to rapid plasticity at the cellular level. We highlight the differences between neocortical and hippocampal mechanisms of insight. We argue that the suddenness of insight can be related to the sudden emergence of place fields in the hippocampus. According to our hypothesis, insights are supported by a state of mind-wandering that can be tied to the process of combining knowledge pieces during sharp-wave ripples (SWRs). Our framework connects the dots between research on creativity, mental navigation, and specific synaptic plasticity mechanisms in the hippocampus.
Collapse
Affiliation(s)
- Jaan Aru
- Institute of Computer Science, University of Tartu, Tartu, Estonia.
| | - Moritz Drüke
- Institute of Biology, Humboldt University Berlin, Berlin, Germany
| | - Juhan Pikamäe
- Institute of Molecular and Cell Biology, University of Tartu, Tartu, Estonia
| | - Matthew E Larkum
- Institute of Biology, Humboldt University Berlin, Berlin, Germany; Neurocure Center for Excellence, Charité Universitätsmedizin Berlin, Berlin, Germany
| |
Collapse
|
47
|
Momennejad I. A rubric for human-like agents and NeuroAI. Philos Trans R Soc Lond B Biol Sci 2023; 378:20210446. [PMID: 36511409 PMCID: PMC9745874 DOI: 10.1098/rstb.2021.0446] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/17/2022] [Accepted: 10/27/2022] [Indexed: 12/15/2022] Open
Abstract
Researchers across cognitive, neuro- and computer sciences increasingly reference 'human-like' artificial intelligence and 'neuroAI'. However, the scope and use of the terms are often inconsistent. Contributed research ranges widely from mimicking behaviour, to testing machine learning methods as neurally plausible hypotheses at the cellular or functional levels, or solving engineering problems. However, it cannot be assumed nor expected that progress on one of these three goals will automatically translate to progress in others. Here, a simple rubric is proposed to clarify the scope of individual contributions, grounded in their commitments to human-like behaviour, neural plausibility or benchmark/engineering/computer science goals. This is clarified using examples of weak and strong neuroAI and human-like agents, and discussing the generative, corroborate and corrective ways in which the three dimensions interact with one another. The author maintains that future progress in artificial intelligence will need strong interactions across the disciplines, with iterative feedback loops and meticulous validity tests-leading to both known and yet-unknown advances that may span decades to come. This article is part of a discussion meeting issue 'New approaches to 3D vision'.
Collapse
Affiliation(s)
- Ida Momennejad
- Microsoft Research NYC, Reinforcement Learning Station, 300 Lafayette, New York, NY 10012, USA
| |
Collapse
|
48
|
Palenciano AF, Senoussi M, Formica S, González-García C. Canonical template tracking: Measuring the activation state of specific neural representations. FRONTIERS IN NEUROIMAGING 2023; 1:974927. [PMID: 37555182 PMCID: PMC10406196 DOI: 10.3389/fnimg.2022.974927] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/21/2022] [Accepted: 12/13/2022] [Indexed: 08/10/2023]
Abstract
Multivariate analyses of neural data have become increasingly influential in cognitive neuroscience since they allow to address questions about the representational signatures of neurocognitive phenomena. Here, we describe Canonical Template Tracking: a multivariate approach that employs independent localizer tasks to assess the activation state of specific representations during the execution of cognitive paradigms. We illustrate the benefits of this methodology in characterizing the particular content and format of task-induced representations, comparing it with standard (cross-)decoding and representational similarity analyses. Then, we discuss relevant design decisions for experiments using this analysis approach, focusing on the nature of the localizer tasks from which the canonical templates are derived. We further provide a step-by-step tutorial of this method, stressing the relevant analysis choices for functional magnetic resonance imaging and magneto/electroencephalography data. Importantly, we point out the potential pitfalls linked to canonical template tracking implementation and interpretation of the results, together with recommendations to mitigate them. To conclude, we provide some examples from previous literature that highlight the potential of this analysis to address relevant theoretical questions in cognitive neuroscience.
Collapse
Affiliation(s)
- Ana F. Palenciano
- Mind, Brain, and Behavior Research Center, University of Granada, Granada, Spain
| | - Mehdi Senoussi
- CLLE Lab, CNRS UMR 5263, University of Toulouse, Toulouse, France
- Department of Experimental Psychology, Ghent University, Ghent, Belgium
| | - Silvia Formica
- Department of Psychology, Berlin School of Mind and Brain, Humboldt Universität zu Berlin, Berlin, Germany
| | | |
Collapse
|
49
|
Roüast NM, Schönauer M. Continuously changing memories: a framework for proactive and non-linear consolidation. Trends Neurosci 2023; 46:8-19. [PMID: 36428193 DOI: 10.1016/j.tins.2022.10.013] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2022] [Revised: 10/10/2022] [Accepted: 10/27/2022] [Indexed: 11/23/2022]
Abstract
The traditional view of long-term memory is that memory traces mature in a predetermined 'linear' process: their neural substrate shifts from rapidly plastic medial temporal regions towards stable neocortical networks. We propose that memories remain malleable, not by repeated reinstantiations of this linear process but instead via dynamic routes of proactive and non-linear consolidation: memories change, their trajectory is flexible and reversible, and their physical basis develops continuously according to anticipated demands. Studies demonstrating memory updating, increasing hippocampal dependence to support adaptive use, and rapid neocortical plasticity provide evidence for continued non-linear consolidation. Although anticipated demand can affect all stages of memory formation, the extent to which it shapes the physical memory trace repeatedly and proactively will require further dedicated research.
Collapse
Affiliation(s)
- Nora Malika Roüast
- Institute for Psychology, Neuropsychology, University of Freiburg, Freiburg, Germany.
| | - Monika Schönauer
- Institute for Psychology, Neuropsychology, University of Freiburg, Freiburg, Germany.
| |
Collapse
|
50
|
Emanuel A, Eldar E. Emotions as computations. Neurosci Biobehav Rev 2023; 144:104977. [PMID: 36435390 PMCID: PMC9805532 DOI: 10.1016/j.neubiorev.2022.104977] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/14/2022] [Revised: 10/26/2022] [Accepted: 11/22/2022] [Indexed: 11/26/2022]
Abstract
Emotions ubiquitously impact action, learning, and perception, yet their essence and role remain widely debated. Computational accounts of emotion aspire to answer these questions with greater conceptual precision informed by normative principles and neurobiological data. We examine recent progress in this regard and find that emotions may implement three classes of computations, which serve to evaluate states, actions, and uncertain prospects. For each of these, we use the formalism of reinforcement learning to offer a new formulation that better accounts for existing evidence. We then consider how these distinct computations may map onto distinct emotions and moods. Integrating extensive research on the causes and consequences of different emotions suggests a parsimonious one-to-one mapping, according to which emotions are integral to how we evaluate outcomes (pleasure & pain), learn to predict them (happiness & sadness), use them to inform our (frustration & content) and others' (anger & gratitude) actions, and plan in order to realize (desire & hope) or avoid (fear & anxiety) uncertain outcomes.
Collapse
Affiliation(s)
- Aviv Emanuel
- Department of Psychology, Hebrew University of Jerusalem, Jerusalem 9190501, Israel; Department of Cognitive and Brain Sciences, Hebrew University of Jerusalem, Jerusalem 9190501, Israel.
| | - Eran Eldar
- Department of Psychology, Hebrew University of Jerusalem, Jerusalem 9190501, Israel; Department of Cognitive and Brain Sciences, Hebrew University of Jerusalem, Jerusalem 9190501, Israel.
| |
Collapse
|