1
|
Zárate-Rochín AM. Contemporary neurocognitive models of memory: A descriptive comparative analysis. Neuropsychologia 2024; 196:108846. [PMID: 38430963 DOI: 10.1016/j.neuropsychologia.2024.108846] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2023] [Revised: 02/27/2024] [Accepted: 02/27/2024] [Indexed: 03/05/2024]
Abstract
The great complexity involved in the study of memory has given rise to numerous hypotheses and models associated with various phenomena at different levels of analysis. This has allowed us to delve deeper in our knowledge about memory but has also made it difficult to synthesize and integrate data from different lines of research. In this context, this work presents a descriptive comparative analysis of contemporary models that address the structure and function of multiple memory systems. The main goal is to outline a panoramic view of the key elements that constitute these models in order to visualize both the current state of research and possible future directions. The elements that stand out from different levels of analysis are distributed neural networks, hierarchical organization, predictive coding, homeostasis, and evolutionary perspective.
Collapse
Affiliation(s)
- Alba Marcela Zárate-Rochín
- Instituto de Investigaciones Cerebrales, Universidad Veracruzana, Dr. Castelazo Ayala s/n, Industrial Animas, 91190, Xalapa-Enríquez, Veracruz, Mexico.
| |
Collapse
|
2
|
McNamee DC. The generative neural microdynamics of cognitive processing. Curr Opin Neurobiol 2024; 85:102855. [PMID: 38428170 DOI: 10.1016/j.conb.2024.102855] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2023] [Revised: 02/06/2024] [Accepted: 02/07/2024] [Indexed: 03/03/2024]
Abstract
The entorhinal cortex and hippocampus form a recurrent network that informs many cognitive processes, including memory, planning, navigation, and imagination. Neural recordings from these regions reveal spatially organized population codes corresponding to external environments and abstract spaces. Aligning the former cognitive functionalities with the latter neural phenomena is a central challenge in understanding the entorhinal-hippocampal circuit (EHC). Disparate experiments demonstrate a surprising level of complexity and apparent disorder in the intricate spatiotemporal dynamics of sequential non-local hippocampal reactivations, which occur particularly, though not exclusively, during immobile pauses and rest. We review these phenomena with a particular focus on their apparent lack of physical simulative realism. These observations are then integrated within a theoretical framework and proposed neural circuit mechanisms that normatively characterize this neural complexity by conceiving different regimes of hippocampal microdynamics as neuromarkers of diverse cognitive computations.
Collapse
|
3
|
Pezzulo G, D'Amato L, Mannella F, Priorelli M, Van de Maele T, Stoianov IP, Friston K. Neural representation in active inference: Using generative models to interact with-and understand-the lived world. Ann N Y Acad Sci 2024; 1534:45-68. [PMID: 38528782 DOI: 10.1111/nyas.15118] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/27/2024]
Abstract
This paper considers neural representation through the lens of active inference, a normative framework for understanding brain function. It delves into how living organisms employ generative models to minimize the discrepancy between predictions and observations (as scored with variational free energy). The ensuing analysis suggests that the brain learns generative models to navigate the world adaptively, not (or not solely) to understand it. Different living organisms may possess an array of generative models, spanning from those that support action-perception cycles to those that underwrite planning and imagination; namely, from explicit models that entail variables for predicting concurrent sensations, like objects, faces, or people-to action-oriented models that predict action outcomes. It then elucidates how generative models and belief dynamics might link to neural representation and the implications of different types of generative models for understanding an agent's cognitive capabilities in relation to its ecological niche. The paper concludes with open questions regarding the evolution of generative models and the development of advanced cognitive abilities-and the gradual transition from pragmatic to detached neural representations. The analysis on offer foregrounds the diverse roles that generative models play in cognitive processes and the evolution of neural representation.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Leo D'Amato
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
- Polytechnic University of Turin, Turin, Italy
| | - Francesco Mannella
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Matteo Priorelli
- Institute of Cognitive Sciences and Technologies, National Research Council, Padua, Italy
| | - Toon Van de Maele
- IDLab, Department of Information Technology, Ghent University - imec, Ghent, Belgium
| | - Ivilin Peev Stoianov
- Institute of Cognitive Sciences and Technologies, National Research Council, Padua, Italy
| | - Karl Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, UK
- VERSES Research Lab, Los Angeles, California, USA
| |
Collapse
|
4
|
Dragoi G. The generative grammar of the brain: a critique of internally generated representations. Nat Rev Neurosci 2024; 25:60-75. [PMID: 38036709 DOI: 10.1038/s41583-023-00763-0] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 10/18/2023] [Indexed: 12/02/2023]
Abstract
The past decade of progress in neurobiology has uncovered important organizational principles for network preconfiguration and neuronal selection that suggest a generative grammar exists in the brain. In this Perspective, I discuss the competence of the hippocampal neural network to generically express temporally compressed sequences of neuronal firing that represent novel experiences, which is envisioned as a form of generative neural syntax supporting a neurobiological perspective on brain function. I compare this neural competence with the hippocampal network performance that represents specific experiences with higher fidelity after new learning during replay, which is envisioned as a form of neural semantic that supports a complementary neuropsychological perspective. I also demonstrate how the syntax of network competence emerges a priori during early postnatal life and is followed by the later development of network performance that enables rapid encoding and memory consolidation. Thus, I propose that this generative grammar of the brain is essential for internally generated representations, which are crucial for the cognitive processes underlying learning and memory, prospection, and inference, which ultimately underlie our reason and representation of the world.
Collapse
Affiliation(s)
- George Dragoi
- Department of Psychiatry, Yale University School of Medicine, New Haven, CT, USA.
- Department of Neuroscience, Yale University School of Medicine, New Haven, CT, USA.
- Wu Tsai Institute, Yale University, New Haven, CT, USA.
| |
Collapse
|
5
|
Coulter ME, Kemere C. The neural basis of mental navigation in rats. Science 2023; 382:517-518. [PMID: 37917674 DOI: 10.1126/science.adl0806] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/04/2023]
Abstract
A brain-machine interface demonstrates volitional control of hippocampal activity.
Collapse
|
6
|
Shaffer C, Barrett LF, Quigley KS. Signal processing in the vagus nerve: Hypotheses based on new genetic and anatomical evidence. Biol Psychol 2023; 182:108626. [PMID: 37419401 PMCID: PMC10563766 DOI: 10.1016/j.biopsycho.2023.108626] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/09/2023] [Revised: 06/25/2023] [Accepted: 07/03/2023] [Indexed: 07/09/2023]
Abstract
Each organism must regulate its internal state in a metabolically efficient way as it interacts in space and time with an ever-changing and only partly predictable world. Success in this endeavor is largely determined by the ongoing communication between brain and body, and the vagus nerve is a crucial structure in that dialogue. In this review, we introduce the novel hypothesis that the afferent vagus nerve is engaged in signal processing rather than just signal relay. New genetic and structural evidence of vagal afferent fiber anatomy motivates two hypotheses: (1) that sensory signals informing on the physiological state of the body compute both spatial and temporal viscerosensory features as they ascend the vagus nerve, following patterns found in other sensory architectures, such as the visual and olfactory systems; and (2) that ascending and descending signals modulate one another, calling into question the strict segregation of sensory and motor signals, respectively. Finally, we discuss several implications of our two hypotheses for understanding the role of viscerosensory signal processing in predictive energy regulation (i.e., allostasis) as well as the role of metabolic signals in memory and in disorders of prediction (e.g., mood disorders).
Collapse
Affiliation(s)
- Clare Shaffer
- Department of Psychology, College of Science, Northeastern University, Boston, MA, USA.
| | - Lisa Feldman Barrett
- Department of Psychology, College of Science, Northeastern University, Boston, MA, USA; Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Charlestown, MA, USA
| | - Karen S Quigley
- Department of Psychology, College of Science, Northeastern University, Boston, MA, USA.
| |
Collapse
|
7
|
Parr T, Holmes E, Friston KJ, Pezzulo G. Cognitive effort and active inference. Neuropsychologia 2023; 184:108562. [PMID: 37080424 PMCID: PMC10636588 DOI: 10.1016/j.neuropsychologia.2023.108562] [Citation(s) in RCA: 5] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/11/2022] [Revised: 04/03/2023] [Accepted: 04/11/2023] [Indexed: 04/22/2023]
Abstract
This paper aims to integrate some key constructs in the cognitive neuroscience of cognitive control and executive function by formalising the notion of cognitive (or mental) effort in terms of active inference. To do so, we call upon a task used in neuropsychology to assess impulse inhibition-a Stroop task. In this task, participants must suppress the impulse to read a colour word and instead report the colour of the text of the word. The Stroop task is characteristically effortful, and we unpack a theory of mental effort in which, to perform this task accurately, participants must overcome prior beliefs about how they would normally act. However, our interest here is not in overt action, but in covert (mental) action. Mental actions change our beliefs but have no (direct) effect on the outside world-much like deploying covert attention. This account of effort as mental action lets us generate multimodal (choice, reaction time, and electrophysiological) data of the sort we might expect from a human participant engaging in this task. We analyse how parameters determining cognitive effort influence simulated responses and demonstrate that-when provided only with performance data-these parameters can be recovered, provided they are within a certain range.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, UK.
| | - Emma Holmes
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, UK
| | - Karl J Friston
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, UK
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| |
Collapse
|
8
|
Katsumi Y, Zhang J, Chen D, Kamona N, Bunce JG, Hutchinson JB, Yarossi M, Tunik E, Dickerson BC, Quigley KS, Barrett LF. Correspondence of functional connectivity gradients across human isocortex, cerebellum, and hippocampus. Commun Biol 2023; 6:401. [PMID: 37046050 PMCID: PMC10097701 DOI: 10.1038/s42003-023-04796-0] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/26/2021] [Accepted: 04/03/2023] [Indexed: 04/14/2023] Open
Abstract
Gradient mapping is an important technique to summarize high dimensional biological features as low dimensional manifold representations in exploring brain structure-function relationships at various levels of the cerebral cortex. While recent studies have characterized the major gradients of functional connectivity in several brain structures using this technique, very few have systematically examined the correspondence of such gradients across structures under a common systems-level framework. Using resting-state functional magnetic resonance imaging, here we show that the organizing principles of the isocortex, and those of the cerebellum and hippocampus in relation to the isocortex, can be described using two common functional gradients. We suggest that the similarity in functional connectivity gradients across these structures can be meaningfully interpreted within a common computational framework based on the principles of predictive processing. The present results, and the specific hypotheses that they suggest, represent an important step toward an integrative account of brain function.
Collapse
Affiliation(s)
- Yuta Katsumi
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, 02114, USA.
| | - Jiahe Zhang
- Department of Psychology, Northeastern University, Boston, MA, 02115, USA
| | - Danlei Chen
- Department of Psychology, Northeastern University, Boston, MA, 02115, USA
| | - Nada Kamona
- Department of Psychology, Northeastern University, Boston, MA, 02115, USA
| | - Jamie G Bunce
- Department of Biology, Northeastern University, Boston, MA, 02115, USA
| | | | - Mathew Yarossi
- Department of Electrical and Computer Engineering, Northeastern University, Boston, MA, 02115, USA
- Department of Physical Therapy, Movement, and Rehabilitation Science, Northeastern University, Boston, MA, 02115, USA
| | - Eugene Tunik
- Department of Physical Therapy, Movement, and Rehabilitation Science, Northeastern University, Boston, MA, 02115, USA
| | - Bradford C Dickerson
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, 02114, USA
- Department of Psychiatry, Massachusetts General Hospital and Harvard Medical School, Boston, MA, 02114, USA
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Boston, MA, 02114, USA
| | - Karen S Quigley
- Department of Psychology, Northeastern University, Boston, MA, 02115, USA
| | - Lisa Feldman Barrett
- Department of Psychology, Northeastern University, Boston, MA, 02115, USA
- Department of Psychiatry, Massachusetts General Hospital and Harvard Medical School, Boston, MA, 02114, USA
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Boston, MA, 02114, USA
| |
Collapse
|
9
|
Chen ZS, Wilson MA. How our understanding of memory replay evolves. J Neurophysiol 2023; 129:552-580. [PMID: 36752404 PMCID: PMC9988534 DOI: 10.1152/jn.00454.2022] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Revised: 01/20/2023] [Accepted: 01/20/2023] [Indexed: 02/09/2023] Open
Abstract
Memory reactivations and replay, widely reported in the hippocampus and cortex across species, have been implicated in memory consolidation, planning, and spatial and skill learning. Technological advances in electrophysiology, calcium imaging, and human neuroimaging techniques have enabled neuroscientists to measure large-scale neural activity with increasing spatiotemporal resolution and have provided opportunities for developing robust analytic methods to identify memory replay. In this article, we first review a large body of historically important and representative memory replay studies from the animal and human literature. We then discuss our current understanding of memory replay functions in learning, planning, and memory consolidation and further discuss the progress in computational modeling that has contributed to these improvements. Next, we review past and present analytic methods for replay analyses and discuss their limitations and challenges. Finally, looking ahead, we discuss some promising analytic methods for detecting nonstereotypical, behaviorally nondecodable structures from large-scale neural recordings. We argue that seamless integration of multisite recordings, real-time replay decoding, and closed-loop manipulation experiments will be essential for delineating the role of memory replay in a wide range of cognitive and motor functions.
Collapse
Affiliation(s)
- Zhe Sage Chen
- Department of Psychiatry, New York University Grossman School of Medicine, New York, New York, United States
- Department of Neuroscience and Physiology, New York University Grossman School of Medicine, New York, New York, United States
- Neuroscience Institute, New York University Grossman School of Medicine, New York, New York, United States
- Department of Biomedical Engineering, New York University Tandon School of Engineering, Brooklyn, New York, United States
| | - Matthew A Wilson
- Department of Brain and Cognitive Sciences, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
- Picower Institute for Learning and Memory, Massachusetts Institute of Technology, Cambridge, Massachusetts, United States
| |
Collapse
|
10
|
Rens N, Lancia GL, Eluchans M, Schwartenbeck P, Cunnington R, Pezzulo G. Evidence for entropy maximisation in human free choice behaviour. Cognition 2023; 232:105328. [PMID: 36463639 DOI: 10.1016/j.cognition.2022.105328] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/06/2021] [Revised: 11/10/2022] [Accepted: 11/12/2022] [Indexed: 12/05/2022]
Abstract
The freedom to choose between options is strongly linked to notions of free will. Accordingly, several studies have shown that individuals demonstrate a preference for choice, or the availability of multiple options, over and above utilitarian value. Yet we lack a decision-making framework that integrates preference for choice with traditional utility maximisation in free choice behaviour. Here we test the predictions of an inference-based model of decision-making in which an agent actively seeks states yielding entropy (availability of options) in addition to utility (economic reward). We designed a study in which participants freely navigated a virtual environment consisting of two consecutive choices leading to reward locations in separate rooms. Critically, the choice of one room always led to two final doors while, in the second room, only one door was permissible to choose. This design allowed us to separately determine the influence of utility and entropy on participants' choice behaviour and their self-evaluation of free will. We found that choice behaviour was better predicted by an inference-based model than by expected utility alone, and that both the availability of options and the value of the context positively influenced participants' perceived freedom of choice. Moreover, this consideration of options was apparent in the ongoing motion dynamics as individuals navigated the environment. In a second study, in which participants selected between rooms that gave access to three or four doors, we observed a similar pattern of results, with participants preferring the room that gave access to more options and feeling freer in it. These results suggest that free choice behaviour is well explained by an inference-based framework in which both utility and entropy are optimised and supports the idea that the feeling of having free will is tightly related to options availability.
Collapse
Affiliation(s)
- Natalie Rens
- Queensland Brain Institute, The University of Queensland, St Lucia, Queensland 4072, Australia
| | - Gian Luca Lancia
- Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia, 44, 00185 Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Mattia Eluchans
- Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia, 44, 00185 Rome, Italy; University of Rome "La Sapienza", Rome, Italy
| | - Philipp Schwartenbeck
- Wellcome Centre for Human Neuroimaging, University College London, London, United Kingdom; Oxford Centre for Functional MRI of the Brain, Nuffield Department of Clinical Neurosciences, University of Oxford, Oxford, United Kingdom; University of Tübingen, Tübingen, Germany; Max Planck Institute for Biological Cybernetics, Tübingen, Baden-Württemberg, Germany
| | - Ross Cunnington
- School of Psychology, The University of Queensland, St Lucia, Queensland 4072, Australia
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Via S. Martino della Battaglia, 44, 00185 Rome, Italy.
| |
Collapse
|
11
|
Comrie AE, Frank LM, Kay K. Imagination as a fundamental function of the hippocampus. Philos Trans R Soc Lond B Biol Sci 2022; 377:20210336. [PMID: 36314152 PMCID: PMC9620759 DOI: 10.1098/rstb.2021.0336] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Accepted: 04/20/2022] [Indexed: 08/25/2023] Open
Abstract
Imagination is a biological function that is vital to human experience and advanced cognition. Despite this importance, it remains unknown how imagination is realized in the brain. Substantial research focusing on the hippocampus, a brain structure traditionally linked to memory, indicates that firing patterns in spatially tuned neurons can represent previous and upcoming paths in space. This work has generally been interpreted under standard views that the hippocampus implements cognitive abilities primarily related to actual experience, whether in the past (e.g. recollection, consolidation), present (e.g. spatial mapping) or future (e.g. planning). However, relatively recent findings in rodents identify robust patterns of hippocampal firing corresponding to a variety of alternatives to actual experience, in many cases without overt reference to the past, present or future. Given these findings, and others on hippocampal contributions to human imagination, we suggest that a fundamental function of the hippocampus is to generate a wealth of hypothetical experiences and thoughts. Under this view, traditional accounts of hippocampal function in episodic memory and spatial navigation can be understood as particular applications of a more general system for imagination. This view also suggests that the hippocampus contributes to a wider range of cognitive abilities than previously thought. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.
Collapse
Affiliation(s)
- Alison E. Comrie
- Neuroscience Graduate Program, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Kavli Institute for Fundamental Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Center for Integrative Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Departments of Physiology and Psychiatry, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
| | - Loren M. Frank
- Kavli Institute for Fundamental Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Center for Integrative Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Departments of Physiology and Psychiatry, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Howard Hughes Medical Institute, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
| | - Kenneth Kay
- Zuckerman Institute, Center for Theoretical Neuroscience, Columbia University, 3227 Broadway, New York, NY 10027, USA
| |
Collapse
|
12
|
Katsumi Y, Theriault JE, Quigley KS, Barrett LF. Allostasis as a core feature of hierarchical gradients in the human brain. Netw Neurosci 2022; 6:1010-1031. [PMID: 38800458 PMCID: PMC11117115 DOI: 10.1162/netn_a_00240] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/01/2021] [Accepted: 02/11/2022] [Indexed: 05/29/2024] Open
Abstract
This paper integrates emerging evidence from two broad streams of scientific literature into one common framework: (a) hierarchical gradients of functional connectivity that reflect the brain's large-scale structural architecture (e.g., a lamination gradient in the cerebral cortex); and (b) approaches to predictive processing and one of its specific instantiations called allostasis (i.e., the predictive regulation of energetic resources in the service of coordinating the body's internal systems). This synthesis begins to sketch a coherent, neurobiologically inspired framework suggesting that predictive energy regulation is at the core of human brain function, and by extension, psychological and behavioral phenomena, providing a shared vocabulary for theory building and knowledge accumulation.
Collapse
Affiliation(s)
- Yuta Katsumi
- Department of Neurology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| | | | - Karen S. Quigley
- Department of Psychology, Northeastern University, Boston, MA, USA
| | - Lisa Feldman Barrett
- Department of Psychology, Northeastern University, Boston, MA, USA
- Department of Psychiatry, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
- Athinoula A. Martinos Center for Biomedical Imaging, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA
| |
Collapse
|
13
|
Stoianov I, Maisto D, Pezzulo G. The hippocampal formation as a hierarchical generative model supporting generative replay and continual learning. Prog Neurobiol 2022; 217:102329. [PMID: 35870678 DOI: 10.1016/j.pneurobio.2022.102329] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/09/2021] [Revised: 07/15/2022] [Accepted: 07/19/2022] [Indexed: 11/28/2022]
Abstract
We advance a novel computational theory of the hippocampal formation as a hierarchical generative model that organizes sequential experiences, such as rodent trajectories during spatial navigation, into coherent spatiotemporal contexts. We propose that the hippocampal generative model is endowed with inductive biases to identify individual items of experience (first hierarchical layer), organize them into sequences (second layer) and cluster them into maps (third layer). This theory entails a novel characterization of hippocampal reactivations as generative replay: the offline resampling of fictive sequences from the generative model, which supports the continual learning of multiple sequential experiences. We show that the model learns and efficiently retains multiple spatial navigation trajectories, by organizing them into spatial maps. Furthermore, the model reproduces flexible and prospective aspects of hippocampal dynamics that are challenging to explain within existing frameworks. This theory reconciles multiple roles of the hippocampal formation in map-based navigation, episodic memory and imagination.
Collapse
Affiliation(s)
- Ivilin Stoianov
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Domenico Maisto
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| |
Collapse
|
14
|
Massi E, Barthélemy J, Mailly J, Dromnelle R, Canitrot J, Poniatowski E, Girard B, Khamassi M. Model-Based and Model-Free Replay Mechanisms for Reinforcement Learning in Neurorobotics. Front Neurorobot 2022; 16:864380. [PMID: 35812782 PMCID: PMC9263850 DOI: 10.3389/fnbot.2022.864380] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/28/2022] [Accepted: 05/05/2022] [Indexed: 11/22/2022] Open
Abstract
Experience replay is widely used in AI to bootstrap reinforcement learning (RL) by enabling an agent to remember and reuse past experiences. Classical techniques include shuffled-, reversed-ordered- and prioritized-memory buffers, which have different properties and advantages depending on the nature of the data and problem. Interestingly, recent computational neuroscience work has shown that these techniques are relevant to model hippocampal reactivations recorded during rodent navigation. Nevertheless, the brain mechanisms for orchestrating hippocampal replay are still unclear. In this paper, we present recent neurorobotics research aiming to endow a navigating robot with a neuro-inspired RL architecture (including different learning strategies, such as model-based (MB) and model-free (MF), and different replay techniques). We illustrate through a series of numerical simulations how the specificities of robotic experimentation (e.g., autonomous state decomposition by the robot, noisy perception, state transition uncertainty, non-stationarity) can shed new lights on which replay techniques turn out to be more efficient in different situations. Finally, we close the loop by raising new hypotheses for neuroscience from such robotic models of hippocampal replay.
Collapse
|
15
|
Parr T, Pezzulo G. Understanding, Explanation, and Active Inference. Front Syst Neurosci 2021; 15:772641. [PMID: 34803619 PMCID: PMC8602880 DOI: 10.3389/fnsys.2021.772641] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/08/2021] [Accepted: 10/15/2021] [Indexed: 11/13/2022] Open
Abstract
While machine learning techniques have been transformative in solving a range of problems, an important challenge is to understand why they arrive at the decisions they output. Some have argued that this necessitates augmenting machine intelligence with understanding such that, when queried, a machine is able to explain its behaviour (i.e., explainable AI). In this article, we address the issue of machine understanding from the perspective of active inference. This paradigm enables decision making based upon a model of how data are generated. The generative model contains those variables required to explain sensory data, and its inversion may be seen as an attempt to explain the causes of these data. Here we are interested in explanations of one's own actions. This implies a deep generative model that includes a model of the world, used to infer policies, and a higher-level model that attempts to predict which policies will be selected based upon a space of hypothetical (i.e., counterfactual) explanations-and which can subsequently be used to provide (retrospective) explanations about the policies pursued. We illustrate the construct validity of this notion of understanding in relation to human understanding by highlighting the similarities in computational architecture and the consequences of its dysfunction.
Collapse
Affiliation(s)
- Thomas Parr
- Wellcome Centre for Human Neuroimaging, Queen Square Institute of Neurology, University College London, London, United Kingdom
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| |
Collapse
|
16
|
Pezzulo G, Zorzi M, Corbetta M. The secret life of predictive brains: what's spontaneous activity for? Trends Cogn Sci 2021; 25:730-743. [PMID: 34144895 PMCID: PMC8363551 DOI: 10.1016/j.tics.2021.05.007] [Citation(s) in RCA: 63] [Impact Index Per Article: 21.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/26/2020] [Revised: 05/14/2021] [Accepted: 05/19/2021] [Indexed: 01/23/2023]
Abstract
Brains at rest generate dynamical activity that is highly structured in space and time. We suggest that spontaneous activity, as in rest or dreaming, underlies top-down dynamics of generative models. During active tasks, generative models provide top-down predictive signals for perception, cognition, and action. When the brain is at rest and stimuli are weak or absent, top-down dynamics optimize the generative models for future interactions by maximizing the entropy of explanations and minimizing model complexity. Spontaneous fluctuations of correlated activity within and across brain regions may reflect transitions between 'generic priors' of the generative model: low dimensional latent variables and connectivity patterns of the most common perceptual, motor, cognitive, and interoceptive states. Even at rest, brains are proactive and predictive.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Roma, Italy.
| | - Marco Zorzi
- Department of General Psychology and Padova Neuroscience Center (PNC), University of Padova, Padova, Italy; IRCCS San Camillo Hospital, Venice, Italy
| | - Maurizio Corbetta
- Department of Neuroscience and Padova Neuroscience Center (PNC), University of Padova, Padova, Italy; Venetian Institute of Molecular Medicine (VIMM), Fondazione Biomedica, Padova, Italy
| |
Collapse
|
17
|
Berners-Lee A, Wu X, Foster DJ. Prefrontal Cortical Neurons Are Selective for Non-Local Hippocampal Representations during Replay and Behavior. J Neurosci 2021; 41:5894-5908. [PMID: 34035138 PMCID: PMC8265798 DOI: 10.1523/jneurosci.1158-20.2021] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/12/2020] [Revised: 04/24/2021] [Accepted: 04/26/2021] [Indexed: 11/21/2022] Open
Abstract
Diverse functions such as decision-making and memory consolidation may depend on communication between neurons in the hippocampus (HP) and prefrontal cortex (PFC). HP replay is a candidate mechanism to facilitate this communication, however details remain largely unknown because of the technical challenges of recording sufficient numbers of HP neurons for replay while also recording PFC neurons. Here, we implanted male rats with 40-tetrode drives, split between HP and PFC, during learning of a Y-maze spatial memory task. Surprisingly, we found that in contrast to their non-selectivity for maze arm during movement, a portion of PFC neurons were highly selective for HP replay of different arms. Moreover, PFC neurons' selectivity to HP non-local arm representation during running tended to match their replay arm selectivity and was predictive of future choice. Thus, PFC activity that is tuned to HP activity is best explained by non-local HP position representations rather than HP representation of actual position, providing a new potential mechanism of HP-PFC coordination during HP replay.SIGNIFICANCE STATEMENT The hippocampus (HP) is implicated in spatial learning while the prefrontal cortex (PFC) is implicated in decision-making. The question of how the two areas interact has been of great interest. A specific activity type in HP called replay is particularly interesting because it resembles internal exploration of non-local experiences, but is technically challenging to study, requiring recordings from large numbers of HP neurons simultaneously. Here, we combined replay recordings from HP with prefrontal recordings, to reveal a surprising degree of selectivity for replay, and a pattern of coordination that supports some conceptions of hippocampocortical interaction and challenges others.
Collapse
Affiliation(s)
- Alice Berners-Lee
- Helen Wills Neuroscience Institute and Department of Psychology, University of California, Berkeley, California 94720
- Solomon H Snyder Department of Neuroscience, Johns Hopkins University School of Medicine, Baltimore, Maryland 21205
| | - Xiaojing Wu
- Solomon H Snyder Department of Neuroscience, Johns Hopkins University School of Medicine, Baltimore, Maryland 21205
| | - David J Foster
- Helen Wills Neuroscience Institute and Department of Psychology, University of California, Berkeley, California 94720
| |
Collapse
|
18
|
Pezzulo G, LaPalme J, Durant F, Levin M. Bistability of somatic pattern memories: stochastic outcomes in bioelectric circuits underlying regeneration. Philos Trans R Soc Lond B Biol Sci 2021; 376:20190765. [PMID: 33550952 PMCID: PMC7935058 DOI: 10.1098/rstb.2019.0765] [Citation(s) in RCA: 18] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/06/2020] [Indexed: 02/06/2023] Open
Abstract
Nervous systems' computational abilities are an evolutionary innovation, specializing and speed-optimizing ancient biophysical dynamics. Bioelectric signalling originated in cells' communication with the outside world and with each other, enabling cooperation towards adaptive construction and repair of multicellular bodies. Here, we review the emerging field of developmental bioelectricity, which links the field of basal cognition to state-of-the-art questions in regenerative medicine, synthetic bioengineering and even artificial intelligence. One of the predictions of this view is that regeneration and regulative development can restore correct large-scale anatomies from diverse starting states because, like the brain, they exploit bioelectric encoding of distributed goal states-in this case, pattern memories. We propose a new interpretation of recent stochastic regenerative phenotypes in planaria, by appealing to computational models of memory representation and processing in the brain. Moreover, we discuss novel findings showing that bioelectric changes induced in planaria can be stored in tissue for over a week, thus revealing that somatic bioelectric circuits in vivo can implement a long-term, re-writable memory medium. A consideration of the mechanisms, evolution and functionality of basal cognition makes novel predictions and provides an integrative perspective on the evolution, physiology and biomedicine of information processing in vivo. This article is part of the theme issue 'Basal cognition: multicellularity, neurons and the cognitive lens'.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Joshua LaPalme
- Allen Discovery Center, Tufts University, Medford, MA, USA
| | - Fallon Durant
- Allen Discovery Center, Tufts University, Medford, MA, USA
| | - Michael Levin
- Allen Discovery Center, Tufts University, Medford, MA, USA
| |
Collapse
|
19
|
Duvelle É, Grieves RM, Liu A, Jedidi-Ayoub S, Holeniewska J, Harris A, Nyberg N, Donnarumma F, Lefort JM, Jeffery KJ, Summerfield C, Pezzulo G, Spiers HJ. Hippocampal place cells encode global location but not connectivity in a complex space. Curr Biol 2021; 31:1221-1233.e9. [PMID: 33581073 PMCID: PMC7988036 DOI: 10.1016/j.cub.2021.01.005] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/13/2020] [Revised: 12/22/2020] [Accepted: 01/05/2021] [Indexed: 11/20/2022]
Abstract
Flexible navigation relies on a cognitive map of space, thought to be implemented by hippocampal place cells: neurons that exhibit location-specific firing. In connected environments, optimal navigation requires keeping track of one's location and of the available connections between subspaces. We examined whether the dorsal CA1 place cells of rats encode environmental connectivity in four geometrically identical boxes arranged in a square. Rats moved between boxes by pushing saloon-type doors that could be locked in one or both directions. Although rats demonstrated knowledge of environmental connectivity, their place cells did not respond to connectivity changes, nor did they represent doorways differently from other locations. Place cells coded location in a global reference frame, with a different map for each box and minimal repetitive fields despite the repetitive geometry. These results suggest that CA1 place cells provide a spatial map that does not explicitly include connectivity.
Collapse
Affiliation(s)
- Éléonore Duvelle
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA.
| | - Roddy M Grieves
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK; Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | - Anyi Liu
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | - Selim Jedidi-Ayoub
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | - Joanna Holeniewska
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | - Adam Harris
- Department of Experimental Psychology, University of Oxford, OX2 6BW Oxford, UK
| | - Nils Nyberg
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | - Francesco Donnarumma
- Institute of Cognitive Sciences and Technologies, National Research Council, via S. Martino d. Battaglia 44, 00185 Rome, Italy
| | - Julie M Lefort
- University College London, Department of Cell and Developmental Biology, London, UK
| | - Kate J Jeffery
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK
| | | | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, via S. Martino d. Battaglia 44, 00185 Rome, Italy
| | - Hugo J Spiers
- Department of Experimental Psychology, Institute of Behavioural Neuroscience, University College London, London, UK.
| |
Collapse
|
20
|
Donnarumma F, Prevete R, Maisto D, Fuscone S, Irvine EM, van der Meer MAA, Kemere C, Pezzulo G. A framework to identify structured behavioral patterns within rodent spatial trajectories. Sci Rep 2021; 11:468. [PMID: 33432100 PMCID: PMC7801653 DOI: 10.1038/s41598-020-79744-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2020] [Accepted: 11/10/2020] [Indexed: 11/09/2022] Open
Abstract
Animal behavior is highly structured. Yet, structured behavioral patterns-or "statistical ethograms"-are not immediately apparent from the full spatiotemporal data that behavioral scientists usually collect. Here, we introduce a framework to quantitatively characterize rodent behavior during spatial (e.g., maze) navigation, in terms of movement building blocks or motor primitives. The hypothesis that we pursue is that rodent behavior is characterized by a small number of motor primitives, which are combined over time to produce open-ended movements. We assume motor primitives to be organized in terms of two sparsity principles: each movement is controlled using a limited subset of motor primitives (sparse superposition) and each primitive is active only for time-limited, time-contiguous portions of movements (sparse activity). We formalize this hypothesis using a sparse dictionary learning method, which we use to extract motor primitives from rodent position and velocity data collected during spatial navigation, and successively to reconstruct past trajectories and predict novel ones. Three main results validate our approach. First, rodent behavioral trajectories are robustly reconstructed from incomplete data, performing better than approaches based on standard dimensionality reduction methods, such as principal component analysis, or single sparsity. Second, the motor primitives extracted during one experimental session generalize and afford the accurate reconstruction of rodent behavior across successive experimental sessions in the same or in modified mazes. Third, in our approach the number of motor primitives associated with each maze correlates with independent measures of maze complexity, hence showing that our formalism is sensitive to essential aspects of task structure. The framework introduced here can be used by behavioral scientists and neuroscientists as an aid for behavioral and neural data analysis. Indeed, the extracted motor primitives enable the quantitative characterization of the complexity and similarity between different mazes and behavioral patterns across multiple trials (i.e., habit formation). We provide example uses of this computational framework, showing how it can be used to identify behavioural effects of maze complexity, analyze stereotyped behavior, classify behavioral choices and predict place and grid cell displacement in novel environments.
Collapse
Affiliation(s)
- Francesco Donnarumma
- Institute of Cognitive Sciences and Technologies (ISTC), National Research Council (CNR), Via San Martino della Battaglia 44, 00185, Rome, Italy
| | - Roberto Prevete
- Department of Electric Engineering and Information Technologies (DIETI), Università degli Studi di Napoli Federico II, Naples, Italy
| | - Domenico Maisto
- Institute for High Performance Computing and Networking (ICAR), National Research Council (CNR), Via Pietro Castellino 111, 80131, Naples, Italy
| | | | - Emily M Irvine
- Department of Psychological and Brain Sciences, Dartmouth College, Hanover, NH, USA
| | | | - Caleb Kemere
- Electrical and Computer Engineering, Rice University, Houston, TX, USA
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies (ISTC), National Research Council (CNR), Via San Martino della Battaglia 44, 00185, Rome, Italy.
| |
Collapse
|
21
|
Tessereau C, O’Dea R, Coombes S, Bast T. Reinforcement learning approaches to hippocampus-dependent flexible spatial navigation. Brain Neurosci Adv 2021; 5:2398212820975634. [PMID: 33954259 PMCID: PMC8042550 DOI: 10.1177/2398212820975634] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2020] [Accepted: 10/21/2020] [Indexed: 11/17/2022] Open
Abstract
Humans and non-human animals show great flexibility in spatial navigation, including the ability to return to specific locations based on as few as one single experience. To study spatial navigation in the laboratory, watermaze tasks, in which rats have to find a hidden platform in a pool of cloudy water surrounded by spatial cues, have long been used. Analogous tasks have been developed for human participants using virtual environments. Spatial learning in the watermaze is facilitated by the hippocampus. In particular, rapid, one-trial, allocentric place learning, as measured in the delayed-matching-to-place variant of the watermaze task, which requires rodents to learn repeatedly new locations in a familiar environment, is hippocampal dependent. In this article, we review some computational principles, embedded within a reinforcement learning framework, that utilise hippocampal spatial representations for navigation in watermaze tasks. We consider which key elements underlie their efficacy, and discuss their limitations in accounting for hippocampus-dependent navigation, both in terms of behavioural performance (i.e. how well do they reproduce behavioural measures of rapid place learning) and neurobiological realism (i.e. how well do they map to neurobiological substrates involved in rapid place learning). We discuss how an actor-critic architecture, enabling simultaneous assessment of the value of the current location and of the optimal direction to follow, can reproduce one-trial place learning performance as shown on watermaze and virtual delayed-matching-to-place tasks by rats and humans, respectively, if complemented with map-like place representations. The contribution of actor-critic mechanisms to delayed-matching-to-place performance is consistent with neurobiological findings implicating the striatum and hippocampo-striatal interaction in delayed-matching-to-place performance, given that the striatum has been associated with actor-critic mechanisms. Moreover, we illustrate that hierarchical computations embedded within an actor-critic architecture may help to account for aspects of flexible spatial navigation. The hierarchical reinforcement learning approach separates trajectory control via a temporal-difference error from goal selection via a goal prediction error and may account for flexible, trial-specific, navigation to familiar goal locations, as required in some arm-maze place memory tasks, although it does not capture one-trial learning of new goal locations, as observed in open field, including watermaze and virtual, delayed-matching-to-place tasks. Future models of one-shot learning of new goal locations, as observed on delayed-matching-to-place tasks, should incorporate hippocampal plasticity mechanisms that integrate new goal information with allocentric place representation, as such mechanisms are supported by substantial empirical evidence.
Collapse
Affiliation(s)
- Charline Tessereau
- School of Mathematical Sciences, University of Nottingham, Nottingham, UK
- School of Psychology, University of Nottingham, Nottingham, UK
- Neuroscience@Nottingham
| | - Reuben O’Dea
- School of Mathematical Sciences, University of Nottingham, Nottingham, UK
- Neuroscience@Nottingham
| | - Stephen Coombes
- School of Mathematical Sciences, University of Nottingham, Nottingham, UK
- Neuroscience@Nottingham
| | - Tobias Bast
- School of Psychology, University of Nottingham, Nottingham, UK
- Neuroscience@Nottingham
| |
Collapse
|
22
|
Dutta S, Gao S, Chu JP, Kemere C. Tracing a Path for Memory in the Hippocampus. Neuron 2020; 107:199-201. [PMID: 32702341 DOI: 10.1016/j.neuron.2020.06.034] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
The hippocampal activity supporting trace fear conditioning has long been mysterious, but a leading hypothesis posits "time-cell"-like sequential patterns. In this issue of Neuron, Ahmed et al. (2020) present new data suggesting that, at least during the first session of learning, a subset of neurons coalesce to selectively encode the task but without expressing reliable sequences.
Collapse
Affiliation(s)
- Shayok Dutta
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA
| | - Sibo Gao
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA
| | - Joshua P Chu
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA
| | - Caleb Kemere
- Department of Electrical and Computer Engineering, Rice University, Houston, TX, USA; Neuroscience Program, Baylor College of Medicine, Houston, TX, USA.
| |
Collapse
|
23
|
Barron HC, Auksztulewicz R, Friston K. Prediction and memory: A predictive coding account. Prog Neurobiol 2020; 192:101821. [PMID: 32446883 PMCID: PMC7305946 DOI: 10.1016/j.pneurobio.2020.101821] [Citation(s) in RCA: 67] [Impact Index Per Article: 16.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2019] [Revised: 02/26/2020] [Accepted: 04/29/2020] [Indexed: 01/27/2023]
Abstract
The hippocampus is crucial for episodic memory, but it is also involved in online prediction. Evidence suggests that a unitary hippocampal code underlies both episodic memory and predictive processing, yet within a predictive coding framework the hippocampal-neocortical interactions that accompany these two phenomena are distinct and opposing. Namely, during episodic recall, the hippocampus is thought to exert an excitatory influence on the neocortex, to reinstate activity patterns across cortical circuits. This contrasts with empirical and theoretical work on predictive processing, where descending predictions suppress prediction errors to 'explain away' ascending inputs via cortical inhibition. In this hypothesis piece, we attempt to dissolve this previously overlooked dialectic. We consider how the hippocampus may facilitate both prediction and memory, respectively, by inhibiting neocortical prediction errors or increasing their gain. We propose that these distinct processing modes depend upon the neuromodulatory gain (or precision) ascribed to prediction error units. Within this framework, memory recall is cast as arising from fictive prediction errors that furnish training signals to optimise generative models of the world, in the absence of sensory data.
Collapse
Affiliation(s)
- Helen C Barron
- Medical Research Council Brain Network Dynamics Unit, Nuffield Department of Clinical Neurosciences, University of Oxford, Mansfield Road, Oxford, OX1 3TH, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, FMRIB, John Radcliffe Hospital, Oxford, OX3 9DU, UK.
| | - Ryszard Auksztulewicz
- Max Planck Institute for Empirical Aesthetics, Frankfurt Am Main, 60322, Germany; Department of Biomedical Sciences, City University of Hong Kong, Hong Kong
| | - Karl Friston
- The Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, WC1N 3BG, UK
| |
Collapse
|
24
|
Khamassi M, Girard B. Modeling awake hippocampal reactivations with model-based bidirectional search. BIOLOGICAL CYBERNETICS 2020; 114:231-248. [PMID: 32065253 DOI: 10.1007/s00422-020-00817-x] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/30/2019] [Accepted: 01/21/2020] [Indexed: 06/10/2023]
Abstract
Hippocampal offline reactivations during reward-based learning, usually categorized as replay events, have been found to be important for performance improvement over time and for memory consolidation. Recent computational work has linked these phenomena to the need to transform reward information into state-action values for decision making and to propagate it to all relevant states of the environment. Nevertheless, it is still unclear whether an integrated reinforcement learning mechanism could account for the variety of awake hippocampal reactivations, including variety in order (forward and reverse reactivated trajectories) and variety in the location where they occur (reward site or decision-point). Here, we present a model-based bidirectional search model which accounts for a variety of hippocampal reactivations. The model combines forward trajectory sampling from current position and backward sampling through prioritized sweeping from states associated with large reward prediction errors until the two trajectories connect. This is repeated until stabilization of state-action values (convergence), which could explain why hippocampal reactivations drastically diminish when the animal's performance stabilizes. Simulations in a multiple T-maze task show that forward reactivations are prominently found at decision-points while backward reactivations are exclusively generated at reward sites. Finally, the model can generate imaginary trajectories that are not allowed to the agent during task performance. We raise some experimental predictions and implications for future studies of the role of the hippocampo-prefronto-striatal network in learning.
Collapse
Affiliation(s)
- Mehdi Khamassi
- Institute of Intelligent Systems and Robotics (ISIR), Sorbonne Université and CNRS (Centre National de la Recherche Scientifique), 75005, Paris, France.
| | - Benoît Girard
- Institute of Intelligent Systems and Robotics (ISIR), Sorbonne Université and CNRS (Centre National de la Recherche Scientifique), 75005, Paris, France
| |
Collapse
|
25
|
The role of sleep in the formation and updating of abstract mental representations. Behav Brain Sci 2020; 43:e151. [DOI: 10.1017/s0140525x19003005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
Abstract
According to Gilead and colleagues, to be efficient abstraction requires a hierarchical organization of information into long-term memory. But, how and when are abstract representations consolidated into long-term memory and how are they integrated with pre-existing abstracta are questions not discussed by Gilead and colleagues. Here, we propose that these processes occur preferentially during offline periods such as sleep.
Collapse
|
26
|
Kaplan R, Tauste Campo A, Bush D, King J, Principe A, Koster R, Ley Nacher M, Rocamora R, Friston KJ. Human hippocampal theta oscillations reflect sequential dependencies during spatial planning. Cogn Neurosci 2019; 11:122-131. [DOI: 10.1080/17588928.2019.1676711] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/29/2022]
Affiliation(s)
- Raphael Kaplan
- Wellcome Centre for Human Neuroimaging, UCL Institute of Neurology, University College London, London, UK
- Kavli Institute for Systems Neuroscience, Norwegian University of Science and Technology, Trondheim, Norway
| | - Adrià Tauste Campo
- Center for Brain and Cognition, Department of Information and Communication Technologies, Universitat Pompeu Fabra, Barcelona, Spain
- Epilepsy Unit, Department of Neurology, Hospital del Mar Medical Research Institute (IMIM), Barcelona, Spain
- Barcelonaβeta Brain Research Center, Pasqual Maragall Foundation, Barcelona, Spain
| | - Daniel Bush
- UCL Institute of Cognitive Neuroscience, University College London, London, UK
- UCL Queen Square Institute of Neurology, University College London, London, UK
| | - John King
- UCL Institute of Cognitive Neuroscience, University College London, London, UK
- Clinical, Education and Health Psychology, University College London, London, UK
| | - Alessandro Principe
- Epilepsy Unit, Department of Neurology, Hospital del Mar Medical Research Institute (IMIM), Barcelona, Spain
| | - Raphael Koster
- Wellcome Centre for Human Neuroimaging, UCL Institute of Neurology, University College London, London, UK
- UCL Institute of Cognitive Neuroscience, University College London, London, UK
| | - Miguel Ley Nacher
- Epilepsy Unit, Department of Neurology, Hospital del Mar Medical Research Institute (IMIM), Barcelona, Spain
| | - Rodrigo Rocamora
- Epilepsy Unit, Department of Neurology, Hospital del Mar Medical Research Institute (IMIM), Barcelona, Spain
| | - Karl J. Friston
- Wellcome Centre for Human Neuroimaging, UCL Institute of Neurology, University College London, London, UK
| |
Collapse
|
27
|
Pezzulo G, Donnarumma F, Maisto D, Stoianov I. Planning at decision time and in the background during spatial navigation. Curr Opin Behav Sci 2019. [DOI: 10.1016/j.cobeha.2019.04.009] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
|
28
|
Abstract
A popular distinction in the human and animal learning literature is between deliberate (or willed) and habitual (or automatic) modes of control. Extensive evidence indicates that, after sufficient learning, living organisms develop behavioural habits that permit them saving computational resources. Furthermore, humans and other animals are able to transfer control from deliberate to habitual modes (and vice versa), trading off efficiently flexibility and parsimony - an ability that is currently unparalleled by artificial control systems. Here, we discuss a computational implementation of habit formation, and the transfer of control from deliberate to habitual modes (and vice versa) within Active Inference: a computational framework that merges aspects of cybernetic theory and of Bayesian inference. To model habit formation, we endow an Active Inference agent with a mechanism to "cache" (or memorize) policy probabilities from previous trials, and reuse them to skip - in part or in full - the inferential steps of deliberative processing. We exploit the fact that the relative quality of policies, conditioned upon hidden states, is constant over trials; provided that contingencies and prior preferences do not change. This means the only quantity that can change policy selection is the prior distribution over the initial state - where this prior is based upon the posterior beliefs from previous trials. Thus, an agent that caches the quality (or the probability) of policies can safely reuse cached values to save on cognitive and computational resources - unless contingencies change. Our simulations illustrate the computational benefits, but also the limits, of three caching schemes under Active Inference. They suggest that key aspects of habitual behaviour - such as perseveration - can be explained in terms of caching policy probabilities. Furthermore, they suggest that there may be many kinds (or stages) of habitual behaviour, each associated with a different caching scheme; for example, caching associated or not associated with contextual estimation. These schemes are more or less impervious to contextual and contingency changes.
Collapse
Affiliation(s)
- D Maisto
- Institute for High Performance Computing and Networking, National Research Council, Via P. Castellino, 111, Naples 80131, Italy
| | - K Friston
- The Wellcome Trust Centre for Neuroimaging, Institute of Neurology, University College London, London, UK
| | - G Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Via San Martino della Battaglia 44, Rome 00185, Italy
| |
Collapse
|
29
|
Reward revaluation biases hippocampal replay content away from the preferred outcome. Nat Neurosci 2019; 22:1450-1459. [DOI: 10.1038/s41593-019-0464-6] [Citation(s) in RCA: 44] [Impact Index Per Article: 8.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2018] [Accepted: 07/09/2019] [Indexed: 02/01/2023]
|
30
|
Heller AS. From Conditioning to Emotion: Translating Animal Models of Learning to Human Psychopathology. Neuroscientist 2019; 26:43-56. [DOI: 10.1177/1073858419866820] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/25/2022]
Abstract
Emotional responses are not static but change as a consequence of learning. Organisms adapt to emotional events and these adaptations influence the way we think, behave, and feel when we encounter similar situations in the future. Integrating recent work from rodent models and research on human psychopathology, this article lays out a model describing how affective events cause learning and can lead to anxiety and depression: affective events are linked to conditioned stimuli and contexts. Affective experiences entrain oscillatory synchrony across distributed neural circuits, including the prefrontal cortex, hippocampus, amygdala, and nucleus accumbens, which form associations that constitute the basis of emotional memories. Consolidation of these experiences appears to be supported by replay in the hippocampus—a process by which hippocampal firing patterns recreate the firing pattern that occurred previously. Generalization of learning occurs to never before experienced contexts when associations form across distinct but related conditioned stimuli. The process of generalization, which requires cortical structures, can cause memories to become abstracted. During abstraction, the latent, overlapping features of the learned associations remain and result in the formation of schemas. Schemas are adaptive because they facilitate the rapid processing of conditioned stimuli and prime behavioral, cognitive, and affective responses that are the manifestations of the accumulation of an individual’s conditioned experiences. However, schemas can be maladaptive when the generalization of aversive emotional responses are applied to stimuli and contexts in which affective reactions are unnecessary. I describe how this process can lead to not only mood and anxiety disorders but also psychotherapeutic treatment.
Collapse
Affiliation(s)
- Aaron S. Heller
- Department of Psychology, Department of Psychiatry & Behavioral Sciences, Graduate Program in Neuroscience, University of Miami, Coral Gables, FL, USA
| |
Collapse
|
31
|
Above and beyond the concrete: The diverse representational substrates of the predictive brain. Behav Brain Sci 2019; 43:e121. [PMID: 31317839 DOI: 10.1017/s0140525x19002000] [Citation(s) in RCA: 45] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Abstract
In recent years, scientists have increasingly taken to investigate the predictive nature of cognition. We argue that prediction relies on abstraction, and thus theories of predictive cognition need an explicit theory of abstract representation. We propose such a theory of the abstract representational capacities that allow humans to transcend the "here-and-now." Consistent with the predictive cognition literature, we suggest that the representational substrates of the mind are built as a hierarchy, ranging from the concrete to the abstract; however, we argue that there are qualitative differences between elements along this hierarchy, generating meaningful, often unacknowledged, diversity. Echoing views from philosophy, we suggest that the representational hierarchy can be parsed into: modality-specific representations, instantiated on perceptual similarity; multimodal representations, instantiated primarily on the discovery of spatiotemporal contiguity; and categorical representations, instantiated primarily on social interaction. These elements serve as the building blocks of complex structures discussed in cognitive psychology (e.g., episodes, scripts) and are the inputs for mental representations that behave like functions, typically discussed in linguistics (i.e., predicators). We support our argument for representational diversity by explaining how the elements in our ontology are all required to account for humans' predictive cognition (e.g., in subserving logic-based prediction; in optimizing the trade-off between accurate and detailed predictions) and by examining how the neuroscientific evidence coheres with our account. In doing so, we provide a testable model of the neural bases of conceptual cognition and highlight several important implications to research on self-projection, reinforcement learning, and predictive-processing models of psychopathology.
Collapse
|
32
|
Gómez CM, Arjona A, Donnarumma F, Maisto D, Rodríguez-Martínez EI, Pezzulo G. Tracking the Time Course of Bayesian Inference With Event-Related Potentials:A Study Using the Central Cue Posner Paradigm. Front Psychol 2019; 10:1424. [PMID: 31275215 PMCID: PMC6593096 DOI: 10.3389/fpsyg.2019.01424] [Citation(s) in RCA: 9] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/04/2018] [Accepted: 06/03/2019] [Indexed: 11/25/2022] Open
Abstract
In this study, we asked whether the event-related potentials associated to cue and target stimuli of a Central Cue Posner Paradigm (CCPP) may encode key parameters of Bayesian inference – prior expectation and surprise – on a trial-by-trial basis. Thirty-two EEG channel were recorded in a sample of 19 young adult subjects while performing a CCPP, in which a cue indicated (validly or invalidly) the position of an incoming auditory target. Three different types of blocks with validities of 50%, 64%, and 88%, respectively, were presented. Estimates of prior expectation and surprise were obtained on a trial-by-trial basis from participants’ responses, using a computational model implementing Bayesian learning. These two values were correlated on a trial-by-trial basis with the EEG values in all the electrodes and time bins. Therefore, a Spearman correlation metrics of the relationship between Bayesian parameters and the EEG was obtained. We report that the surprise parameter was able to classify the different validity blocks. Furthermore, the prior expectation parameter showed a significant correlation with the EEG in the cue-target period, in which the Contingent Negative Variation develops. Finally, in the post-target period the surprise parameter showed a significant correlation in the latencies and electrodes in which different event-related potentials are induced. Our results suggest that Bayesian parameters are coded in the EEG signals; and namely, the CNV would be related to prior expectation, while the post-target components P2a, P2, P3a, P3b, and SW would be related to surprise. This study thus provides novel support to the idea that human electrophysiological neural activity may implement a (Bayesian) predictive processing scheme.
Collapse
Affiliation(s)
- Carlos M Gómez
- Human Psychobiology Lab, Department of Experimental Psychology, University of Seville, Seville, Spain
| | - Antonio Arjona
- Human Psychobiology Lab, Department of Experimental Psychology, University of Seville, Seville, Spain
| | - Francesco Donnarumma
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Domenico Maisto
- Institute for High Performance Computing and Networking, National Research Council, Naples, Italy
| | | | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| |
Collapse
|
33
|
Making the Environment an Informative Place: A Conceptual Analysis of Epistemic Policies and Sensorimotor Coordination. ENTROPY 2019; 21:e21040350. [PMID: 33267064 PMCID: PMC7514834 DOI: 10.3390/e21040350] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 12/20/2018] [Revised: 03/20/2019] [Accepted: 03/25/2019] [Indexed: 01/02/2023]
Abstract
How do living organisms decide and act with limited and uncertain information? Here, we discuss two computational approaches to solving these challenging problems: a "cognitive" and a "sensorimotor" enrichment of stimuli, respectively. In both approaches, the key notion is that agents can strategically modulate their behavior in informative ways, e.g., to disambiguate amongst alternative hypotheses or to favor the perception of stimuli providing the information necessary to later act appropriately. We discuss how, despite their differences, both approaches appeal to the notion that actions must obey both epistemic (i.e., information-gathering or uncertainty-reducing) and pragmatic (i.e., goal- or reward-maximizing) imperatives and balance them. Our computationally-guided analysis reveals that epistemic behavior is fundamental to understanding several facets of cognitive processing, including perception, decision making, and social interaction.
Collapse
|
34
|
Donnarumma F, Dindo H, Pezzulo G. Sensorimotor Communication for Humans and Robots: Improving Interactive Skills by Sending Coordination Signals. IEEE Trans Cogn Dev Syst 2018. [DOI: 10.1109/tcds.2017.2756107] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/06/2022]
|
35
|
Ciliberti D, Michon F, Kloosterman F. Real-time classification of experience-related ensemble spiking patterns for closed-loop applications. eLife 2018; 7:36275. [PMID: 30373716 PMCID: PMC6207426 DOI: 10.7554/elife.36275] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/27/2018] [Accepted: 09/27/2018] [Indexed: 02/06/2023] Open
Abstract
Communication in neural circuits across the cortex is thought to be mediated by spontaneous temporally organized patterns of population activity lasting ~50 –200 ms. Closed-loop manipulations have the unique power to reveal direct and causal links between such patterns and their contribution to cognition. Current brain–computer interfaces, however, are not designed to interpret multi-neuronal spiking patterns at the millisecond timescale. To bridge this gap, we developed a system for classifying ensemble patterns in a closed-loop setting and demonstrated its application in the online identification of hippocampal neuronal replay sequences in the rat. Our system decodes multi-neuronal patterns at 10 ms resolution, identifies within 50 ms experience-related patterns with over 70% sensitivity and specificity, and classifies their content with 95% accuracy. This technology scales to high-count electrode arrays and will help to shed new light on the contribution of internally generated neural activity to coordinated neural assembly interactions and cognition.
Collapse
Affiliation(s)
- Davide Ciliberti
- Neuro-Electronics Research Flanders, Leuven, Belgium.,Brain and Cognition, KU Leuven, Leuven, Belgium.,VIB, Leuven, Belgium
| | - Frédéric Michon
- Neuro-Electronics Research Flanders, Leuven, Belgium.,Brain and Cognition, KU Leuven, Leuven, Belgium.,VIB, Leuven, Belgium
| | - Fabian Kloosterman
- Neuro-Electronics Research Flanders, Leuven, Belgium.,Brain and Cognition, KU Leuven, Leuven, Belgium.,VIB, Leuven, Belgium.,imec, Leuven, Belgium
| |
Collapse
|
36
|
Stoianov IP, Pennartz CMA, Lansink CS, Pezzulo G. Model-based spatial navigation in the hippocampus-ventral striatum circuit: A computational analysis. PLoS Comput Biol 2018; 14:e1006316. [PMID: 30222746 PMCID: PMC6160242 DOI: 10.1371/journal.pcbi.1006316] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2017] [Revised: 09/27/2018] [Accepted: 06/20/2018] [Indexed: 12/26/2022] Open
Abstract
While the neurobiology of simple and habitual choices is relatively well known, our current understanding of goal-directed choices and planning in the brain is still limited. Theoretical work suggests that goal-directed computations can be productively associated to model-based (reinforcement learning) computations, yet a detailed mapping between computational processes and neuronal circuits remains to be fully established. Here we report a computational analysis that aligns Bayesian nonparametrics and model-based reinforcement learning (MB-RL) to the functioning of the hippocampus (HC) and the ventral striatum (vStr)-a neuronal circuit that increasingly recognized to be an appropriate model system to understand goal-directed (spatial) decisions and planning mechanisms in the brain. We test the MB-RL agent in a contextual conditioning task that depends on intact hippocampus and ventral striatal (shell) function and show that it solves the task while showing key behavioral and neuronal signatures of the HC-vStr circuit. Our simulations also explore the benefits of biological forms of look-ahead prediction (forward sweeps) during both learning and control. This article thus contributes to fill the gap between our current understanding of computational algorithms and biological realizations of (model-based) reinforcement learning.
Collapse
Affiliation(s)
- Ivilin Peev Stoianov
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Cyriel M. A. Pennartz
- University of Amsterdam, Swammerdam Institute for Life Sciences–Center for Neuroscience Amsterdam, The Netherlands
| | - Carien S. Lansink
- University of Amsterdam, Swammerdam Institute for Life Sciences–Center for Neuroscience Amsterdam, The Netherlands
| | - Giovani Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| |
Collapse
|
37
|
Liu K, Sibille J, Dragoi G. Generative Predictive Codes by Multiplexed Hippocampal Neuronal Tuplets. Neuron 2018; 99:1329-1341.e6. [PMID: 30146305 DOI: 10.1016/j.neuron.2018.07.047] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2018] [Revised: 06/03/2018] [Accepted: 07/27/2018] [Indexed: 01/20/2023]
Abstract
Rapid internal representations are continuously formed based on single experiential episodes in space and time, but the neuronal ensemble mechanisms enabling rapid encoding without constraining the capacity for multiple distinct representations are unknown. We developed a probabilistic statistical model of hippocampal spontaneous sequential activity and revealed existence of an internal model of generative predictive codes for the regularities of multiple future novel spatial sequences. During navigation, the inferred difference between external stimuli and the internal model was encoded by emergence of intrinsic-unlikely, novel functional connections, which updated the model by preferentially potentiating post-experience. This internal model and these predictive codes depended on neuronal organization into inferred modules of short, high-repeat sequential neuronal "tuplets" operating as "neuro-codons." We propose that flexible multiplexing of neuronal tuplets into repertoires of extended sequences vastly expands the capacity of hippocampal predictive codes, which could initiate top-down hierarchical cortical loops for spatial and mental navigation and rapid learning.
Collapse
Affiliation(s)
- Kefei Liu
- Department of Psychiatry, Yale School of Medicine, New Haven, CT 06511, USA
| | - Jeremie Sibille
- Department of Psychiatry, Yale School of Medicine, New Haven, CT 06511, USA
| | - George Dragoi
- Department of Psychiatry, Yale School of Medicine, New Haven, CT 06511, USA; Department of Neuroscience and Interdepartmental Neuroscience Program, Yale School of Medicine, New Haven, CT 06511, USA.
| |
Collapse
|
38
|
Pezzulo G. Commentary: The Problem of Mental Action: Predictive Control Without Sensory Sheets. Front Psychol 2018; 9:1291. [PMID: 30090081 PMCID: PMC6068400 DOI: 10.3389/fpsyg.2018.01291] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2018] [Accepted: 07/05/2018] [Indexed: 01/29/2023] Open
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| |
Collapse
|
39
|
Corcoran AW, Pezzulo G, Hohwy J. Commentary: Respiration-Entrained Brain Rhythms Are Global but Often Overlooked. Front Syst Neurosci 2018; 12:25. [PMID: 29937718 PMCID: PMC6003246 DOI: 10.3389/fnsys.2018.00025] [Citation(s) in RCA: 27] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/04/2018] [Accepted: 05/16/2018] [Indexed: 11/13/2022] Open
Affiliation(s)
- Andrew W. Corcoran
- Cognition and Philosophy Laboratory, School of Philosophical, Historical and International Studies, Monash University, Melbourne, VIC, Australia
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| | - Jakob Hohwy
- Cognition and Philosophy Laboratory, School of Philosophical, Historical and International Studies, Monash University, Melbourne, VIC, Australia
| |
Collapse
|
40
|
Pezzulo G, Rigoli F, Friston KJ. Hierarchical Active Inference: A Theory of Motivated Control. Trends Cogn Sci 2018; 22:294-306. [PMID: 29475638 PMCID: PMC5870049 DOI: 10.1016/j.tics.2018.01.009] [Citation(s) in RCA: 135] [Impact Index Per Article: 22.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2017] [Revised: 01/23/2018] [Accepted: 01/30/2018] [Indexed: 12/17/2022]
Abstract
Motivated control refers to the coordination of behaviour to achieve affectively valenced outcomes or goals. The study of motivated control traditionally assumes a distinction between control and motivational processes, which map to distinct (dorsolateral versus ventromedial) brain systems. However, the respective roles and interactions between these processes remain controversial. We offer a novel perspective that casts control and motivational processes as complementary aspects - goal propagation and prioritization, respectively - of active inference and hierarchical goal processing under deep generative models. We propose that the control hierarchy propagates prior preferences or goals, but their precision is informed by the motivational context, inferred at different levels of the motivational hierarchy. The ensuing integration of control and motivational processes underwrites action and policy selection and, ultimately, motivated behaviour, by enabling deep inference to prioritize goals in a context-sensitive way.
Collapse
Affiliation(s)
- Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy.
| | - Francesco Rigoli
- City, University of London, London, UK; Wellcome Trust Centre for Neuroimaging, UCL, London, UK
| | | |
Collapse
|
41
|
Genon S, Reid A, Langner R, Amunts K, Eickhoff SB. How to Characterize the Function of a Brain Region. Trends Cogn Sci 2018; 22:350-364. [PMID: 29501326 PMCID: PMC7978486 DOI: 10.1016/j.tics.2018.01.010] [Citation(s) in RCA: 107] [Impact Index Per Article: 17.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2017] [Revised: 01/29/2018] [Accepted: 01/31/2018] [Indexed: 12/12/2022]
Abstract
Many brain regions have been defined, but a comprehensive formalization of each region's function in relation to human behavior is still lacking. Current knowledge comes from various fields, which have diverse conceptions of 'functions'. We briefly review these fields and outline how the heterogeneity of associations could be harnessed to disclose the computational function of any region. Aggregating activation data from neuroimaging studies allows us to characterize the functional engagement of a region across a range of experimental conditions. Furthermore, large-sample data can disclose covariation between brain region features and ecological behavioral phenotyping. Combining these two approaches opens a new perspective to determine the behavioral associations of a brain region, and hence its function and broader role within large-scale functional networks.
Collapse
Affiliation(s)
- Sarah Genon
- Institute of Neuroscience and Medicine (INM-1, INM-7), Research Centre Jülich, Jülich, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany.
| | - Andrew Reid
- School of Psychology, University of Nottingham, Nottingham, UK
| | - Robert Langner
- Institute of Neuroscience and Medicine (INM-1, INM-7), Research Centre Jülich, Jülich, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Katrin Amunts
- Institute of Neuroscience and Medicine (INM-1, INM-7), Research Centre Jülich, Jülich, Germany; C. and O. Vogt Institute for Brain Research, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| | - Simon B Eickhoff
- Institute of Neuroscience and Medicine (INM-1, INM-7), Research Centre Jülich, Jülich, Germany; Institute of Systems Neuroscience, Medical Faculty, Heinrich Heine University Düsseldorf, Düsseldorf, Germany
| |
Collapse
|
42
|
Zielinski MC, Tang W, Jadhav SP. The role of replay and theta sequences in mediating hippocampal-prefrontal interactions for memory and cognition. Hippocampus 2018; 30:60-72. [PMID: 29251801 DOI: 10.1002/hipo.22821] [Citation(s) in RCA: 37] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/03/2017] [Revised: 12/03/2017] [Accepted: 12/10/2017] [Indexed: 11/05/2022]
Abstract
Sequential activity is seen in the hippocampus during multiple network patterns, prominently as replay activity during both awake and sleep sharp-wave ripples (SWRs), and as theta sequences during active exploration. Although various mnemonic and cognitive functions have been ascribed to these hippocampal sequences, evidence for these proposed functions remains primarily phenomenological. Here, we briefly review current knowledge about replay events and theta sequences in spatial memory tasks. We reason that in order to gain a mechanistic and causal understanding of how these patterns influence memory and cognitive processing, it is important to consider how these sequences influence activity in other regions, and in particular, the prefrontal cortex, which is crucial for memory-guided behavior. For spatial memory tasks, we posit that hippocampal-prefrontal interactions mediated by replay and theta sequences play complementary and overlapping roles at different stages in learning, supporting memory encoding and retrieval, deliberative decision making, planning, and guiding future actions. This framework offers testable predictions for future physiology and closed-loop feedback inactivation experiments for specifically targeting hippocampal sequences as well as coordinated prefrontal activity in different network states, with the potential to reveal their causal roles in memory-guided behavior.
Collapse
Affiliation(s)
- Mark C Zielinski
- Graduate Program in Neuroscience, Brandeis University, Waltham, Massachusetts, 02453
| | - Wenbo Tang
- Graduate Program in Neuroscience, Brandeis University, Waltham, Massachusetts, 02453
| | - Shantanu P Jadhav
- Neuroscience Program, Department of Psychology and Volen National Center for Complex Systems, Brandeis University, Waltham, Massachusetts, 02453
| |
Collapse
|
43
|
Goh JOS, Hung HY, Su YS. A conceptual consideration of the free energy principle in cognitive maps: How cognitive maps help reduce surprise. PSYCHOLOGY OF LEARNING AND MOTIVATION 2018. [DOI: 10.1016/bs.plm.2018.09.005] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
44
|
|
45
|
Donnarumma F, Dindo H, Pezzulo G. Sensorimotor Coarticulation in the Execution and Recognition of Intentional Actions. Front Psychol 2017; 8:237. [PMID: 28280475 PMCID: PMC5322223 DOI: 10.3389/fpsyg.2017.00237] [Citation(s) in RCA: 8] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/29/2016] [Accepted: 02/07/2017] [Indexed: 11/13/2022] Open
Abstract
Humans excel at recognizing (or inferring) another's distal intentions, and recent experiments suggest that this may be possible using only subtle kinematic cues elicited during early phases of movement. Still, the cognitive and computational mechanisms underlying the recognition of intentional (sequential) actions are incompletely known and it is unclear whether kinematic cues alone are sufficient for this task, or if it instead requires additional mechanisms (e.g., prior information) that may be more difficult to fully characterize in empirical studies. Here we present a computationally-guided analysis of the execution and recognition of intentional actions that is rooted in theories of motor control and the coarticulation of sequential actions. In our simulations, when a performer agent coarticulates two successive actions in an action sequence (e.g., "reach-to-grasp" a bottle and "grasp-to-pour"), he automatically produces kinematic cues that an observer agent can reliably use to recognize the performer's intention early on, during the execution of the first part of the sequence. This analysis lends computational-level support for the idea that kinematic cues may be sufficiently informative for early intention recognition. Furthermore, it suggests that the social benefits of coarticulation may be a byproduct of a fundamental imperative to optimize sequential actions. Finally, we discuss possible ways a performer agent may combine automatic (coarticulation) and strategic (signaling) ways to facilitate, or hinder, an observer's action recognition processes.
Collapse
Affiliation(s)
- Francesco Donnarumma
- Institute of Cognitive Sciences and Technologies, National Research Council Rome, Italy
| | - Haris Dindo
- Computer Science Engineering, University of Palermo Palermo, Italy
| | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council Rome, Italy
| |
Collapse
|