1
|
Deng X, Liu YX, Yang ZZ, Zhao YF, Xu YT, Fu MY, Shen Y, Qu K, Guan Z, Tong WY, Zhang YY, Chen BB, Zhong N, Xiang PH, Duan CG. Spatial evolution of the proton-coupled Mott transition in correlated oxides for neuromorphic computing. SCIENCE ADVANCES 2024; 10:eadk9928. [PMID: 38820158 PMCID: PMC11141630 DOI: 10.1126/sciadv.adk9928] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/22/2023] [Accepted: 04/29/2024] [Indexed: 06/02/2024]
Abstract
The proton-electron coupling effect induces rich spectrums of electronic states in correlated oxides, opening tempting opportunities for exploring novel devices with multifunctions. Here, via modest Pt-aided hydrogen spillover at room temperature, amounts of protons are introduced into SmNiO3-based devices. In situ structural characterizations together with first-principles calculation reveal that the local Mott transition is reversibly driven by migration and redistribution of the predoped protons. The accompanying giant resistance change results in excellent memristive behaviors under ultralow electric fields. Hierarchical tree-like memory states, an instinct displayed in bio-synapses, are further realized in the devices by spatially varying the proton concentration with electric pulses, showing great promise in artificial neural networks for solving intricate problems. Our research demonstrates the direct and effective control of proton evolution using extremely low electric field, offering an alternative pathway for modifying the functionalities of correlated oxides and constructing low-power consumption intelligent devices and neural network circuits.
Collapse
Affiliation(s)
- Xing Deng
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Yu-Xiang Liu
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Zhen-Zhong Yang
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Yi-Feng Zhao
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Ya-Ting Xu
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Meng-Yao Fu
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Yu Shen
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Ke Qu
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Zhao Guan
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Wen-Yi Tong
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Yuan-Yuan Zhang
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Bin-Bin Chen
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
| | - Ni Zhong
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
- Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan, Shanxi 030006, China
| | - Ping-Hua Xiang
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
- Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan, Shanxi 030006, China
| | - Chun-Gang Duan
- Key Laboratory of Polar Materials and Devices (Ministry of Education), Shanghai Center of Brain-Inspired Intelligent Materials and Devices, Department of Electronics, East China Normal University, Shanghai 200241, China
- Collaborative Innovation Center of Extreme Optics, Shanxi University, Taiyuan, Shanxi 030006, China
| |
Collapse
|
2
|
Tsutsui K, Tanaka R, Takeda K, Fujii K. Collaborative hunting in artificial agents with deep reinforcement learning. eLife 2024; 13:e85694. [PMID: 38711355 DOI: 10.7554/elife.85694] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2022] [Accepted: 03/19/2024] [Indexed: 05/08/2024] Open
Abstract
Collaborative hunting, in which predators play different and complementary roles to capture prey, has been traditionally believed to be an advanced hunting strategy requiring large brains that involve high-level cognition. However, recent findings that collaborative hunting has also been documented in smaller-brained vertebrates have placed this previous belief under strain. Here, using computational multi-agent simulations based on deep reinforcement learning, we demonstrate that decisions underlying collaborative hunts do not necessarily rely on sophisticated cognitive processes. We found that apparently elaborate coordination can be achieved through a relatively simple decision process of mapping between states and actions related to distance-dependent internal representations formed by prior experience. Furthermore, we confirmed that this decision rule of predators is robust against unknown prey controlled by humans. Our computational ecological results emphasize that collaborative hunting can emerge in various intra- and inter-specific interactions in nature, and provide insights into the evolution of sociality.
Collapse
Affiliation(s)
- Kazushi Tsutsui
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
- Institute for Advanced Research, Nagoya University, Nagoya, Japan
| | - Ryoya Tanaka
- Institute for Advanced Research, Nagoya University, Nagoya, Japan
- Graduate School of Science, Nagoya University, Nagoya, Japan
| | - Kazuya Takeda
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
- Institute of Innovation for Future Society, Nagoya University, Nagoya, Japan
| | - Keisuke Fujii
- Graduate School of Informatics, Nagoya University, Nagoya, Japan
- Institute for Advanced Research, Nagoya University, Nagoya, Japan
- RIKEN Center for Advanced Intelligence Project, Tokyo, Japan
- PRESTO, Japan Science and Technology Agency, Tokyo, Japan
| |
Collapse
|
3
|
Chu J, Tenenbaum JB, Schulz LE. In praise of folly: flexible goals and human cognition. Trends Cogn Sci 2024:S1364-6613(24)00059-7. [PMID: 38616478 DOI: 10.1016/j.tics.2024.03.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2022] [Revised: 03/13/2024] [Accepted: 03/13/2024] [Indexed: 04/16/2024]
Abstract
Humans often pursue idiosyncratic goals that appear remote from functional ends, including information gain. We suggest that this is valuable because goals (even prima facie foolish or unachievable ones) contain structured information that scaffolds thinking and planning. By evaluating hypotheses and plans with respect to their goals, humans can discover new ideas that go beyond prior knowledge and observable evidence. These hypotheses and plans can be transmitted independently of their original motivations, adapted across generations, and serve as an engine of cultural evolution. Here, we review recent empirical and computational research underlying goal generation and planning and discuss the ways that the flexibility of our motivational system supports cognitive gains for both individuals and societies.
Collapse
Affiliation(s)
- Junyi Chu
- Massachusetts Institute of Technology, Cambridge, MA, USA; Harvard University, Cambridge, MA, USA.
| | | | - Laura E Schulz
- Massachusetts Institute of Technology, Cambridge, MA, USA
| |
Collapse
|
4
|
Thomas T, Straub D, Tatai F, Shene M, Tosik T, Kersting K, Rothkopf CA. Modelling dataset bias in machine-learned theories of economic decision-making. Nat Hum Behav 2024; 8:679-691. [PMID: 38216691 PMCID: PMC11045447 DOI: 10.1038/s41562-023-01784-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2022] [Accepted: 11/14/2023] [Indexed: 01/14/2024]
Abstract
Normative and descriptive models have long vied to explain and predict human risky choices, such as those between goods or gambles. A recent study reported the discovery of a new, more accurate model of human decision-making by training neural networks on a new online large-scale dataset, choices13k. Here we systematically analyse the relationships between several models and datasets using machine-learning methods and find evidence for dataset bias. Because participants' choices in stochastically dominated gambles were consistently skewed towards equipreference in the choices13k dataset, we hypothesized that this reflected increased decision noise. Indeed, a probabilistic generative model adding structured decision noise to a neural network trained on data from a laboratory study transferred best, that is, outperformed all models apart from those trained on choices13k. We conclude that a careful combination of theory and data analysis is still required to understand the complex interactions of machine-learning models and data of human risky choices.
Collapse
Affiliation(s)
- Tobias Thomas
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany.
- Hessian Center for Artificial Intelligence, Darmstadt, Germany.
| | - Dominik Straub
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
| | - Fabian Tatai
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
| | - Megan Shene
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
| | - Tümer Tosik
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
| | - Kristian Kersting
- Hessian Center for Artificial Intelligence, Darmstadt, Germany
- Centre for Cognitive Science and Computer Science Department, Technical University of Darmstadt, Darmstadt, Germany
| | - Constantin A Rothkopf
- Centre for Cognitive Science and Institute of Psychology, Technical University of Darmstadt, Darmstadt, Germany
- Hessian Center for Artificial Intelligence, Darmstadt, Germany
| |
Collapse
|
5
|
Lai AT, Espinosa G, Wink GE, Angeloni CF, Dombeck DA, MacIver MA. A robot-rodent interaction arena with adjustable spatial complexity for ethologically relevant behavioral studies. Cell Rep 2024; 43:113671. [PMID: 38280195 DOI: 10.1016/j.celrep.2023.113671] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2023] [Revised: 10/19/2023] [Accepted: 12/26/2023] [Indexed: 01/29/2024] Open
Abstract
Outside of the laboratory, animals behave in spaces where they can transition between open areas and coverage as they interact with others. Replicating these conditions in the laboratory can be difficult to control and record. This has led to a dominance of relatively simple, static behavioral paradigms that reduce the ethological relevance of behaviors and may alter the engagement of cognitive processes such as planning and decision-making. Therefore, we developed a method for controllable, repeatable interactions with others in a reconfigurable space. Mice navigate a large honeycomb lattice of adjustable obstacles as they interact with an autonomous robot coupled to their actions. We illustrate the system using the robot as a pseudo-predator, delivering airpuffs to the mice. The combination of obstacles and a mobile threat elicits a diverse set of behaviors, such as increased path diversity, peeking, and baiting, providing a method to explore ethologically relevant behaviors in the laboratory.
Collapse
Affiliation(s)
- Alexander T Lai
- Department of Biomedical Engineering, Technological Institute E311, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA
| | - German Espinosa
- Department of Computer Science, Northwestern University, Seeley Mudd 3219, 2233 Tech Drive, Evanston, IL 60208, USA
| | - Gabrielle E Wink
- Department of Mechanical Engineering, Technological Institute B224, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA
| | - Christopher F Angeloni
- Department of Neurobiology, Northwestern University, Hogan 2-160, 2205 Tech Drive, Evanston, IL 60208, USA
| | - Daniel A Dombeck
- Department of Neurobiology, Northwestern University, Hogan 2-160, 2205 Tech Drive, Evanston, IL 60208, USA.
| | - Malcolm A MacIver
- Department of Biomedical Engineering, Technological Institute E311, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA; Department of Computer Science, Northwestern University, Seeley Mudd 3219, 2233 Tech Drive, Evanston, IL 60208, USA; Department of Mechanical Engineering, Technological Institute B224, Northwestern University, 2145 Sheridan Road, Evanston, IL 60208, USA; Department of Neurobiology, Northwestern University, Hogan 2-160, 2205 Tech Drive, Evanston, IL 60208, USA.
| |
Collapse
|
6
|
Wise T, Emery K, Radulescu A. Naturalistic reinforcement learning. Trends Cogn Sci 2024; 28:144-158. [PMID: 37777463 PMCID: PMC10878983 DOI: 10.1016/j.tics.2023.08.016] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2023] [Revised: 08/23/2023] [Accepted: 08/24/2023] [Indexed: 10/02/2023]
Abstract
Humans possess a remarkable ability to make decisions within real-world environments that are expansive, complex, and multidimensional. Human cognitive computational neuroscience has sought to exploit reinforcement learning (RL) as a framework within which to explain human decision-making, often focusing on constrained, artificial experimental tasks. In this article, we review recent efforts that use naturalistic approaches to determine how humans make decisions in complex environments that better approximate the real world, providing a clearer picture of how humans navigate the challenges posed by real-world decisions. These studies purposely embed elements of naturalistic complexity within experimental paradigms, rather than focusing on simplification, generating insights into the processes that likely underpin humans' ability to navigate complex, multidimensional real-world environments so successfully.
Collapse
Affiliation(s)
- Toby Wise
- Department of Neuroimaging, King's College London, London, UK.
| | - Kara Emery
- Center for Data Science, New York University, New York, NY, USA
| | - Angela Radulescu
- Center for Computational Psychiatry, Icahn School of Medicine at Mt. Sinai, New York, NY, USA
| |
Collapse
|
7
|
Chen D, Axmacher N, Wang L. Grid codes underlie multiple cognitive maps in the human brain. Prog Neurobiol 2024; 233:102569. [PMID: 38232782 DOI: 10.1016/j.pneurobio.2024.102569] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2023] [Revised: 01/07/2024] [Accepted: 01/10/2024] [Indexed: 01/19/2024]
Abstract
Grid cells fire at multiple positions that organize the vertices of equilateral triangles tiling a 2D space and are well studied in rodents. The last decade witnessed rapid progress in two other research lines on grid codes-empirical studies on distributed human grid-like representations in physical and multiple non-physical spaces, and cognitive computational models addressing the function of grid cells based on principles of efficient and predictive coding. Here, we review the progress in these fields and integrate these lines into a systematic organization. We also discuss the coordinate mechanisms of grid codes in the human entorhinal cortex and medial prefrontal cortex and their role in neurological and psychiatric diseases.
Collapse
Affiliation(s)
- Dong Chen
- CAS Key Laboratory of Mental Health, Institute of Psychology, 100101, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, 100101, Beijing, China
| | - Nikolai Axmacher
- Department of Neuropsychology, Institute of Cognitive Neuroscience, Faculty of Psychology, Ruhr University Bochum, 44801, Bochum, Germany
| | - Liang Wang
- CAS Key Laboratory of Mental Health, Institute of Psychology, 100101, Beijing, China; Department of Psychology, University of Chinese Academy of Sciences, 100101, Beijing, China.
| |
Collapse
|
8
|
Londei F, Arena G, Ferrucci L, Russo E, Ceccarelli F, Genovesio A. Connecting the dots in the zona incerta: A study of neural assemblies and motifs of inter-area coordination in mice. iScience 2024; 27:108761. [PMID: 38274403 PMCID: PMC10808920 DOI: 10.1016/j.isci.2023.108761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/19/2023] [Revised: 10/23/2023] [Accepted: 11/11/2023] [Indexed: 01/27/2024] Open
Abstract
The zona incerta (ZI), a subthalamic area connected to numerous brain regions, has raised clinical interest because its stimulation alleviates the motor symptoms of Parkinson's disease. To explore its coordinative nature, we studied the assembly formation in a dataset of neural recordings in mice and quantified the degree of functional coordination of ZI with other 24 brain areas. We found that the ZI is a highly integrative area. The analysis in terms of "loop-like" motifs, directional assemblies composed of three neurons spanning two areas, has revealed reciprocal functional interactions with reentrant signals that, in most cases, start and end with the activation of ZI units. In support of its proposed integrative role, we found that almost one-third of the ZI's neurons formed assemblies with more than half of the other recorded areas and that loop-like assemblies may stand out as hyper-integrative motifs compared to other types of activation patterns.
Collapse
Affiliation(s)
- Fabrizio Londei
- Department of Physiology and Pharmacology, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
- PhD Program in Behavioral Neuroscience, Sapienza University of Rome, Rome, Italy
| | - Giulia Arena
- Department of Physiology and Pharmacology, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
- PhD Program in Behavioral Neuroscience, Sapienza University of Rome, Rome, Italy
| | - Lorenzo Ferrucci
- Department of Physiology and Pharmacology, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
| | - Eleonora Russo
- The BioRobotics Institute, Department of Excellence in Robotics and AI, Scuola Superiore Sant’Anna, 56127 Pisa, Italy
| | - Francesco Ceccarelli
- Department of Physiology and Pharmacology, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
| | - Aldo Genovesio
- Department of Physiology and Pharmacology, Sapienza University of Rome, Piazzale Aldo Moro 5, 00185 Rome, Italy
| |
Collapse
|
9
|
Gordon J, Chierichetti F, Panconesi A, Pezzulo G. Information foraging with an oracle. PLoS One 2023; 18:e0295005. [PMID: 38153955 PMCID: PMC10754449 DOI: 10.1371/journal.pone.0295005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2023] [Accepted: 11/13/2023] [Indexed: 12/30/2023] Open
Abstract
During ecological decisions, such as when foraging for food or selecting a weekend activity, we often have to balance the costs and benefits of exploiting known options versus exploring novel ones. Here, we ask how individuals address such cost-benefit tradeoffs during tasks in which we can either explore by ourselves or seek external advice from an oracle (e.g., a domain expert or recommendation system). To answer this question, we designed two studies in which participants chose between inquiring (at a cost) for expert advice from an oracle, or to search for options without guidance, under manipulations affecting the optimal choice. We found that participants showed a greater propensity to seek expert advice when it was instrumental to increase payoff (study A), and when it reduced choice uncertainty, above and beyond payoff maximization (study B). This latter result was especially apparent in participants with greater trait-level intolerance of uncertainty. Taken together, these results suggest that we seek expert advice for both economic goals (i.e., payoff maximization) and epistemic goals (i.e., uncertainty minimization) and that our decisions to ask or not ask for advice are sensitive to cost-benefit tradeoffs.
Collapse
Affiliation(s)
- Jeremy Gordon
- University of California, Berkeley, Berkeley, CA, United States of America
| | | | | | - Giovanni Pezzulo
- Institute of Cognitive Sciences and Technologies, National Research Council, Rome, Italy
| |
Collapse
|
10
|
Prince SM, Yassine TA, Katragadda N, Roberts TC, Singer AC. New information triggers prospective codes to adapt for flexible navigation. BIORXIV : THE PREPRINT SERVER FOR BIOLOGY 2023:2023.10.31.564814. [PMID: 37961524 PMCID: PMC10634986 DOI: 10.1101/2023.10.31.564814] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/15/2023]
Abstract
Navigating a dynamic world requires rapidly updating choices by integrating past experiences with new information. In hippocampus and prefrontal cortex, neural activity representing future goals is theorized to support planning. However, it remains unknown how prospective goal representations incorporate new, pivotal information. Accordingly, we designed a novel task that precisely introduces new information using virtual reality, and we recorded neural activity as mice flexibly adapted their planned destinations. We found that new information triggered increased hippocampal prospective representations of both possible goals; while in prefrontal cortex, new information caused prospective representations of choices to rapidly shift to the new choice. When mice did not flexibly adapt, prefrontal choice codes failed to switch, despite relatively intact hippocampal goal representations. Prospective code updating depended on the commitment to the initial choice and degree of adaptation needed. Thus, we show how prospective codes update with new information to flexibly adapt ongoing navigational plans.
Collapse
Affiliation(s)
- Stephanie M. Prince
- Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, 30332, United States
| | - Teema A. Yassine
- Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, 30332, United States
| | - Navya Katragadda
- Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, 30332, United States
| | - Tyler C. Roberts
- Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, 30332, United States
| | - Annabelle C. Singer
- Coulter Department of Biomedical Engineering, Emory University and Georgia Institute of Technology, Atlanta, GA, 30332, United States
| |
Collapse
|
11
|
Ruesseler M, Weber LA, Marshall TR, O'Reilly J, Hunt LT. Quantifying decision-making in dynamic, continuously evolving environments. eLife 2023; 12:e82823. [PMID: 37883173 PMCID: PMC10602589 DOI: 10.7554/elife.82823] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/18/2022] [Accepted: 10/13/2023] [Indexed: 10/27/2023] Open
Abstract
During perceptual decision-making tasks, centroparietal electroencephalographic (EEG) potentials report an evidence accumulation-to-bound process that is time locked to trial onset. However, decisions in real-world environments are rarely confined to discrete trials; they instead unfold continuously, with accumulation of time-varying evidence being recency-weighted towards its immediate past. The neural mechanisms supporting recency-weighted continuous decision-making remain unclear. Here, we use a novel continuous task design to study how the centroparietal positivity (CPP) adapts to different environments that place different constraints on evidence accumulation. We show that adaptations in evidence weighting to these different environments are reflected in changes in the CPP. The CPP becomes more sensitive to fluctuations in sensory evidence when large shifts in evidence are less frequent, and the potential is primarily sensitive to fluctuations in decision-relevant (not decision-irrelevant) sensory input. A complementary triphasic component over occipito-parietal cortex encodes the sum of recently accumulated sensory evidence, and its magnitude covaries with parameters describing how different individuals integrate sensory evidence over time. A computational model based on leaky evidence accumulation suggests that these findings can be accounted for by a shift in decision threshold between different environments, which is also reflected in the magnitude of pre-decision EEG activity. Our findings reveal how adaptations in EEG responses reflect flexibility in evidence accumulation to the statistics of dynamic sensory environments.
Collapse
Affiliation(s)
- Maria Ruesseler
- Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford Centre for Human Brain Activity (OHBA) University Department of Psychiatry Warneford HospitalOxfordUnited Kingdom
| | - Lilian Aline Weber
- Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford Centre for Human Brain Activity (OHBA) University Department of Psychiatry Warneford HospitalOxfordUnited Kingdom
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory QuarterOxfordUnited Kingdom
| | - Tom Rhys Marshall
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory QuarterOxfordUnited Kingdom
- Centre for Human Brain Health, University of BirminghamBirminghamUnited Kingdom
| | - Jill O'Reilly
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory QuarterOxfordUnited Kingdom
| | - Laurence Tudor Hunt
- Wellcome Centre for Integrative Neuroimaging, Department of Psychiatry, University of Oxford, Oxford Centre for Human Brain Activity (OHBA) University Department of Psychiatry Warneford HospitalOxfordUnited Kingdom
- Department of Experimental Psychology, University of Oxford, Anna Watts Building, Radcliffe Observatory QuarterOxfordUnited Kingdom
| |
Collapse
|
12
|
Teichroeb JA, Smeltzer EA, Mathur V, Anderson KA, Fowler EJ, Adams FV, Vasey EN, Tamara Kumpan L, Stead SM, Arseneau-Robar TJM. How can we apply decision-making theories to wild animal behavior? Predictions arising from dual process theory and Bayesian decision theory. Am J Primatol 2023:e23565. [PMID: 37839050 DOI: 10.1002/ajp.23565] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/26/2023] [Revised: 09/19/2023] [Accepted: 10/03/2023] [Indexed: 10/17/2023]
Abstract
Our understanding of decision-making processes and cognitive biases is ever increasing, thanks to an accumulation of testable models and a large body of research over the last several decades. The vast majority of this work has been done in humans and laboratory animals because these study subjects and situations allow for tightly controlled experiments. However, it raises questions about how this knowledge can be applied to wild animals in their complex environments. Here, we review two prominent decision-making theories, dual process theory and Bayesian decision theory, to assess the similarities in these approaches and consider how they may apply to wild animals living in heterogenous environments within complicated social groupings. In particular, we wanted to assess when wild animals are likely to respond to a situation with a quick heuristic decision and when they are likely to spend more time and energy on the decision-making process. Based on the literature and evidence from our multi-destination routing experiments on primates, we find that individuals are likely to make quick, heuristic decisions when they encounter routine situations, or signals/cues that accurately predict a certain outcome, or easy problems that experience or evolutionary history has prepared them for. Conversely, effortful decision-making is likely in novel or surprising situations, when signals and cues have unpredictable or uncertain relationships to an outcome, and when problems are computationally complex. Though if problems are overly complex, satisficing via heuristics is likely, to avoid costly mental effort. We present hypotheses for how animals with different socio-ecologies may have to distribute their cognitive effort. Finally, we examine the conservation implications and potential cognitive overload for animals experiencing increasingly novel situations caused by current human-induced rapid environmental change.
Collapse
Affiliation(s)
- Julie A Teichroeb
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Anthropology, University of Toronto, Toronto, Ontario, Canada
| | - Eve A Smeltzer
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Anthropology, University of Toronto, Toronto, Ontario, Canada
| | - Virendra Mathur
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Anthropology, University of Toronto, Toronto, Ontario, Canada
| | - Karyn A Anderson
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Anthropology, University of Toronto, Toronto, Ontario, Canada
| | - Erica J Fowler
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Anthropology, University of Toronto, Toronto, Ontario, Canada
| | - Frances V Adams
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Anthropology, University of Toronto, Toronto, Ontario, Canada
| | - Eric N Vasey
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Anthropology, University of Toronto, Toronto, Ontario, Canada
| | - Ludmila Tamara Kumpan
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Anthropology, University of Toronto, Toronto, Ontario, Canada
| | - Samantha M Stead
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Anthropology, University of Toronto, Toronto, Ontario, Canada
| | - T Jean M Arseneau-Robar
- Department of Anthropology, University of Toronto Scarborough, Toronto, Ontario, Canada
- Department of Biology, Concordia University, Montréal, Quebec, Canada
| |
Collapse
|
13
|
Braem S, Held L, Shenhav A, Frömer R. Learning how to reason and deciding when to decide. Behav Brain Sci 2023; 46:e115. [PMID: 37462203 PMCID: PMC10597599 DOI: 10.1017/s0140525x22003090] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/20/2023]
Abstract
Research on human reasoning has both popularized and struggled with the idea that intuitive and deliberate thoughts stem from two different systems, raising the question how people switch between them. Inspired by research on cognitive control and conflict monitoring, we argue that detecting the need for further thought relies on an intuitive, context-sensitive process that is learned in itself.
Collapse
Affiliation(s)
- Senne Braem
- Department of Experimental Psychology, Universiteit Gent, Gent, Belgium ; https://users.ugent.be/~sbraem/
| | - Leslie Held
- Department of Experimental Psychology, Universiteit Gent, Gent, Belgium ; https://users.ugent.be/~sbraem/
| | - Amitai Shenhav
- Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, RI, USA ; https://www.shenhavlab.org
| | - Romy Frömer
- Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, RI, USA ; https://www.shenhavlab.org
- Centre for Human Brain Health, School of Psychology, University of Birmingham, Birmingham, UK
| |
Collapse
|
14
|
Nelli S, Braun L, Dumbalska T, Saxe A, Summerfield C. Neural knowledge assembly in humans and neural networks. Neuron 2023; 111:1504-1516.e9. [PMID: 36898375 PMCID: PMC10618408 DOI: 10.1016/j.neuron.2023.02.014] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/29/2022] [Revised: 12/21/2022] [Accepted: 02/09/2023] [Indexed: 03/11/2023]
Abstract
Human understanding of the world can change rapidly when new information comes to light, such as when a plot twist occurs in a work of fiction. This flexible "knowledge assembly" requires few-shot reorganization of neural codes for relations among objects and events. However, existing computational theories are largely silent about how this could occur. Here, participants learned a transitive ordering among novel objects within two distinct contexts before exposure to new knowledge that revealed how they were linked. Blood-oxygen-level-dependent (BOLD) signals in dorsal frontoparietal cortical areas revealed that objects were rapidly and dramatically rearranged on the neural manifold after minimal exposure to linking information. We then adapt online stochastic gradient descent to permit similar rapid knowledge assembly in a neural network model.
Collapse
Affiliation(s)
- Stephanie Nelli
- Department of Cognitive Science, Occidental College, Los Angeles, CA 90041, USA; Department of Experimental Psychology, University of Oxford, Oxford OX2 6GC, UK.
| | - Lukas Braun
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GC, UK
| | | | - Andrew Saxe
- Department of Experimental Psychology, University of Oxford, Oxford OX2 6GC, UK; Gatsby Unit & Sainsbury Wellcome Centre, University College London, London W1T 4JG, UK; CIFAR Azrieli Global Scholars Program, CIFAR, Toronto, ON M5G 1M1, Canada
| | | |
Collapse
|
15
|
Zhu SL, Lakshminarasimhan KJ, Angelaki DE. Computational cross-species views of the hippocampal formation. Hippocampus 2023; 33:586-599. [PMID: 37038890 PMCID: PMC10947336 DOI: 10.1002/hipo.23535] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/10/2023] [Revised: 03/17/2023] [Accepted: 03/21/2023] [Indexed: 04/12/2023]
Abstract
The discovery of place cells and head direction cells in the hippocampal formation of freely foraging rodents has led to an emphasis of its role in encoding allocentric spatial relationships. In contrast, studies in head-fixed primates have additionally found representations of spatial views. We review recent experiments in freely moving monkeys that expand upon these findings and show that postural variables such as eye/head movements strongly influence neural activity in the hippocampal formation, suggesting that the function of the hippocampus depends on where the animal looks. We interpret these results in the light of recent studies in humans performing challenging navigation tasks which suggest that depending on the context, eye/head movements serve one of two roles-gathering information about the structure of the environment (active sensing) or externalizing the contents of internal beliefs/deliberation (embodied cognition). These findings prompt future experimental investigations into the information carried by signals flowing between the hippocampal formation and the brain regions controlling postural variables, and constitute a basis for updating computational theories of the hippocampal system to accommodate the influence of eye/head movements.
Collapse
Affiliation(s)
- Seren L Zhu
- Center for Neural Science, New York University, New York, New York, USA
| | - Kaushik J Lakshminarasimhan
- Center for Theoretical Neuroscience, Zuckerman Mind Brain Behavior Institute, Columbia University, New York, New York, USA
| | - Dora E Angelaki
- Center for Neural Science, New York University, New York, New York, USA
- Mechanical and Aerospace Engineering, Tandon School of Engineering, New York University, New York, New York, USA
| |
Collapse
|
16
|
Miyamoto K, Rushworth MFS, Shea N. Imagining the future self through thought experiments. Trends Cogn Sci 2023; 27:446-455. [PMID: 36801162 DOI: 10.1016/j.tics.2023.01.005] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/05/2022] [Revised: 01/23/2023] [Accepted: 01/23/2023] [Indexed: 02/19/2023]
Abstract
The ability of the mind to conceptualize what is not present is essential. It allows us to reason counterfactually about what might have happened had events unfolded differently or had another course of action been taken. It allows us to think about what might happen - to perform 'Gedankenexperimente' (thought experiments) - before we act. However, the cognitive and neural mechanisms mediating this ability are poorly understood. We suggest that the frontopolar cortex (FPC) keeps track of and evaluates alternative choices (what we might have done), whereas the anterior lateral prefrontal cortex (alPFC) compares simulations of possible future scenarios (what we might do) and evaluates their reward values. Together, these brain regions support the construction of suppositional scenarios.
Collapse
Affiliation(s)
- Kentaro Miyamoto
- Laboratory for Imagination and Executive Functions, RIKEN Center for Brain Science, Wako, Japan.
| | - Matthew F S Rushworth
- Department of Experimental Psychology, University of Oxford, Oxford, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | - Nicholas Shea
- Institute of Philosophy, School of Advanced Study, University of London, London, UK; Faculty of Philosophy, University of Oxford, Oxford, UK
| |
Collapse
|
17
|
Kurth-Nelson Z, Behrens T, Wayne G, Miller K, Luettgau L, Dolan R, Liu Y, Schwartenbeck P. Replay and compositional computation. Neuron 2023; 111:454-469. [PMID: 36640765 DOI: 10.1016/j.neuron.2022.12.028] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/29/2022] [Revised: 08/11/2022] [Accepted: 12/18/2022] [Indexed: 01/15/2023]
Abstract
Replay in the brain has been viewed as rehearsal or, more recently, as sampling from a transition model. Here, we propose a new hypothesis: that replay is able to implement a form of compositional computation where entities are assembled into relationally bound structures to derive qualitatively new knowledge. This idea builds on recent advances in neuroscience, which indicate that the hippocampus flexibly binds objects to generalizable roles and that replay strings these role-bound objects into compound statements. We suggest experiments to test our hypothesis, and we end by noting the implications for AI systems which lack the human ability to radically generalize past experience to solve new problems.
Collapse
Affiliation(s)
- Zeb Kurth-Nelson
- DeepMind, London, UK; Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK.
| | - Timothy Behrens
- Wellcome Centre for Human Neuroimaging, University College London, London, UK; Wellcome Centre for Integrative Neuroimaging, University of Oxford, Oxford, UK
| | | | - Kevin Miller
- DeepMind, London, UK; Institute of Ophthalmology, University College London, London, UK
| | - Lennart Luettgau
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK
| | - Ray Dolan
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, London, UK; Wellcome Centre for Human Neuroimaging, University College London, London, UK
| | - Yunzhe Liu
- State Key Laboratory of Cognitive Neuroscience and Learning, IDG/McGovern Institute for Brain Research, Beijing Normal University, Beijing, China; Chinese Institute for Brain Research, Beijing, China
| | - Philipp Schwartenbeck
- Max Planck Institute for Biological Cybernetics, Tubingen, Germany; University of Tubingen, Tubingen, Germany
| |
Collapse
|
18
|
Levenstein D, Alvarez VA, Amarasingham A, Azab H, Chen ZS, Gerkin RC, Hasenstaub A, Iyer R, Jolivet RB, Marzen S, Monaco JD, Prinz AA, Quraishi S, Santamaria F, Shivkumar S, Singh MF, Traub R, Nadim F, Rotstein HG, Redish AD. On the Role of Theory and Modeling in Neuroscience. J Neurosci 2023; 43:1074-1088. [PMID: 36796842 PMCID: PMC9962842 DOI: 10.1523/jneurosci.1179-22.2022] [Citation(s) in RCA: 8] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/16/2022] [Revised: 12/14/2022] [Accepted: 12/18/2022] [Indexed: 02/18/2023] Open
Abstract
In recent years, the field of neuroscience has gone through rapid experimental advances and a significant increase in the use of quantitative and computational methods. This growth has created a need for clearer analyses of the theory and modeling approaches used in the field. This issue is particularly complex in neuroscience because the field studies phenomena that cross a wide range of scales and often require consideration at varying degrees of abstraction, from precise biophysical interactions to the computations they implement. We argue that a pragmatic perspective of science, in which descriptive, mechanistic, and normative models and theories each play a distinct role in defining and bridging levels of abstraction, will facilitate neuroscientific practice. This analysis leads to methodological suggestions, including selecting a level of abstraction that is appropriate for a given problem, identifying transfer functions to connect models and data, and the use of models themselves as a form of experiment.
Collapse
Affiliation(s)
- Daniel Levenstein
- Montreal Neurological Institute, McGill University, Montreal, Quebec H3A 2B4, Canada
| | - Veronica A Alvarez
- Laboratory on Neurobiology of Compulsive Behaviors, National Institute on Alcohol Abuse and Alcoholism, National Institutes of Health, Bethesda, Maryland 20892
| | - Asohan Amarasingham
- Departments of Mathematics and Biology, City College and the Graduate Center, City University of New York, New York, New York 10032
| | - Habiba Azab
- Department of Neuroscience, Center for Magnetic Resonance Research, University of Minnesota, Minneapolis, Minnesota 55455
| | - Zhe S Chen
- Department of Psychiatry, Neuroscience & Physiology, New York University School of Medicine, New York, New York, 10016
| | - Richard C Gerkin
- School of Life Sciences, Arizona State University, Tempe, Arizona 85281
| | - Andrea Hasenstaub
- Department of Otolaryngology-Head and Neck Surgery, University of California San Francisco, San Francisco, California 94115
| | | | - Renaud B Jolivet
- Maastricht Centre for Systems Biology, Maastricht University, Maastricht, The Netherlands
| | - Sarah Marzen
- W. M. Keck Science Department, Pitzer, Scripps, and Claremont McKenna Colleges, Claremont, California 91711
| | - Joseph D Monaco
- Department of Biomedical Engineering, Johns Hopkins University School of Medicine, Baltimore, Maryland 21218
| | - Astrid A Prinz
- Department of Biology, Emory University, Atlanta, Georgia 30322
| | - Salma Quraishi
- Neuroscience, Developmental and Regnerative Biology Department, University of Texas at San Antonio, San Antonio, Texas 78249
| | - Fidel Santamaria
- Neuroscience, Developmental and Regnerative Biology Department, University of Texas at San Antonio, San Antonio, Texas 78249
| | - Sabyasachi Shivkumar
- Brain and Cognitive Sciences, University of Rochester, Rochester, New York 14627
| | - Matthew F Singh
- Department of Psychological & Brain Sciences, Department of Electrical & Systems Engineering, Washington University in St. Louis, St. Louis, Missouri 63112
| | - Roger Traub
- IBM T.J. Watson Research Center, AI Foundations, Yorktown Heights, New York 10598
| | - Farzan Nadim
- Montreal Neurological Institute, McGill University, Montreal, Quebec H3A 2B4, Canada
- Department of Otolaryngology-Head and Neck Surgery, University of California San Francisco, San Francisco, California 94115
| | - Horacio G Rotstein
- Montreal Neurological Institute, McGill University, Montreal, Quebec H3A 2B4, Canada
- Department of Otolaryngology-Head and Neck Surgery, University of California San Francisco, San Francisco, California 94115
| | - A David Redish
- Department of Neuroscience, University of Minnesota, Minneapolis, Minnesota 55455
| |
Collapse
|
19
|
Humans account for cognitive costs when finding shortcuts: An information-theoretic analysis of navigation. PLoS Comput Biol 2023; 19:e1010829. [PMID: 36608145 PMCID: PMC9851521 DOI: 10.1371/journal.pcbi.1010829] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/22/2022] [Revised: 01/19/2023] [Accepted: 12/19/2022] [Indexed: 01/09/2023] Open
Abstract
When faced with navigating back somewhere we have been before we might either retrace our steps or seek a shorter path. Both choices have costs. Here, we ask whether it is possible to characterize formally the choice of navigational plans as a bounded rational process that trades off the quality of the plan (e.g., its length) and the cognitive cost required to find and implement it. We analyze the navigation strategies of two groups of people that are firstly trained to follow a "default policy" taking a route in a virtual maze and then asked to navigate to various known goal destinations, either in the way they want ("Go To Goal") or by taking novel shortcuts ("Take Shortcut"). We address these wayfinding problems using InfoRL: an information-theoretic approach that formalizes the cognitive cost of devising a navigational plan, as the informational cost to deviate from a well-learned route (the "default policy"). In InfoRL, optimality refers to finding the best trade-off between route length and the amount of control information required to find it. We report five main findings. First, the navigational strategies automatically identified by InfoRL correspond closely to different routes (optimal or suboptimal) in the virtual reality map, which were annotated by hand in previous research. Second, people deliberate more in places where the value of investing cognitive resources (i.e., relevant goal information) is greater. Third, compared to the group of people who receive the "Go To Goal" instruction, those who receive the "Take Shortcut" instruction find shorter but less optimal solutions, reflecting the intrinsic difficulty of finding optimal shortcuts. Fourth, those who receive the "Go To Goal" instruction modulate flexibly their cognitive resources, depending on the benefits of finding the shortcut. Finally, we found a surprising amount of variability in the choice of navigational strategies and resource investment across participants. Taken together, these results illustrate the benefits of using InfoRL to address navigational planning problems from a bounded rational perspective.
Collapse
|
20
|
Comrie AE, Frank LM, Kay K. Imagination as a fundamental function of the hippocampus. Philos Trans R Soc Lond B Biol Sci 2022; 377:20210336. [PMID: 36314152 PMCID: PMC9620759 DOI: 10.1098/rstb.2021.0336] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/13/2021] [Accepted: 04/20/2022] [Indexed: 08/25/2023] Open
Abstract
Imagination is a biological function that is vital to human experience and advanced cognition. Despite this importance, it remains unknown how imagination is realized in the brain. Substantial research focusing on the hippocampus, a brain structure traditionally linked to memory, indicates that firing patterns in spatially tuned neurons can represent previous and upcoming paths in space. This work has generally been interpreted under standard views that the hippocampus implements cognitive abilities primarily related to actual experience, whether in the past (e.g. recollection, consolidation), present (e.g. spatial mapping) or future (e.g. planning). However, relatively recent findings in rodents identify robust patterns of hippocampal firing corresponding to a variety of alternatives to actual experience, in many cases without overt reference to the past, present or future. Given these findings, and others on hippocampal contributions to human imagination, we suggest that a fundamental function of the hippocampus is to generate a wealth of hypothetical experiences and thoughts. Under this view, traditional accounts of hippocampal function in episodic memory and spatial navigation can be understood as particular applications of a more general system for imagination. This view also suggests that the hippocampus contributes to a wider range of cognitive abilities than previously thought. This article is part of the theme issue 'Thinking about possibilities: mechanisms, ontogeny, functions and phylogeny'.
Collapse
Affiliation(s)
- Alison E. Comrie
- Neuroscience Graduate Program, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Kavli Institute for Fundamental Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Center for Integrative Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Departments of Physiology and Psychiatry, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
| | - Loren M. Frank
- Kavli Institute for Fundamental Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Center for Integrative Neuroscience, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Departments of Physiology and Psychiatry, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
- Howard Hughes Medical Institute, University of California San Francisco, 675 Nelson Rising Lane, San Francisco, CA 94158, USA
| | - Kenneth Kay
- Zuckerman Institute, Center for Theoretical Neuroscience, Columbia University, 3227 Broadway, New York, NY 10027, USA
| |
Collapse
|
21
|
Ujfalussy BB, Orbán G. Sampling motion trajectories during hippocampal theta sequences. eLife 2022; 11:74058. [DOI: 10.7554/elife.74058] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/20/2021] [Accepted: 09/28/2022] [Indexed: 11/06/2022] Open
Abstract
Efficient planning in complex environments requires that uncertainty associated with current inferences and possible consequences of forthcoming actions is represented. Representation of uncertainty has been established in sensory systems during simple perceptual decision making tasks but it remains unclear if complex cognitive computations such as planning and navigation are also supported by probabilistic neural representations. Here, we capitalized on gradually changing uncertainty along planned motion trajectories during hippocampal theta sequences to capture signatures of uncertainty representation in population responses. In contrast with prominent theories, we found no evidence of encoding parameters of probability distributions in the momentary population activity recorded in an open-field navigation task in rats. Instead, uncertainty was encoded sequentially by sampling motion trajectories randomly and efficiently in subsequent theta cycles from the distribution of potential trajectories. Our analysis is the first to demonstrate that the hippocampus is well equipped to contribute to optimal planning by representing uncertainty.
Collapse
Affiliation(s)
- Balazs B Ujfalussy
- Laboratory of Biological Computation, Institute of Experimental Medicine
- Laboratory of Neuronal Signalling, Institute of Experimental Medicine, Budapest
| | - Gergő Orbán
- Computational Systems Neuroscience Lab, Wigner Research Center for Physics, Budapest
| |
Collapse
|
22
|
Oversampled and undersolved: Depressive rumination from an active inference perspective. Neurosci Biobehav Rev 2022; 142:104873. [PMID: 36116573 DOI: 10.1016/j.neubiorev.2022.104873] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/10/2022] [Revised: 09/12/2022] [Accepted: 09/12/2022] [Indexed: 11/22/2022]
Abstract
Rumination is a widely recognized cognitive deviation in depression. Despite the recognition, researchers have struggled to explain why patients cannot disengage from the process, although it depresses their mood and fails to lead to effective problem-solving. We rethink rumination as repetitive but unsuccessful problem-solving attempts. Appealing to an active inference account, we suggest that adaptive problem-solving is based on the generation, evaluation, and performance of candidate policies that increase an organism's knowledge of its environment. We argue that the problem-solving process is distorted during rumination. Specifically, rumination is understood as engaging in excessive yet unsuccessful oversampling of policy candidates that do not resolve uncertainty. Because candidates are sampled from policies that were selected in states resembling one's current state, "bad" starting points (e.g., depressed mood, physical inactivity) make the problem-solving process vulnerable for generating a ruminative "halting problem". This problem leads to high opportunity costs, learned helplessness and diminished overt behavior. Besides reviewing evidence for the conceptual paths of this model, we discuss its neurophysiological correlates and point towards clinical implications.
Collapse
|
23
|
Klein-Flügge MC, Bongioanni A, Rushworth MFS. Medial and orbital frontal cortex in decision-making and flexible behavior. Neuron 2022; 110:2743-2770. [PMID: 35705077 DOI: 10.1016/j.neuron.2022.05.022] [Citation(s) in RCA: 22] [Impact Index Per Article: 11.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/08/2022] [Revised: 05/17/2022] [Accepted: 05/19/2022] [Indexed: 11/15/2022]
Abstract
The medial frontal cortex and adjacent orbitofrontal cortex have been the focus of investigations of decision-making, behavioral flexibility, and social behavior. We review studies conducted in humans, macaques, and rodents and argue that several regions with different functional roles can be identified in the dorsal anterior cingulate cortex, perigenual anterior cingulate cortex, anterior medial frontal cortex, ventromedial prefrontal cortex, and medial and lateral parts of the orbitofrontal cortex. There is increasing evidence that the manner in which these areas represent the value of the environment and specific choices is different from subcortical brain regions and more complex than previously thought. Although activity in some regions reflects distributions of reward and opportunities across the environment, in other cases, activity reflects the structural relationships between features of the environment that animals can use to infer what decision to take even if they have not encountered identical opportunities in the past.
Collapse
Affiliation(s)
- Miriam C Klein-Flügge
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Tinsley Building, Mansfield Road, Oxford OX1 3TA, UK; Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), University of Oxford, Nuffield Department of Clinical Neurosciences, Level 6, West Wing, John Radcliffe Hospital, Oxford OX3 9DU, UK; Department of Psychiatry, University of Oxford, Warneford Lane, Headington, Oxford OX3 7JX, UK.
| | - Alessandro Bongioanni
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Tinsley Building, Mansfield Road, Oxford OX1 3TA, UK
| | - Matthew F S Rushworth
- Wellcome Centre for Integrative Neuroimaging (WIN), Department of Experimental Psychology, University of Oxford, Tinsley Building, Mansfield Road, Oxford OX1 3TA, UK; Wellcome Centre for Integrative Neuroimaging (WIN), Centre for Functional MRI of the Brain (FMRIB), University of Oxford, Nuffield Department of Clinical Neurosciences, Level 6, West Wing, John Radcliffe Hospital, Oxford OX3 9DU, UK
| |
Collapse
|
24
|
Zhu S, Lakshminarasimhan KJ, Arfaei N, Angelaki DE. Eye movements reveal spatiotemporal dynamics of visually-informed planning in navigation. eLife 2022; 11:73097. [PMID: 35503099 PMCID: PMC9135400 DOI: 10.7554/elife.73097] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 05/01/2022] [Indexed: 11/28/2022] Open
Abstract
Goal-oriented navigation is widely understood to depend upon internal maps. Although this may be the case in many settings, humans tend to rely on vision in complex, unfamiliar environments. To study the nature of gaze during visually-guided navigation, we tasked humans to navigate to transiently visible goals in virtual mazes of varying levels of difficulty, observing that they took near-optimal trajectories in all arenas. By analyzing participants’ eye movements, we gained insights into how they performed visually-informed planning. The spatial distribution of gaze revealed that environmental complexity mediated a striking trade-off in the extent to which attention was directed towards two complimentary aspects of the world model: the reward location and task-relevant transitions. The temporal evolution of gaze revealed rapid, sequential prospection of the future path, evocative of neural replay. These findings suggest that the spatiotemporal characteristics of gaze during navigation are significantly shaped by the unique cognitive computations underlying real-world, sequential decision making.
Collapse
Affiliation(s)
- Seren Zhu
- Center for Neural Science, New York University, New York, United States
| | | | - Nastaran Arfaei
- Department of Psychology, New York University, New York, United States
| | - Dora E Angelaki
- Center for Neural Science, New York University, New York, United States
| |
Collapse
|
25
|
Callaway F, van Opheusden B, Gul S, Das P, Krueger PM, Lieder F, Griffiths TL. Rational use of cognitive resources in human planning. Nat Hum Behav 2022; 6:1112-1125. [PMID: 35484209 DOI: 10.1038/s41562-022-01332-8] [Citation(s) in RCA: 15] [Impact Index Per Article: 7.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/03/2021] [Accepted: 03/03/2022] [Indexed: 12/19/2022]
Abstract
Making good decisions requires thinking ahead, but the huge number of actions and outcomes one could consider makes exhaustive planning infeasible for computationally constrained agents, such as humans. How people are nevertheless able to solve novel problems when their actions have long-reaching consequences is thus a long-standing question in cognitive science. To address this question, we propose a model of resource-constrained planning that allows us to derive optimal planning strategies. We find that previously proposed heuristics such as best-first search are near optimal under some circumstances but not others. In a mouse-tracking paradigm, we show that people adapt their planning strategies accordingly, planning in a manner that is broadly consistent with the optimal model but not with any single heuristic model. We also find systematic deviations from the optimal model that might result from additional cognitive constraints that are yet to be uncovered.
Collapse
Affiliation(s)
| | | | - Sayan Gul
- Department of Psychology, University of California, Berkeley, CA, USA
| | - Priyam Das
- Department of Cognitive Sciences, University of California, Irvine, CA, USA
| | - Paul M Krueger
- Department of Psychology, Princeton University, Princeton, NJ, USA
| | - Falk Lieder
- Max Planck Institute for Intelligent Systems, Tübingen, Germany
| | | |
Collapse
|
26
|
MacIver MA, Finlay BL. The neuroecology of the water-to-land transition and the evolution of the vertebrate brain. Philos Trans R Soc Lond B Biol Sci 2022; 377:20200523. [PMID: 34957852 PMCID: PMC8710882 DOI: 10.1098/rstb.2020.0523] [Citation(s) in RCA: 13] [Impact Index Per Article: 6.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/15/2022] Open
Abstract
The water-to-land transition in vertebrate evolution offers an unusual opportunity to consider computational affordances of a new ecology for the brain. All sensory modalities are changed, particularly a greatly enlarged visual sensorium owing to air versus water as a medium, and expanded by mobile eyes and neck. The multiplication of limbs, as evolved to exploit aspects of life on land, is a comparable computational challenge. As the total mass of living organisms on land is a hundredfold larger than the mass underwater, computational improvements promise great rewards. In water, the midbrain tectum coordinates approach/avoid decisions, contextualized by water flow and by the animal's body state and learning. On land, the relative motions of sensory surfaces and effectors must be resolved, adding on computational architectures from the dorsal pallium, such as the parietal cortex. For the large-brained and long-living denizens of land, making the right decision when the wrong one means death may be the basis of planning, which allows animals to learn from hypothetical experience before enactment. Integration of value-weighted, memorized panoramas in basal ganglia/frontal cortex circuitry, with allocentric cognitive maps of the hippocampus and its associated cortices becomes a cognitive habit-to-plan transition as substantial as the change in ecology. This article is part of the theme issue 'Systems neuroscience through the lens of evolutionary theory'.
Collapse
Affiliation(s)
- Malcolm A. MacIver
- Center for Robotics and Biosystems, Northwestern University, Evanston, IL 60208, USA
| | - Barbara L. Finlay
- Department of Psychology, Behavioral and Evolutionary Neuroscience Group, Cornell University, Ithaca, NY 14850, USA
| |
Collapse
|
27
|
Abstract
Recent breakthroughs in artificial intelligence (AI) have enabled machines to plan in tasks previously thought to be uniquely human. Meanwhile, the planning algorithms implemented by the brain itself remain largely unknown. Here, we review neural and behavioral data in sequential decision-making tasks that elucidate the ways in which the brain does-and does not-plan. To systematically review available biological data, we create a taxonomy of planning algorithms by summarizing the relevant design choices for such algorithms in AI. Across species, recording techniques, and task paradigms, we find converging evidence that the brain represents future states consistent with a class of planning algorithms within our taxonomy-focused, depth-limited, and serial. However, we argue that current data are insufficient for addressing more detailed algorithmic questions. We propose a new approach leveraging AI advances to drive experiments that can adjudicate between competing candidate algorithms.
Collapse
|