1
|
Monosov IE, Zimmermann J, Frank MJ, Mathis MW, Baker JT. Ethological computational psychiatry: Challenges and opportunities. Curr Opin Neurobiol 2024; 86:102881. [PMID: 38696972 PMCID: PMC11162904 DOI: 10.1016/j.conb.2024.102881] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/26/2023] [Revised: 04/02/2024] [Accepted: 04/03/2024] [Indexed: 05/04/2024]
Abstract
Studying the intricacies of individual subjects' moods and cognitive processing over extended periods of time presents a formidable challenge in medicine. While much of systems neuroscience appropriately focuses on the link between neural circuit functions and well-constrained behaviors over short timescales (e.g., trials, hours), many mental health conditions involve complex interactions of mood and cognition that are non-stationary across behavioral contexts and evolve over extended timescales. Here, we discuss opportunities, challenges, and possible future directions in computational psychiatry to quantify non-stationary continuously monitored behaviors. We suggest that this exploratory effort may contribute to a more precision-based approach to treating mental disorders and facilitate a more robust reverse translation across animal species. We conclude with ethical considerations for any field that aims to bridge artificial intelligence and patient monitoring.
Collapse
Affiliation(s)
- Ilya E. Monosov
- Departments of Neuroscience, Biomedical Engineering, Electrical Engineering, and Neurosurgery, Washington University School of Medicine, St. Louis, MO, USA
| | - Jan Zimmermann
- Department of Neuroscience, University of Minnesota, Minneapolis, MN, USA
| | - Michael J. Frank
- Carney Center for Computational Brain Science, Brown University, Providence, RI, USA
| | | | | |
Collapse
|
2
|
Gregorová K, Eldar E, Deserno L, Reiter AMF. A cognitive-computational account of mood swings in adolescence. Trends Cogn Sci 2024; 28:290-303. [PMID: 38503636 DOI: 10.1016/j.tics.2024.02.006] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/22/2022] [Revised: 02/06/2024] [Accepted: 02/12/2024] [Indexed: 03/21/2024]
Abstract
Teenagers have a reputation for being fickle, in both their choices and their moods. This variability may help adolescents as they begin to independently navigate novel environments. Recently, however, adolescent moodiness has also been linked to psychopathology. Here, we consider adolescents' mood swings from a novel computational perspective, grounded in reinforcement learning (RL). This model proposes that mood is determined by surprises about outcomes in the environment, and how much we learn from these surprises. It additionally suggests that mood biases learning and choice in a bidirectional manner. Integrating independent lines of research, we sketch a cognitive-computational account of how adolescents' mood, learning, and choice dynamics influence each other, with implications for normative and psychopathological development.
Collapse
Affiliation(s)
- Klára Gregorová
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital, Würzburg 97080, Germany; Department of Psychology, Julius-Maximilians-Universität, Würzburg 97070, Germany; German Center of Prevention Research on Mental Health, Würzburg 97080, Germany
| | - Eran Eldar
- Department of Psychology, Hebrew University of Jerusalem, Jerusalem 9190501, Israel; Department of Cognitive & Brain Sciences, Hebrew University of Jerusalem, Jerusalem 9190501, Israel
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital, Würzburg 97080, Germany; Department of Psychology, Julius-Maximilians-Universität, Würzburg 97070, Germany; Department of Cognitive & Brain Sciences, Hebrew University of Jerusalem, Jerusalem 9190501, Israel; Department of Psychiatry and Psychotherapy, Technical University of Dresden, Dresden 01069, Germany
| | - Andrea M F Reiter
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, University Hospital, Würzburg 97080, Germany; Department of Psychology, Julius-Maximilians-Universität, Würzburg 97070, Germany; German Center of Prevention Research on Mental Health, Würzburg 97080, Germany; Collaborative Research Centre 940 Volition and Cognitive Control, Technical University of Dresden, Dresden 01069, Germany.
| |
Collapse
|
3
|
Simoens J, Verguts T, Braem S. Learning environment-specific learning rates. PLoS Comput Biol 2024; 20:e1011978. [PMID: 38517916 PMCID: PMC10990245 DOI: 10.1371/journal.pcbi.1011978] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 04/03/2024] [Accepted: 03/09/2024] [Indexed: 03/24/2024] Open
Abstract
People often have to switch back and forth between different environments that come with different problems and volatilities. While volatile environments require fast learning (i.e., high learning rates), stable environments call for lower learning rates. Previous studies have shown that people adapt their learning rates, but it remains unclear whether they can also learn about environment-specific learning rates, and instantaneously retrieve them when revisiting environments. Here, using optimality simulations and hierarchical Bayesian analyses across three experiments, we show that people can learn to use different learning rates when switching back and forth between two different environments. We even observe a signature of these environment-specific learning rates when the volatility of both environments is suddenly the same. We conclude that humans can flexibly adapt and learn to associate different learning rates to different environments, offering important insights for developing theories of meta-learning and context-specific control.
Collapse
Affiliation(s)
- Jonas Simoens
- Department of Experimental Psychology, Ghent University, Belgium
| | - Tom Verguts
- Department of Experimental Psychology, Ghent University, Belgium
| | - Senne Braem
- Department of Experimental Psychology, Ghent University, Belgium
| |
Collapse
|
4
|
Katyal S, Fleming SM. The future of metacognition research: Balancing construct breadth with measurement rigor. Cortex 2024; 171:223-234. [PMID: 38041921 PMCID: PMC11139654 DOI: 10.1016/j.cortex.2023.11.002] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/20/2023] [Revised: 10/20/2023] [Accepted: 11/02/2023] [Indexed: 12/04/2023]
Abstract
Foundational work in the psychology of metacognition identified a distinction between metacognitive knowledge (stable beliefs about one's capacities) and metacognitive experiences (local evaluations of performance). More recently, the field has focused on developing tasks and metrics that seek to identify metacognitive capacities from momentary estimates of confidence in performance, and providing precise computational accounts of metacognitive failure. However, this notable progress in formalising models of metacognitive judgments may come at a cost of ignoring broader elements of the psychology of metacognition - such as how stable meta-knowledge is formed, how social cognition and metacognition interact, and how we evaluate affective states that do not have an obvious ground truth. We propose that construct breadth in metacognition research can be restored while maintaining rigour in measurement, and highlight promising avenues for expanding the scope of metacognition research. Such a research programme is well placed to recapture qualitative features of metacognitive knowledge and experience while maintaining the psychophysical rigor that characterises modern research on confidence and performance monitoring.
Collapse
Affiliation(s)
- Sucharit Katyal
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Wellcome Centre for Human Neuroimaging, University College London, London, UK.
| | - Stephen M Fleming
- Max Planck UCL Centre for Computational Psychiatry and Ageing Research, University College London, London, UK; Wellcome Centre for Human Neuroimaging, University College London, London, UK; Department of Experimental Psychology, University College London, London, UK.
| |
Collapse
|
5
|
Knobe J, Cushman F. The common effect of value on prioritized memory and category representation. Trends Cogn Sci 2023; 27:892-900. [PMID: 37460339 DOI: 10.1016/j.tics.2023.06.007] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/15/2023] [Revised: 06/20/2023] [Accepted: 06/21/2023] [Indexed: 09/15/2023]
Abstract
The way we represent categories depends on both the frequency and value of the members of that category. Thus, for instance, prototype representations can be impacted by both information about what is statistically frequent and judgments about what is valuable. Notably, recent research on memory suggests that prioritized memory is also influenced by both statistical frequency and value judgments. Although work on conceptual representation and work on prioritized memory have so far proceeded almost entirely independently, the patterns of existing findings provide evidence for a link between these two phenomena. In particular, these patterns provide evidence for the hypothesis that the impact of value on conceptual representation arises from its co-dependent relationship with prioritized memory.
Collapse
|
6
|
Sinclair AH, Wang YC, Adcock RA. Instructed motivational states bias reinforcement learning and memory formation. Proc Natl Acad Sci U S A 2023; 120:e2304881120. [PMID: 37490530 PMCID: PMC10401012 DOI: 10.1073/pnas.2304881120] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 06/19/2023] [Indexed: 07/27/2023] Open
Abstract
Motivation influences goals, decisions, and memory formation. Imperative motivation links urgent goals to actions, narrowing the focus of attention and memory. Conversely, interrogative motivation integrates goals over time and space, supporting rich memory encoding for flexible future use. We manipulated motivational states via cover stories for a reinforcement learning task: The imperative group imagined executing a museum heist, whereas the interrogative group imagined planning a future heist. Participants repeatedly chose among four doors, representing different museum rooms, to sample trial-unique paintings with variable rewards (later converted to bonus payments). The next day, participants performed a surprise memory test. Crucially, only the cover stories differed between the imperative and interrogative groups; the reinforcement learning task was identical, and all participants had the same expectations about how and when bonus payments would be awarded. In an initial sample and a preregistered replication, we demonstrated that imperative motivation increased exploitation during reinforcement learning. Conversely, interrogative motivation increased directed (but not random) exploration, despite the cost to participants' earnings. At test, the interrogative group was more accurate at recognizing paintings and recalling associated values. In the interrogative group, higher value paintings were more likely to be remembered; imperative motivation disrupted this effect of reward modulating memory. Overall, we demonstrate that a prelearning motivational manipulation can bias learning and memory, bearing implications for education, behavior change, clinical interventions, and communication.
Collapse
Affiliation(s)
- Alyssa H. Sinclair
- Department of Psychology & Neuroscience, Duke University, Durham, NC27710
| | - Yuxi C. Wang
- Department of Psychology & Neuroscience, Duke University, Durham, NC27710
| | - R. Alison Adcock
- Department of Psychology & Neuroscience, Duke University, Durham, NC27710
- Department of Psychiatry & Behavioral Sciences, Duke University, Durham, NC27710
| |
Collapse
|
7
|
Towner E, Chierchia G, Blakemore SJ. Sensitivity and specificity in affective and social learning in adolescence. Trends Cogn Sci 2023:S1364-6613(23)00092-X. [PMID: 37198089 DOI: 10.1016/j.tics.2023.04.002] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/24/2022] [Revised: 03/23/2023] [Accepted: 04/05/2023] [Indexed: 05/19/2023]
Abstract
Adolescence is a period of heightened affective and social sensitivity. In this review we address how this increased sensitivity influences associative learning. Based on recent evidence from human and rodent studies, as well as advances in computational biology, we suggest that, compared to other age groups, adolescents show features of heightened Pavlovian learning but tend to perform worse than adults at instrumental learning. Because Pavlovian learning does not involve decision-making, whereas instrumental learning does, we propose that these developmental differences might be due to heightened sensitivity to rewards and threats in adolescence, coupled with a lower specificity of responding. We discuss the implications of these findings for adolescent mental health and education.
Collapse
Affiliation(s)
- Emily Towner
- Department of Psychology, University of Cambridge, Downing Street, Cambridge, UK.
| | - Gabriele Chierchia
- Department of Brain and Behavioral Sciences, University of Pavia, Pavia, Italy; Department of Psychology, University of Cambridge, Downing Street, Cambridge, UK
| | | |
Collapse
|
8
|
Chierchia G, Soukupová M, Kilford EJ, Griffin C, Leung J, Palminteri S, Blakemore SJ. Confirmatory reinforcement learning changes with age during adolescence. Dev Sci 2023; 26:e13330. [PMID: 36194156 PMCID: PMC7615280 DOI: 10.1111/desc.13330] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2021] [Revised: 07/26/2022] [Accepted: 09/20/2022] [Indexed: 11/26/2022]
Abstract
Understanding how learning changes during human development has been one of the long-standing objectives of developmental science. Recently, advances in computational biology have demonstrated that humans display a bias when learning to navigate novel environments through rewards and punishments: they learn more from outcomes that confirm their expectations than from outcomes that disconfirm them. Here, we ask whether confirmatory learning is stable across development, or whether it might be attenuated in developmental stages in which exploration is beneficial, such as in adolescence. In a reinforcement learning (RL) task, 77 participants aged 11-32 years (four men, mean age = 16.26) attempted to maximize monetary rewards by repeatedly sampling different pairs of novel options, which varied in their reward/punishment probabilities. Mixed-effect models showed an age-related increase in accuracy as long as learning contingencies remained stable across trials, but less so when they reversed halfway through the trials. Age was also associated with a greater tendency to stay with an option that had just delivered a reward, more than to switch away from an option that had just delivered a punishment. At the computational level, a confirmation model provided increasingly better fit with age. This model showed that age differences are captured by decreases in noise or exploration, rather than in the magnitude of the confirmation bias. These findings provide new insights into how learning changes during development and could help better tailor learning environments to people of different ages. RESEARCH HIGHLIGHTS: Reinforcement learning shows age-related improvement during adolescence, but more in stable learning environments compared with volatile learning environments. People tend to stay with an option after a win more than they shift from an option after a loss, and this asymmetry increases with age during adolescence. Computationally, these changes are captured by a developing confirmatory learning style, in which people learn more from outcomes that confirm rather than disconfirm their choices. Age-related differences in confirmatory learning are explained by decreases in stochasticity, rather than changes in the magnitude of the confirmation bias.
Collapse
Affiliation(s)
- Gabriele Chierchia
- Department of Psychology, University of Cambridge, UK
- Institute of Cognitive Neuroscience, University College London, UK
| | | | - Emma J. Kilford
- Institute of Cognitive Neuroscience, University College London, UK
- Department of Clinical, Educational and Health Psychology, University College London, UK
| | - Cait Griffin
- Institute of Cognitive Neuroscience, University College London, UK
| | - Jovita Leung
- Institute of Cognitive Neuroscience, University College London, UK
| | - Stefano Palminteri
- Institute of Cognitive Neuroscience, University College London, UK
- Department of Cognitive Science, École Normale Supérieure, FR
- Institute of Cognitive Neuroscience, HSE, Moscow, Federation of Russia
| | - Sarah-Jayne Blakemore
- Department of Psychology, University of Cambridge, UK
- Institute of Cognitive Neuroscience, University College London, UK
| |
Collapse
|
9
|
Goldway N, Eldar E, Shoval G, Hartley CA. Computational Mechanisms of Addiction and Anxiety: A Developmental Perspective. Biol Psychiatry 2023; 93:739-750. [PMID: 36775050 PMCID: PMC10038924 DOI: 10.1016/j.biopsych.2023.02.004] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/20/2022] [Revised: 02/05/2023] [Accepted: 02/07/2023] [Indexed: 02/12/2023]
Abstract
A central goal of computational psychiatry is to identify systematic relationships between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. Most psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.
Collapse
Affiliation(s)
- Noam Goldway
- Department of Psychology, New York University, New York, New York
| | - Eran Eldar
- Department of Psychology, The Hebrew University of Jerusalem, Jerusalem, Israel; Department of Cognitive and Brain Sciences, The Hebrew University of Jerusalem, Jerusalem, Israel
| | - Gal Shoval
- Princeton Neuroscience Institute, Princeton University, Princeton, New Jersey; Child and Adolescent Division, Geha Mental Health Center, Petah Tikva, Israel; Department of Psychiatry, Sackler Faculty of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Catherine A Hartley
- Department of Psychology, New York University, New York, New York; Center for Neural Science, New York University, New York, New York.
| |
Collapse
|
10
|
Scholz V, Waltmann M, Herzog N, Reiter A, Horstmann A, Deserno L. Cortical Grey Matter Mediates Increases in Model-Based Control and Learning from Positive Feedback from Adolescence to Adulthood. J Neurosci 2023; 43:2178-2189. [PMID: 36823039 PMCID: PMC10039741 DOI: 10.1523/jneurosci.1418-22.2023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Revised: 12/20/2022] [Accepted: 01/13/2023] [Indexed: 02/25/2023] Open
Abstract
Cognition and brain structure undergo significant maturation from adolescence into adulthood. Model-based (MB) control is known to increase across development, which is mediated by cognitive abilities. Here, we asked two questions unaddressed in previous developmental studies. First, what are the brain structural correlates of age-related increases in MB control? Second, how are age-related increases in MB control from adolescence to adulthood influenced by motivational context? A human developmental sample (n = 103; age, 12-50, male/female, 55:48) completed structural MRI and an established task to capture MB control. The task was modified with respect to outcome valence by including (1) reward and punishment blocks to manipulate the motivational context and (2) an additional choice test to assess learning from positive versus negative feedback. After replicating that an age-dependent increase in MB control is mediated by cognitive abilities, we demonstrate first-time evidence that gray matter density (GMD) in the parietal cortex mediates the increase of MB control with age. Although motivational context did not relate to age-related changes in MB control, learning from positive feedback improved with age. Meanwhile, negative feedback learning showed no age effects. We present a first report that an age-related increase in positive feedback learning was mediated by reduced GMD in the parietal, medial, and dorsolateral prefrontal cortex. Our findings indicate that brain maturation, putatively reflected in lower GMD, in distinct and partially overlapping brain regions could lead to a more efficient brain organization and might thus be a key developmental step toward age-related increases in planning and value-based choice.SIGNIFICANCE STATEMENT Changes in model-based decision-making are paralleled by extensive maturation in cognition and brain structure across development. Still, to date the neuroanatomical underpinnings of these changes remain unclear. Here, we demonstrate for the first time that parietal GMD mediates age-dependent increases in model-based control. Age-related increases in positive feedback learning were mediated by reduced GMD in the parietal, medial, and dorsolateral prefrontal cortex. A manipulation of motivational context did not have an impact on age-related changes in model-based control. These findings highlight that brain maturation in distinct and overlapping cortical regions constitutes a key developmental step toward improved value-based choices.
Collapse
Affiliation(s)
- Vanessa Scholz
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, 97080 Würzburg, Germany
- Donders Institute for Brain, Cognition and Behaviour, Radboud University, 6525 GD Nijmegen, The Netherlands
| | - Maria Waltmann
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, 97080 Würzburg, Germany
- Max Planck Institute for Cognition and Neuroscience, D-04103 Leipzig, Germany
| | - Nadine Herzog
- Max Planck Institute for Cognition and Neuroscience, D-04103 Leipzig, Germany
- Integrated Research and Treatment Center AdiposityDiseases, Leipzig University Medical Center, 04103 Leipzig, Germany
| | - Andrea Reiter
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, 97080 Würzburg, Germany
- Collaborative Research Center-940 Volition and Cognitive Control, Faculty of Psychology, Technical University Dresden, 01069 Dresden, Germany
| | - Annette Horstmann
- Max Planck Institute for Cognition and Neuroscience, D-04103 Leipzig, Germany
- Integrated Research and Treatment Center AdiposityDiseases, Leipzig University Medical Center, 04103 Leipzig, Germany
- Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, 00014 Helsinki, Finland
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, 97080 Würzburg, Germany
- Max Planck Institute for Cognition and Neuroscience, D-04103 Leipzig, Germany
- Integrated Research and Treatment Center AdiposityDiseases, Leipzig University Medical Center, 04103 Leipzig, Germany
- Department of Psychiatry and Psychotherapy, University Hospital Carl Gustav Carus, Technical University Dresden, 01069 Dresden, Germany
| |
Collapse
|
11
|
Waltmann M, Herzog N, Reiter AMF, Villringer A, Horstmann A, Deserno L. Diminished reinforcement sensitivity in adolescence is associated with enhanced response switching and reduced coding of choice probability in the medial frontal pole. Dev Cogn Neurosci 2023; 60:101226. [PMID: 36905874 PMCID: PMC10005907 DOI: 10.1016/j.dcn.2023.101226] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2022] [Revised: 03/06/2023] [Accepted: 03/06/2023] [Indexed: 03/09/2023] Open
Abstract
Precisely charting the maturation of core neurocognitive functions such as reinforcement learning (RL) and flexible adaptation to changing action-outcome contingencies is key for developmental neuroscience and adjacent fields like developmental psychiatry. However, research in this area is both sparse and conflicted, especially regarding potentially asymmetric development of learning for different motives (obtain wins vs avoid losses) and learning from valenced feedback (positive vs negative). In the current study, we investigated the development of RL from adolescence to adulthood, using a probabilistic reversal learning task modified to experimentally separate motivational context and feedback valence, in a sample of 95 healthy participants between 12 and 45. We show that adolescence is characterized by enhanced novelty seeking and response shifting especially after negative feedback, which leads to poorer returns when reward contingencies are stable. Computationally, this is accounted for by reduced impact of positive feedback on behavior. We also show, using fMRI, that activity of the medial frontopolar cortex reflecting choice probability is attenuated in adolescence. We argue that this can be interpreted as reflecting diminished confidence in upcoming choices. Interestingly, we find no age-related differences between learning in win and loss contexts.
Collapse
Affiliation(s)
- Maria Waltmann
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany.
| | - Nadine Herzog
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany
| | - Andrea M F Reiter
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Würzburg, Germany; CRC-940 Volition and Cognitive Control, Faculty of Psychology, Technical University of Dresden, Dresden, Germany
| | - Arno Villringer
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; MindBrainBody Institute, Berlin School of Mind and Brain, Charité-Universitätsmedizin Berlin and Humboldt-Universität zu Berlin, Berlin, Germany
| | - Annette Horstmann
- Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; Department of Psychology and Logopedics, Faculty of Medicine, University of Helsinki, Helsinki, Finland
| | - Lorenz Deserno
- Department of Child and Adolescent Psychiatry, Psychosomatics and Psychotherapy, Centre of Mental Health, University of Würzburg, Würzburg, Germany; Max Planck Institute for Human Cognitive and Brain Sciences, Leipzig, Germany; Neuroimaging Center, Technical University of Dresden, Dresden, Germany
| |
Collapse
|
12
|
El Damaty S, Darcey VL, McQuaid GA, Picci G, Stoianova M, Mucciarone V, Chun Y, Laws ML, Campano V, Van Hecke K, Ryan M, Rose EJ, Fishbein DH, VanMeter AS. Introducing an adolescent cognitive maturity index. Front Psychol 2022; 13:1017317. [PMID: 36571021 PMCID: PMC9771453 DOI: 10.3389/fpsyg.2022.1017317] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/11/2022] [Accepted: 10/25/2022] [Indexed: 12/12/2022] Open
Abstract
Children show substantial variation in the rate of physical, cognitive, and social maturation as they traverse adolescence and enter adulthood. Differences in developmental paths are thought to underlie individual differences in later life outcomes, however, there remains a lack of consensus on the normative trajectory of cognitive maturation in adolescence. To address this problem, we derive a Cognitive Maturity Index (CMI), to estimate the difference between chronological and cognitive age predicted with latent factor estimates of inhibitory control, risky decision-making and emotional processing measured with standard neuropsychological instruments. One hundred and forty-one children from the Adolescent Development Study (ADS) were followed longitudinally across three time points from ages 11-14, 13-16, and 14-18. Age prediction with latent factor estimates of cognitive skills approximated age within ±10 months (r = 0.71). Males in advanced puberty displayed lower cognitive maturity relative to peers of the same age; manifesting as weaker inhibitory control, greater risk-taking, desensitization to negative affect, and poor recognition of positive affect.
Collapse
Affiliation(s)
- Shady El Damaty
- Interdisciplinary Program in Neuroscience, Georgetown University, Washington, DC, United States,Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States,*Correspondence: Shady El Damaty
| | - Valerie L. Darcey
- Interdisciplinary Program in Neuroscience, Georgetown University, Washington, DC, United States,Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States
| | - Goldie A. McQuaid
- Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States
| | - Giorgia Picci
- Institute for Human Neuroscience, Boys Town National Research Hospital, Boys Town, NE, United States,Center for Pediatric Brain Health, Boys Town National Research Hospital, Boys Town, NE, United States
| | - Maria Stoianova
- Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States
| | - Veronica Mucciarone
- Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States
| | - Yewon Chun
- Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States
| | - Marissa L. Laws
- Interdisciplinary Program in Neuroscience, Georgetown University, Washington, DC, United States
| | - Victor Campano
- Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States
| | - Kinney Van Hecke
- Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States
| | - Mary Ryan
- Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States
| | - Emma Jane Rose
- Department of Psychology, The Pennsylvania State University, University Park, PA, United States
| | - Diana H. Fishbein
- Translational Neuro-Prevention Research, Frank Porter Graham Child Development Institute, University of Northern Carolina, Chapel Hill, NC, United States
| | - Ashley S. VanMeter
- Interdisciplinary Program in Neuroscience, Georgetown University, Washington, DC, United States,Center for Functional & Molecular Imaging, Georgetown University Medical Center, Department of Neurology, Washington, DC, United States
| |
Collapse
|
13
|
Saragosa-Harris NM, Cohen AO, Reneau TR, Villano WJ, Heller AS, Hartley CA. Real-World Exploration Increases Across Adolescence and Relates to Affect, Risk Taking, and Social Connectivity. Psychol Sci 2022; 33:1664-1679. [DOI: 10.1177/09567976221102070] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022] Open
Abstract
Cross-species research suggests that exploratory behaviors increase during adolescence and relate to the social, affective, and risky behaviors characteristic of this developmental stage. However, how these typical adolescent behaviors manifest and relate in real-world settings remains unclear. Using geolocation tracking to quantify exploration—variability in daily movement patterns—over a 3-month period in 58 adolescents and adults (ages 13–27) in New York City, we investigated whether daily exploration varied with age and whether exploration related to social connectivity, risk taking, and momentary positive affect. In our cross-sectional sample, we found an association between daily exploration and age, with individuals near the transition to legal adulthood exhibiting the highest exploration levels. Days of higher exploration were associated with greater positive affect irrespective of age. Higher mean exploration was associated with greater social connectivity in all participants but was linked to higher risk taking selectively among adolescents. Our results highlight the interplay of exploration and socioemotional behaviors across development and suggest that societal norms may modulate their expression in naturalistic contexts.
Collapse
Affiliation(s)
| | | | | | | | | | - Catherine A. Hartley
- Department of Psychology, New York University
- Center for Neural Science, New York University
| |
Collapse
|
14
|
Nussenbaum K, Velez JA, Washington BT, Hamling HE, Hartley CA. Flexibility in valenced reinforcement learning computations across development. Child Dev 2022; 93:1601-1615. [PMID: 35596654 PMCID: PMC9831067 DOI: 10.1111/cdev.13791] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Abstract
Optimal integration of positive and negative outcomes during learning varies depending on an environment's reward statistics. The present study investigated the extent to which children, adolescents, and adults (N = 142 8-25 year-olds, 55% female, 42% White, 31% Asian, 17% mixed race, and 8% Black; data collected in 2021) adapt their weighting of better-than-expected and worse-than-expected outcomes when learning from reinforcement. Participants made choices across two contexts: one in which weighting positive outcomes more heavily than negative outcomes led to better performance, and one in which the reverse was true. Reinforcement learning modeling revealed that across age, participants shifted their valence biases in accordance with environmental structure. Exploratory analyses revealed strengthening of context-dependent flexibility with increasing age.
Collapse
Affiliation(s)
| | | | | | | | - Catherine A. Hartley
- Corresponding Author: Catherine A. Hartley, Department of Psychology, New York University, 6 Washington Place, Room 871A, New York, NY, 10003.
| |
Collapse
|
15
|
Raab HA, Foord C, Ligneul R, Hartley CA. Developmental shifts in computations used to detect environmental controllability. PLoS Comput Biol 2022; 18:e1010120. [PMID: 35648788 PMCID: PMC9191713 DOI: 10.1371/journal.pcbi.1010120] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/04/2021] [Revised: 06/13/2022] [Accepted: 04/19/2022] [Indexed: 12/13/2022] Open
Abstract
Accurate assessment of environmental controllability enables individuals to adaptively adjust their behavior—exploiting rewards when desirable outcomes are contingent upon their actions and minimizing costly deliberation when their actions are inconsequential. However, it remains unclear how estimation of environmental controllability changes from childhood to adulthood. Ninety participants (ages 8–25) completed a task that covertly alternated between controllable and uncontrollable conditions, requiring them to explore different actions to discover the current degree of environmental controllability. We found that while children were able to distinguish controllable and uncontrollable conditions, accuracy of controllability assessments improved with age. Computational modeling revealed that whereas younger participants’ controllability assessments relied on evidence gleaned through random exploration, older participants more effectively recruited their task structure knowledge to make highly informative interventions. Age-related improvements in working memory mediated this qualitative shift toward increased use of an inferential strategy. Collectively, these findings reveal an age-related shift in the cognitive processes engaged to assess environmental controllability. Improved detection of environmental controllability may foster increasingly adaptive behavior over development by revealing when actions can be leveraged for one’s benefit. The ability to determine when one’s actions are consequential organizes learning and decision making across the lifespan. However, few studies have examined how the ability to detect control over our environment changes from childhood to adulthood. Here, we leveraged a computational modeling framework to characterize the component learning processes underlying controllability assessment in children, adolescents, and adults. We observed age-related improvements in controllability assessment that stemmed from an increasing ability to represent contingencies between states and actions and to use that knowledge to make informative interventions that yield diagnostic evidence of the current degree of control. Increasing ability to accurately assess environmental controllability may confer greater recognition of opportunities to adaptively pursue rewards through goal-directed action across development.
Collapse
Affiliation(s)
- Hillary A. Raab
- Department of Psychology, New York University, New York, New York, United States of America
| | - Careen Foord
- Center for Neural Science, New York University, New York, New York, United States of America
| | - Romain Ligneul
- Champalimaud Research, Champalimaud Center for the Unknown, Lisbon, Portugal
| | - Catherine A. Hartley
- Department of Psychology, New York University, New York, New York, United States of America
- Center for Neural Science, New York University, New York, New York, United States of America
- * E-mail:
| |
Collapse
|
16
|
Palminteri S, Lebreton M. The computational roots of positivity and confirmation biases in reinforcement learning. Trends Cogn Sci 2022; 26:607-621. [PMID: 35662490 DOI: 10.1016/j.tics.2022.04.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2021] [Revised: 04/13/2022] [Accepted: 04/18/2022] [Indexed: 12/16/2022]
Abstract
Humans do not integrate new information objectively: outcomes carrying a positive affective value and evidence confirming one's own prior belief are overweighed. Until recently, theoretical and empirical accounts of the positivity and confirmation biases assumed them to be specific to 'high-level' belief updates. We present evidence against this account. Learning rates in reinforcement learning (RL) tasks, estimated across different contexts and species, generally present the same characteristic asymmetry, suggesting that belief and value updating processes share key computational principles and distortions. This bias generates over-optimistic expectations about the probability of making the right choices and, consequently, generates over-optimistic reward expectations. We discuss the normative and neurobiological roots of these RL biases and their position within the greater picture of behavioral decision-making theories.
Collapse
Affiliation(s)
- Stefano Palminteri
- Laboratoire de Neurosciences Cognitives et Computationnelles, Institut National de la Santé et Recherche Médicale, Paris, France; Département d'Études Cognitives, Ecole Normale Supérieure, Paris, France; Université de Recherche Paris Sciences et Lettres, Paris, France.
| | - Maël Lebreton
- Paris School of Economics, Paris, France; LabNIC, Department of Fundamental Neurosciences, University of Geneva, Geneva, Switzerland; Swiss Center for Affective Science, Geneva, Switzerland.
| |
Collapse
|
17
|
Jepma M, Schaaf JV, Visser I, Huizenga HM. Impaired learning to dissociate advantageous and disadvantageous risky choices in adolescents. Sci Rep 2022; 12:6490. [PMID: 35443773 PMCID: PMC9021244 DOI: 10.1038/s41598-022-10100-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/20/2021] [Accepted: 03/25/2022] [Indexed: 11/09/2022] Open
Abstract
Adolescence is characterized by a surge in maladaptive risk-taking behaviors, but whether and how this relates to developmental changes in experience-based learning is largely unknown. In this preregistered study, we addressed this issue using a novel task that allowed us to separate the learning-driven optimization of risky choice behavior over time from overall risk-taking tendencies. Adolescents (12-17 years old) learned to dissociate advantageous from disadvantageous risky choices less well than adults (20-35 years old), and this impairment was stronger in early than mid-late adolescents. Computational modeling revealed that adolescents' suboptimal performance was largely due to an inefficiency in core learning and choice processes. Specifically, adolescents used a simpler, suboptimal, expectation-updating process and a more stochastic choice policy. In addition, the modeling results suggested that adolescents, but not adults, overvalued the highest rewards. Finally, an exploratory latent-mixture model analysis indicated that a substantial proportion of the participants in each age group did not engage in experience-based learning but used a gambler's fallacy strategy, stressing the importance of analyzing individual differences. Our results help understand why adolescents tend to make more, and more persistent, maladaptive risky decisions than adults when the values of these decisions have to be learned from experience.
Collapse
Affiliation(s)
- Marieke Jepma
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands.
| | - Jessica V Schaaf
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
| | - Ingmar Visser
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
| | - Hilde M Huizenga
- Department of Psychology, University of Amsterdam, Amsterdam, the Netherlands
| |
Collapse
|
18
|
Nicholas J, Daw ND, Shohamy D. Uncertainty alters the balance between incremental learning and episodic memory. eLife 2022; 11:81679. [PMID: 36458809 PMCID: PMC9810331 DOI: 10.7554/elife.81679] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2022] [Accepted: 12/01/2022] [Indexed: 12/04/2022] Open
Abstract
A key question in decision-making is how humans arbitrate between competing learning and memory systems to maximize reward. We address this question by probing the balance between the effects, on choice, of incremental trial-and-error learning versus episodic memories of individual events. Although a rich literature has studied incremental learning in isolation, the role of episodic memory in decision-making has only recently drawn focus, and little research disentangles their separate contributions. We hypothesized that the brain arbitrates rationally between these two systems, relying on each in circumstances to which it is most suited, as indicated by uncertainty. We tested this hypothesis by directly contrasting contributions of episodic and incremental influence to decisions, while manipulating the relative uncertainty of incremental learning using a well-established manipulation of reward volatility. Across two large, independent samples of young adults, participants traded these influences off rationally, depending more on episodic information when incremental summaries were more uncertain. These results support the proposal that the brain optimizes the balance between different forms of learning and memory according to their relative uncertainties and elucidate the circumstances under which episodic memory informs decisions.
Collapse
Affiliation(s)
- Jonathan Nicholas
- Department of Psychology, Columbia UniversityNew YorkUnited States,Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia UniversityNew YorkUnited States
| | - Nathaniel D Daw
- Department of Psychology, Princeton UniversityPrincetonUnited States,Princeton Neuroscience Institute, Princeton UniversityPrincetonUnited States
| | - Daphna Shohamy
- Department of Psychology, Columbia UniversityNew YorkUnited States,Mortimer B. Zuckerman Mind, Brain, Behavior Institute, Columbia UniversityNew YorkUnited States,The Kavli Institute for Brain Science, Columbia UniversityNew YorkUnited States
| |
Collapse
|