Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory

Reinforcement learning increasingly relates to memory specificity from childhood to adulthood

In some contexts, abstract stimulus representations can effectively promote reward pursuit, whereas in others, detailed representations are needed to guide choice. Here, we ask how, across development, the reward statistics of the environment influence the specificity of both value-guided learning computations and recognition memory. Across two experiments (N = 224), we show that participants ages 8 - 25 years adaptively up- and down-weight detailed versus broader stimulus representations and that these learning computations relate to mnemonic specificity. When participants place greater weight on granular representations during learning, they better remember stimulus details, whereas when they place greater weight on broader representations, they show enhanced memory only for categorical information. Moreover, the strength of the coupling between learning and memory specificity increases with age. We demonstrate that from early in life, reward shapes the granularity with which the world is partitioned, and increasingly across development, the specificity with which experiences are remembered.

Distorted learning from local metacognition supports transdiagnostic underconfidence

Individuals experiencing symptoms of anxiety and depression have been shown to exhibit persistent underconfidence. The origin of such metacognitive biases presents a puzzle, given that individuals should be able to learn appropriate levels of confidence from observing their own performance. In two large general population samples (N = 230 and N = 278), we measure both 'local' confidence in individual task instances and 'global' confidence as longer-run self-performance estimates while manipulating external feedback. Global confidence is sensitive to both local confidence and feedback valence-more frequent positive (negative) feedback increases (respectively decreases) global confidence, with asymmetries in feedback also leading to shifts in affective self-beliefs. Notably, however, global confidence exhibits reduced sensitivity to instances of higher local confidence in individuals with greater subclinical anxious-depression symptomatology, despite sensitivity to feedback valence remaining intact. Our finding of blunted sensitivity to increases in local confidence offers a mechanistic basis for how persistent underconfidence is maintained in the face of intact performance.

The preference for surprise in reinforcement learning underlies the differences in developmental changes in risk preference between autistic and neurotypical youth

BACKGROUND

Risk preference changes nonlinearly across development. Although extensive developmental research on the neurotypical (NTP) population has shown that risk preference is highest during adolescence, developmental changes in risk preference in autistic (AUT) people, who tend to prefer predictable behaviors, have not been investigated. Here, we aimed to investigate these changes and underlying computational mechanisms.

METHOD

We ran a game-like risk-sensitive reinforcement learning task on 75 participants aged 6-30 years (AUT group, n = 31; NTP group, n = 44). Focusing on choices between alternatives with the same objective value but different risks, we calculated the risk preference and stay probability of a risky choice after a rewarding or non-rewarding outcome. Analyses using t-tests and multiple regression analyses were conducted. Using the choice-related data of each participant, we fit four reinforcement learning models and compared the fit of each model to the data. Furthermore, we validated the results of model fitting with multiple methods, model recovery, parameter recovery, and posterior predictive check.

RESULTS

We found a significant difference in nonlinear developmental changes in risk preference between the AUT and NTP groups. The computational modeling approach with reinforcement learning models revealed that individual preferences for surprise modulated such preferences.

CONCLUSIONS

These findings indicate that for NTP people, adolescence is a developmental period involving risk preference, possibly due to lower surprise aversion. Conversely, for AUT people, who show opposite developmental change of risk preference, adolescence could be a developmental period involving risk avoidance because of low surprise preference.

The connecting brain in context: How adolescent plasticity supports learning and development

Puberty initiates significant neurobiological changes that amplify adolescents' responsiveness to their environment, facilitating neural adaptation through processes like synaptic pruning, myelination, and neuronal reorganization. This heightened neuroplasticity, combined with their burgeoning social curiosity and appetite for risk, propels adolescents to explore diverse new environments and forge social bonds. Such exploration can accelerate experiential learning and the formation of social networks as adolescents prepare for adult independence. This review examines the complex interplay between adolescent neuroplasticity, environmental influences, and learning processes, synthesizing findings from recent studies that illustrate how factors such as social interactions, school environments, and neighborhood contexts influence both the transient activation and enduring organization of the developing brain. We advocate for incorporating social interaction into adolescent-tailored interventions, leveraging their social plasticity to optimize learning and development during this critical phase. Going forward, we discuss the importance of longitudinal studies that employ multimodal approaches to characterize the dynamic interactions between development and environment, highlighting recent advancements in quantifying environmental impacts in studies of developmental neuroscience. Ultimately, this paper provides an updated synopsis of adolescent neuroplasticity and the environment, underscoring the potential for environmental enrichment programs to support healthy brain development and resilience at this critical development stage.

Decrease in decision noise from adolescence into adulthood mediates an increase in more sophisticated choice behaviors and performance gain

Learning and decision-making undergo substantial developmental changes, with adolescence being a particular vulnerable window of opportunity. In adolescents, developmental changes in specific choice behaviors have been observed (e.g., goal-directed behavior, motivational influences over choice). Elevated levels of decision noise, i.e., choosing suboptimal options, were reported consistently in adolescents. However, it remains unknown whether these observations, the development of specific and more sophisticated choice processes and higher decision noise, are independent or related. It is conceivable, but has not yet been investigated, that the development of specific choice processes might be impacted by age-dependent changes in decision noise. To answer this, we examined 93 participants (12 to 42 years) who completed 3 reinforcement learning (RL) tasks: a motivational Go/NoGo task assessing motivational influences over choices, a reversal learning task capturing adaptive decision-making in response to environmental changes, and a sequential choice task measuring goal-directed behavior. This allowed testing of (1) cross-task generalization of computational parameters focusing on decision noise; and (2) assessment of mediation effects of noise on specific choice behaviors. Firstly, we found only noise levels to be strongly correlated across RL tasks. Second, and critically, noise levels mediated age-dependent increases in more sophisticated choice behaviors and performance gain. Our findings provide novel insights into the computational processes underlying developmental changes in decision-making: namely a vital role of seemingly unspecific changes in noise in the specific development of more complex choice components. Studying the neurocomputational mechanisms of how varying levels of noise impact distinct aspects of learning and decision processes may also be key to better understand the developmental onset of psychiatric diseases.

Revisiting the musical reminiscence bump: insights from neurocognitive and social brain development in adolescence

Music listening is enjoyed across the lifespan and around the world. This has spurred many theories on the evolutionary purpose of music. The Music for Social Bonding hypothesis posits that the human capacity to make music evolved for the purpose of creating and preserving relationships between one another. Considering different time periods of music use across the lifespan, adolescence is especially a period of social reorientation away from family towards peers, characterized by new social bonds and increased prosocial behavior. This shift is accompanied by notable structural and functional changes in brain networks supporting reward processing and prosocial behavior. Reviewing the extant literature on developmental cognitive neuroscience and adolescent music use, we propose that neurocognitive changes in the reward system make adolescence an ideal developmental time window for investigating interactions between prosocial behavior and reward processing, as adolescence constitutes a time of relative increase in music reward valuation. Testing this hypothesis may clarify our understanding of developmental trajectories in music reward valuation, and offer insights into why music from adults' adolescence holds a great deal of personal significance.

Revisiting adolescence as a sensitive period for sociocultural processing

Waves of research and public discourse have characterized adolescence as periods of developmental risk and opportunity. Underlying this discussion is the recognition that adolescence is a period of major biological and social transition when experience may have an outsized effect on development. This article updates and expands upon prior work suggesting that adolescence may be a sensitive period for sociocultural processing specifically. By integrating evidence from developmental psychology and neuroscience, we identify how trajectories of social and neurobiological development may relate to adolescents' ability to adapt to and learn from their social environments. However, we also highlight gaps in the literature, including challenges in attributing developmental change to adolescent experiences. We discuss the importance of better understanding variability in biology (e.g., pubertal development) and cultural environments, as well as distinguishing between sensitive periods and periods of heightened sensitivity. Finally, we look toward future directions and translational implications of this research.

Sensitivity to the Instrumental Value of Choice Increases Across Development

Across development, people tend to demonstrate a preference for contexts in which they have the opportunity to make choices. However, it is not clear how children, adolescents, and adults learn to calibrate this preference based on the costs and benefits of agentic choice. Here, in both a primary, in-person, reinforcement-learning experiment (N = 92; age range = 10-25 years) and a preregistered online replication study (N = 150; age range = 8-25 years), we found that participants overvalued agentic choice but also calibrated their agency decisions to the reward structure of the environment, increasingly selecting agentic choice when choice had greater instrumental value. Regression analyses and computational modeling of participant choices revealed that participants' bias toward agentic choice-reflecting its intrinsic value-remained consistent across age, whereas sensitivity to the instrumental value of agentic choice increased from childhood to early adulthood.

Positive affect modulates memory by regulating the influence of reward prediction errors

How our decisions impact our memories is not well understood. Reward prediction errors (RPEs), the difference between expected and obtained reward, help us learn to make optimal decisions-providing a signal that may influence subsequent memory. To measure this influence and how it might go awry in mood disorders, we recruited a large cohort of human participants to perform a decision-making task in which perceptually memorable stimuli were associated with probabilistic rewards, followed by a recognition test for those stimuli. Computational modeling revealed that positive RPEs enhanced both the accuracy of memory and the temporal efficiency of memory search, beyond the contribution of perceptual information. Critically, positive affect upregulated the beneficial effect of RPEs on memory. These findings demonstrate how affect selectively regulates the impact of RPEs on memory, providing a computational mechanism for biased memory in mood disorders.

Ethological computational psychiatry: Challenges and opportunities

Studying the intricacies of individual subjects' moods and cognitive processing over extended periods of time presents a formidable challenge in medicine. While much of systems neuroscience appropriately focuses on the link between neural circuit functions and well-constrained behaviors over short timescales (e.g., trials, hours), many mental health conditions involve complex interactions of mood and cognition that are non-stationary across behavioral contexts and evolve over extended timescales. Here, we discuss opportunities, challenges, and possible future directions in computational psychiatry to quantify non-stationary continuously monitored behaviors. We suggest that this exploratory effort may contribute to a more precision-based approach to treating mental disorders and facilitate a more robust reverse translation across animal species. We conclude with ethical considerations for any field that aims to bridge artificial intelligence and patient monitoring.

A cognitive-computational account of mood swings in adolescence

Teenagers have a reputation for being fickle, in both their choices and their moods. This variability may help adolescents as they begin to independently navigate novel environments. Recently, however, adolescent moodiness has also been linked to psychopathology. Here, we consider adolescents' mood swings from a novel computational perspective, grounded in reinforcement learning (RL). This model proposes that mood is determined by surprises about outcomes in the environment, and how much we learn from these surprises. It additionally suggests that mood biases learning and choice in a bidirectional manner. Integrating independent lines of research, we sketch a cognitive-computational account of how adolescents' mood, learning, and choice dynamics influence each other, with implications for normative and psychopathological development.

Learning environment-specific learning rates

People often have to switch back and forth between different environments that come with different problems and volatilities. While volatile environments require fast learning (i.e., high learning rates), stable environments call for lower learning rates. Previous studies have shown that people adapt their learning rates, but it remains unclear whether they can also learn about environment-specific learning rates, and instantaneously retrieve them when revisiting environments. Here, using optimality simulations and hierarchical Bayesian analyses across three experiments, we show that people can learn to use different learning rates when switching back and forth between two different environments. We even observe a signature of these environment-specific learning rates when the volatility of both environments is suddenly the same. We conclude that humans can flexibly adapt and learn to associate different learning rates to different environments, offering important insights for developing theories of meta-learning and context-specific control.

The future of metacognition research: Balancing construct breadth with measurement rigor

Foundational work in the psychology of metacognition identified a distinction between metacognitive knowledge (stable beliefs about one's capacities) and metacognitive experiences (local evaluations of performance). More recently, the field has focused on developing tasks and metrics that seek to identify metacognitive capacities from momentary estimates of confidence in performance, and providing precise computational accounts of metacognitive failure. However, this notable progress in formalising models of metacognitive judgments may come at a cost of ignoring broader elements of the psychology of metacognition - such as how stable meta-knowledge is formed, how social cognition and metacognition interact, and how we evaluate affective states that do not have an obvious ground truth. We propose that construct breadth in metacognition research can be restored while maintaining rigour in measurement, and highlight promising avenues for expanding the scope of metacognition research. Such a research programme is well placed to recapture qualitative features of metacognitive knowledge and experience while maintaining the psychophysical rigor that characterises modern research on confidence and performance monitoring.

The common effect of value on prioritized memory and category representation

The way we represent categories depends on both the frequency and value of the members of that category. Thus, for instance, prototype representations can be impacted by both information about what is statistically frequent and judgments about what is valuable. Notably, recent research on memory suggests that prioritized memory is also influenced by both statistical frequency and value judgments. Although work on conceptual representation and work on prioritized memory have so far proceeded almost entirely independently, the patterns of existing findings provide evidence for a link between these two phenomena. In particular, these patterns provide evidence for the hypothesis that the impact of value on conceptual representation arises from its co-dependent relationship with prioritized memory.

Instructed motivational states bias reinforcement learning and memory formation

Motivation influences goals, decisions, and memory formation. Imperative motivation links urgent goals to actions, narrowing the focus of attention and memory. Conversely, interrogative motivation integrates goals over time and space, supporting rich memory encoding for flexible future use. We manipulated motivational states via cover stories for a reinforcement learning task: The imperative group imagined executing a museum heist, whereas the interrogative group imagined planning a future heist. Participants repeatedly chose among four doors, representing different museum rooms, to sample trial-unique paintings with variable rewards (later converted to bonus payments). The next day, participants performed a surprise memory test. Crucially, only the cover stories differed between the imperative and interrogative groups; the reinforcement learning task was identical, and all participants had the same expectations about how and when bonus payments would be awarded. In an initial sample and a preregistered replication, we demonstrated that imperative motivation increased exploitation during reinforcement learning. Conversely, interrogative motivation increased directed (but not random) exploration, despite the cost to participants' earnings. At test, the interrogative group was more accurate at recognizing paintings and recalling associated values. In the interrogative group, higher value paintings were more likely to be remembered; imperative motivation disrupted this effect of reward modulating memory. Overall, we demonstrate that a prelearning motivational manipulation can bias learning and memory, bearing implications for education, behavior change, clinical interventions, and communication.

Sensitivity and specificity in affective and social learning in adolescence

Adolescence is a period of heightened affective and social sensitivity. In this review we address how this increased sensitivity influences associative learning. Based on recent evidence from human and rodent studies, as well as advances in computational biology, we suggest that, compared to other age groups, adolescents show features of heightened Pavlovian learning but tend to perform worse than adults at instrumental learning. Because Pavlovian learning does not involve decision-making, whereas instrumental learning does, we propose that these developmental differences might be due to heightened sensitivity to rewards and threats in adolescence, coupled with a lower specificity of responding. We discuss the implications of these findings for adolescent mental health and education.

Confirmatory reinforcement learning changes with age during adolescence

Understanding how learning changes during human development has been one of the long-standing objectives of developmental science. Recently, advances in computational biology have demonstrated that humans display a bias when learning to navigate novel environments through rewards and punishments: they learn more from outcomes that confirm their expectations than from outcomes that disconfirm them. Here, we ask whether confirmatory learning is stable across development, or whether it might be attenuated in developmental stages in which exploration is beneficial, such as in adolescence. In a reinforcement learning (RL) task, 77 participants aged 11-32 years (four men, mean age = 16.26) attempted to maximize monetary rewards by repeatedly sampling different pairs of novel options, which varied in their reward/punishment probabilities. Mixed-effect models showed an age-related increase in accuracy as long as learning contingencies remained stable across trials, but less so when they reversed halfway through the trials. Age was also associated with a greater tendency to stay with an option that had just delivered a reward, more than to switch away from an option that had just delivered a punishment. At the computational level, a confirmation model provided increasingly better fit with age. This model showed that age differences are captured by decreases in noise or exploration, rather than in the magnitude of the confirmation bias. These findings provide new insights into how learning changes during development and could help better tailor learning environments to people of different ages. RESEARCH HIGHLIGHTS: Reinforcement learning shows age-related improvement during adolescence, but more in stable learning environments compared with volatile learning environments. People tend to stay with an option after a win more than they shift from an option after a loss, and this asymmetry increases with age during adolescence. Computationally, these changes are captured by a developing confirmatory learning style, in which people learn more from outcomes that confirm rather than disconfirm their choices. Age-related differences in confirmatory learning are explained by decreases in stochasticity, rather than changes in the magnitude of the confirmation bias.

Computational Mechanisms of Addiction and Anxiety: A Developmental Perspective

A central goal of computational psychiatry is to identify systematic relationships between transdiagnostic dimensions of psychiatric symptomatology and the latent learning and decision-making computations that inform individuals' thoughts, feelings, and choices. Most psychiatric disorders emerge prior to adulthood, yet little work has extended these computational approaches to study the development of psychopathology. Here, we lay out a roadmap for future studies implementing this approach by developing empirically and theoretically informed hypotheses about how developmental changes in model-based control of action and Pavlovian learning processes may modulate vulnerability to anxiety and addiction. We highlight how insights from studies leveraging computational approaches to characterize the normative developmental trajectories of clinically relevant learning and decision-making processes may suggest promising avenues for future developmental computational psychiatry research.

Cortical Grey Matter Mediates Increases in Model-Based Control and Learning from Positive Feedback from Adolescence to Adulthood

Cognition and brain structure undergo significant maturation from adolescence into adulthood. Model-based (MB) control is known to increase across development, which is mediated by cognitive abilities. Here, we asked two questions unaddressed in previous developmental studies. First, what are the brain structural correlates of age-related increases in MB control? Second, how are age-related increases in MB control from adolescence to adulthood influenced by motivational context? A human developmental sample (n = 103; age, 12-50, male/female, 55:48) completed structural MRI and an established task to capture MB control. The task was modified with respect to outcome valence by including (1) reward and punishment blocks to manipulate the motivational context and (2) an additional choice test to assess learning from positive versus negative feedback. After replicating that an age-dependent increase in MB control is mediated by cognitive abilities, we demonstrate first-time evidence that gray matter density (GMD) in the parietal cortex mediates the increase of MB control with age. Although motivational context did not relate to age-related changes in MB control, learning from positive feedback improved with age. Meanwhile, negative feedback learning showed no age effects. We present a first report that an age-related increase in positive feedback learning was mediated by reduced GMD in the parietal, medial, and dorsolateral prefrontal cortex. Our findings indicate that brain maturation, putatively reflected in lower GMD, in distinct and partially overlapping brain regions could lead to a more efficient brain organization and might thus be a key developmental step toward age-related increases in planning and value-based choice.SIGNIFICANCE STATEMENT Changes in model-based decision-making are paralleled by extensive maturation in cognition and brain structure across development. Still, to date the neuroanatomical underpinnings of these changes remain unclear. Here, we demonstrate for the first time that parietal GMD mediates age-dependent increases in model-based control. Age-related increases in positive feedback learning were mediated by reduced GMD in the parietal, medial, and dorsolateral prefrontal cortex. A manipulation of motivational context did not have an impact on age-related changes in model-based control. These findings highlight that brain maturation in distinct and overlapping cortical regions constitutes a key developmental step toward improved value-based choices.

Diminished reinforcement sensitivity in adolescence is associated with enhanced response switching and reduced coding of choice probability in the medial frontal pole

Precisely charting the maturation of core neurocognitive functions such as reinforcement learning (RL) and flexible adaptation to changing action-outcome contingencies is key for developmental neuroscience and adjacent fields like developmental psychiatry. However, research in this area is both sparse and conflicted, especially regarding potentially asymmetric development of learning for different motives (obtain wins vs avoid losses) and learning from valenced feedback (positive vs negative). In the current study, we investigated the development of RL from adolescence to adulthood, using a probabilistic reversal learning task modified to experimentally separate motivational context and feedback valence, in a sample of 95 healthy participants between 12 and 45. We show that adolescence is characterized by enhanced novelty seeking and response shifting especially after negative feedback, which leads to poorer returns when reward contingencies are stable. Computationally, this is accounted for by reduced impact of positive feedback on behavior. We also show, using fMRI, that activity of the medial frontopolar cortex reflecting choice probability is attenuated in adolescence. We argue that this can be interpreted as reflecting diminished confidence in upcoming choices. Interestingly, we find no age-related differences between learning in win and loss contexts.

Introducing an adolescent cognitive maturity index

Children show substantial variation in the rate of physical, cognitive, and social maturation as they traverse adolescence and enter adulthood. Differences in developmental paths are thought to underlie individual differences in later life outcomes, however, there remains a lack of consensus on the normative trajectory of cognitive maturation in adolescence. To address this problem, we derive a Cognitive Maturity Index (CMI), to estimate the difference between chronological and cognitive age predicted with latent factor estimates of inhibitory control, risky decision-making and emotional processing measured with standard neuropsychological instruments. One hundred and forty-one children from the Adolescent Development Study (ADS) were followed longitudinally across three time points from ages 11-14, 13-16, and 14-18. Age prediction with latent factor estimates of cognitive skills approximated age within ±10 months (r = 0.71). Males in advanced puberty displayed lower cognitive maturity relative to peers of the same age; manifesting as weaker inhibitory control, greater risk-taking, desensitization to negative affect, and poor recognition of positive affect.

Real-World Exploration Increases Across Adolescence and Relates to Affect, Risk Taking, and Social Connectivity

Cross-species research suggests that exploratory behaviors increase during adolescence and relate to the social, affective, and risky behaviors characteristic of this developmental stage. However, how these typical adolescent behaviors manifest and relate in real-world settings remains unclear. Using geolocation tracking to quantify exploration—variability in daily movement patterns—over a 3-month period in 58 adolescents and adults (ages 13–27) in New York City, we investigated whether daily exploration varied with age and whether exploration related to social connectivity, risk taking, and momentary positive affect. In our cross-sectional sample, we found an association between daily exploration and age, with individuals near the transition to legal adulthood exhibiting the highest exploration levels. Days of higher exploration were associated with greater positive affect irrespective of age. Higher mean exploration was associated with greater social connectivity in all participants but was linked to higher risk taking selectively among adolescents. Our results highlight the interplay of exploration and socioemotional behaviors across development and suggest that societal norms may modulate their expression in naturalistic contexts.

Flexibility in valenced reinforcement learning computations across development

Optimal integration of positive and negative outcomes during learning varies depending on an environment's reward statistics. The present study investigated the extent to which children, adolescents, and adults (N = 142 8-25 year-olds, 55% female, 42% White, 31% Asian, 17% mixed race, and 8% Black; data collected in 2021) adapt their weighting of better-than-expected and worse-than-expected outcomes when learning from reinforcement. Participants made choices across two contexts: one in which weighting positive outcomes more heavily than negative outcomes led to better performance, and one in which the reverse was true. Reinforcement learning modeling revealed that across age, participants shifted their valence biases in accordance with environmental structure. Exploratory analyses revealed strengthening of context-dependent flexibility with increasing age.

Developmental shifts in computations used to detect environmental controllability

Accurate assessment of environmental controllability enables individuals to adaptively adjust their behavior—exploiting rewards when desirable outcomes are contingent upon their actions and minimizing costly deliberation when their actions are inconsequential. However, it remains unclear how estimation of environmental controllability changes from childhood to adulthood. Ninety participants (ages 8–25) completed a task that covertly alternated between controllable and uncontrollable conditions, requiring them to explore different actions to discover the current degree of environmental controllability. We found that while children were able to distinguish controllable and uncontrollable conditions, accuracy of controllability assessments improved with age. Computational modeling revealed that whereas younger participants’ controllability assessments relied on evidence gleaned through random exploration, older participants more effectively recruited their task structure knowledge to make highly informative interventions. Age-related improvements in working memory mediated this qualitative shift toward increased use of an inferential strategy. Collectively, these findings reveal an age-related shift in the cognitive processes engaged to assess environmental controllability. Improved detection of environmental controllability may foster increasingly adaptive behavior over development by revealing when actions can be leveraged for one’s benefit.

The ability to determine when one’s actions are consequential organizes learning and decision making across the lifespan. However, few studies have examined how the ability to detect control over our environment changes from childhood to adulthood. Here, we leveraged a computational modeling framework to characterize the component learning processes underlying controllability assessment in children, adolescents, and adults. We observed age-related improvements in controllability assessment that stemmed from an increasing ability to represent contingencies between states and actions and to use that knowledge to make informative interventions that yield diagnostic evidence of the current degree of control. Increasing ability to accurately assess environmental controllability may confer greater recognition of opportunities to adaptively pursue rewards through goal-directed action across development.

The computational roots of positivity and confirmation biases in reinforcement learning

Humans do not integrate new information objectively: outcomes carrying a positive affective value and evidence confirming one's own prior belief are overweighed. Until recently, theoretical and empirical accounts of the positivity and confirmation biases assumed them to be specific to 'high-level' belief updates. We present evidence against this account. Learning rates in reinforcement learning (RL) tasks, estimated across different contexts and species, generally present the same characteristic asymmetry, suggesting that belief and value updating processes share key computational principles and distortions. This bias generates over-optimistic expectations about the probability of making the right choices and, consequently, generates over-optimistic reward expectations. We discuss the normative and neurobiological roots of these RL biases and their position within the greater picture of behavioral decision-making theories.

Impaired learning to dissociate advantageous and disadvantageous risky choices in adolescents

Adolescence is characterized by a surge in maladaptive risk-taking behaviors, but whether and how this relates to developmental changes in experience-based learning is largely unknown. In this preregistered study, we addressed this issue using a novel task that allowed us to separate the learning-driven optimization of risky choice behavior over time from overall risk-taking tendencies. Adolescents (12-17 years old) learned to dissociate advantageous from disadvantageous risky choices less well than adults (20-35 years old), and this impairment was stronger in early than mid-late adolescents. Computational modeling revealed that adolescents' suboptimal performance was largely due to an inefficiency in core learning and choice processes. Specifically, adolescents used a simpler, suboptimal, expectation-updating process and a more stochastic choice policy. In addition, the modeling results suggested that adolescents, but not adults, overvalued the highest rewards. Finally, an exploratory latent-mixture model analysis indicated that a substantial proportion of the participants in each age group did not engage in experience-based learning but used a gambler's fallacy strategy, stressing the importance of analyzing individual differences. Our results help understand why adolescents tend to make more, and more persistent, maladaptive risky decisions than adults when the values of these decisions have to be learned from experience.

Uncertainty alters the balance between incremental learning and episodic memory

A key question in decision-making is how humans arbitrate between competing learning and memory systems to maximize reward. We address this question by probing the balance between the effects, on choice, of incremental trial-and-error learning versus episodic memories of individual events. Although a rich literature has studied incremental learning in isolation, the role of episodic memory in decision-making has only recently drawn focus, and little research disentangles their separate contributions. We hypothesized that the brain arbitrates rationally between these two systems, relying on each in circumstances to which it is most suited, as indicated by uncertainty. We tested this hypothesis by directly contrasting contributions of episodic and incremental influence to decisions, while manipulating the relative uncertainty of incremental learning using a well-established manipulation of reward volatility. Across two large, independent samples of young adults, participants traded these influences off rationally, depending more on episodic information when incremental summaries were more uncertain. These results support the proposal that the brain optimizes the balance between different forms of learning and memory according to their relative uncertainties and elucidate the circumstances under which episodic memory informs decisions.