Effect of aging on stimulus-reward association learning

Impact of a Nutrient Formulation on Longitudinal Myelination, Cognition, and Behavior from Birth to 2 Years: A Randomized Clinical Trial

Observation studies suggest differences in myelination in relation to differences in early life nutrition. This two-center randomized controlled trial investigates the effect of a 12-month nutritional intervention on longitudinal changes in myelination, cognition, and behavior. Eighty-one full-term, neurotypical infants were randomized into an investigational (N = 42) or a control group (N = 39), receiving higher versus lower levels of a blend of nutrients. Non-randomized breastfed infants (N = 108) served as a reference group. Main outcomes were myelination (MRI), neurodevelopment (Bayley-III), social-emotional development (ASQ:SE-2), infant and toddler behavior (IBQ-R and TBAQ), and infant sleep (BISQ) during the first 2 years of life. The full analysis set comprised N = 67 infants from the randomized groups, with 81 myelin-sensitive MRI sequences. Significantly higher myelination was observed in the investigational compared to the control group at 6, 12, 18, and 24 months of life, as well as significantly higher gray matter volume at 24 months, a reduced number of night awakenings at 6 months, increased day sleep at 12 months, and reduced social fearfulness at 24 months. The results suggest that brain development may be modifiable with brain- and age-relevant nutritional approaches in healthy infants and young children, which may be foundational for later learning outcomes.

Multivariate associations between dopamine receptor availability and risky investment decision-making across adulthood

Enhancing dopamine increases financial risk taking across adulthood but it is unclear whether baseline individual differences in dopamine function are related to risky financial decisions. Here, thirty-five healthy adults completed an incentive-compatible risky investment decision task and a PET scan at rest using [11C]FLB457 to assess dopamine D2-like receptor availability. Participants made choices between a safe asset (bond) and a risky asset (stock) with either an expected value less than the bond ("bad stock") or expected value greater than the bond ("good stock"). Five measures of behavior (choice inflexibility, risk seeking, suboptimal investment) and beliefs (absolute error, optimism) were computed and D2-like binding potential was extracted from four brain regions of interest (midbrain, amygdala, anterior cingulate, insula). We used canonical correlation analysis to evaluate multivariate associations between decision-making and dopamine function controlling for age. Decomposition of the first dimension (r = 0.76) revealed that the strongest associations were between measures of choice inflexibility, incorrect choice, optimism, amygdala binding potential, and age. Follow-up univariate analyses revealed that amygdala binding potential and age were both independently associated with choice inflexibility. The findings suggest that individual differences in dopamine function may be associated with financial risk taking in healthy adults.

Neural regions associated with gain-loss frequency and average reward in older and younger adults

Research on the biological basis of reinforcement-learning has focused on how brain regions track expected value based on average reward. However, recent work suggests that humans are more attuned to reward frequency. Furthermore, older adults are less likely to use expected values to guide choice than younger adults. This raises the question of whether brain regions assumed to be sensitive to average reward, like the medial and lateral PFC, also track reward frequency, and whether there are age-based differences. Older (60-81 years) and younger (18-30 years) adults performed the Soochow Gambling task, which separates reward frequency from average reward, while undergoing fMRI. Overall, participants preferred options that provided negative net payoffs, but frequent gains. Older adults improved less over time, were more reactive to recent negative outcomes, and showed greater frequency-related activation in several regions, including DLPFC. We also found broader recruitment of prefrontal and parietal regions associated with frequency value and reward prediction errors in older adults, which may indicate compensation. The results suggest greater reliance on average reward for younger adults than older adults.

The Effects of Cannabis Use on Cognitive Function in Healthy Aging: A Systematic Scoping Review

BACKGROUND

Older adults (≥50 years) represent the fastest-growing population of people who use cannabis, potentially due to the increasing promotion of cannabis as medicine by dispensaries and cannabis websites. Given healthy aging and cannabis use are both associated with cognitive decline, it is important to establish the effects of cannabis on cognition in healthy aging.

OBJECTIVE

This systematic scoping review used preferred reporting items for systematic reviews and meta-analyses guidelines to critically examine the extent of literature on this topic and highlight areas for future research.

METHOD

A search of six databases (PubMed, EMBASE, PsycINFO, Web of Science, Family and Society Studies Worldwide, and CINAHL) for articles published by September 2019, yielded 1,014 unique results.

RESULTS

Six articles reported findings for older populations (three human and three rodent studies), highlighting the paucity of research in this area. Human studies revealed largely null results, likely due to several methodological limitations. Better-controlled rodent studies indicate that the relationship between ∆9-tetrahydrocannabinol (THC) and cognitive function in healthy aging depends on age and level of THC exposure. Extremely low doses of THC improved cognition in very old rodents. Somewhat higher chronic doses improved cognition in moderately aged rodents. No studies examined the effects of cannabidiol (CBD) or high-CBD cannabis on cognition.

CONCLUSIONS

This systematic scoping review provides crucial, timely direction for future research on this emerging issue. Future research that combines neuroimaging and cognitive assessment would serve to advance understanding of the effects of age and quantity of THC and CBD on cognition in healthy aging.

Ageing is associated with disrupted reinforcement learning whilst learning to help others is preserved

Reinforcement learning is a fundamental mechanism displayed by many species. However, adaptive behaviour depends not only on learning about actions and outcomes that affect ourselves, but also those that affect others. Using computational reinforcement learning models, we tested whether young (age 18-36) and older (age 60-80, total n = 152) adults learn to gain rewards for themselves, another person (prosocial), or neither individual (control). Detailed model comparison showed that a model with separate learning rates for each recipient best explained behaviour. Young adults learned faster when their actions benefitted themselves, compared to others. Compared to young adults, older adults showed reduced self-relevant learning rates but preserved prosocial learning. Moreover, levels of subclinical self-reported psychopathic traits (including lack of concern for others) were lower in older adults and the core affective-interpersonal component of this measure negatively correlated with prosocial learning. These findings suggest learning to benefit others is preserved across the lifespan with implications for reinforcement learning and theories of healthy ageing.

Target value and prevalence influence visual foraging in younger and older age

The prevalence and reward-value of targets have an influence on visual search. The strength of the effect of an item's reward-value on attentional selection varies substantially between individuals and is potentially sensitive to aging. We investigated individual and age differences in a hybrid foraging task, in which the prevalence and value of multiple target types was varied. Using optimal foraging theory measures, foraging was more efficient overall in younger than older observers. However, the influence of prevalence and value on target selections was similar across age groups, suggesting that the underlying cognitive mechanisms are preserved in older age. When prevalence was varied but target value was balanced, younger and older observers preferably selected the most frequent target type and were biased to select another instance of the previously selected target type. When value was varied, younger and older observers showed a tendency to select high-value targets, but preferences were more diverse between individuals. When value and prevalence were inversely related, some observers showed particularly strong preferences for high-valued target types, while others showed a preference for high-prevalent, albeit low-value, target types. In younger adults, individual differences in the selection choices correlated with a personality index, suggesting that avoiding selections of low-value targets may be related to reward-seeking behaviour.

Age Differences in the Subcomponents of Executive Functioning

OBJECTIVES

Across the life span, deficits in executive functioning (EF) are associated with poor behavioral control and failure to achieve goals. Though EF is often discussed as one broad construct, a prominent model of EF suggests that it is composed of three subdomains: inhibition, set shifting, and updating. These subdomains are seen in both younger (YA) and older adults (OA), with performance deficits across subdomains in OA. Therefore, our goal was to investigate whether subdomains of EF might be differentially affected by age, and how these differences may relate to broader global age differences in EF.

METHODS

To assess these age differences, we conducted a meta-analysis at multiple levels, including task level, subdomain level, and of global EF. Based on previous work, we hypothesized that there would be overall differences in EF in OA.

RESULTS

Using 1,268 effect sizes from 401 articles, we found overall differences in EF with age. Results suggested that differences in performance are not uniform, such that variability in age effects emerged at the task level, and updating was not as affected by age as other subdomains.

DISCUSSION

These findings advance our understanding of age differences in EF, and stand to inform early detection of EF decline.

Diminished Feedback Evaluation and Knowledge Updating Underlying Age-Related Differences in Choice Behavior During Feedback Learning

In our daily lives, we continuously evaluate feedback information, update our knowledge, and adapt our behavior in order to reach desired goals. This ability to learn from feedback information, however, declines with age. Previous research has indicated that certain higher-level learning processes, such as feedback evaluation, integration of feedback information, and updating of knowledge, seem to be affected by age, and recent studies have shown how the adaption of choice behavior following feedback can differ with age. The neural mechanisms underlying this age-related change in choice behavior during learning, however, remain unclear. The aim of this study is therefore to investigate the relation between learning-related neural processes and choice behavior during feedback learning in two age groups. Behavioral and fMRI data were collected, while a group of young (age 18–30) and older (age 60–75) adults performed a probabilistic learning task consisting of 10 blocks of 20 trials each. On each trial, the participants chose between a house and a face, after which they received visual feedback (loss vs. gain). In each block, either the house or the face image had a higher probability of yielding a reward (62.5 vs. 37.5%). Participants were instructed to try to maximize their gains. Our results showed that less successful learning in older adults, as indicated by a lower learning rate, corresponded with a higher tendency to switch to the other stimulus option, and with a reduced adaptation of this switch choice behavior following positive feedback. At the neural level, activation following positive and negative feedback was found to be less distinctive in the older adults, due to a smaller feedback-evaluation response to positive feedback in this group. Furthermore, whereas young adults displayed increased levels of knowledge updating prior to adapting choice behavior, we did not find this effect in older adults. Together, our results suggest that diminished learning performance with age corresponds with diminished evaluation of positive feedback and reduced knowledge updating related to changes in choice behavior, indicating how such differences in feedback processing at the trial level in older adults might lead to reduced learning performance across trials.

Reversal learning in young and middle-age neurotypicals: Individual difference reaction time considerations

Reversal learning is frequently used to assess components of executive function that contribute to understanding age-related cognitive differences. Reaction time (RT) is less characterized in the reversal learning literature, perhaps due to the daunting task of analyzing the entire RT distribution, but has been deemed a generally sensitive measure of cognitive aging. The current study extends our prior work to further characterize distributional properties of the reversal RT distribution and to distinguish groups of individuals with fractionated profiles of performance, which may be of clinical importance within the context of cognitive aging. Participant sample included young (n = 43) and community-dwelling, healthy, middle-aged (n = 139) adults. To explore individual differences, recursive partitioning analysis achieved a high classification rate by specifying decision tree rules that split participants into young and middle-aged groups. Mu (μ, efficient RT) was the most successful parameter in distinguishing age groups while sigma ( σ ) and tau ( τ , ex-Gaussian indices of intra-individual variability) revealed more subtle individual differences. Accuracy measures did not contribute to separating the groups, suggesting that fractionated components of RT, as opposed to accuracy, can distinguish differences between young and middle-aged participants.

Corticostriatal White Matter Integrity and Dopamine D1 Receptor Availability Predict Age Differences in Prefrontal Value Signaling during Reward Learning

Probabilistic reward learning reflects the ability to adapt choices based on probabilistic feedback. The dopaminergically innervated corticostriatal circuit in the brain plays an important role in supporting successful probabilistic reward learning. Several components of the corticostriatal circuit deteriorate with age, as it does probabilistic reward learning. We showed previously that D1 receptor availability in NAcc predicts the strength of anticipatory value signaling in vmPFC, a neural correlate of probabilistic learning that is attenuated in older participants and predicts probabilistic reward learning performance. We investigated how white matter integrity in the pathway between nucleus accumbens (NAcc) and ventromedial prefrontal cortex (vmPFC) relates to the strength of anticipatory value signaling in vmPFC in younger and older participants. We found that in a sample of 22 old and 23 young participants, fractional anisotropy in the pathway between NAcc and vmPFC predicted the strength of value signaling in vmPFC independently from D1 receptor availability in NAcc. These findings provide tentative evidence that integrity in the dopaminergic and white matter pathways of corticostriatal circuitry supports the expression of value signaling in vmPFC which supports reward learning, however, the limited sample size calls for independent replication. These and future findings could add to the improved understanding of how corticostriatal integrity contributes to reward learning ability.

Relevance of working memory for reinforcement learning in older adults varies with timescale of learning

In young adults, individual differences in working memory (WM) contribute to reinforcement learning (RL). Age-related RL changes, however, are mostly attributed to decreased reward prediction-error (RPE) signaling. Here, we investigated the contribution of WM to RL in young (18-35) and older (≥65) adults. Because WM supports maintenance across a limited timescale, we only expected a relation between RL and WM with short delays between stimulus repetitions. Our results demonstrated better learning with short than long delays. A week later, however, long-delay associations were remembered better. Computational modeling corroborated that during learning, WM was more engaged by young adults in the short-delay condition than in any other age-condition combination. Crucially, both model-derived and neuropsychological assessments of WM predicted short-delay learning in older adults, who further benefitted from using self-conceived learning strategies. Thus, depending on the timescale of learning, age-related RL changes may not only reflect decreased RPE signaling but also WM decline.

Age-related variability in decision-making: Insights from neurochemistry

Despite dopamine's significant role in models of value-based decision-making and findings demonstrating loss of dopamine function in aging, evidence of systematic changes in decision-making over the life span remains elusive. Previous studies attempting to resolve the neural basis of age-related alteration in decision-making have typically focused on physical age, which can be a poor proxy for age-related effects on neural systems. There is growing appreciation that aging has heterogeneous effects on distinct components of the dopamine system within subject in addition to substantial variability between subjects. We propose that some of the conflicting findings in age-related effects on decision-making may be reconciled if we can observe the underlying dopamine components within individuals. This can be achieved by incorporating in vivo imaging techniques including positron emission tomography (PET) and neuromelanin-sensitive MR. Further, we discuss how affective factors may contribute to individual differences in decision-making performance among older adults. Specifically, we propose that age-related shifts in affective attention ("positivity effect") can, in some cases, counteract the impact of altered dopamine function on specific decision-making processes, contributing to variability in findings. In an effort to provide clarity to the field and advance productive hypothesis testing, we propose ways in which in vivo dopamine imaging can be leveraged to disambiguate dopaminergic influences on decision-making, and suggest strategies for assessing individual differences in the contribution of affective attentional focus.

Emotional feedback ameliorates older adults' feedback-induced learning

In older age, learning and feedback processing are usually impaired. This is thought to be due to impairments in the dopaminergic system and the anterior cingulate cortex. By contrast, processing of affective information seems to remain relatively intact. Recent research has also demonstrated that cognitive functioning can be influenced by affective materials or contexts and lead to an enhancement in diverse cognitive tasks. Hence, the aim of the present study was to explore, whether emotional feedback would counteract age-related learning deficits and strengthen early and later phases of feedback processing as reflected in the feedback-related negativity (FRN) and P3b of the event-related potential (ERP). Younger and older participants conducted a probabilistic reinforcement learning task in which the accurate responses had to be learned via feedback. In emotional trials, feedback stimuli consisted of faces with smiling and disgusted expressions, and in a non-emotional condition, positive and negative feedback was indicated by the background color of faces with neutral expressions. Our main results were that older adults showed better learning performance in the emotional feedback condition and a larger P3b after emotional than non-emotional feedback indexing heightened working memory updating after task relevant events.

A distributional and theoretical analysis of reaction time in the reversal task across adulthood

Reversal learning assesses components of executive function important for understanding cognitive changes with age. Extant reversal learning literature has largely assessed measures of accuracy, but reaction time (RT) has not yet been well characterized, perhaps due to the daunting task of analyzing non-normal RT distributions. The current study contributes to the literature by examining distributional and theoretical aspects of the entire RT distribution in addition to accuracy. Participant sample included young (N = 43) and community-dwelling, healthy, middle-aged (N = 139) adults. Results showed a Normal-3 Mixture distribution best fits the sample as a whole, with the ex-Gaussian distribution passing visual inspection. Age related significantly to various measures of RT (p's < 0.5); older age was associated with higher both efficient and overall RT, perhaps due to a more conservative criterion of decision-making. In a generalized adaptive elastic net regression, RT explained age-related differences in performance while accuracy did not contribute. Specifically, middle-aged adults were slower in efficient RT and had increased intra-individual variability which has been previously linked to poorer frontal lobe processes and age-related cognitive decline. Overall, these findings highlight the importance of examining the entire RT distribution and measuring RT as a fractionated construct to further explain age-related differences in reversal learning, even in middle-aged individuals.

The Role of the Striatum in Learning to Orthogonalize Action and Valence: A Combined PET and 7 T MRI Aging Study

Pavlovian biases influence instrumental learning by coupling reward seeking with action invigoration and punishment avoidance with action suppression. Using a probabilistic go/no-go task designed to orthogonalize action (go/no-go) and valence (reward/punishment), recent studies have shown that the interaction between the two is dependent on the striatum and its key neuromodulator dopamine. Using this task, we sought to identify how structural and neuromodulatory age-related differences in the striatum may influence Pavlovian biases and instrumental learning in 25 young and 31 older adults. Computational modeling revealed a significant age-related reduction in reward and punishment sensitivity and marked (albeit not significant) reduction in learning rate and lapse rate (irreducible noise). Voxel-based morphometry analysis using 7 Tesla MRI images showed that individual differences in learning rate in older adults were related to the volume of the caudate nucleus. In contrast, dopamine synthesis capacity in the dorsal striatum, assessed using [18F]-DOPA positron emission tomography in 22 of these older adults, was not associated with learning performance and did not moderate the relationship between caudate volume and learning rate. This multiparametric approach suggests that age-related differences in striatal volume may influence learning proficiency in old age.

Intact Reinforcement Learning But Impaired Attentional Control During Multidimensional Probabilistic Learning in Older Adults

To efficiently learn optimal behavior in complex environments, humans rely on an interplay of learning and attention. Healthy aging has been shown to independently affect both of these functions. Here, we investigate how reinforcement learning and selective attention interact during learning from trial and error across age groups. We acquired behavioral and fMRI data from older and younger adults (male and female) performing two probabilistic learning tasks with varying attention demands. Although learning in the unidimensional task did not differ across age groups, older adults performed worse than younger adults in the multidimensional task, which required high levels of selective attention. Computational modeling showed that choices of older adults are better predicted by reinforcement learning than Bayesian inference, and that older adults rely more on reinforcement learning-based predictions than younger adults. Conversely, a higher proportion of younger adults' choices was predicted by a computationally demanding Bayesian approach. In line with the behavioral findings, we observed no group differences in reinforcement-learning related fMRI activation. Specifically, prediction-error activation in the nucleus accumbens was similar across age groups, and numerically higher in older adults. However, activation in the default mode was less suppressed in older adults in for higher attentional task demands, and the level of suppression correlated with behavioral performance. Our results indicate that healthy aging does not significantly impair simple reinforcement learning. However, in complex environments, older adults rely more heavily on suboptimal reinforcement-learning strategies supported by the ventral striatum, whereas younger adults use attention processes supported by cortical networks.SIGNIFICANCE STATEMENT Changes in the way that healthy human aging affects how we learn to optimally behave are not well understood; it has been suggested that age-related declines in dopaminergic function may impair older adult's ability to learn from reinforcement. In the present fMRI experiment, we show that learning and nucleus accumbens activation in a simple unidimensional reinforcement-learning task was not significantly affected by age. However, in a more complex multidimensional task, older adults showed worse performance and relied more on reinforcement-learning strategies than younger adults, while failing to disengage their default-mode network during learning. These results imply that older adults are only impaired in reinforcement learning if they additionally need to learn which dimensions of the environment are currently important.

Impulsivity and compulsivity are differentially associated with automaticity and routine on the Creature of Habit Scale

Habits may develop when meaningful action patterns are frequently repeated in a stable environment. We measured the differing tendencies of people to form habits in a population sample of n = 533 using the Creature of Habit Scale (COHS). We confirmed the high reliability of the two latent factors measured by the COHS, automaticity and routines. Whilst automatic behaviours are triggered by context and do not serve a particular purpose or goal, routines often have purpose, and because they have been performed so often in a given context, they become automatic only after their action sequence has been activated. We found that both types of habitual behaviours are influenced by the frequency of their occurrence and they are differentially influenced by personality traits. Compulsive personality is associated with an increase in both aspects of habitual tendency, whereas impulsivity is linked with increased automaticity, but reduced routine behaviours. Our findings provide further evidence that the COHS is a useful tool for understanding habitual tendencies in the general population and may inform the development of therapeutic strategies that capitalise on functional habits and help to treat dysfunctional ones.

Effect of Age and Dietary Intervention on Discrimination Learning in Pet Dogs

Aging is associated with a decline in cognitive functions such as learning, memory, attention, cognitive flexibility, and executive functions. Recent evidence indicates that interventions such as exercise, diet and cognitive training can be used to reduce the rate of age-dependent cognitive decline. In this study, we examined the changes in discrimination learning in older pet dogs, tested whether a dietary intervention counteracts a potential decline in learning and evaluated the influence of lifelong training on learning speed and cognitive flexibility. We included 115 pet dogs (>6 years) of 30 different breeds into one of two treatment groups: either a diet enriched with antioxidants, docosahexaenoic acid (DHA), Phosphatidylserine and tryptophan or a control diet for 1 year. Lifelong training was calculated for each dog using a questionnaire where owners filled their dog’s training experiences over years. Dogs were trained to discriminate different pictures at the start of the dietary intervention using a touch screen methodology. After 1 year of dietary intervention, they were tested on a main picture discrimination task where they were confronted with a discrimination of four new pictures. We used the total number of sessions needed to reach learning criterion as a measure of learning speed and the rate of correction trials as a measure of deficit in learning from feedback/cognitive flexibility. In the main discrimination task, we found an influence of neither age nor diet on the speed of learning and deficit in learning from feedback. We did not find any influence of lifelong training either. The null findings were further corroborated by Bayesian statistics. The null findings might be due to the fact that pet dogs live in a stimulating environment which may reduce the rate of cognitive decline and hinder finding an age or diet effect. Also, the similarity between the training and the main discrimination task might have made the main task too easy for the animals to solve. Further studies are warranted to assess the effect of enriched diets on pet dogs using tasks that measure cognitive functions with a higher sensitivity.

One-year change in cognitive flexibility and fine motor function in middle-aged male and female marmosets (Callithrix jacchus)

The common marmoset (Callithrix jacchus) is uniquely suited for longitudinal studies of cognitive aging, due to a relatively short lifespan, sophisticated cognitive abilities, and patterns of brain aging that resemble those of humans. We examined cognitive function and fine motor skills in male and female marmosets (mean age ∼5 at study entry) followed longitudinally for 2 years. Each year, monkeys were tested on a reversal learning task with three pairs of stimuli (n = 18, 9 females) and a fine motor task requiring them to grasp small rewards from two staircases (Hill and Valley test, n = 12, 6 females). There was little evidence for a decline in cognitive flexibility between the two time points, in part because of practice effects. However, independent of year of testing, females took longer than males to reach criterion in the reversals, indicating impaired cognitive flexibility. Motivation was unlikely to contribute to this effect, as males refused a greater percentage of trials than females in the reversals. With regards to motor function, females were significantly faster than males in the Hill and Valley task. From Year 1 to Year 2, a slight slowing of motor function was observed in both sexes, but accuracy decreased significantly in males only. This study (1) demonstrates that marmosets exhibit sex differences in cognitive flexibility and fine motor function that resemble those described in humans; (2) that changes in fine motor function can already be detected at middle-age; and (3) that males may experience greater age-related changes in fine motor skills than females. Additional data points will determine whether these sex and age differences persist over time.

Feedback-Based Learning in Aging: Contributions and Trajectories of Change in Striatal and Hippocampal Systems

The striatum supports learning from immediate feedback by coding prediction errors (PEs), whereas the hippocampus (HC) plays a parallel role in learning from delayed feedback. Both regions show evidence of decline in human aging, but behavioral research suggests greater decline in HC versus striatal functions. The present study included male and female humans and used fMRI to examine younger and older adults' brain activation patterns during a learning task with choice feedback presented immediately or after a brief delay. Participants then completed a surprise memory task that tested their recognition of trial-unique feedback stimuli, followed by assessments of postlearning cue preference, outcome probability awareness, and willingness to pay. The study yielded three main findings. First, behavioral measures indicated similar rates of learning in younger and older adults across conditions, but postlearning measures indicated impairment in older adults' ability to subsequently apply learning to discriminate between cues. Second, PE signals in the striatum were greater for immediate versus delayed feedback in both age groups, but PE signals in the HC were greater for delayed versus immediate feedback only in younger adults. Third, unlike younger adults, older adults failed to exhibit enhanced episodic memory for outcome stimuli in the delayed-feedback condition. Together, these findings indicate that HC circuits supporting learning and memory decline more than striatal circuits in healthy aging, which suggests that declines in HC learning signals may be an important predictor of deficits in learning-dependent economic decisions among older adults.SIGNIFICANCE STATEMENT The hippocampus (HC) and striatum play distinct and critical roles in learning. Substantial research suggests that age-related decline in learning supported by the HC outpaces decline in learning supported by the striatum; however, such inferences have been drawn by comparing performance in tasks with fundamentally different structures. The present study overcomes this obstacle by implementing a single fMRI-learning paradigm with a subtle variation in feedback timing to examine differential age effects on memory supported by the HC and striatum. Our results provide converging behavioral and brain-imaging evidence showing that HC circuits supporting learning and memory decline more than striatal circuits in healthy aging and that declines in HC learning signals may predict early deficits in learning-dependent decisions among older adults.

Age-dependent Pavlovian biases influence motor decision-making

Motor decision-making is an essential component of everyday life which requires weighing potential rewards and punishments against the probability of successfully executing an action. To achieve this, humans rely on two key mechanisms; a flexible, instrumental, value-dependent process and a hardwired, Pavlovian, value-independent process. In economic decision-making, age-related decline in risk taking is explained by reduced Pavlovian biases that promote action toward reward. Although healthy ageing has also been associated with decreased risk-taking in motor decision-making, it is currently unknown whether this is a result of changes in Pavlovian biases, instrumental processes or a combination of both. Using a newly established approach-avoidance computational model together with a novel app-based motor decision-making task, we measured sensitivity to reward and punishment when participants (n = 26,532) made a ‘go/no-go’ motor gamble based on their perceived ability to execute a complex action. We show that motor decision-making can be better explained by a model with both instrumental and Pavlovian parameters, and reveal age-related changes across punishment- and reward-based instrumental and Pavlovian processes. However, the most striking effect of ageing was a decrease in Pavlovian attraction towards rewards, which was associated with a reduction in optimality of choice behaviour. In a subset of participants who also played an independent economic decision-making task (n = 17,220), we found similar decision-making tendencies for motor and economic domains across a majority of age groups. Pavlovian biases, therefore, play an important role in not only explaining motor decision-making behaviour but also the changes which occur through normal ageing. This provides a deeper understanding of the mechanisms which shape motor decision-making across the lifespan.

Decisions in everyday life often require weighing the probability of successfully executing an action (e.g., successfully crossing a street) against potential rewards and punishments. Although older individuals take fewer risks during such motor decision-making scenarios, the underlying mechanism remains unclear. Similar age-related changes in economic decision-making are explained by a decrease in Pavlovian attraction toward reward. However, despite the role of Pavlovian biases in linking action with reward and avoidance with punishment, their impact on motor decision-making is unclear. To address this, we developed a novel app-based motor decision-making task (n = 26,532). We found that motor decision-making was subject to Pavlovian influences. Although we found age-related changes for both punishment and reward-based decision-making processes, the most striking effect of ageing was a decrease in the facilitatory effect of Pavlovian attraction on action in pursuit of reward. Using data from an independent economic decision task in the same individuals (n = 17,220), we demonstrate similar decision-making tendencies for motor and economic domains across a majority of age groups. Hence, Pavlovian biases play an essential role in not only explaining motor decision-making behaviour but also the changes which occur through normal ageing.

Age affects reinforcement learning through dopamine-based learning imbalance and high decision noise-not through Parkinsonian mechanisms

Probabilistic reinforcement learning declines in healthy cognitive aging. While some findings suggest impairments are especially conspicuous in learning from rewards, resembling deficits in Parkinson's disease, others also show impairments in learning from punishments. To reconcile these findings, we tested 252 adults from 3 age groups on a probabilistic reinforcement learning task, analyzed trial-by-trial performance with a Q-reinforcement learning model, and correlated both fitted model parameters and behavior to polymorphisms in dopamine-related genes. Analyses revealed that learning from both positive and negative feedback declines with age but through different mechanisms: when learning from negative feedback, older adults were slower due to noisy decision-making; when learning from positive feedback, they tended to settle for a nonoptimal solution due to an imbalance in learning from positive and negative prediction errors. The imbalance was associated with polymorphisms in the DARPP-32 gene and appeared to arise from mechanisms different from those previously attributed to Parkinson's disease. Moreover, this imbalance predicted previous findings on aging using the Probabilistic Selection Task, which were misattributed to Parkinsonian mechanisms.

Independent effects of age and levodopa on reversal learning in healthy volunteers

The dopamine overdose hypothesis has provided an important theoretical framework for understanding cognition in Parkinson's disease. It posits that effects of dopaminergic therapy on cognition in Parkinson's disease depend on baseline dopamine levels in brain regions that support different functions. Although functions performed by more severely dopamine-depleted brain regions improve with medication, those associated with less dopamine deficient areas are actually worsened. It is presumed that medication-related worsening of cognition owes to dopamine overdose. We investigated whether age-related changes in baseline dopamine levels would modulate effects of dopaminergic therapy on reward learning in healthy volunteers. In a double-blind, crossover design, healthy younger and older adults completed a probabilistic reversal learning task after treatment with 100/25 mg of levodopa/carbidopa versus placebo. Older adults learned more poorly than younger adults at baseline, being more likely to shift responses after misleading punishment. Levodopa worsened stimulus-reward learning relative to placebo to the same extent in both groups, irrespective of differences in baseline performance and expected dopamine levels. When order effects were eliminated, levodopa induced response shifts after reward more often than placebo. Our results reveal independent deleterious effects of age group and exogenous dopamine on reward learning, suggesting a more complex scenario than predicted by the dopamine overdose hypothesis.

Age-differences in cognitive flexibility when overcoming a preexisting bias through feedback

INTRODUCTION

Older adults are often worse than younger adults at adapting to changing situational demands, and this difference is commonly attributed to an age-related decline in acquiring and updating information. Previous research on aging and cognitive flexibility has used measures that require adapting to novel associations learned during a laboratory task (e.g., choice X led to positive outcomes but now leads to negative outcomes). However, in everyday life people must frequently overcome associations based on preexisting beliefs and biases (e.g., you like to eat cake, but your doctor said to limit your sugar intake). The goal of the present study was to examine possible age-differences in overcoming a preexisting bias and determine whether age-related changes in the acquisition and updating of information influence this form of flexibility.

METHOD

Older (n = 20) and younger (n = 20) adults completed a novel task in which repeated choices were made between a sure option (gain or loss) and one of two risky options that were initially ambiguous. Optimal performance required overcoming a framing bias toward being risk seeking to avoid a sure loss and risk averse when offered a sure gain. Probe questions assessed knowledge of choice outcomes, while skin conductance assessed physiological reactions to choices and choice outcomes.

RESULTS

Both older and younger adults demonstrated flexibility by reducing the impact of bias over trials, but younger adults had better performance overall. Age-differences were associated with distinct aspects of processing. Young adults had more precise knowledge of choice outcomes and developed skin conductance responses in anticipation of bad choices that were not apparent in older adults.

CONCLUSIONS

Older adults showed significant improvement over trials in their ability to decrease bias-driven choices, but younger showed greater flexibility. Age-differences in task performance were based on differences in learning and corresponding representations of task-relevant information.

Attenuation of dopamine-modulated prefrontal value signals underlies probabilistic reward learning deficits in old age

Probabilistic reward learning is characterised by individual differences that become acute in aging. This may be due to age-related dopamine (DA) decline affecting neural processing in striatum, prefrontal cortex, or both. We examined this by administering a probabilistic reward learning task to younger and older adults, and combining computational modelling of behaviour, fMRI and PET measurements of DA D1 availability. We found that anticipatory value signals in ventromedial prefrontal cortex (vmPFC) were attenuated in older adults. The strength of this signal predicted performance beyond age and was modulated by D1 availability in nucleus accumbens. These results uncover that a value-anticipation mechanism in vmPFC declines in aging, and that this mechanism is associated with DA D1 receptor availability.

The association between endogenous testosterone level and behavioral flexibility in young men - Evidence from stimulus-outcome reversal learning

The capacity to flexibly adapt responding to unexpected changes in the environment is crucial for survival. Several neurotransmitters have been implicated in stimulus-outcome reversal learning. Yet, it remains an open question whether inter-individual differences in the neuroactive hormone testosterone may also be related to this type of behavioral flexibility. In this study we assessed the association between endogenous testosterone level and reversal learning in young healthy men. We used an observer reversal learning task, in which subjects viewed computer-based decisions between two stimuli, of which one was currently rewarded while the other one was punished. Contingencies reversed unpredictably every 5-9 trials. Subjects had to indicate the current outcome association before the actual outcome was revealed. In the trial following an unexpected reversal either the same stimulus from the reversal (experienced reversal), or its alternative, for which the reversal had not yet been shown (inferred reversal), could be chosen by the computer, and subjects had to adapt responding accordingly. We found that testosterone predicted better post-reversal performance. This correlation was strongest in the more difficult inferred reversal condition, particularly in impulsive individuals. Collectively, these data support the view that endogenous testosterone may enhance behavioral flexibility in men, particularly when working memory demand is high and subjects have to update several stimulus-outcome contingencies at the same time. It remains to be further elucidated whether this testosterone effect was achieved through an interaction with dopaminergic transmission or through direct interplay with androgen receptors in the brain regions implicated in reversal learning.

Adult age differences in decision making across domains: Increased discounting of social and health-related rewards

Although research on aging and decision making continues to grow, the majority of studies examine decisions made to maximize monetary earnings or points. It is not clear whether these results generalize to other types of rewards. To investigate this, we examined adult age differences in 92 healthy participants aged 22 to 83. Participants completed 9 hypothetical discounting tasks, which included 3 types of discounting factors (time, probability, effort) across 3 reward domains (monetary, social, health). Participants made choices between a smaller magnitude reward with a shorter time delay/higher probability/lower level of physical effort required and a larger magnitude reward with a longer time delay/lower probability/higher level of physical effort required. Older compared with younger individuals were more likely to choose options that involved shorter time delays or higher probabilities of experiencing an interaction with a close social partner or receiving health benefits from a hypothetical drug. These findings suggest that older adults may be more motivated than young adults to obtain social and health rewards immediately and with certainty. (PsycINFO Database Record

Frontostriatal anatomical connections predict age- and difficulty-related differences in reinforcement learning

Reinforcement learning (RL) is supported by a network of striatal and frontal cortical structures that are connected through white-matter fiber bundles. With age, the integrity of these white-matter connections declines. The role of structural frontostriatal connectivity in individual and age-related differences in RL is unclear, although local white-matter density and diffusivity have been linked to individual differences in RL. Here we show that frontostriatal tract counts in young human adults (aged 18-28), as assessed noninvasively with diffusion-weighted magnetic resonance imaging and probabilistic tractography, positively predicted individual differences in RL when learning was difficult (70% valid feedback). In older adults (aged 63-87), in contrast, learning under both easy (90% valid feedback) and difficult conditions was predicted by tract counts in the same frontostriatal network. Furthermore, network-level analyses showed a double dissociation between the task-relevant networks in young and older adults, suggesting that older adults relied on different frontostriatal networks than young adults to obtain the same task performance. These results highlight the importance of successful information integration across striatal and frontal regions during RL, especially with variable outcomes.

Adult Age Differences in Dual Information Processes: Implications for the Role of Affective and Deliberative Processes in Older Adults' Decision Making

Age differences in affective/experiential and deliberative processes have important theoretical implications for judgment and decision theory and important pragmatic implications for older-adult decision making. Age-related declines in the efficiency of deliberative processes predict poorer-quality decisions as we age. However, age-related adaptive processes, including motivated selectivity in the use of deliberative capacity, an increased focus on emotional goals, and greater experience, predict better or worse decisions for older adults depending on the situation. The aim of the current review is to examine adult age differences in affective and deliberative information processes in order to understand their potential impact on judgments and decisions. We review evidence for the role of these dual processes in judgment and decision making and then review two representative life-span perspectives (based on aging-related changes to cognitive or motivational processes) on the interplay between these processes. We present relevant predictions for older-adult decisions and make note of contradictions and gaps that currently exist in the literature. Finally, we review the sparse evidence about age differences in decision making and how theories and findings regarding dual processes could be applied to decision theory and decision aiding. In particular, we focus on prospect theory ( Kahneman & Tversky, 1979 ) and how prospect theory and theories regarding age differences in information processing can inform one another.

Age differences in learning emerge from an insufficient representation of uncertainty in older adults

Healthy aging can lead to impairments in learning that affect many laboratory and real-life tasks. These tasks often involve the acquisition of dynamic contingencies, which requires adjusting the rate of learning to environmental statistics. For example, learning rate should increase when expectations are uncertain (uncertainty), outcomes are surprising (surprise) or contingencies are more likely to change (hazard rate). In this study, we combine computational modelling with an age-comparative behavioural study to test whether age-related learning deficits emerge from a failure to optimize learning according to the three factors mentioned above. Our results suggest that learning deficits observed in healthy older adults are driven by a diminished capacity to represent and use uncertainty to guide learning. These findings provide insight into age-related cognitive changes and demonstrate how learning deficits can emerge from a failure to accurately assess how much should be learned.

The ability to learn decreases with old age especially in a dynamically changing environment, however the precise nature of this decline is not understood. Nassar and colleagues report that older adults show a reduced ability to learn from uncertain outcomes compared to younger adults.

Aging effects on discrimination learning, logical reasoning and memory in pet dogs

In laboratory dogs, aging leads to a decline in various cognitive domains such as learning, memory and behavioural flexibility. However, much less is known about aging in pet dogs, i.e. dogs that are exposed to different home environments by their caregivers. We used tasks on a touchscreen apparatus to detect differences in various cognitive functions across pet Border Collies aged from 5 months to 13 years. Ninety-five dogs were divided into five age groups and tested in four tasks: (1) underwater photo versus drawing discrimination, (2) clip art picture discrimination, (3) inferential reasoning by exclusion and (4) a memory test with a retention interval of 6 months. The tasks were designed to test three cognitive abilities: visual discrimination learning, logical reasoning and memory. The total number of sessions to reach criterion and the number of correction trials needed in the two discrimination tasks were compared across age groups. The results showed that both measures increased linearly with age, with dogs aged over 13 years displaying slower learning and reduced flexibility in comparison to younger dogs. Inferential reasoning ability increased with age, but less than 10 % of dogs showed patterns of choice consistent with inference by exclusion. No age effect was found in the long-term memory test. In conclusion, the discrimination learning tests used are suitable to detect cognitive aging in pet dogs, which can serve as a basis for comparison to help diagnose cognition-related problems and as a tool to assist with the development of treatments to delay cognitive decline.

The Affective Neuroscience of Aging

Although aging is associated with clear declines in physical and cognitive processes, emotional functioning fares relatively well. Consistent with this behavioral profile, two core emotional brain regions, the amygdala and ventromedial prefrontal cortex, show little structural and functional decline in aging, compared with other regions. However, emotional processes depend on interacting systems of neurotransmitters and brain regions that go beyond these structures. This review examines how age-related brain changes influence processes such as attending to and remembering emotional stimuli, regulating emotion, and recognizing emotional expressions, as well as empathy, risk taking, impulsivity, behavior change, and attentional focus.

Aging and loss decision making: increased risk aversion and decreased use of maximizing information, with correlated rationality and value maximization

We investigated how adult aging specifically alters economic decision-making, focusing on examining alterations in uncertainty preferences (willingness to gamble) and choice strategies (what gamble information influences choices) within both the gains and losses domains. Within each domain, participants chose between certain monetary outcomes and gambles with uncertain outcomes. We examined preferences by quantifying how uncertainty modulates choice behavior as if altering the subjective valuation of gambles. We explored age-related preferences for two types of uncertainty, risk, and ambiguity. Additionally, we explored how aging may alter what information participants utilize to make their choices by comparing the relative utilization of maximizing and satisficing information types through a choice strategy metric. Maximizing information was the ratio of the expected value of the two options, while satisficing information was the probability of winning. We found age-related alterations of economic preferences within the losses domain, but no alterations within the gains domain. Older adults (OA; 61–80 years old) were significantly more uncertainty averse for both risky and ambiguous choices. OA also exhibited choice strategies with decreased use of maximizing information. Within OA, we found a significant correlation between risk preferences and choice strategy. This linkage between preferences and strategy appears to derive from a convergence to risk neutrality driven by greater use of the effortful maximizing strategy. As utility maximization and value maximization intersect at risk neutrality, this result suggests that OA are exhibiting a relationship between enhanced rationality and enhanced value maximization. While there was variability in economic decision-making measures within OA, these individual differences were unrelated to variability within examined measures of cognitive ability. Our results demonstrate that aging alters economic decision-making for losses through changes in both individual preferences and the strategies individuals employ.

Neural correlates of positive and negative performance feedback in younger and older adults

BACKGROUND

Recent studies with younger adults have shown that performance feedback can serve as a reward, and it elicits reward-related brain activations. This study investigated whether performance feedback is processed similarly in younger and older adults and whether there are differential aging effects for positive and negative performance feedback.

METHODS

We used event-related fMRI in a choice reaction-time task and provided performance feedback after each trial.

RESULTS

Although younger and older adults differed in task-related activation, they showed comparable reward-related activation. Positive performance feedback elicited the strongest striatal and amygdala activation, which was reflected behaviorally in slightly faster reaction times.

CONCLUSIONS

These results suggest that performance feedback serves as a reward in both younger and older adults.

Impact of aging on frontostriatal reward processing

Healthy aging is associated with a progressive decline across a range of cognitive functions. An important factor underlying this decline may be the age-related impairment in stimulus-reward processing. Several studies have investigated age-related effects, but compared young versus old subjects. This is the first study to investigate the effect of aging on brain activation during reward processing within a continuous segment of the adult life span. We scanned 49 healthy adults aged 40-70 years, using functional MRI. We adopted a simple reward task, which allowed separate evaluation of neural responses to reward anticipation and receipt. The effect of reward on performance accuracy and speed was not related to age, indicating that all subjects could perform the task correctly. We identified a whole-brain significant age-related decline of ventral striatum activation during reward anticipation as compared to neutral anticipation. Importantly, the specificity of this finding was underscored by the observation that there was no general decline in activation during anticipation. Activation in the ventral striatum increased with age during reward receipt as compared to receiving neutral outcome. Finally, activation in the ventromedial prefrontal cortex during outcome was not affected by age. Our data demonstrate that the typical shift in striatal activation from reward receipt to reward anticipation in young adults disappears with healthy aging. These changes are consistent the well-ocumented age-related decline of striatal dopamine availability, and may provide a stepping stone for further research of age-related neurodegenerative diseases.

Reward processing in neurodegenerative disease

Representation of reward value involves a distributed network including cortical and subcortical structures. Because neurodegenerative illnesses target specific anatomic networks that partially overlap with the reward circuit, they would be predicted to have distinct impairments in reward processing. This review presents the existing evidence of reward processing changes in neurodegenerative diseases including mild cognitive impairment (MCI), Alzheimer's disease, frontotemporal dementia, amyotrophic lateral sclerosis (ALS), Parkinson's disease, and Huntington's disease, as well as in healthy aging. Carefully distinguishing the different aspects of reward processing (primary rewards, secondary rewards, reward-based learning, and reward-based decision-making) and using tasks that differentiate the stages of processing reward will lead to improved understanding of this fundamental process and clarify a contributing cause of behavioral change in these illnesses.

From colour photographs to black-and-white line drawings: an assessment of chimpanzees' (Pan troglodytes') transfer behaviour

Over two experiments, we investigated the ability of two adolescent and two adult chimpanzees to generalise a learnt, pictorial categorisation to increasingly degraded and abstract stimuli. In Experiment 2, we further assessed the ability of the adolescent chimpanzees to engage in open-ended categorisation of black-and-white line drawings. The current results confirmed and extended previous findings, showing that sub-adult chimpanzees outperform adult chimpanzees in the categorisation of pictorial stimuli, particularly when the stimuli are more degraded and abstract in nature. However, none of the four chimpanzees showed positive transfer of their category learning to a set of black-and-white line drawings, and neither of the adolescent chimpanzees evidenced reliable open-ended categorisation of the black-and-white line drawings. The latter findings suggest that both sub-adult and adult chimpanzees find it difficult to recognise black-and-white line drawings, and that open-ended categorisation of black-and-white line drawings is challenging for chimpanzees.

Adult age differences in subjective and objective measures of strategy use on a sequentially cued prediction task

Differences in strategy use are thought to underlie age-related performance deficits on many learning and decision-making tasks. Recently, age-related differences in learning to make predictions were reported on the Triplets Prediction Task (TPT). Notably, deficits appeared early in training and continued with experience. To assess if age differences were due to early strategy use, neural networks were used to objectively assess the strategies implemented by participants during Session 1. Then, the relationship between these strategies and performance was examined. Results revealed that older adults were more likely to implement a disadvantageous strategy early in learning, and this led to poorer task performance. Importantly, the relationship between age and task performance was partially mediated by early strategy use, suggesting that early strategy selection played a role in the lower quality of predictions in older adults.

Bringing the cognitive estimation task into the 21st century: normative data on two new parallel forms

The Cognitive Estimation Test (CET) is widely used by clinicians and researchers to assess the ability to produce reasonable cognitive estimates. Although several studies have published normative data for versions of the CET, many of the items are now outdated and parallel forms of the test do not exist to allow cognitive estimation abilities to be assessed on more than one occasion. In the present study, we devised two new 9-item parallel forms of the CET. These versions were administered to 184 healthy male and female participants aged 18–79 years with 9–22 years of education. Increasing age and years of education were found to be associated with successful CET performance as well as gender, intellect, naming, arithmetic and semantic memory abilities. To validate that the parallel forms of the CET were sensitive to frontal lobe damage, both versions were administered to 24 patients with frontal lobe lesions and 48 age-, gender- and education-matched controls. The frontal patients’ error scores were significantly higher than the healthy controls on both versions of the task. This study provides normative data for parallel forms of the CET for adults which are also suitable for assessing frontal lobe dysfunction on more than one occasion without practice effects.

Scaffolding across the lifespan in history-dependent decision-making

We examined the relationship between pressure and age-related changes in decision-making using a task for which currently available rewards depend on the participant's previous history of choices. Optimal responding in this task requires the participant to learn how his or her current choices affect changes in the future rewards given for each option. Building on the scaffolding theory of aging and cognition, we predicted that when additional frontal resources are available, compensatory recruitment leads to increased monitoring and increased use of heuristic-based strategies, ultimately leading to better performance. Specifically, we predicted that scaffolding would result in an age-related performance advantage under no pressure conditions. We also predicted that, although younger adults would engage in scaffolding under pressure, older adults would not have additional resources available for increased scaffolding under pressure-packed conditions, leading to an age-related performance deficit. Both predictions were supported by the data. In addition, computational models were used to evaluate decision-making strategies employed by each participant group. As expected, older adults under no pressure conditions and younger adults under pressure conditions showed increased use of heuristic-based strategies relative to older adults under pressure and younger adults under no pressure, respectively. These results are consistent with the notion that scaffolding can occur across the life span in the face of an environmental challenge.

Age-related similarities and differences in brain activity underlying reversal learning

The ability to update associative memory is an important aspect of episodic memory and a critical skill for social adaptation. Previous research with younger adults suggests that emotional arousal alters brain mechanisms underlying memory updating; however, it is unclear whether this applies to older adults. Given that the ability to update associative information declines with age, it is important to understand how emotion modulates the brain processes underlying memory updating in older adults. The current study investigated this question using reversal learning tasks, where younger and older participants (age ranges 19–35 and 61–78, respectively) learn a stimulus–outcome association and then update their response when contingencies change. We found that younger and older adults showed similar patterns of activation in the frontopolar OFC and the amygdala during emotional reversal learning. In contrast, when reversal learning did not involve emotion, older adults showed greater parietal cortex activity than did younger adults. Thus, younger and older adults show more similarities in brain activity during memory updating involving emotional stimuli than during memory updating not involving emotional stimuli.

Dopamine restores reward prediction errors in old age

Senescence affects the ability to utilize information about the likelihood of rewards for optimal decision-making. In a human functional magnetic resonance imaging (fMRI) study, we show that healthy older adults have an abnormal signature of expected value resulting in an incomplete reward prediction error signal in the nucleus accumbens, a brain region receiving rich input projections from substantia nigra/ventral tegmental area (SN/VTA) dopaminergic neurons. Structural connectivity between SN/VTA and striatum measured with diffusion tensor imaging (DTI) was tightly coupled to inter-individual differences in the expression of this expected reward value signal. The dopamine precursor levodopa (L-DOPA) increased the task-based learning rate and task performance in some older adults to a level shown by young adults. Critically this drug-effect was linked to restoration of a canonical neural reward prediction error. Thus we identify a neurochemical signature underlying abnormal reward processing in older adults and show this can be modulated by L-DOPA.