How Reward and Aversion Shape Motivation and Decision Making: A Computational Account

Hypothalamic neuronal outputs transmit sensorimotor signals at the onset of locomotor initiation

The lateral hypothalamus (LH) plays a critical role in sensory integration to organize behavior responses. However, how projection-defined LH neuronal outputs dynamically transmit sensorimotor signals to major downstream targets to organize behavior is unknown. Here, using multi-fiber photometry, we show that three major LH neuronal outputs projecting to the dorsal raphe nucleus (DRN), ventral tegmental area (VTA), and lateral habenula (LHb) exhibit significant coherent activity in mice engaging sensory-evoked or self-initiated motor responses. Increased activity at LH axon terminals precedes movement initiation during active coping responses and the activity of serotonin neurons and dopamine neurons. The optogenetic activation of LH axon terminals in either of the DRN, VTA, or LHb was sufficient to increase motor initiation but had different effects on passive avoidance and sucrose consumption. Our findings support the complementary role of three projection-defined LH neuronal outputs in the transmission of sensorimotor signals to major downstream regions at movement onset.

The role of the orbitofrontal cortex in exercise addiction and exercise motivation: A brain imaging study based on multimodal magnetic resonance imaging

BACKGROUND

Excessive exercise may also lead to exercise addiction (EXA), which is harmful to people's physical and mental health. Behavioral and neuroimaging studies have demonstrated that addictive disorders are essentially motivational problems. However, little is known about the neuropsychological mechanism of EXA and the effects of motivation on EXA.

METHODS

We investigated 130 regularly exercised participants with EXA symptoms to explore the neurobiological basis of EXA and its association with motivation. The correlation between EXA and gray matter volume (GMV) was evaluated by whole-brain regression analysis based on voxel-based morphometry. Then, regional brain function was extracted and the relationship between brain structure-function-EXA was analyzed. Finally, mediation analysis was performed to further detect the relationship between the brain, motivation, and EXA.

RESULTS

Whole-brain correlation analyses showed that the GMV of the right orbitofrontal cortex (OFC) was negatively correlated with EXA. The function of the right OFC played an indirect role in EXA and affected EXA via the GMV of the OFC. Importantly, the GMV of the right OFC played a mediating role in the relationship between ability motivation and EXA. These results remain significant even when adjusting for sex, age, body mass index, family socioeconomic status, general intelligence, total intracranial volume, and head motion.

LIMITATION

The results should be interpreted carefully because only the people with EXA symptoms were included.

CONCLUSION

This study provided evidence for the underlying neuropsychological mechanism of the important role of the right OFC in EXA and revealed that there may be a decrease in executive control function in EXA.

Motivating Machines: The Potential of Modeling Motivation as MoA for Behavior Change Systems

The pathway through which behavior change techniques have an effect on the behavior of an individual is referred to as the Mechanism of Action (MoA). Digitally enabled behavior change interventions could potentially benefit from explicitly modelling the MoA to achieve more effective, adaptive, and personalized interventions. For example, if ‘motivation’ is proposed as the targeted construct in any behavior change intervention, how can a model of this construct be used to act as a mechanism of action, mediating the intervention effect using various behavior change techniques? This article discusses a computational model for motivation based on the neural reward pathway with the aim to make it act as a mediator between behavior change techniques and target behavior. This model’s formal description and parametrization are described from a neurocomputational sciences prospect and elaborated with the help of a sub-question, i.e., what parameters/processes of the model are crucial for the generation and maintenance of motivation. An intervention scenario is simulated to show how an explicit model of ‘motivation’ and its parameters can be used to achieve personalization and adaptivity. A computational representation of motivation as a mechanism of action may also further advance the design, evaluation, and effectiveness of personalized and adaptive digital behavior change interventions.

Children's value-based decision making

To effectively navigate their environments, infants and children learn how to recognize events predict salient outcomes, such as rewards or punishments. Relatively little is known about how children acquire this ability to attach value to the stimuli they encounter. Studies often examine children’s ability to learn about rewards and threats using either classical conditioning or behavioral choice paradigms. Here, we assess both approaches and find that they yield different outcomes in terms of which individuals had efficiently learned the value of information presented to them. The findings offer new insights into understanding how to assess different facets of value learning in children.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Unique and Transdiagnostic Dimensions of Reward Functioning in Attention-Deficit/Hyperactivity Disorder and Alcohol Use Disorder Symptoms

AIMS

Contemporary theories of attention-deficit/hyperactivity disorder (ADHD) and alcohol use disorder (AUD) emphasize core dysfunctions in reward-related processes and behaviors as pathognomonic characteristics. However, to date, it is unclear which domains of reward functioning are unique to ADHD versus AUD symptom dimensions, and which represent underlying shared correlates.

METHODS

The current study employed secondary data analyses from a large community sample of emerging adults (N = 602; 57.3% female) and novel transdiagnostic modeling (i.e. bi-factor confirmatory factor analyses and structural equation modeling) of ADHD, AUD and shared symptom dimensions to identify unique and common reward-related dimensions: environmental suppressors, reward probability, hedonic capacity, proportionate substance-related reinforcement and delay discounting.

RESULTS

The presence of environmental suppressors was the only reward-related construct that correlated with the underlying ADHD-AUD shared dimension. The AUD symptom dimension was uniquely associated with proportionate substance-related reinforcement, whereas the ADHD symptom dimension was uniquely associated with limited reward probability. No significant associations were found for delay discounting or hedonic capacity.

CONCLUSIONS

These novel findings highlight specific aspects of reward-related functioning in ADHD, AUD and shared symptom dimensions. In so doing, this work meaningfully advances theoretical conceptualizations of these two commonly co-occurring presentations and suggests future directions for research on transdiagnostic correlates. Future longitudinal studies should include clinical samples with diagnoses of AUD and ADHD to further identify underlying correlates over time.

Modulation of value-based decision making behavior by subregions of the rat prefrontal cortex

RATIONALE

During value-based decision-making, organisms make choices on the basis of reward expectations, which have been formed during prior action-outcome learning. Although it is known that neuronal manipulations of different subregions of the rat prefrontal cortex (PFC) have qualitatively different effects on behavioral tasks involving value-based decision-making, it is unclear how these regions contribute to the underlying component processes.

OBJECTIVES

Assessing how different regions of the rodent PFC contribute to component processes of value-based decision-making behavior, including reward (or positive feedback) learning, punishment (or negative feedback) learning, response persistence, and exploration versus exploitation.

METHODS

We performed behavioral modeling of data of rats in a probabilistic reversal learning task after pharmacological inactivation of five PFC subregions, to assess how inactivation of these different regions affected the structure of responding of animals in the task.

RESULTS

Our results show reductions in reward and punishment learning after PFC subregion inactivation. The prelimbic, infralimbic, lateral orbital, and medial orbital PFC particularly contributed to punishment learning, and the prelimbic and lateral orbital PFC to reward learning. In addition, response persistence depended on the infralimbic and medial orbital PFC. As a result, pharmacological inactivation of the infralimbic and lateral orbitofrontal cortex reduced the number of reversals achieved, whereas inactivation of the prelimbic and medial orbitofrontal cortex decreased the number of rewards obtained. Finally, using simulated data, we explain discrepancies with a previous study and demonstrate complex, interacting relationships between conventional measures of probabilistic reversal learning performance, such as win-stay/lose-switch behavior, and component processes of value-based decision-making.

CONCLUSIONS

Together, our data suggest that distinct components of value-based learning and decision-making are generated in medial and orbital PFC regions, displaying functional specialization and overlap, with a prominent role of large parts of the PFC in negative feedback processing.

Dopaminergic contributions to behavioral control under threat of punishment in rats

RATIONALE

Excessive intake of rewards, such as food and drugs, often has explicit negative consequences, including the development of obesity and addiction, respectively. Thus, choosing not to pursue reward is the result of a cost/benefit decision, proper execution of which requires inhibition of behavior. An extensive body of preclinical and clinical evidence implicates dopamine in certain forms of inhibition of behavior, but it is not fully known how it contributes to behavioral inhibition under threat of explicit punishment.

OBJECTIVES

To assess the involvement of midbrain dopamine neurons and their corticostriatal output regions, the ventral striatum and prefrontal cortex, in control over behavior under threat of explicit (foot shock) punishment in rats.

METHODS

We used a recently developed behavioral inhibition task, which assesses the ability of rats to exert behavioral restraint at the mere sight of food reward, under threat of foot shock punishment. Using in vivo fiber photometry, chemogenetics, c-Fos immunohistochemistry, and behavioral pharmacology, we investigated how dopamine neurons in the ventral tegmental area, as well as its output areas, the ventral striatum and prefrontal cortex, contribute to behavior in this task.

RESULTS

Using this multidisciplinary approach, we found little evidence for a direct involvement of ascending midbrain dopamine neurons in inhibitory control over behavior under threat of punishment. For example, photometry recordings suggested that VTA DA neurons do not directly govern control over behavior in the task, as no differences were observed in neuronal population activity during successful versus unsuccessful behavioral control. In addition, chemogenetic and pharmacological manipulations of the mesocorticolimbic DA system had little or no effect on the animals' ability to exert inhibitory control over behavior. Rather, the dopamine system appeared to have a role in the motivational components of reward pursuit.

CONCLUSIONS

Together, our data provide insight into the mesocorticolimbic mechanisms behind motivated behaviors by showing a modulatory role of dopamine in the expression of cost/benefit decisions. In contrast to our expectations, dopamine did not appear to directly mediate the type of behavioral control that is tested in our task.

Differential contributions of striatal dopamine D1 and D2 receptors to component processes of value-based decision making

Dopamine has been implicated in value-based learning and decision making by signaling reward prediction errors and facilitating cognitive flexibility, incentive motivation, and voluntary movement. Dopamine receptors can roughly be divided into the D1 and D2 subtypes, and it has been hypothesized that these two types of receptors have an opposite function in facilitating reward-related and aversion-related behaviors, respectively. Here, we tested the contribution of striatal dopamine D1 and D2 receptors to processes underlying value-based learning and decision making in rats, employing a probabilistic reversal learning paradigm. Using computational trial-by-trial analysis of task behavior after systemic or intracranial treatment with dopamine D1 and D2 receptor agonists and antagonists, we show that negative feedback learning can be modulated through D2 receptor signaling and positive feedback learning through D1 receptor signaling in the ventral striatum. Furthermore, stimulation of D2 receptors in the ventral or dorsolateral (but not dorsomedial) striatum promoted explorative choice behavior, suggesting an additional function of dopamine in these areas in value-based decision making. Finally, treatment with most dopaminergic drugs affected response latencies and number of trials completed, which was also seen after infusion of D2, but not D1 receptor-acting drugs into the striatum. Together, our data support the idea that dopamine D1 and D2 receptors have complementary functions in learning on the basis of emotionally valenced feedback, and provide evidence that dopamine facilitates value-based and motivated behaviors through distinct striatal regions.

Insensitivity to Losses: A Core Feature in Patients With Anorexia Nervosa?

BACKGROUND

Patients with anorexia nervosa (AN) demonstrate aberrations in choice behavior, including impairments in laboratory measures of decision making. Although a wealth of studies suggest that these aberrations arise from alterations in value processing, it remains unclear by which core component of value processing this is mediated.

METHODS

We fit trial-by-trial data of patients with AN (n = 60 first cohort, n = 216 second cohort) and healthy control participants (n = 55) performing the Iowa Gambling Task to a computational model based on prospect utility theory. We determined, per participant, the best-fit model parameters and compared these between the groups.

RESULTS

Analyses revealed a decreased estimate of model parameter λ in patients with AN, indicative of an attenuation of loss-aversive behavior in the Iowa Gambling Task. In comparison, measures of reward sensitivity, value-based learning, and exploration versus exploitation were unaltered in patients with AN. A measurement in a second independent cohort replicated the finding that loss aversion, typically observed in healthy individuals, is reduced in patients with AN.

CONCLUSIONS

We show that patients with AN, in contrast to healthy control participants, demonstrate reduced loss-aversive behavior. This finding provides important fundamental insights into the decision-making capacity of patients with AN, suggesting alterations in the mechanisms involved in value processing related to negative feedback.