Category-specific general Pavlovian-instrumental transfer

Modern living is characterized by easy access to highly palatable energy-dense foods. Environmental cues associated with palatable foods increase seeking of those foods (specific transfer) and other palatable foods (general transfer). We conducted a series of studies testing the boundaries of food cue-reactivity by evaluating the impact of broader flavor associations (i.e. saltiness, sweetness) in eliciting general transfer effects. Experiment 1 was an online experiment with fictive rewards that tested if two actions associated with different food rewards (chip and chocolate points) could be provoked by images of other foods that were either similar or distinct in flavor from the foods associated with these instrumental actions. We observed that response excitation was only elicited by similarly flavored food cues, whereas distinctly flavored food cues inhibited response rates relative to control cues. Experiment 2 confirmed this observation in a classroom setting where real food rewards were contingent on task performance. Experiment 3 was an online study that further confirmed the reliability of the effects with a well powered sample. There were moderate-to-strong associations between specific and general transfer effects across all studies, suggesting overlapping cognitive processes are responsible for both transfer effects. These data improve the mechanistic understanding of how broad category associations can moderate the impact of food cues on food choices. This knowledge could be helpful for improving the precision of psychological interventions that seek to mitigate the impact of food cue-reactivity.

Sex differences in motivational biases over instrumental actions

Motivational (i.e., appetitive or aversive) cues can bias value-based decisions by affecting either direction and intensity of instrumental actions. Despite several findings describing important interindividual differences in these biases, whether biological sex can also play a role is still up to debate. By comparing females and males in both appetitive and aversive Pavlovian-to-Instrumental Transfer paradigms we found that, while motivational cues similarly bias the direction of instrumental actions in both sexes, the intensity of such actions is increased by the cue in male participants only. The present results constitute compelling evidence that a crucial motivational bias of daily actions directed to obtaining rewards or avoiding punishments is modulated by biological sex. This evidence sheds new light on the role of sex in motivational processes that underlie decision-making, highlighting the importance of considering sex as a crucial factor in future research on this topic.

Value Modulation of Self-Defeating Impulsivity

BACKGROUND

Impulse control is a critical aspect of cognitive functioning. Intuitively, whether an action is executed prematurely depends on its associated reward, yet the link between value and impulsivity remains poorly understood. Three frameworks for impulsivity offer contrasting views: impulsive behavior may be valuable because it is associated with hidden internal reward (e.g., reduction of mental effort). Alternatively, it can emerge from exploration, which is disadvantageous in the short term but can yield long-term benefits. Finally, impulsivity may reflect Pavlovian bias, an inherent tendency that occurs even when its outcome is negative.

METHODS

To test these hypotheses, we trained 17 male mice to withhold licking while anticipating variable rewards. We then measured and optogenetically manipulated dopamine release in the ventral striatum.

RESULTS

We found that higher reward magnitudes correlated with increased impulsivity. This behavior was well explained by a Pavlovian bias model. Furthermore, we observed negative dopamine signals during premature licking, suggesting that in this task, impulsivity is not merely an unsuccessful attempt at obtaining a reward. Rather, it is a failure to overcome the urge to act prematurely despite knowledge of the negative consequences of such impulsive actions.

CONCLUSIONS

Our findings underscore the integral role value plays in regulating impulsivity and suggest that the dopaminergic system influences impulsivity through the mediation of value learning.

The selective serotonin reuptake inhibitor sertraline alters learning from aversive reinforcements in patients with depression: evidence from a randomized controlled trial

BACKGROUND

Selective serotonin reuptake inhibitors (SSRIs) are first-line pharmacological treatments for depression and anxiety. However, little is known about how pharmacological action is related to cognitive and affective processes. Here, we examine whether specific reinforcement learning processes mediate the treatment effects of SSRIs.

METHODS

The PANDA trial was a multicentre, double-blind, randomized clinical trial in UK primary care comparing the SSRI sertraline with placebo for depression and anxiety. Participants (N = 655) performed an affective Go/NoGo task three times during the trial and computational models were used to infer reinforcement learning processes.

RESULTS

There was poor task performance: only 54% of the task runs were informative, with more informative task runs in the placebo than in the active group. There was no evidence for the preregistered hypothesis that Pavlovian inhibition was affected by sertraline. Exploratory analyses revealed that in the sertraline group, early increases in Pavlovian inhibition were associated with improvements in depression after 12 weeks. Furthermore, sertraline increased how fast participants learned from losses and faster learning from losses was associated with more severe generalized anxiety symptoms.

CONCLUSIONS

The study findings indicate a relationship between aversive reinforcement learning mechanisms and aspects of depression, anxiety, and SSRI treatment, but these relationships did not align with the initial hypotheses. Poor task performance limits the interpretability and likely generalizability of the findings, and highlights the critical importance of developing acceptable and reliable tasks for use in clinical studies.

FUNDING

This article presents research supported by NIHR Program Grants for Applied Research (RP-PG-0610-10048), the NIHR BRC, and UCL, with additional support from IMPRS COMP2PSYCH (JM, QH) and a Wellcome Trust grant (QH).

Functional Magnetic Resonance Imaging Signatures of Pavlovian and Instrumental Valuation Systems during a Modified Orthogonalized Go/No-go Task

Motivational (i.e., Pavlovian) values interfere with instrumental responding and can lead to suboptimal decision-making. In humans, task-based neuroimaging studies have only recently started illuminating the functional neuroanatomy of Pavlovian biasing of instrumental control. To provide a mechanistic understanding of the neural dynamics underlying the Pavlovian and instrumental valuation systems, analysis of neuroimaging data has been informed by computational modeling of conditioned behavior. Nonetheless, because of collinearities in Pavlovian and instrumental predictions, previous research failed to tease out hemodynamic activity that is parametrically and dynamically modulated by coexistent Pavlovian and instrumental value expectations. Moreover, neural correlates of Pavlovian to instrumental transfer effects have so far only been identified in extinction (i.e., in the absence of learning). In this study, we devised a modified version of the orthogonalized go/no-go paradigm, which introduced Pavlovian-only catch trials to better disambiguate trial-by-trial Pavlovian and instrumental predictions in both sexes. We found that hemodynamic activity in the ventromedial pFC covaried uniquely with the model-derived Pavlovian value expectations. Notably, modulation of neural activity encoding for instrumental predictions in the supplementary motor cortex was linked to successful action selection in conflict conditions. Furthermore, hemodynamic activity in regions pertaining to the limbic system and medial pFC was correlated with synergistic Pavlovian and instrumental predictions and improved conditioned behavior during congruent trials. Altogether, our results provide new insights into the functional neuroanatomy of decision-making and corroborate the validity of our variant of the orthogonalized go/no-go task as a behavioral assay of the Pavlovian and instrumental valuation systems.

Elevated Amygdala Responses During De Novo Pavlovian Conditioning in Alcohol Use Disorder Are Associated With Pavlovian-to-Instrumental Transfer and Relapse Latency

Background

Contemporary learning theories of drug addiction ascribe a key role to Pavlovian learning mechanisms in the development, maintenance, and relapse of addiction. In fact, cue-reactivity research has demonstrated the power of alcohol-associated cues to activate the brain's reward system, which has been linked to craving and subsequent relapse. However, whether de novo Pavlovian conditioning is altered in alcohol use disorder (AUD) has rarely been investigated.

Methods

To characterize de novo Pavlovian conditioning in AUD, 62 detoxified patients with AUD and 63 matched healthy control participants completed a Pavlovian learning task as part of a Pavlovian-to-instrumental transfer paradigm during a functional magnetic resonance imaging session. Patients were followed up for 12 months to assess drinking behavior and relapse status.

Results

While patients and healthy controls did not differ in their ability to explicitly acquire the contingencies between conditioned and unconditioned stimuli, patients with AUD displayed significantly stronger amygdala responses toward Pavlovian cues, an effect primarily driven by stronger blood oxygen level-dependent differentiation during learning from reward compared with punishment. Moreover, in patients compared with controls, differential amygdala responses during conditioning were positively related to the ability of Pavlovian stimuli to influence ongoing instrumental choice behavior measured during a subsequent Pavlovian-to-instrumental transfer test. Finally, patients who relapsed within the 12-month follow-up period showed an inverse association between amygdala activity during conditioning and relapse latency.

Conclusions

We provide evidence of altered neural correlates of de novo Pavlovian conditioning in patients with AUD, especially for appetitive stimuli. Thus, heightened processing of Pavlovian cues might constitute a behaviorally relevant mechanism in alcohol addiction.

Modifying Pavlovian-to-instrumental transfer by approach avoidance training in healthy subjects: a proof of concept study

The modulation of instrumental action by conditioned Pavlovian cues is hypothesized to play a role in the emergence and maintenance of maladaptive behavior. The Pavlovian to Instrumental transfer task (PIT) is designed to examine the magnitude of the influence of cues on behavior and we aim to manipulate the motivational value of Pavlovian cues to reduce their effect on instrumental responding. To this end, we utilized a joystick-based modification of approach and avoidance propensities that has shown success in clinical populations. To examine changes in PIT, we subjected 35 healthy participants to a series of experimental procedures: (1) Instrumental training was followed by (2) Pavlovian conditioning of neutral stimuli that were associated with monetary reward or loss. (3) In a subsequent joystick task, approach and avoidance tendencies toward conditioned cues were assessed. (4) In a transfer test, the PIT effect as the impact of conditioned cues on instrumental behavior was measured. (5) The explicit knowledge of cue-reward contingencies was assessed in a forced-choice phase. (6, 7) systematic joystick training was followed by a posttest (8) the transfer task and forced-choice test were repeated. We found no effect of training on approach-avoidance propensities in the context of this proof of concept study. A higher response rate towards negative stimuli during PIT after systematic training compared to sham training was seen. On the other hand, we saw an increased PIT effect after sham training. These results contribute to the understanding of the strength of the influence of cues on instrumental behavior. Our findings further stress the importance of context, instructions and operationalization of instrumental behavior in the framework of transfer effects.

The Association of Non-Drug-Related Pavlovian-to-Instrumental Transfer Effect in Nucleus Accumbens With Relapse in Alcohol Dependence: A Replication

BACKGROUND

The Pavlovian-to-instrumental transfer (PIT) paradigm measures the effects of Pavlovian conditioned cues on instrumental behavior in the laboratory. A previous study conducted by our research group observed activity in the left nucleus accumbens (NAcc) elicited by a non-drug-related PIT task across patients with alcohol dependence (AD) and healthy control subjects, and the left NAcc PIT effect differentiated patients who subsequently relapsed from those who remained abstinent. In this study, we aimed to examine whether such effects were present in a larger sample collected at a later date.

METHODS

A total of 129 recently detoxified patients with AD (21 females) and 74 healthy, age- and gender-matched control subjects (12 females) performing a PIT task during functional magnetic resonance imaging were examined. After task assessments, patients were followed for 6 months. Forty-seven patients relapsed and 37 remained abstinent.

RESULTS

We found a significant behavioral non-drug-related PIT effect and PIT-related activity in the NAcc across all participants. Moreover, subsequent relapsers showed stronger behavioral and left NAcc PIT effects than abstainers. These findings are consistent with our previous findings.

CONCLUSIONS

Behavioral non-drug-related PIT and neural PIT correlates are associated with prospective relapse risk in AD. This study replicated previous findings and provides evidence for the clinical relevance of PIT mechanisms to treatment outcome in AD. The observed difference between prospective relapsers and abstainers in the NAcc PIT effect in our study is small overall. Future studies are needed to further elucidate the mechanisms and the possible modulators of neural PIT in relapse in AD.

Susceptibility to interference between Pavlovian and instrumental control predisposes risky alcohol use developmental trajectory from ages 18 to 24

Pavlovian cues can influence ongoing instrumental behaviour via Pavlovian-to-instrumental transfer (PIT) processes. While appetitive Pavlovian cues tend to promote instrumental approach, they are detrimental when avoidance behaviour is required, and vice versa for aversive cues. We recently reported that susceptibility to interference between Pavlovian and instrumental control assessed via a PIT task was associated with risky alcohol use at age 18. We now investigated whether such susceptibility also predicts drinking trajectories until age 24, based on AUDIT (Alcohol Use Disorders Identification Test) consumption and binge drinking (gramme alcohol/drinking occasion) scores. The interference PIT effect, assessed at ages 18 and 21 during fMRI, was characterized by increased error rates (ER) and enhanced neural responses in the ventral striatum (VS), the lateral and dorsomedial prefrontal cortices (dmPFC) during conflict, that is, when an instrumental approach was required in the presence of an aversive Pavlovian cue or vice versa. We found that a stronger VS response during conflict at age 18 was associated with a higher starting point of both drinking trajectories but predicted a decrease in binge drinking. At age 21, high ER and enhanced neural responses in the dmPFC were associated with increasing AUDIT-C scores over the next 3 years until age 24. Overall, susceptibility to interference between Pavlovian and instrumental control might be viewed as a predisposing mechanism towards hazardous alcohol use during young adulthood, and the identified high-risk group may profit from targeted interventions.

Pavlovian-to-Instrumental Transfer across Mental Disorders: A Review

A mechanism known as Pavlovian-to-instrumental transfer (PIT) describes a phenomenon by which the values of environmental cues acquired through Pavlovian conditioning can motivate instrumental behavior. PIT may be one basic mechanism of action control that can characterize mental disorders on a dimensional level beyond current classification systems. Therefore, we review human PIT studies investigating subclinical and clinical mental syndromes. The literature prevails an inhomogeneous picture concerning PIT. While enhanced PIT effects seem to be present in non-substance-related disorders, overweight people, and most studies with AUD patients, no altered PIT effects were reported in tobacco use disorder and obesity. Regarding AUD and relapsing alcohol-dependent patients, there is mixed evidence of enhanced or no PIT effects. Additionally, there is evidence for aberrant corticostriatal activation and genetic risk, e.g., in association with high-risk alcohol consumption and relapse after alcohol detoxification. In patients with anorexia nervosa, stronger PIT effects elicited by low caloric stimuli were associated with increased disease severity. In patients with depression, enhanced aversive PIT effects and a loss of action-specificity associated with poorer treatment outcomes were reported. Schizophrenic patients showed disrupted specific but intact general PIT effects. Patients with chronic back pain showed reduced PIT effects. We provide possible reasons to understand heterogeneity in PIT effects within and across mental disorders. Further, we strengthen the importance of reliable experimental tasks and provide test-retest data of a PIT task showing moderate to good reliability. Finally, we point toward stress as a possible underlying factor that may explain stronger PIT effects in mental disorders, as there is some evidence that stress per se interacts with the impact of environmental cues on behavior by selectively increasing cue-triggered wanting. To conclude, we discuss the results of the literature review in the light of Research Domain Criteria, suggesting future studies that comprehensively assess PIT across psychopathological dimensions.

The effects of induced positive and negative affect on Pavlovian-instrumental interactions

Across species, animals have an intrinsic drive to approach appetitive stimuli and to withdraw from aversive stimuli. In affective science, influential theories of emotion link positive affect with strengthened behavioural approach and negative affect with avoidance. Based on these theories, we predicted that individuals' positive and negative affect levels should particularly influence their behaviour when innate Pavlovian approach/avoidance tendencies conflict with learned instrumental behaviours. Here, across two experiments - exploratory Experiment 1 (N = 91) and a preregistered confirmatory Experiment 2 (N = 335) - we assessed how induced positive and negative affect influenced Pavlovian-instrumental interactions in a reward/punishment Go/No-Go task. Contrary to our hypotheses, we found no evidence for a main effect of positive/negative affect on either approach/avoidance behaviour or Pavlovian-instrumental interactions. However, we did find evidence that the effects of induced affect on behaviour were moderated by individual differences in self-reported behavioural inhibition and gender. Exploratory computational modelling analyses explained these demographic moderating effects as arising from positive correlations between demographic factors and individual differences in the strength of Pavlovian-instrumental interactions. These findings serve to sharpen our understanding of the effects of positive and negative affect on instrumental behaviour.

Psychopathic tendency in violent offenders is associated with reduced aversive Pavlovian inhibition of behavior and associated striatal BOLD signal

Background

Violent offenders with psychopathic tendencies are characterized by instrumental, i.e., planned, callous, and unemotional (aggressive) behavior and have been shown to exhibit abnormal aversive processing. However, the consequences of abnormal aversive processing for instrumental action and associated neural mechanisms are unclear.

Materials and methods

Here we address this issue by using event-related functional magnetic resonance imaging (fMRI) in 15 violent offenders with high psychopathic tendencies and 18 matched controls during the performance of an aversive Pavlovian-to-instrumental transfer paradigm. This paradigm allowed us to assess the degree to which aversive Pavlovian cues affect instrumental action and associated neural signaling.

Results

Psychopathic tendency scores were associated with an attenuation of aversive Pavlovian inhibition of instrumental action. Moreover, exploratory analyses revealed an anomalous positive association between aversive inhibition of action and aversive inhibition of BOLD signal in the caudate nucleus of violent offenders with psychopathic tendencies. In addition, psychopathic tendency also correlated positively with amygdala reactivity during aversive versus neutral cues in Pavlovian training.

Conclusion

These findings strengthen the hypothesis that psychopathic tendencies in violent offenders are related to abnormal impact of aversive processing on instrumental behavior. The neural effects raise the possibility that this reflects deficient transfer of aversive Pavlovian inhibitory biases onto neural systems that implement instrumental action, including the caudate nucleus.

Amygdala response predicts clinical symptom reduction in patients with borderline personality disorder: A pilot fMRI study

Borderline personality disorder (BPD) is a prevalent, devastating, and heterogeneous psychiatric disorder. Treatment success is highly variable within this patient group. A cognitive neuroscientific approach to BPD might contribute to precision psychiatry by identifying neurocognitive factors that predict who will benefit from a specific treatment. Here, we build on observations that BPD is accompanied by the enhanced impact of the aversive effect on behavior and abnormal neural signaling in the amygdala. We assessed whether BPD is accompanied by abnormal aversive regulation of instrumental behavior and associated neural signaling, in a manner that is predictive of symptom reduction after therapy. We tested a clinical sample of 15 female patients with BPD, awaiting dialectical behavior therapy (DBT), and 16 matched healthy controls using fMRI and an aversive Pavlovian-to-instrumental transfer (PIT) task that assesses how instrumental behaviors are influenced by aversive Pavlovian stimuli. Patients were assessed 1 year after the start of DBT to quantify changes in BPD symptom severity. At baseline, behavioral aversive PIT and associated neural signaling did not differ between groups. However, the BOLD signal in the amygdala measured during aversive PIT was associated with symptom reduction at 1-year follow-up: higher PIT-related aversive amygdala signaling before treatment was associated with reduced clinical improvement at follow-up. Thus, within the evaluated group of BPD patients, the BOLD signal in the amygdala before treatment was related to clinical symptom reduction 1 year after the start of treatment. The results suggest that less PIT-related responsiveness of the amygdala increases the chances of treatment success. We note that the relatively small sample size is a limitation of this study and that replication is warranted.

General Pavlovian-to-instrumental transfer in humans: Evidence from Bayesian inference

When repeatedly paired with rewarding outcomes (i.e., Pavlovian conditioning), environmental cues may acquire predictive and motivational significance and later enhance instrumental responding for the same (i.e., outcome-specific transfer) or motivationally similar (i.e., general transfer) outcomes. Although outcome-specific and general Pavlovian-to-Instrumental Transfer (PIT) are characterized by different neural substrates and behavioral mechanisms, general transfer has never been studied in isolation from outcome-specific transfer in humans. The first aim of the present study was to test whether the general transfer effect could emerge in isolation and independently of outcome-specific transfer. Our results showed that general transfer can be elicited without the concurrent presence of outcome-specific transfer, supporting the idea that outcome-specific and general transfer can be studied independently of each other. The second aim of the present study was to clarify whether the affordance-like properties of the outcomes can affect the general transfer. In fact, a critical difference in current studies on general transfer concerns the use of cues associated with outcomes for which an action was previously learned (or not) during the instrumental training. This apparently minor difference affects the affordance-like properties of the outcome and may also be transferred to the cue, in turn impacting general transfer. Results revealed a general transfer of the same magnitude regardless of whether cues were associated with reward earned or not during instrumental conditioning. These findings increase the current knowledge on the incentive motivational mechanism behind general transfer, indicating that it is independent of the motor features of the outcome.

Aversive Pavlovian inhibition in adult attention-deficit/hyperactivity disorder and its restoration by mindfulness-based cognitive therapy

Background

Control over the tendency to make or withhold responses guided by contextual Pavlovian information plays a key role in understanding impulsivity and hyperactivity. Here we set out to assess (1) the understudied relation between contextual Pavlovian inhibitory control and hyperactivity/impulsivity in adults with ADHD and (2) whether this inhibition can be enhanced by mindfulness based cognitive therapy (MBCT).

Methods

Within the framework of a randomized controlled trial 50 Adult ADHD patients were assessed before and after 8 weeks of treatment as usual (TAU) with (n = 24) or without (n = 26) MBCT. We employed a well-established behavioral Pavlovian-to-instrumental transfer task that quantifies Pavlovian inhibitory control over instrumental behavior.

Results

Task results revealed (1) less aversive Pavlovian inhibition in ADHD patients with clinically relevant hyperactivity/impulsivity than in those without; and (2) enhanced Pavlovian inhibition across all ADHD patients after TAU+MBCT compared with TAU.

Conclusion

These findings offer new insights in the neurocognitive mechanisms of hyperactivity/impulsivity in ADHD and its treatment: We reveal a role for Pavlovian inhibitory mechanisms in understanding hyperactive/impulsive behaviors in ADHD and point toward MBCT as an intervention that might influence these mechanisms.

Alcohol approach bias is associated with both behavioral and neural Pavlovian-to-instrumental transfer effects in alcohol-dependent patients

Background

Even after qualified detoxification, alcohol-dependent (AD) patients may relapse to drinking alcohol despite their decision to abstain. Two mechanisms may play important roles. First, the impact of environmental cues on instrumental behavior (i.e., Pavlovian-to-instrumental transfer [PIT] effect), which was found to be stronger in prospectively relapsing AD patients than in abstaining patients. Second, an automatic approach bias toward alcohol stimuli was observed in AD patients, and interventions targeting this bias reduced the relapse risk in some studies. Previous findings suggest a potential behavioral and neurobiological overlap between these two mechanisms.

Methods

In this study, we examined the association between alcohol approach bias and both behavioral and neural non-drug-related PIT effects in AD patients after detoxification. A total of 100 AD patients (17 females) performed a PIT task and an alcohol approach/avoidance task. Patients were followed for 6 months.

Results

A stronger alcohol approach bias was associated with both a more pronounced behavioral PIT effect and stronger PIT-related neural activity in the right nucleus accumbens. Moreover, the association between alcohol approach bias and behavioral PIT increased with the severity of alcohol dependence and trait impulsivity and was stronger in patients who relapsed during follow-up in the exploratory analysis.

Conclusions

These findings indicate partial behavioral and neurobiological overlap between alcohol approach bias and the PIT effect assessed with our tasks. The association was stronger in patients with more severe alcohol dependence.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

Reward prediction error in the ERP following unconditioned aversive stimuli

Reinforcement learning in humans and other animals is driven by reward prediction errors: deviations between the amount of reward or punishment initially expected and that which is obtained. Temporal difference methods of reinforcement learning generate this reward prediction error at the earliest time at which a revision in reward or punishment likelihood is signalled, for example by a conditioned stimulus. Midbrain dopamine neurons, believed to compute reward prediction errors, generate this signal in response to both conditioned and unconditioned stimuli, as predicted by temporal difference learning. Electroencephalographic recordings of human participants have suggested that a component named the feedback-related negativity (FRN) is generated when this signal is carried to the cortex. If this is so, the FRN should be expected to respond equivalently to conditioned and unconditioned stimuli. However, very few studies have attempted to measure the FRN's response to unconditioned stimuli. The present study attempted to elicit the FRN in response to a primary aversive stimulus (electric shock) using a design that varied reward prediction error while holding physical intensity constant. The FRN was strongly elicited, but earlier and more transiently than typically seen, suggesting that it may incorporate other processes than the midbrain dopamine system.

Susceptibility to interference between Pavlovian and instrumental control is associated with early hazardous alcohol use

Pavlovian-to-instrumental transfer (PIT) tasks examine the influence of Pavlovian stimuli on ongoing instrumental behaviour. Previous studies reported associations between a strong PIT effect, high-risk drinking and alcohol use disorder. This study investigated whether susceptibility to interference between Pavlovian and instrumental control is linked to risky alcohol use in a community sample of 18-year-old male adults. Participants (N = 191) were instructed to 'collect good shells' and 'leave bad shells' during the presentation of appetitive (monetary reward), aversive (monetary loss) or neutral Pavlovian stimuli. We compared instrumental error rates (ER) and functional magnetic resonance imaging (fMRI) brain responses between the congruent and incongruent conditions, as well as among high-risk and low-risk drinking groups. On average, individuals showed a substantial PIT effect, that is, increased ER when Pavlovian cues and instrumental stimuli were in conflict compared with congruent trials. Neural PIT correlates were found in the ventral striatum and the dorsomedial and lateral prefrontal cortices (lPFC). Importantly, high-risk drinking was associated with a stronger behavioural PIT effect, a decreased lPFC response and an increased neural response in the ventral striatum on the trend level. Moreover, high-risk drinkers showed weaker connectivity from the ventral striatum to the lPFC during incongruent trials. Our study links interference during PIT to drinking behaviour in healthy, young adults. High-risk drinkers showed higher susceptibility to Pavlovian cues, especially when they conflicted with instrumental behaviour, indicating lower interference control abilities. Increased activity in the ventral striatum (bottom-up), decreased lPFC response (top-down), and their altered interplay may contribute to poor interference control in the high-risk drinkers.

Anticipating control over aversive stimuli is mediated by the medial prefrontal cortex: An fMRI study with healthy adults

The anticipation of control over aversive events in life is relevant for our mental health. Insights on the underlying neural mechanisms remain limited. We developed a new functional magnetic resonance imaging (fMRI) task that uses auditory stimuli to explore the neural correlates of (1) the anticipation of control over aversion and (2) the processing of aversion. In a sample of 25 healthy adults, we observed increased neural activation in the medial prefrontal cortex (ventromedial prefrontal cortex and rostral anterior cingulate cortex), other brain areas relevant for reward anticipation (ventral striatum, brainstem [ventral tegmental area], midcingulate cortex), and the posterior cingulate cortex when they anticipated control over aversion compared with anticipating no control (1). The processing of aversive sounds compared to neutral sounds (2) was associated with increased neural activation in the bilateral posterior insula. Our findings provide evidence for the important role of medial prefrontal regions in control anticipation and highlight the relevance of conceiving the neural mechanisms involved within a reward-based framework.

Imaging of Functional Brain Circuits during Acquisition and Memory Retrieval in an Aversive Feedback Learning Task: Single Photon Emission Computed Tomography of Regional Cerebral Blood Flow in Freely Behaving Rats

Active avoidance learning is a complex form of aversive feedback learning that in humans and other animals is essential for actively coping with unpleasant, aversive, or dangerous situations. Since the functional circuits involved in two-way avoidance (TWA) learning have not yet been entirely identified, the aim of this study was to obtain an overall picture of the brain circuits that are involved in active avoidance learning. In order to obtain a longitudinal assessment of activation patterns in the brain of freely behaving rats during different stages of learning, we applied single-photon emission computed tomography (SPECT). We were able to identify distinct prefrontal cortical, sensory, and limbic circuits that were specifically recruited during the acquisition and retrieval phases of the two-way avoidance learning task.

Approach-Avoidance Decisions Under Threat: The Role of Autonomic Psychophysiological States

Acutely challenging or threatening situations frequently require approach-avoidance decisions. Acute threat triggers fast autonomic changes that prepare the body to freeze, fight or flee. However, such autonomic changes may also influence subsequent instrumental approach-avoidance decisions. Since defensive bodily states are often not considered in value-based decision-making models, it remains unclear how they influence the decision-making process. Here, we aim to bridge this gap by discussing the existing literature on the potential role of threat-induced bodily states on decision making and provide a new neurocomputational framework explaining how these effects can facilitate or bias approach-avoid decisions under threat. Theoretical accounts have stated that threat-induced parasympathetic activity is involved in information gathering and decision making. Parasympathetic dominance over sympathetic activity is particularly seen during threat-anticipatory freezing, an evolutionarily conserved response to threat demonstrated across species and characterized by immobility and bradycardia. Although this state of freezing has been linked to altered information processing and action preparation, a full theoretical treatment of the interactions with value-based decision making has not yet been achieved. Our neural framework, which we term the Threat State/Value Integration (TSI) Model, will illustrate how threat-induced bodily states may impact valuation of competing incentives at three stages of the decision-making process, namely at threat evaluation, integration of rewards and threats, and action initiation. Additionally, because altered parasympathetic activity and decision biases have been shown in anxious populations, we will end with discussing how biases in this system can lead to characteristic patterns of avoidance seen in anxiety-related disorders, motivating future pre-clinical and clinical research.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

Forced choices reveal a trade-off between cognitive effort and physical pain

Cognitive effort is described as aversive, and people will generally avoid it when possible. This aversion to effort is believed to arise from a cost-benefit analysis of the actions available. The comparison of cognitive effort against other primary aversive experiences, however, remains relatively unexplored. Here, we offered participants choices between performing a cognitively demanding task or experiencing thermal pain. We found that cognitive effort can be traded off for physical pain and that people generally avoid exerting high levels of cognitive effort. We also used computational modelling to examine the aversive subjective value of effort and its effects on response behaviours. Applying this model to decision times revealed asymmetric effects of effort and pain, suggesting that cognitive effort may not share the same basic influences on avoidance behaviour as more primary aversive stimuli such as physical pain.

Intact corticostriatal control of goal-directed action in Alcohol Use Disorder: a Pavlovian-to-instrumental transfer and outcome-devaluation study

Deficits in instrumental, goal-directed control, combined with the influence of drug-associated Pavlovian-conditioned stimuli, are thought to influence the development and maintenance of addiction. However, direct evidence has mainly come from animal studies. We sought to establish whether alcohol use disorder (AUD) is characterized by behavioral or neurobiological deficits in (i) the integration of Pavlovian and instrumental values and (ii) goal-directed control; and (iii) whether duration or severity of AUD is associated with such deficits. The influence of cues predicting food rewards on instrumental action was assessed in a Pavlovian-to-instrumental transfer (PIT) test, measuring both specific and general PIT, and goal-directed behavior in an outcome-devaluation test. Brain activity was measured using functional MRI in 38 abstinent individuals with AUD and 22 matched healthy control individuals (HCs). We found significant specific and general PIT and outcome-devaluation effects across groups indicating goal-directed control, mediated by distinct corticostriatal signals, but no significant differences between individuals with AUD and healthy controls. Bayesian analyses provided substantial-to-strong evidence for the absence of group differences for these effects, or any relationship with duration or severity of AUD. These results suggest intact ability to integrate action-outcome associations on specific and general PIT and goal-directed learning in AUD during abstinence.

An Adaptation of Pavlovian-to-Instrumental Transfer (PIT) Methodology to Examine the Energizing Effects of Reward-Predicting Cues on Behavior in Young Adults

There is growing recognition that much of human behavior is governed by the presence of classically conditioned cues. The Pavlovian-to-Instrumental Transfer (PIT) paradigm offers a way to measure the effects of classically conditioned stimuli on behavior. In the current study, a novel behavioral task, an adaptation of the PIT framework, was developed for use in conjunction with an fMRI classical conditioning task. Twenty-four healthy young adults completed (1) instrumental training, (2) Pavlovian conditioning, and (3) a Transfer test. During instrumental training, participants learned to apply force to a handgrip to win money from slot machines pictured on a computer screen. During Pavlovian conditioning, slot machines appeared with one of two abstract symbols (cues), one symbol was predictive of monetary reward. During the Transfer test, participants again applied force to a handgrip to win money. This time, the slot machines were presented with the Pavlovian cues, but with the outcomes hidden. The results indicated increased effort on the instrumental task, i.e. higher response frequency and greater force, in the presence of the reward-predicting cue. Our findings add to the growing number of studies demonstrating PIT effects in humans. This new paradigm is effective in measuring the effects of a conditioned stimulus on behavioral activation.

Dysfunctional approach behavior triggered by alcohol-unrelated Pavlovian cues predicts long-term relapse in alcohol dependence

We demonstrated that alcohol-dependent patients who relapsed within 1 year after detoxification showed stronger PIT effects compared with abstainers and controls. Relapsers particularly failed to correctly perform in trials where an instrumental stimulus required inhibition while a Pavlovian background cue indicated a monetary gain. Under that condition, relapsers approached the instrumental stimulus, independent of the expected punishment. The failure of inhibiting an aversive stimulus in favor of approaching an appetitive context cue reflects dysfunctional altered learning mechanisms in relapsers.

Neural Response Patterns During Pavlovian-to-Instrumental Transfer Predict Alcohol Relapse and Young Adult Drinking

BACKGROUND

Pavlovian-to-instrumental transfer (PIT) describes the influence of conditioned stimuli on instrumental behaviors and is discussed as a key process underlying substance abuse. Here, we tested whether neural responses during alcohol-related PIT predict future relapse in alcohol-dependent patients and future drinking behavior in adolescents.

METHODS

Recently detoxified alcohol-dependent patients (n = 52) and young adults without dependence (n = 136) underwent functional magnetic resonance imaging during an alcohol-related PIT paradigm, and their drinking behavior was assessed in a 12-month follow-up. To predict future drinking behavior from PIT activation patterns, we used a multivoxel classification scheme based on linear support vector machines.

RESULTS

When training and testing the classification scheme in patients, PIT activation patterns predicted future relapse with 71.2% accuracy. Feature selection revealed that classification was exclusively based on activation patterns in medial prefrontal cortex. To probe the generalizability of this functional magnetic resonance imaging-based prediction of future drinking behavior, we applied the support vector machine classifier that had been trained on patients to PIT functional magnetic resonance imaging data from adolescents. An analysis of cross-classification predictions revealed that those young social drinkers who were classified as abstainers showed a greater reduction in alcohol consumption at 12-month follow-up than those classified as relapsers (Δ = -24.4 ± 6.0 g vs. -5.7 ± 3.6 g; p = .019).

CONCLUSIONS

These results suggest that neural responses during PIT could constitute a generalized prognostic marker for future drinking behavior in established alcohol use disorder and in at-risk states.

Biasing Actions by Incentive Valence in an Approach/Avoidance Task

The present study investigates interactions between incentive valence and action, which mirror well-known valence-action biases in the emotional domain. In three joystick experiments, incentive valence (win/loss) and action type (approach/avoid) were signaled by distinct orthogonal stimulus features. By combining several design aspects, i.e., the use of bi-directional joystick movements, the inclusion of no-incentive baseline trials, and cue-locked versus target-locked valence and action signals, we tried to bridge between paradigms used in the emotional and motivational domain, and to understand previous, partly inconsistent results. In the first task variant (Experiment 1), we observed performance benefits for compatible mappings (win-approach; loss-avoid) relative to incompatible ones (loss-approach; win-avoid) when valence and action signals were target-locked, consistent with a fairly automatic response activation that can benefit or impair task performance. In contrast, cue-locked valence signals led to response facilitation (relative to a no-incentive baseline) more or less independent of actual valence (win/loss) and action type (approach/avoid), which is reminiscent of general facilitation effects of incentive cues across diverse cognitive tasks. Slight design variations did not change this main result pattern, indicating that it was neither driven by the close proximity between target and performance feedback (Experiment 2), nor by mere temporal coincidence of valence and action signals (Experiment 3), but rather by differences between preparatory (cued) and immediate (non-cued) effects of incentive valence. The present study provides novel insights regarding the nature of valence-action biases in the motivational domain and helps to integrate previous, partly inconsistent findings across domains.

Pavlovian-To-Instrumental Transfer and Alcohol Consumption in Young Male Social Drinkers: Behavioral, Neural and Polygenic Correlates

In animals and humans, behavior can be influenced by irrelevant stimuli, a phenomenon called Pavlovian-to-instrumental transfer (PIT). In subjects with substance use disorder, PIT is even enhanced with functional activation in the nucleus accumbens (NAcc) and amygdala. While we observed enhanced behavioral and neural PIT effects in alcohol-dependent subjects, we here aimed to determine whether behavioral PIT is enhanced in young men with high-risk compared to low-risk drinking and subsequently related functional activation in an a-priori region of interest encompassing the NAcc and amygdala and related to polygenic risk for alcohol consumption. A representative sample of 18-year old men (n = 1937) was contacted: 445 were screened, 209 assessed: resulting in 191 valid behavioral, 139 imaging and 157 genetic datasets. None of the subjects fulfilled criteria for alcohol dependence according to the Diagnostic and Statistical Manual of Mental Disorders-IV-TextRevision (DSM-IV-TR). We measured how instrumental responding for rewards was influenced by background Pavlovian conditioned stimuli predicting action-independent rewards and losses. Behavioral PIT was enhanced in high-compared to low-risk drinkers (b = 0.09, SE = 0.03, z = 2.7, p < 0.009). Across all subjects, we observed PIT-related neural blood oxygen level-dependent (BOLD) signal in the right amygdala (t = 3.25, p_SVC = 0.04, x = 26, y = -6, z = -12), but not in NAcc. The strength of the behavioral PIT effect was positively correlated with polygenic risk for alcohol consumption (r_s = 0.17, p = 0.032). We conclude that behavioral PIT and polygenic risk for alcohol consumption might be a biomarker for a subclinical phenotype of risky alcohol consumption, even if no drug-related stimulus is present. The association between behavioral PIT effects and the amygdala might point to habitual processes related to out PIT task. In non-dependent young social drinkers, the amygdala rather than the NAcc is activated during PIT; possible different involvement in association with disease trajectory should be investigated in future studies.

Pavlovian-to-instrumental transfer after human threat conditioning

Threat conditioning is a common associative learning model with translational relevance. How threat-conditioned cues impact on formally unrelated instrumental behavior in humans is not well known. Such an effect is known as Pavlovian-to-instrumental transfer (PIT). While PIT with aversive primary Pavlovian reinforcers is established in nonhuman animals, this is less clear in humans, where secondary reinforcers or instructed instrumental responses are most often investigated. We modified an existing human PIT procedure to include primary reinforcers. Participants first learned to obtain (or avoid losing) appetitive instrumental reinforcement (chocolate) by appropriate approach or avoidance actions. They either had to act (Go) or to withhold an action (NoGo), and in the Go condition either to approach a reward target to collect it or to withdraw from the reward target to avoid losing it. Then they learned to associate screen color (CS) with aversive Pavlovian reinforcement (electric shock US). In the transfer phase, we conducted the instrumental task during the presence of Pavlovian CS. In a first experiment, we show that the aversive Pavlovian CS+, compared to CS−, increased response rate in Go-Withdraw trials, i.e., induce conditioned facilitation of avoidance responses. This finding was confirmed in a second and independent experiment with an increased number of Go-Withdraw trials. Notably, we observed no appreciable conditioned suppression of approach responses. Effect size to distinguish CS+/CS− in Go-Withdraw trials was d = 0.42 in the confirmation sample. This would require n = 37 participants to demonstrate threat learning with 80% power. Thus, the effect size is on a practically useful scale although smaller than for model-based analysis of autonomic measures. In summary, our results indicate conditioned facilitation of formally unrelated instrumental avoidance behavior in humans and provide a novel behavioral threat learning measure that requires only key presses.

Neural correlates of instrumental responding in the context of alcohol-related cues index disorder severity and relapse risk

The influence of Pavlovian conditioned stimuli on ongoing behavior may contribute to explaining how alcohol cues stimulate drug seeking and intake. Using a Pavlovian-instrumental transfer task, we investigated the effects of alcohol-related cues on approach behavior (i.e., instrumental response behavior) and its neural correlates, and related both to the relapse after detoxification in alcohol-dependent patients. Thirty-one recently detoxified alcohol-dependent patients and 24 healthy controls underwent instrumental training, where approach or non-approach towards initially neutral stimuli was reinforced by monetary incentives. Approach behavior was tested during extinction with either alcohol-related or neutral stimuli (as Pavlovian cues) presented in the background during functional magnetic resonance imaging (fMRI). Patients were subsequently followed up for 6 months. We observed that alcohol-related background stimuli inhibited the approach behavior in detoxified alcohol-dependent patients (t = - 3.86, p < .001), but not in healthy controls (t = - 0.92, p = .36). This behavioral inhibition was associated with neural activation in the nucleus accumbens (NAcc) (t₍₃₀₎ = 2.06, p < .05). Interestingly, both the effects were only present in subsequent abstainers, but not relapsers and in those with mild but not severe dependence. Our data show that alcohol-related cues can acquire inhibitory behavioral features typical of aversive stimuli despite being accompanied by a stronger NAcc activation, suggesting salience attribution. The fact that these findings are restricted to abstinence and milder illness suggests that they may be potential resilience factors.Clinical trial: LeAD study, http://www.lead-studie.de , NCT01679145.

Social threat learning transfers to decision making in humans

In today’s world, indirect exposure to threatening situations is more common than ever, as illustrated by footage of terror and disaster in social media. How do such social threat learning experiences shape our decisions? We found that learning about threats from both observation and verbal information strongly influenced decision making. As with learning from our own experience, this influence could be either adaptive or maladaptive depending on whether the social information provided accurate expectations about the environment. Our findings can help explain both adaptive and pathological behaviors resulting from the indirect exposure to threatening events.

In today’s world, mass-media and online social networks present us with unprecedented exposure to second-hand, vicarious experiences and thereby the chance of forming associations between previously innocuous events (e.g., being in a subway station) and aversive outcomes (e.g., footage or verbal reports from a violent terrorist attack) without direct experience. Such social threat, or fear, learning can have dramatic consequences, as manifested in acute stress symptoms and maladaptive fears. However, most research has so far focused on socially acquired threat responses that are expressed as increased arousal rather than active behavior. In three experiments (n = 120), we examined the effect of indirect experiences on behaviors by establishing a link between social threat learning and instrumental decision making. We contrasted learning from direct experience (i.e., Pavlovian conditioning) (experiment 1) against two common forms of social threat learning—social observation (experiment 2) and verbal instruction (experiment 3)—and how this learning transferred to subsequent instrumental decision making using behavioral experiments and computational modeling. We found that both types of social threat learning transfer to decision making in a strong and surprisingly inflexible manner. Notably, computational modeling indicated that the transfer of observational and instructed threat learning involved different computational mechanisms. Our results demonstrate the strong influence of others’ expressions of fear on one’s own decisions and have important implications for understanding both healthy and pathological human behaviors resulting from the indirect exposure to threatening events.

Common and distinct neural substrates of the money illusion in win and loss domains

People often evaluate money based on its face value and overlook its real purchasing power, a phenomenon known as the money illusion. In the present study, using functional magnetic resonance imaging (fMRI) combined with a gambling task, we examined the neural signatures of the money illusion in both win and loss domains. Behavioral results showed that self-reported satisfaction with outcomes was modulated by the face value but not the true value of money in both win and loss domains. At the neural level, activity in the posterior insula was associated with the true value of money in the win domain, but not in the loss domain. Importantly, we found that the ventral striatum, ventromedial prefrontal cortex (vmPFC) and amygdala encoded the money illusion in both domains, indicating a domain-general rather than domain-specific neural signature. Moreover, participants with a larger degree of money illusion at the behavioral level showed stronger functional connectivity between the ventral striatum and ventral anterior cingulate cortex (vACC) in the win domain, but stronger functional connectivity between the ventral striatum and amygdala in the loss domain. Our findings highlight the overlapping and distinct neural substrates underlying the money illusion in the context of wins and losses.

Reward and avoidance learning in the context of aversive environments and possible implications for depressive symptoms

BACKGROUND

Aversive stimuli in the environment influence human actions. This includes valence-dependent influences on action selection, e.g., increased avoidance but decreased approach behavior. However, it is yet unclear how aversive stimuli interact with complex learning and decision-making in the reward and avoidance domain. Moreover, the underlying computational mechanisms of these decision-making biases are unknown.

METHODS

To elucidate these mechanisms, 54 healthy young male subjects performed a two-step sequential decision-making task, which allows to computationally model different aspects of learning, e.g., model-free, habitual, and model-based, goal-directed learning. We used a within-subject design, crossing task valence (reward vs. punishment learning) with emotional context (aversive vs. neutral background stimuli). We analyzed choice data, applied a computational model, and performed simulations.

RESULTS

Whereas model-based learning was not affected, aversive stimuli interacted with model-free learning in a way that depended on task valence. Thus, aversive stimuli increased model-free avoidance learning but decreased model-free reward learning. The computational model confirmed this effect: the parameter lambda that indicates the influence of reward prediction errors on decision values was increased in the punishment condition but decreased in the reward condition when aversive stimuli were present. Further, by using the inferred computational parameters to simulate choice data, our effects were captured. Exploratory analyses revealed that the observed biases were associated with subclinical depressive symptoms.

CONCLUSION

Our data show that aversive environmental stimuli affect complex learning and decision-making, which depends on task valence. Further, we provide a model of the underlying computations of this affective modulation. Finally, our finding of increased decision-making biases in subjects reporting subclinical depressive symptoms matches recent reports of amplified Pavlovian influences on action selection in depression and suggests a potential vulnerability factor for mood disorders. We discuss our findings in the light of the involvement of the neuromodulators serotonin and dopamine.

Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning

The brain’s dopaminergic pathways are crucially important for adaptive behavior. They are thought to enable us to approach rewards and stay away from punishments. During learning, dopaminergic reward prediction errors are thought to reinforce previously rewarded actions, so they become easier to repeat. This dopaminergic activity could lead to a systematic bias by which rewarded actions are more readily learned than rewarded inactions. We present two findings. First, dopamine receptors in cortex, dorsal striatum, and nucleus accumbens provide distinct sources of variance in the human brain. Second, the boost in an individual’s learning rate from previously rewarded actions is dependent on the dopamine receptor density in dorsal striatum, a central structure in the dopaminergic circuit.

Learning to act to obtain reward and inhibit to avoid punishment is easier compared with learning the opposite contingencies. This coupling of action and valence is often thought of as a Pavlovian bias, although recent research has shown it may also emerge through instrumental mechanisms. We measured this learning bias with a rewarded go/no-go task in 60 adults of different ages. Using computational modeling, we characterized the bias as being instrumental. To assess the role of endogenous dopamine (DA) in the expression of this bias, we quantified DA D1 receptor availability using positron emission tomography (PET) with the radioligand [¹¹C]SCH23390. Using principal-component analysis on the binding potentials in a number of cortical and striatal regions of interest, we demonstrated that cortical, dorsal striatal, and ventral striatal areas provide independent sources of variance in DA D1 receptor availability. Interindividual variation in the dorsal striatal component was related to the strength of the instrumental bias during learning. These data suggest at least three anatomical sources of variance in DA D1 receptor availability separable using PET in humans, and we provide evidence that human dorsal striatal DA D1 receptors are involved in the modulation of instrumental learning biases.

Motivational sensitivity of outcome-response priming: Experimental research and theoretical models

Outcome-response (O-R) priming is at the core of various associative theories of human intentional action. This is a simple and parsimonious mechanism by which activation of outcome representations (e.g. thinking about the light coming on) leads to activation of the associated motor patterns required to achieve it (e.g. pushing the light switch). In the current manuscript, we review the evidence for such O-R associative links demonstrated by converging (yet until now, separate) strands of research. While there is a wealth of evidence that both the perceptual and motivational properties of an outcome can be encoded in the O-R association and mediate O-R priming, we critically examine the integration of these mechanisms and the conditions under which motivational factors constrain the sensory O-R priming effect. We discuss the clinical relevance of this O-R priming mechanism, whether it can satisfactorily account for human goal-directed behaviour, and the implications for theories of human action control.

Active Confirmation Bias in the Evaluative Processing of Food Images

Predictive processing is fundamental to many aspects of the human mind, including perception and decision-making. It remains to be elucidated, however, in which way predictive information impacts on evaluative processing, particularly in tasks that employ bivalent stimulus sets. Various accounts, including framing, proactive interference, and cognitive control, appear to imply contradictory proposals on the relation between prediction and preference formation. To disambiguate whether predictive cues produce congruent biases versus opponent mechanisms in evaluative processing, we conducted two experiments in which participants were asked to rate individual food images. The image database included appetitive and aversive items. In each trial, a cue predicted, with varying degrees of reliability, the valence of the impending food image. In both experiments, we found that the ratings exhibited congruent biases as a function of the reliability of the predictive cue, with the highest evaluations following the most reliable positive-valence predictions. Eye prepositioning further showed a selective spatial bias suggestive of response preparation in line with the predictions. The response times also exhibited a pattern of results consistent with selective preparation, producing slow responses following invalid predictions. The data suggested an active form of evaluative processing, implementing a confirmation bias that aims to accommodate the prediction.

Frontal network dynamics reflect neurocomputational mechanisms for reducing maladaptive biases in motivated action

Motivation exerts control over behavior by eliciting Pavlovian responses, which can either match or conflict with instrumental action. We can overcome maladaptive motivational influences putatively through frontal cognitive control. However, the neurocomputational mechanisms subserving this control are unclear; does control entail up-regulating instrumental systems, down-regulating Pavlovian systems, or both? We combined electroencephalography (EEG) recordings with a motivational Go/NoGo learning task (N = 34), in which multiple Go options enabled us to disentangle selective action learning from nonselective Pavlovian responses. Midfrontal theta-band (4 Hz–8 Hz) activity covaried with the level of Pavlovian conflict and was associated with reduced Pavlovian biases rather than reduced instrumental learning biases. Motor and lateral prefrontal regions synchronized to the midfrontal cortex, and these network dynamics predicted the reduction of Pavlovian biases over and above local, midfrontal theta activity. This work links midfrontal processing to detecting Pavlovian conflict and highlights the importance of network processing in reducing the impact of maladaptive, Pavlovian biases.

The anticipation of reward and punishment are key drivers of behavior: we tend to take action for rewards while holding back in the face of punishment. This motivational bias might have an overall evolutionary advantage but can also work against us in specific situations. Here, we first asked whether this motivational bias relies on innate, automatic action tendencies or whether this bias might actually itself be learned. Secondly, we studied which brain processes reduce the impact of these motivational drives when they become dysfunctional. By comparing the actions of human participants to the predictions of several mathematical models, we showed that the motivational bias in action relies partly on automatic tendencies and partly on asymmetric learning from experience. We then observed that activity over the midfrontal cortex specifically increased as a function of how dysfunctional the automatic tendencies were. Additionally, this midfrontal cortex activity was functionally connected to the motor and lateral frontal cortices, which play a role in activating and inhibiting behavior. By incorporating this connectivity into the mathematical models, we showed that stronger midfrontal connectivity predicted reduced impact of dysfunctional automatic tendencies on behavior. We propose that the midfrontal cortex detects dysfunctional action tendencies and implements cognitive control by signaling across the network.

Depression is associated with enhanced aversive Pavlovian control over instrumental behaviour

The dynamic modulation of instrumental behaviour by conditioned Pavlovian cues is an important process in decision-making. Patients with major depressive disorder (MDD) are known to exhibit mood-congruent biases in information processing, which may occur due to Pavlovian influences, but this hypothesis has never been tested directly in an unmedicated sample. To address this we tested unmedicated MDD patients and healthy volunteers on a computerized Pavlovian-Instrumental Transfer (PIT) task designed to separately examine instrumental approach and withdrawal actions in the context of Pavlovian appetitive and aversive cues. This design allowed us to directly measure the degree to which Pavlovian cues influence instrumental responding. Depressed patients were profoundly influenced by aversive Pavlovian stimuli, to a significantly greater degree than healthy volunteers. This was the case for instrumental behaviour both in the approach condition (in which aversive Pavlovian cues inhibited ‘go’ responses), and in the withdrawal condition (in which aversive Pavlovian cues facilitated ‘go’ responses). Exaggerated aversive PIT provides a potential cognitive mechanism for biased emotion processing in major depression. This finding also has wider significance for the understanding of disrupted motivational processing in neuropsychiatric disorders.

Interactions of Motivation and Cognitive Control

There is general agreement that both motivation and cognitive control play critical roles in shaping goal-directed behavior, but only recently has scientific interest focused around the question of motivation-control interactions. Here we briefly survey this literature, organizing contemporary findings around three issues: 1) whether motivation preferentially impacts cognitive control processes, 2) the neural mechanisms that underlie motivation-cognition interactions, and 3) why motivation might be relevant for overcoming the costs of control. Dopamine (DA) is discussed as a key neuromodulator in these motivation-cognition interactions. We conclude by highlighting open issues, specifically Pavlovian versus instrumental control distinctions and effects of motivational valence and conflict, which could benefit from future research attention.

Pavlovian Control of Escape and Avoidance

To survive in complex environments, animals need to have mechanisms to select effective actions quickly, with minimal computational costs. As perhaps the computationally most parsimonious of these systems, Pavlovian control accomplishes this by hardwiring specific stereotyped responses to certain classes of stimuli. It is well documented that appetitive cues initiate a Pavlovian bias toward vigorous approach; however, Pavlovian responses to aversive stimuli are less well understood. Gaining a deeper understanding of aversive Pavlovian responses, such as active avoidance, is important given the critical role these behaviors play in several psychiatric conditions. The goal of the current study was to establish a behavioral and computational framework to examine aversive Pavlovian responses (activation vs. inhibition) depending on the proximity of an aversive state (escape vs. avoidance). We introduce a novel task in which participants are exposed to primary aversive (noise) stimuli and characterized behavior using a novel generative computational model. This model combines reinforcement learning and drift-diffusion models so as to capture effects of invigoration/inhibition in both explicit choice behavior as well as changes in RT. Choice and RT results both suggest that escape is associated with a bias for vigorous action, whereas avoidance is associated with behavioral inhibition. These results lay a foundation for future work seeking insights into typical and atypical aversive Pavlovian responses involved in psychiatric disorders, allowing us to quantify both implicit and explicit indices of vigorous choice behavior in the context of aversion.

The Neural Basis of Aversive Pavlovian Guidance during Planning

Important real-world decisions are often arduous as they frequently involve sequences of choices, with initial selections affecting future options. Evaluating every possible combination of choices is computationally intractable, particularly for longer multistep decisions. Therefore, humans frequently use heuristics to reduce the complexity of decisions. We recently used a goal-directed planning task to demonstrate the profound behavioral influence and ubiquity of one such shortcut, namely aversive pruning, a reflexive Pavlovian process that involves neglecting parts of the decision space residing beyond salient negative outcomes. However, how the brain implements this important decision heuristic and what underlies individual differences have hitherto remained unanswered. Therefore, we administered an adapted version of the same planning task to healthy male and female volunteers undergoing functional magnetic resonance imaging (fMRI) to determine the neural basis of aversive pruning. Through both computational and standard categorical fMRI analyses, we show that when planning was influenced by aversive pruning, the subgenual cingulate cortex was robustly recruited. This neural signature was distinct from those associated with general planning and valuation, two fundamental cognitive components elicited by our task but which are complementary to aversive pruning. Furthermore, we found that individual variation in levels of aversive pruning was associated with the responses of insula and dorsolateral prefrontal cortices to the receipt of large monetary losses, and also with subclinical levels of anxiety. In summary, our data reveal the neural signatures of an important reflexive Pavlovian process that shapes goal-directed evaluations and thereby determines the outcome of high-level sequential cognitive processes.SIGNIFICANCE STATEMENT Multistep decisions are complex because initial choices constrain future options. Evaluating every path for long decision sequences is often impractical; thus, cognitive shortcuts are often essential. One pervasive and powerful heuristic is aversive pruning, in which potential decision-making avenues are curtailed at immediate negative outcomes. We used neuroimaging to examine how humans implement such pruning. We found it to be associated with activity in the subgenual cingulate cortex, with neural signatures that were distinguishable from those covarying with planning and valuation. Individual variations in aversive pruning levels related to subclinical anxiety levels and insular cortex activation. These findings reveal the neural mechanisms by which basic negative Pavlovian influences guide decision-making during planning, with implications for disrupted decision-making in psychiatric disorders.

Strong seduction: impulsivity and the impact of contextual cues on instrumental behavior in alcohol dependence

Alcohol-related cues acquire incentive salience through Pavlovian conditioning and then can markedly affect instrumental behavior of alcohol-dependent patients to promote relapse. However, it is unclear whether similar effects occur with alcohol-unrelated cues. We tested 116 early-abstinent alcohol-dependent patients and 91 healthy controls who completed a delay discounting task to assess choice impulsivity, and a Pavlovian-to-instrumental transfer (PIT) paradigm employing both alcohol-unrelated and alcohol-related stimuli. To modify instrumental choice behavior, we tiled the background of the computer screen either with conditioned stimuli (CS) previously generated by pairing abstract pictures with pictures indicating monetary gains or losses, or with pictures displaying alcohol or water beverages. CS paired to money gains and losses affected instrumental choices differently. This PIT effect was significantly more pronounced in patients compared to controls, and the group difference was mainly driven by highly impulsive patients. The PIT effect was particularly strong in trials in which the instrumental stimulus required inhibition of instrumental response behavior and the background CS was associated to monetary gains. Under that condition, patients performed inappropriate approach behavior, contrary to their previously formed behavioral intention. Surprisingly, the effect of alcohol and water pictures as background stimuli resembled that of aversive and appetitive CS, respectively. These findings suggest that positively valenced background CS can provoke dysfunctional instrumental approach behavior in impulsive alcohol-dependent patients. Consequently, in real life they might be easily seduced by environmental cues to engage in actions thwarting their long-term goals. Such behaviors may include, but are not limited to, approaching alcohol.

Catecholaminergic challenge uncovers distinct Pavlovian and instrumental mechanisms of motivated (in)action

Catecholamines modulate the impact of motivational cues on action. Such motivational biases have been proposed to reflect cue-based, 'Pavlovian' effects. Here, we assess whether motivational biases may also arise from asymmetrical instrumental learning of active and passive responses following reward and punishment outcomes. We present a novel paradigm, allowing us to disentangle the impact of reward and punishment on instrumental learning from Pavlovian response biasing. Computational analyses showed that motivational biases reflect both Pavlovian and instrumental effects: reward and punishment cues promoted generalized (in)action in a Pavlovian manner, whereas outcomes enhanced instrumental (un)learning of chosen actions. These cue- and outcome-based biases were altered independently by the catecholamine enhancer melthylphenidate. Methylphenidate's effect varied across individuals with a putative proxy of baseline dopamine synthesis capacity, working memory span. Our study uncovers two distinct mechanisms by which motivation impacts behaviour, and helps refine current models of catecholaminergic modulation of motivated action.