Phasic transition from goal-directed to habitual control over drug-seeking produced by conflicting reinforcer expectancy

High-mind wandering correlates with high risk for problematic alcohol use in China and Germany

Everyone experiences the natural ebb and flow of task-unrelated thoughts. Given how common the fluctuations in these thoughts are, surprisingly, we know very little about how they shape individuals' responses to alcohol use. Here, we investigated if mind wandering is associated with a risk of developing problematic alcohol use. We undertook an online survey among the general population in China (N = 1123) and Germany (N = 1018) from December 2021 to February 2022 and examined the subjective experience of mind wandering and problematic alcohol use through the Mind Wandering Questionnaire (MWQ) and the Alcohol Use Disorders Identification Test (AUDIT). We compared mind wandering and problematic alcohol use between two countries and investigated the association between MWQ scores with AUDIT scores. We found higher scores on the MWQ and a high percentage of problematic alcohol use (i.e., AUDIT score ≥ 8) in Germany (22.5%) as compared to in China (14.5%). Higher self-reported mind wandering was associated with higher AUDIT scores. AUDIT scores were increased mostly in male, elder, and high-mind wandering people. Our findings highlight that mind wandering and problematic alcohol use enhanced in Germany as compared to in China. Our study sheds light on the relationship between mind wandering and problematic alcohol use that may help to further investigate causal effects of interventions.

The effects of age on the severity of problem drinking: Mediating effects of positive alcohol expectancy and neural correlates

Aging is associated with reduction in the severity of alcohol misuse. However, the psychological and neural mechanisms underlying the age-related changes remain unclear. Here, we tested the hypothesis that age-related diminution of positive alcohol expectancy (AE) mediated the effects of age on problem drinking and investigated the neural correlates of the mediating effects. Ninety-six drinkers 21-85 years of age, including social drinkers and those with mild/moderate alcohol use disorder (AUD), were assessed for global positive (GP) AE and problem drinking, each with the Alcohol Expectancy Questionnaire and Alcohol Use Disorders Identification Test (AUDIT), and with brain imaging during alcohol cue exposure. We processed imaging data with published routines; identified the correlates shared between whole-brain regression against age, GP and AUDIT scores; and performed mediation and path analyses to explore the interrelationships between the clinical and neural variables. The results showed that age was negatively correlated with both GP and AUDIT scores, with GP score completely mediating the correlation between age and AUDIT score. Lower age and higher GP correlated with shared cue responses in bilateral parahippocampal gyrus and left middle occipital cortex (PHG/OC). Further, higher GP and AUDIT scores were associated with shared cue responses in bilateral rostral anterior cingulate cortex and caudate head (ACC/caudate). Path analyses demonstrated models with significant statistical fit and PHG/OC and ACC/caudate each interrelating age to GP and GP to AUDIT scores. These findings confirmed change in positive AE as a psychological mechanism mitigating alcohol misuse as individuals age and highlighted the neural processes of cue-reactivity interrelating age and alcohol use severity.

Pavlovian-to-Instrumental Transfer across Mental Disorders: A Review

A mechanism known as Pavlovian-to-instrumental transfer (PIT) describes a phenomenon by which the values of environmental cues acquired through Pavlovian conditioning can motivate instrumental behavior. PIT may be one basic mechanism of action control that can characterize mental disorders on a dimensional level beyond current classification systems. Therefore, we review human PIT studies investigating subclinical and clinical mental syndromes. The literature prevails an inhomogeneous picture concerning PIT. While enhanced PIT effects seem to be present in non-substance-related disorders, overweight people, and most studies with AUD patients, no altered PIT effects were reported in tobacco use disorder and obesity. Regarding AUD and relapsing alcohol-dependent patients, there is mixed evidence of enhanced or no PIT effects. Additionally, there is evidence for aberrant corticostriatal activation and genetic risk, e.g., in association with high-risk alcohol consumption and relapse after alcohol detoxification. In patients with anorexia nervosa, stronger PIT effects elicited by low caloric stimuli were associated with increased disease severity. In patients with depression, enhanced aversive PIT effects and a loss of action-specificity associated with poorer treatment outcomes were reported. Schizophrenic patients showed disrupted specific but intact general PIT effects. Patients with chronic back pain showed reduced PIT effects. We provide possible reasons to understand heterogeneity in PIT effects within and across mental disorders. Further, we strengthen the importance of reliable experimental tasks and provide test-retest data of a PIT task showing moderate to good reliability. Finally, we point toward stress as a possible underlying factor that may explain stronger PIT effects in mental disorders, as there is some evidence that stress per se interacts with the impact of environmental cues on behavior by selectively increasing cue-triggered wanting. To conclude, we discuss the results of the literature review in the light of Research Domain Criteria, suggesting future studies that comprehensively assess PIT across psychopathological dimensions.

Subliminal determinants of cue-guided choice

By anticipating potential rewards, external cues can guide behavior to achieve a goal. Whether the conscious elaboration of these cues is necessary to elicit cue-guided choices is still unknown. The goal of the present study is to test whether the subliminal presentation of a visual cue previously paired with a reward is sufficient to bias responses that can lead to the same or a similar reward. To this aim, three experiments compared the subliminal and supraliminal presentation of reward-associated cues during a Pavlovian-to-Instrumental Transfer task. In line with previous evidence, results showed that the supraliminal presentation of reward-associated Pavlovian cues biased participant’s choice towards motivationally similar rewards (general transfer) as well as towards rewards sharing the precise sensory-specific properties of the cue (outcome-specific transfer). In striking contrast, subliminal cues biased choice only towards motivationally similar rewards (general transfer). Taken together, these findings suggest that cue-guided choices are modulated by the level of perceptual threshold (i.e., subliminal vs supraliminal) of reward-associated cues. Although conscious elaboration of the cue is necessary to guide choice towards a specific reward, subliminal processing is still sufficient to push towards choices sharing the motivational properties of the cue. Implications for everyday life, clinical conditions, and theoretical accounts of cue-guided choices are discussed.

Addiction is driven by excessive goal-directed drug choice under negative affect: translational critique of habit and compulsion theory

Drug addiction may be a goal-directed choice driven by excessive drug value in negative affective states, a habit driven by strong stimulus-response associations, or a compulsion driven by insensitivity to costs imposed on drug seeking. Laboratory animal and human evidence for these three theories is evaluated. Excessive goal theory is supported by dependence severity being associated with greater drug choice/economic demand. Drug choice is demonstrably goal-directed (driven by the expected value of the drug) and can be augmented by stress/negative mood induction and withdrawal-effects amplified in those with psychiatric symptoms and drug use coping motives. Furthermore, psychiatric symptoms confer risk of dependence, and coping motives mediate this risk. Habit theory of addiction has weaker support. Habitual behaviour seen in drug-exposed animals often does not occur in complex decision scenarios, or where responding is rewarded, so habit is unlikely to explain most human addictive behaviour where these conditions apply. Furthermore, most human studies have not found greater propensity to habitual behaviour in drug users or as a function of dependence severity, and the minority that have can be explained by task disengagement producing impaired explicit contingency knowledge. Compulsion theory of addiction also has weak support. The persistence of punished drug seeking in animals is better explained by greater drug value (evinced by the association with economic demand) than by insensitivity to costs. Furthermore, human studies have provided weak evidence that propensity to discount cost imposed on drug seeking is associated with dependence severity. These data suggest that human addiction is primarily driven by excessive goal-directed drug choice under negative affect, and less by habit or compulsion. Addiction is pathological because negative states powerfully increase expected drug value acutely outweighing abstinence goals.

Intact corticostriatal control of goal-directed action in Alcohol Use Disorder: a Pavlovian-to-instrumental transfer and outcome-devaluation study

Deficits in instrumental, goal-directed control, combined with the influence of drug-associated Pavlovian-conditioned stimuli, are thought to influence the development and maintenance of addiction. However, direct evidence has mainly come from animal studies. We sought to establish whether alcohol use disorder (AUD) is characterized by behavioral or neurobiological deficits in (i) the integration of Pavlovian and instrumental values and (ii) goal-directed control; and (iii) whether duration or severity of AUD is associated with such deficits. The influence of cues predicting food rewards on instrumental action was assessed in a Pavlovian-to-instrumental transfer (PIT) test, measuring both specific and general PIT, and goal-directed behavior in an outcome-devaluation test. Brain activity was measured using functional MRI in 38 abstinent individuals with AUD and 22 matched healthy control individuals (HCs). We found significant specific and general PIT and outcome-devaluation effects across groups indicating goal-directed control, mediated by distinct corticostriatal signals, but no significant differences between individuals with AUD and healthy controls. Bayesian analyses provided substantial-to-strong evidence for the absence of group differences for these effects, or any relationship with duration or severity of AUD. These results suggest intact ability to integrate action-outcome associations on specific and general PIT and goal-directed learning in AUD during abstinence.

Compulsivity in Alcohol Use Disorder and Obsessive Compulsive Disorder: Implications for Neuromodulation

Alcohol use Disorder (AUD) is one of the leading causes of morbidity and mortality worldwide. The progression of the disorder is associated with the development of compulsive alcohol use, which in turn contributes to the high relapse rate and poor longer term functioning reported in most patients, even with treatment. While the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) defines AUD by a cluster of symptoms, parsing its heterogeneous phenotype by domains of behavior such as compulsivity may be a critical step to improve outcomes of this condition. Still, neurobiological underpinnings of compulsivity need to be fully elucidated in AUD in order to better design targeted treatment strategies. In this manuscript, we review and discuss findings supporting common mechanisms between AUD and OCD, dissecting the construct of compulsivity and focusing specifically on characteristic disruptions in habit learning and cognitive control in the two disorders. Finally, neuromodulatory interventions are proposed as a probe to test compulsivity as key pathophysiologic feature of AUD, and as a potential therapy for the subgroup of individuals with compulsive alcohol use, i.e., the more resistant stage of the disorder. This transdiagnostic approach may help to destigmatize the disorder, and suggest potential treatment targets across different conditions.

Nicotine effects on associative learning in human non-smokers

Tobacco smoking is the most common preventable cause of death in the US. Nicotine is considered the primary constituent responsible for tobacco addiction. Its paradoxically high abuse potential may reflect behavioral control by drug-associated stimuli, which appears to play a larger role for tobacco dependence than for other abused drugs. We tested a potential explanation, hypothesizing that nicotine enhances associative learning, the mechanism underlying the conditioning of drug-associated stimuli. Thirty-two non-smokers were exposed to transdermal nicotine (7 mg/24 h) and placebo in a double-blind cross-over study and tested with behavioral paradigms designed to isolate incidental stimulus-stimulus or stimulus-response learning. The stop signal task required speeded gender judgments of face stimuli. A tone signaled when to withhold the response. Unbeknownst to participants, some faces were always paired with stop trials. Nicotine enhanced the facilitation of stop-responses to these stimuli, and the slowing of go-responses when previously stop-associated stimuli were paired with go trials, indicating stronger associations between paired stimuli and the stop signal/stop response. Another task required feedback-based learning of associations between pairs of shape stimuli. Five pairs were made from either ten different stimuli, or from different combinations of two identical sets of five stimuli with correct associations depending on contextual information. Nicotine increased incorrect choices of stimuli that were associated in a different context, indicating stronger stimulus-stimulus associations at the expense of flexible context-adaptive behavior. The results indicate that nicotine can enhance incidental associative learning, a mechanism that may promote the formation of smoking-associated stimuli and cue-controlled drug-taking.

The Role of Habits in Anorexia Nervosa: Where We Are and Where to Go From Here?

PURPOSE OF REVIEW

The persistent maladaptive eating behavior characteristic of anorexia nervosa (AN) can be understood as a learned habit. This review describes the cognitive neuroscience background and the existing data from research in AN.

RECENT FINDINGS

Behavior is habitual after it is frequently repeated and becomes nearly automatic, relatively insensitive to outcome, and mediated by dorsal frontostriatal neural systems. There is evidence for such behavior in AN, in which restrictive intake has been related to dorsal frontostriatal systems. Other neural and neurocognitive data provide mixed findings, some of which suggest disturbances in habit systems in AN. There are compelling behavioral and neural data to suggest that habit systems may underlie the persistence of AN. The habit model needs further research, via more direct behavioral hypothesis testing and probes of the development of habitual behavior. Investigation of the habit-centered model of AN may open avenues for the development of novel treatments.

Synaptic adaptations to chronic ethanol intake in male rhesus monkey dorsal striatum depend on age of drinking onset

One in 12 adults suffer with alcohol use disorder (AUD). Studies suggest the younger the age in which alcohol consumption begins the higher the probability of being diagnosed with AUD. Binge/excessive alcohol drinking involves a transition from flexible to inflexible behavior likely involving the dorsal striatum (caudate and putamen nuclei). A major focus of this study was to examine the effect of age of drinking onset on subsequent chronic, voluntary ethanol intake and dorsal striatal circuitry. Data from rhesus monkeys (n = 45) that started drinking as adolescents, young adults or mature adults confirms an age-related risk for heavy drinking. Striatal neuroadaptations were examined using whole-cell patch clamp electrophysiology to record AMPA receptor-mediated miniature excitatory postsynaptic currents (mEPSCs) and GABAA receptor-mediated miniature inhibitory postsynaptic currents (mIPSCs) from medium-sized spiny projection neurons located in the caudate or putamen nuclei. In controls, greater GABAergic transmission (mIPSC frequency and amplitude) was observed in the putamen compared to the caudate. With advancing age, in the absence of ethanol, an increase in mIPSC frequency concomitant with changes in mIPSC amplitude was observed in both regions. Chronic ethanol drinking decreased mIPSC frequency in the putamen regardless of age of onset. In the caudate, an ethanol drinking-induced increase in mIPSC frequency was only observed in monkeys that began drinking as young adults. Glutamatergic transmission did not differ between the dorsal striatal subregions in controls. With chronic ethanol drinking there was a decrease in the postsynaptic characteristics of rise time and area of mEPSCs in the putamen but an increase in mEPSC frequency in the caudate. Together, the observed changes in striatal physiology indicate a combined disinhibition due to youth and ethanol leading to abnormally strong activation of the putamen that could contribute to the increased risk for problem drinking in younger drinkers.

Depressive statements prime goal-directed alcohol-seeking in individuals who report drinking to cope with negative affect

BACKGROUND

Most variants of negative reinforcement theory predict that acute depressed mood can promote alcohol-seeking behaviour, but the precise mechanisms underpinning this effect remain contested. One possibility is that mood-induced alcohol-seeking is due to the formation of a stimulus-response (S-R) association, enabling depressed mood to elicit alcohol-seeking automatically. A second possibility is that depressed mood undergoes incentive learning, enabling it to enhance the expected value of alcohol and thus promote goal-directed alcohol-seeking.

OBJECTIVES

These two explanations were distinguished using a human outcome-revaluation procedure.

METHODS

One hundred and twenty-eight alcohol drinkers completed questionnaires of alcohol use disorder, drinking to cope with negative affect and depression symptoms. Participants then learned that two responses earned alcohol and food points respectively (baseline) in two alternative forced choice trials. At test, participants rated the valence of randomly sampled negative and positive mood statements and, after each statement, chose between the alcohol- and food-seeking responses in extinction.

RESULTS

The percentage of alcohol- versus food-seeking responses was increased significantly in trials containing negative statements compared to baseline and positive statement trials, in individuals who reported drinking to cope with negative affect (p = .004), but there was no such interaction with indices of alcohol use disorder (p = .87) or depression symptoms (p = .58).

CONCLUSIONS

Individuals who drink to cope with negative affect are more sensitive to the motivational impact of acute depressed mood statements priming goal-directed alcohol-seeking. Negative copers' vulnerability to alcohol dependence may be better explained by excessive affective incentive learning than by S-R habit formation.

Alcohol devaluation has dissociable effects on distinct components of alcohol behaviour

RATIONALE

Substance-related behaviour is often viewed as an appetitive behaviour, motivated by the reinforcing effects of the drug. However, there are various indices of substance motivation (e.g. attentional bias, behavioural economic demand, craving) and it is unclear how these are related or whether they play an important role in all types of substance-related behaviour.

OBJECTIVES

(1) To determine the effect of alcohol devaluation on several indices of alcohol motivation and goal-directed and cue-elicited alcohol behaviour. (2) To investigate which components of motivation mediate any effect of devaluation on behaviour.

METHODS

Sixty-two social drinkers gave baseline measures of alcohol craving, behavioural economic demand and choice for alcohol vs. soft drink. Participants tasted alcohol which was either unadulterated (control) or adulterated with a bitter solution (devaluation) before craving and demand were measured again. Alcohol choice was assessed in several phases: extinction (evaluating goal-directed behaviour), in the presence of drink cues (Pavlovian-to-instrumental transfer (PIT, cue-elicited behaviour)), and reacquisition. Attentional bias (AB) was measured by tracking eye movements towards the drink cues during novel PIT trials where both cues were presented. Finally, consumption was evaluated in a taste test.

RESULTS

Alcohol devaluation reduced alcohol-related demand, AB, alcohol choice in all phases, and consumption. Alcohol cues presented during PIT increased alcohol choice above baseline irrespective of devaluation. AB and demand for alcohol fully mediated the effect of devaluation on alcohol choice during extinction, AB fully mediated the effect on cue-elicited (specific PIT) alcohol choice and alcohol consumption.

CONCLUSIONS

Alcohol behaviour in social drinkers is largely sensitive to devaluation, i.e. governed by current motivational value of the drug (suggesting goal-directed behaviour). However, a dissociable form of stimulus control can also drive alcohol-seeking independently of drug value (specific PIT). Mediation analyses suggests that AB may play a paradoxical role in both forms of alcohol seeking and consumption.

Pavlovian-to-Instrumental Transfer of Nicotine and Food Cues in Deprived Cigarette Smokers

Introduction

Smoking-related cues can promote drug-seeking behavior and curtail attempts to quit. One way to understand the potential impact of such cues is to compare cue-elicited behaviors for smoking and other reinforcers (eg, food) using the Pavlovian-to-instrumental transfer paradigm, which measures how much control cues can exert over reward-seeking responses.

Methods

We tested the influence of appetitive cues on smokers' behavior following 12 hours of abstinence from smoking and eating. First, we equated the value of cigarette and food by assessing a Willingness-to-Pay measure for each reinforcer. Second, we evaluated behavioral differences between cues with Pavlovian-to-instrumental transfer. In two phases, participants learned (1) the association between distinct stimuli and cigarette or food outcomes and, (2) specific instrumental responses that yielded such outcomes. Motivated behavior was probed under extinction in a subsequent transfer test assessing instrumental responding in the presence of the cues.

Results

Participants showed an increase in specific responding (eg, instrumental response associated with cigarette) when faced with the corresponding appetitive cue (eg, stimulus associated with cigarette) despite absence of outcome. Notably, they made more cigarette-seeking than food-seeking instrumental responses, suggesting that cues representative of cigarette outcomes exert stronger influences on behavior than non-drug (food) cues. Using a measure of subjective preference, we also observed that greater preference for cigarette-compared to food-cues correlated with increased cigarette-seeking behavior in the test phase.

Conclusion

Taken together, these results highlight how drug and non-drug cues differentially influence reward-seeking behaviors during deprivation, which has implications for smoking cessation treatment and relapse.

Implications

This study examines the motivational influence of both drug and non-drug cues within a single sample of cigarette smokers. Our results demonstrate that the motivational properties of smoking cues differ from cues relating to other types of reward, such as food. This research informs smoking cessation programs to target the salience of nicotine cues and the maladaptive drug-seeking behaviors prompted by them.

When Habits Are Dangerous: Alcohol Expectancies and Habitual Decision Making Predict Relapse in Alcohol Dependence

BACKGROUND

Addiction is supposedly characterized by a shift from goal-directed to habitual decision making, thus facilitating automatic drug intake. The two-step task allows distinguishing between these mechanisms by computationally modeling goal-directed and habitual behavior as model-based and model-free control. In addicted patients, decision making may also strongly depend upon drug-associated expectations. Therefore, we investigated model-based versus model-free decision making and its neural correlates as well as alcohol expectancies in alcohol-dependent patients and healthy controls and assessed treatment outcome in patients.

METHODS

Ninety detoxified, medication-free, alcohol-dependent patients and 96 age- and gender-matched control subjects underwent functional magnetic resonance imaging during the two-step task. Alcohol expectancies were measured with the Alcohol Expectancy Questionnaire. Over a follow-up period of 48 weeks, 37 patients remained abstinent and 53 patients relapsed as indicated by the Alcohol Timeline Followback method.

RESULTS

Patients who relapsed displayed reduced medial prefrontal cortex activation during model-based decision making. Furthermore, high alcohol expectancies were associated with low model-based control in relapsers, while the opposite was observed in abstainers and healthy control subjects. However, reduced model-based control per se was not associated with subsequent relapse.

CONCLUSIONS

These findings suggest that poor treatment outcome in alcohol dependence does not simply result from a shift from model-based to model-free control but is instead dependent on the interaction between high drug expectancies and low model-based decision making. Reduced model-based medial prefrontal cortex signatures in those who relapse point to a neural correlate of relapse risk. These observations suggest that therapeutic interventions should target subjective alcohol expectancies.

Current major depression is associated with greater sensitivity to the motivational effect of both negative mood induction and abstinence on tobacco-seeking behavior

BACKGROUND

Although depression and smoking commonly co-occur, the mechanisms underpinning this association are poorly understood. One hypothesis is that depression promotes tobacco dependence, persistence and relapse by increasing sensitivity to acute negative mood and abstinence induced tobacco-seeking behavior.

METHODS

Twenty nine daily smokers of >10 cigarettes per day, nine with major depression and 20 without, completed two laboratory sessions one week apart, smoking as normal prior to session 1 (sated session), and 6h abstinent prior to session 2 (abstinent session). In both sessions, tobacco-seeking was measured at baseline by preference to view smoking versus food images. Negative mood was then induced by negative ruminative statements and sad music, before tobacco-seeking was measured again at test.

RESULTS

In the sated session, negative mood induction produced a greater increase in tobacco choice from baseline to test in depressed (p<0.001, η_p²=0.782) compared to non-depressed smokers (p=0.045, η_p²=0.216, interaction: p=0.046, η_p²=0.150). Abstinence also produced a greater increase in baseline tobacco choice between the sated and abstinent sessions in depressed (p=0.002, η_p²=0.771) compared to non-depressed smokers (p=0.22, η_p²=0.089, interaction: p=0.023, η_p²=0.189). These mood and abstinence induced increases in tobacco choice were positively associated with depression symptoms across the sample as a whole (ps≤0.04, η_p²≥0.159), and correlated with each other (r=0.67, p<0.001).

CONCLUSIONS

Current major depression or depression symptoms may promote tobacco dependence, persistence and relapse by increasing sensitivity to both acute negative mood and abstinence induced tobacco-seeking behavior. Treatments should seek to break the association between adverse states and smoking to cope.

Triggering Avoidance: Dissociable Influences of Aversive Pavlovian Conditioned Stimuli on Human Instrumental Behavior

The present study investigates human aversive Pavlovian-to-Instrumental Transfer (PIT) and possible influences of outcome devaluation and instrumental overtraining on this effect. PIT measures the extent to which a Pavlovian conditioned stimulus (CS) can increase instrumental responses independently paired with the same (outcome-specific transfer) or a different (general transfer) reinforcer. Two measures of PIT were obtained: the percentage of instrumental responses and the vigor of such responses. Thirty-eight volunteers performed a standard PIT task sequence. Results showed a double dissociation between outcome-specific and general transfer: the first selectively expressed in the amount of responses, the second in the vigor measure solely. Furthermore, outcome-specific transfer was enhanced by overtraining, but not affected by devaluation. General transfer, on the other hand, was affected by neither overtraining, nor devaluation. A positive correlation between general transfer and sensitivity to punishments was found. Findings are discussed in terms of hypothetically different underlying neurobehavioral mechanisms and their relations to habits and goal-directed behavior.

The propositional basis of cue-controlled reward seeking

Two experiments examined the role of propositional and automatic (ideomotor) processes in cue-elicited responding for rewarding outcomes (beer and chocolate). In a training phase, participants earned either chocolate or beer points by making one of two button-press responses. Rewards were indicated by the presentation of chocolate and beer pictures. On test, each trial began with a picture of beer or chocolate, or a blank screen, and choice of the beer versus chocolate response was assessed in the presence of these three pictures. Participants tended to choose the beer and chocolate response in the presence of the beer and chocolate pictures, respectively. In Experiment 1, instructions signalling that the pictures did not indicate which response would be rewarded significantly reduced the priming effect. In Experiment 2, instructions indicating that the pictures signified which response would not be rewarded resulted in a reversed priming effect. Finally, in both experiments, the priming effect correlated with self-reported beliefs that the cues signalled which response was more likely to be reinforced. These results suggest that cue-elicited response selection is mediated by a propositional belief regarding the efficacy of the response–outcome relationship, rather than an automatic ideomotor mechanism.

Don't Think, Just Feel the Music: Individuals with Strong Pavlovian-to-Instrumental Transfer Effects Rely Less on Model-based Reinforcement Learning

Behavioral choice can be characterized along two axes. One axis distinguishes reflexive, model-free systems that slowly accumulate values through experience and a model-based system that uses knowledge to reason prospectively. The second axis distinguishes Pavlovian valuation of stimuli from instrumental valuation of actions or stimulus-action pairs. This results in four values and many possible interactions between them, with important consequences for accounts of individual variation. We here explored whether individual variation along one axis was related to individual variation along the other. Specifically, we asked whether individuals' balance between model-based and model-free learning was related to their tendency to show Pavlovian interferences with instrumental decisions. In two independent samples with a total of 243 participants, Pavlovian-instrumental transfer effects were negatively correlated with the strength of model-based reasoning in a two-step task. This suggests a potential common underlying substrate predisposing individuals to both have strong Pavlovian interference and be less model-based and provides a framework within which to interpret the observation of both effects in addiction.

Negative mood reverses devaluation of goal-directed drug-seeking favouring an incentive learning account of drug dependence

BACKGROUND

Two theories explain how negative mood primes smoking behaviour. The stimulus-response (S-R) account argues that in the negative mood state, smoking is experienced as more reinforcing, establishing a direct (automatic) association between the negative mood state and smoking behaviour. By contrast, the incentive learning account argues that in the negative mood state smoking is expected to be more reinforcing, which integrates with instrumental knowledge of the response required to produce that outcome.

OBJECTIVES

One differential prediction is that whereas the incentive learning account anticipates that negative mood induction could augment a novel tobacco-seeking response in an extinction test, the S-R account could not explain this effect because the extinction test prevents S-R learning by omitting experience of the reinforcer.

METHODS

To test this, overnight-deprived daily smokers (n = 44) acquired two instrumental responses for tobacco and chocolate points, respectively, before smoking to satiety. Half then received negative mood induction to raise the expected value of tobacco, opposing satiety, whilst the remainder received positive mood induction. Finally, a choice between tobacco and chocolate was measured in extinction to test whether negative mood could augment tobacco choice, opposing satiety, in the absence of direct experience of tobacco reinforcement.

RESULTS

Negative mood induction not only abolished the devaluation of tobacco choice, but participants with a significant increase in negative mood increased their tobacco choice in extinction, despite satiety.

CONCLUSIONS

These findings suggest that negative mood augments drug-seeking by raising the expected value of the drug through incentive learning, rather than through automatic S-R control.

Model-based and model-free decisions in alcohol dependence

BACKGROUND

Human and animal work suggests a shift from goal-directed to habitual decision-making in addiction. However, the evidence for this in human alcohol dependence is as yet inconclusive.

METHODS

Twenty-six healthy controls and 26 recently detoxified alcohol-dependent patients underwent behavioral testing with a 2-step task designed to disentangle goal-directed and habitual response patterns.

RESULTS

Alcohol-dependent patients showed less evidence of goal-directed choices than healthy controls, particularly after losses. There was no difference in the strength of the habitual component. The group differences did not survive controlling for performance on the Digit Symbol Substitution Task.

CONCLUSION

Chronic alcohol use appears to selectively impair goal-directed function, rather than promoting habitual responding. It appears to do so particularly after nonrewards, and this may be mediated by the effects of alcohol on more general cognitive functions subserved by the prefrontal cortex.

Pavlovian-to-instrumental transfer in alcohol dependence: a pilot study

BACKGROUND

Pavlovian processes are thought to play an important role in the development, maintenance and relapse of alcohol dependence, possibly by influencing and usurping ongoing thought and behavior. The influence of pavlovian stimuli on ongoing behavior is paradigmatically measured by pavlovian-to-instrumental transfer (PIT) tasks. These involve multiple stages and are complex. Whether increased PIT is involved in human alcohol dependence is uncertain. We therefore aimed to establish and validate a modified PIT paradigm that would be robust, consistent and tolerated by healthy controls as well as by patients suffering from alcohol dependence, and to explore whether alcohol dependence is associated with enhanced PIT.

METHODS

Thirty-two recently detoxified alcohol-dependent patients and 32 age- and gender-matched healthy controls performed a PIT task with instrumental go/no-go approach behaviors. The task involved both pavlovian stimuli associated with monetary rewards and losses, and images of drinks.

RESULTS

Both patients and healthy controls showed a robust and temporally stable PIT effect. Strengths of PIT effects to drug-related and monetary conditioned stimuli were highly correlated. Patients more frequently showed a PIT effect, and the effect was stronger in response to aversively conditioned CSs (conditioned suppression), but there was no group difference in response to appetitive CSs.

CONCLUSION

The implementation of PIT has favorably robust properties in chronic alcohol-dependent patients and in healthy controls. It shows internal consistency between monetary and drug-related cues. The findings support an association of alcohol dependence with an increased propensity towards PIT.

Individual differences in the influence of task-irrelevant Pavlovian cues on human behavior

Pavlovian-to-instrumental transfer (PIT) refers to the process of a Pavlovian reward-paired cue acquiring incentive motivational proprieties that drive choices. It represents a crucial phenomenon for understanding cue-controlled behavior, and it has both adaptive and maladaptive implications (i.e., drug-taking). In animals, individual differences in the degree to which such cues bias performance have been identified in two types of individuals that exhibit distinct Conditioned Responses (CR) during Pavlovian conditioning: Sign-Trackers (ST) and Goal-Trackers (GT). Using an appetitive PIT procedure with a monetary reward, the present study investigated, for the first time, the extent to which such individual differences might affect the influence of reward-paired cues in humans. In a first task, participants learned an instrumental response leading to reward; then, in a second task, a visual Pavlovian cue was associated with the same reward; finally, in a third task, PIT was tested by measuring the preference for the reward-paired instrumental response when the task-irrelevant reward-paired cue was presented, in the absence of the reward itself. In ST individuals, but not in GT individuals, reward-related cues biased behavior, resulting in an increased likelihood to perform the instrumental response independently paired with the same reward when presented with the task-irrelevant reward-paired cue, even if the reward itself was no longer available (i.e., stronger PIT effect). This finding has important implications for developing individualized treatment for maladaptive behaviors, such as addiction.

A hierarchical instrumental decision theory of nicotine dependence

It is important to characterize the learning processes governing tobacco-seeking in order to understand how best to treat this behavior. Most drug learning theories have adopted a Pavlovian framework wherein the conditioned response is the main motivational process. We favor instead a hierarchical instrumental decision account, wherein expectations about the instrumental contingency between voluntary tobacco-seeking and the receipt of nicotine reward determines the probability of executing this behavior. To support this view, we review titration and nicotine discrimination research showing that internal signals for deprivation/satiation modulate expectations about the current incentive value of smoking, thereby modulating the propensity of this behavior. We also review research on cue-reactivity which has shown that external smoking cues modulate expectations about the probability of the tobacco-seeking response being effective, thereby modulating the propensity of this behavior. Economic decision theory is then considered to elucidate how expectations about the value and probability of response-nicotine contingency are integrated to form an overall utility estimate for that option for comparison with qualitatively different, nonsubstitute reinforcers, to determine response selection. As an applied test for this hierarchical instrumental decision framework, we consider how well it accounts for individual liability to smoking uptake and perseveration, pharmacotherapy, cue-extinction therapies, and plain packaging. We conclude that the hierarchical instrumental account is successful in reconciling this broad range of phenomenon precisely because it accepts that multiple diverse sources of internal and external information must be integrated to shape the decision to smoke.

Plain cigarette packs do not exert Pavlovian to instrumental transfer of control over tobacco-seeking

AIMS

To gain insight into the potential impact of plain tobacco packaging policy, two experiments were undertaken to test whether 'prototype' plain compared with branded UK cigarette pack stimuli would differentially elicit instrumental tobacco-seeking in a nominal Pavlovian to instrumental transfer (PIT) procedure.

DESIGN, SETTING AND PARTICIPANTS

Two experiments were undertaken at the University of Bristol UK, with a convenience sample of adult smokers (experiment 1, n = 23, experiment 2, n = 121).

MEASUREMENT

In both experiments, smokers were trained on a concurrent choice procedure in which two responses earned points for cigarettes and chocolate, respectively, before images of branded and plain packs were tested for capacity to elicit the tobacco-seeking response in extinction. The primary outcome was percentage choice of the tobacco- over the chocolate-seeking response in plain pack, branded pack and no-stimulus conditions.

FINDINGS

Both experiments found that branded packs primed a greater percentage of tobacco-seeking (overall mean = 62%) than plain packs (overall mean = 53%) and the no-stimulus condition (overall mean = 52%; Ps ≤ 0.01, ŋp (2) s ≥ 0.16), and that there was no difference in percentage tobacco-seeking between plain packs and the no-stimulus condition (Ps ≥ 0.17, ŋp (2) s ≤ 0.04). Plain tobacco packs showed an overall 9% reduction in the priming of a tobacco choice response compared to branded tobacco packs.

CONCLUSIONS

Plain packaging may reduce smoking in current smokers by degrading cue-elicited tobacco-seeking.

Habitual responding for alcohol depends upon both AMPA and D2 receptor signaling in the dorsolateral striatum

Chronic alcohol self-administration leads to alcohol-seeking behavior that is habitual and insensitive to changes in the value of the earned alcohol. Such behavior has been shown to rely on the dorsolateral region of the striatum in rats but the specific pharmacological control of output from this region is not yet understood. In the following experiments rats were trained to self-administer unsweetened 10% (v/v) ethanol in daily sessions for 8 weeks prior to testing for sensitivity to outcome devaluation. We examined the role of glutamatergic AMPA-receptor activation by testing the effects of the antagonist NBQX (0.3 and 1.0 μg/μl) infused specifically into the dorsolateral striatum (DLS) before devaluation testing. In a separate group of rats we examined the role of dopaminergic D2-receptor activation using the D2-receptor antagonist raclopride (0.2 and 1.0 μg/μl) infused into the DLS before devaluation testing. Following control (saline) infusions rats’ lever-press performance was insensitive to devaluation of ethanol thus showing evidence of habitual responding. NBQX and racolpride each restored goal-directed control of responding at doses that did not impair overall lever-press rates. These data demonstrate that expression of habitual alcohol seeking relies on glutamatergic inputs to the DLS and D2 receptors within the DLS.

Electrophysiological responses to alcohol cues are not associated with Pavlovian-to-instrumental transfer in social drinkers

Pavlovian to Instrumental Transfer (PIT) refers to the behavioral phenomenon of increased instrumental responding for a reinforcer when in the presence of Pavlovian conditioned stimuli that were separately paired with that reinforcer. PIT effects may play an important role in substance use disorders, but little is known about the brain mechanisms that underlie these effects in alcohol consumers. We report behavioral and electroencephalographic (EEG) data from a group of social drinkers (n = 31) who performed a PIT task in which they chose between two instrumental responses in pursuit of beer and chocolate reinforcers while their EEG reactivity to beer, chocolate and neutral pictorial cues was recorded. We examined two markers of the motivational salience of the pictures: the P300 and slow wave event-related potentials (ERPs). Results demonstrated a behavioral PIT effect: responding for beer was increased when a beer picture was presented. Analyses of ERP amplitudes demonstrated significantly larger slow potentials evoked by beer cues at various electrode clusters. Contrary to hypotheses, there were no significant correlations between behavioral PIT effects, electrophysiological reactivity to the cues, and individual differences in drinking behaviour. Our findings are the first to demonstrate a PIT effect for beer, accompanied by increased slow potentials in response to beer cues, in social drinkers. The lack of relationship between behavioral and EEG measures, and between these measures and individual differences in drinking behaviour may be attributed to methodological features of the PIT task and to characteristics of our sample.

BIS impulsivity and acute nicotine exposure are associated with discounting global consequences in the Harvard game

OBJECTIVE

The Barratt Impulsivity Scale (BIS) provides a transdiagnostic marker for a number of psychiatric conditions and drug abuse, but the precise psychological trait(s) tapped by this questionnaire remain obscure.

METHOD

To address this, 51 smokers completed in counterbalanced order the BIS, a delay discounting task and a Harvard game that measured choice between a response that yielded a high immediate monetary payoff but decreased opportunity to earn money overall (local choice) versus a response that yielded a lower immediate payoff but afforded a greater opportunity to earn overall (global choice).

RESULTS

Individual level of BIS impulsivity and self-elected smoking prior to the study were independently associated with increased preference for the local over the global choice in the Harvard game, but not delay discounting.

CONCLUSIONS

BIS impulsivity and acute nicotine exposure reflect a bias in the governance of choice by immediate reward contingencies over global consequences, consistent with contemporary dual-process instrumental learning theories.