Acute alcohol impairs human goal-directed action

Striatal cell-type specific stability and reorganization underlying agency and habit

Adaptive decision making requires agency, knowledge that actions produce particular outcomes. For well-practiced routines, agency is relinquished in favor of habit. Here, we asked how dorsomedial striatum D1+ and D2/A2A+ neurons contribute to agency and habit. We imaged calcium activity of these neurons as mice learned to lever press with agency and formed habits with overtraining. Whereas many D1+ neurons stably encoded actions throughout learning and developed encoding of reward outcomes, A2A+ neurons reorganized their encoding of actions from initial action-outcome learning to habit formation. Chemogenetic manipulations indicated that both D1+ and A2A+ neurons support action-outcome learning, but only D1+ neurons enable the use of such agency for adaptive, goal-directed decision making. These data reveal coordinated dorsomedial striatum D1+ and A2A+ function for the development of agency, cell-type specific stability and reorganization underlying agency and habit, and important insights into the neuronal circuits of how we learn and decide.

A dual-pathway architecture enables chronic stress to disrupt agency and promote habit formation

Chronic stress can change how we learn and, thus, how we make decisions. Here we investigated the neuronal circuit mechanisms that enable this. Using a multifaceted systems neuroscience approach in male and female mice, we reveal a dual pathway, amygdala-striatal neuronal circuit architecture by which a recent history of chronic stress disrupts the action-outcome learning underlying adaptive agency and promotes the formation of inflexible habits. We found that the basolateral amygdala projection to the dorsomedial striatum is activated by rewarding events to support the action-outcome learning needed for flexible, goal-directed decision making. Chronic stress attenuates this to disrupt action-outcome learning and, therefore, agency. Conversely, the central amygdala projection to the dorsomedial striatum mediates habit formation. Following stress this pathway is progressively recruited to learning to promote the premature formation of inflexible habits. Thus, stress exerts opposing effects on two amygdala-striatal pathways to disrupt agency and promote habit. These data provide neuronal circuit insights into how chronic stress shapes learning and decision making, and help understand how stress can lead to the disrupted decision making and pathological habits that characterize substance use disorders and mental health conditions.

A novel free-operant framework enables experimental habit induction in humans

Habits are a prominent feature of both adaptive and maladaptive behavior. Yet, despite substantial research efforts, there are currently no well-established experimental procedures for habit induction in humans. It is likely that laboratory experimental settings, as well as the session-based structure typically used in controlled experiments (also outside the lab), impose serious constraints on studying habits and other effects that are sensitive to context, motivation, and training duration and frequency. To overcome these challenges, we devised a unique real-world free-operant task structure, implemented through a novel smartphone application, whereby participants could freely enter the app (24 hours a day, 7 days a week) to win rewards. This procedure is free of typical laboratory constraints, yet well controlled. Using the canonical sensitivity to outcome devaluation criterion, we successfully demonstrated habit formation as a function of training duration, a long-standing challenge in the field. Additionally, we show a positive relationship between multiple facets of engagement/motivation and goal-directedness. We suggest that our novel paradigm can be used to study the neurobehavioral and psychological mechanism underlying habits in humans. Moreover, the real-world free-operant framework can potentially be used to examine other instrumental behavior-related questions, with greater face validity in naturalistic conditions.

Behavioral Engagement With Playable Objects Resolves Stress-Induced Adaptive Changes by Reshaping the Reward System

BACKGROUND

The reward system regulates motivated behavior, and repeated practice of specific motivated behavior might conversely modify the reward system. However, the detailed mechanisms by which they reciprocally regulate each other are not clearly understood.

METHODS

Mice subjected to chronic restraint stress show long-lasting depressive-like behavior, which is rescued by continual engagement with playable objects. A series of molecular, pharmacological, genetic, and behavioral analyses, combined with microarray, liquid chromatography, and chemogenetic tools, are used to investigate the neural mechanisms of antidepressive effects of playable objects.

RESULTS

Here, we show that repeated restraint induces dopamine surges into the nucleus accumbens-lateral shell (NAc-lSh), which cause upregulation of the neuropeptide PACAP in the NAc-lSh. As repeated stress is continued, the dopamine surge by stressors is adaptively suppressed without restoring PACAP upregulation, and the resulting enhanced PACAP inputs from NAc-lSh neurons to the ventral pallidum facilitate depressive-like behaviors. Continual engagement with playable objects in mice subjected to chronic stress remediates reduced dopamine response to new stressors, enhanced PACAP upregulation, and depressive-like behaviors. Overactivation of dopamine D₁ receptors over the action of D₂ receptors in the NAc-lSh promotes depressive-like behaviors. Conversely, inhibition of D₁ receptors or PACAP upregulation in the NAc-lSh confers resilience to chronic stress-induced depressive-like behaviors. Histochemical and chemogenetic analyses reveal that engagement with playable objects produces antidepressive effects by reshaping the ventral tegmental area-to-NAc-lSh and NAc-lSh-to-ventral pallidum circuits.

CONCLUSIONS

These results suggest that behavioral engagement with playable objects remediates depressive-like behaviors by resolving stress-induced maladaptive changes in the reward system.

Reduced Glutamate Turnover in the Putamen Is Linked With Automatic Habits in Human Cocaine Addiction

BACKGROUND

The balance between goal-directed behavior and habits has been hypothesized to be biased toward the latter in individuals with cocaine use disorder (CUD), suggesting possible neurochemical changes in the putamen, which may contribute to their compulsive behavior.

METHODS

We assessed habitual behavior in 48 patients with CUD and 42 healthy control participants using a contingency degradation paradigm and the Creature of Habit Scale. In a subgroup of this sample (CUD: n = 21; control participants: n = 22), we also measured glutamate and glutamine concentrations in the left putamen using ultra-high-field (7T) magnetic resonance spectroscopy. We hypothesized that increased habitual tendencies in patients with CUD would be associated with abnormal glutamatergic metabolites in the putamen.

RESULTS

Compared with their non-drug-using peers, patients with CUD exhibited greater habitual tendencies during contingency degradation, which correlated with increased levels of self-reported daily habits. We further identified a significant reduction in glutamate concentration and glutamate turnover (glutamate-to-glutamine ratio) in the putamen in patients with CUD, which was significantly related to the level of self-reported daily habits.

CONCLUSIONS

Patients with CUD exhibit enhanced habitual behavior, as assessed both by questionnaire and by a laboratory paradigm of contingency degradation. This automatic habitual tendency is related to a reduced glutamate turnover in the putamen, suggesting a dysregulation of habits caused by chronic cocaine use.

Addiction is driven by excessive goal-directed drug choice under negative affect: translational critique of habit and compulsion theory

Drug addiction may be a goal-directed choice driven by excessive drug value in negative affective states, a habit driven by strong stimulus-response associations, or a compulsion driven by insensitivity to costs imposed on drug seeking. Laboratory animal and human evidence for these three theories is evaluated. Excessive goal theory is supported by dependence severity being associated with greater drug choice/economic demand. Drug choice is demonstrably goal-directed (driven by the expected value of the drug) and can be augmented by stress/negative mood induction and withdrawal-effects amplified in those with psychiatric symptoms and drug use coping motives. Furthermore, psychiatric symptoms confer risk of dependence, and coping motives mediate this risk. Habit theory of addiction has weaker support. Habitual behaviour seen in drug-exposed animals often does not occur in complex decision scenarios, or where responding is rewarded, so habit is unlikely to explain most human addictive behaviour where these conditions apply. Furthermore, most human studies have not found greater propensity to habitual behaviour in drug users or as a function of dependence severity, and the minority that have can be explained by task disengagement producing impaired explicit contingency knowledge. Compulsion theory of addiction also has weak support. The persistence of punished drug seeking in animals is better explained by greater drug value (evinced by the association with economic demand) than by insensitivity to costs. Furthermore, human studies have provided weak evidence that propensity to discount cost imposed on drug seeking is associated with dependence severity. These data suggest that human addiction is primarily driven by excessive goal-directed drug choice under negative affect, and less by habit or compulsion. Addiction is pathological because negative states powerfully increase expected drug value acutely outweighing abstinence goals.

Working memory moderates the association between condom use intentions and behavior among moderate-to-heavy drinking men who have sex with men

Men who have sex with men (MSM) are a high-risk population for HIV infection and this risk is increased for those who consume alcohol. Condomless anal intercourse (CAI) is the central transmission risk factor for this population. This study examined whether individual differences in working memory moderated the association between intentions to use condoms and the frequency of CAI among MSM who engaged in anal intercourse over a subsequent 6-week period. Moderate- and heavy-drinking MSM (n = 207) completed questionnaires regarding alcohol use and condom use intentions and an operation span task to assess working memory at baseline. Participants then completed 6 weeks of morning surveys via a mobile phone app to assess anal intercourse frequency with and without condoms. Negative binomial regression analyses showed that the association between intentions to use condoms and episodes of CAI during the monitoring period was moderated by working memory such that intentions predicted CAI for those high in working memory but not those low in working memory. These results support the view that self-reported intentions may be less-likely to translate into health behaviors among those with poorer executive functioning skills.

Dorsal Striatal Circuits for Habits, Compulsions and Addictions

Here, we review the neural circuit bases of habits, compulsions, and addictions, behaviors which are all characterized by relatively automatic action performance. We discuss relevant studies, primarily from the rodent literature, and describe how major headway has been made in identifying the brain regions and neural cell types whose activity is modulated during the acquisition and performance of these automated behaviors. The dorsal striatum and cortical inputs to this structure have emerged as key players in the wider basal ganglia circuitry encoding behavioral automaticity, and changes in the activity of different neuronal cell-types in these brain regions have been shown to co-occur with the formation of automatic behaviors. We highlight how disordered functioning of these neural circuits can result in neuropsychiatric disorders, such as obsessive-compulsive disorder (OCD) and drug addiction. Finally, we discuss how the next phase of research in the field may benefit from integration of approaches for access to cells based on their genetic makeup, activity, connectivity and precise anatomical location.

Presynaptic Ethanol Actions: Potential Roles in Ethanol Seeking

Ethanol produces intoxication through actions on numerous molecular and cellular targets. Adaptations involving these and other targets contribute to chronic drug actions that underlie continued and problematic drinking. Among the mechanisms involved in these ethanol actions are alterations in presynaptic mechanisms of synaptic transmission, including presynaptic protein function and excitation-secretion coupling. At synapses in the central nervous system (CNS), excitation-secretion coupling involves ion channel activation followed by vesicle fusion and neurotransmitter release. These mechanisms are altered by presynaptic neurotransmitter receptors and prominently by G protein-coupled receptors (GPCRs). Studies over the last 20-25 years have revealed that acute ethanol exposure alters neurotransmitter secretion, with especially robust effects on synapses that use the neurotransmitter gamma-aminobutyric acid (GABA). Intracellular signaling pathways involving second messengers such as cyclic AMP and calcium are implicated in these acute ethanol actions. Ethanol-induced release of neuropeptides and small molecule neurotransmitters that act on presynaptic GPCRs also contribute to presynaptic potentiation at synapses in the amygdala and hippocampus and inhibition of GABA release in the striatum. Prolonged exposure to ethanol alters neurotransmitter release at many CNS GABAergic and glutamatergic synapses, and changes in GPCR function are implicated in many of these neuroadaptations. These presynaptic neuroadaptations appear to involve compensation for acute drug effects at some synapses, but "allostatic" effects that result in long-term resetting of synaptic efficacy occur at others. Current investigations are determining how presynaptic neuroadaptations contribute to behavioral changes at different stages of alcohol drinking, with increasing focus on circuit adaptations underlying these behaviors. This chapter will discuss the acute and chronic presynaptic effects of ethanol in the CNS, as well as some of the consequences of these effects in amygdala and corticostriatal circuits that are related to excessive seeking/drinking and ethanol abuse.

Compulsivity in Alcohol Use Disorder and Obsessive Compulsive Disorder: Implications for Neuromodulation

Alcohol use Disorder (AUD) is one of the leading causes of morbidity and mortality worldwide. The progression of the disorder is associated with the development of compulsive alcohol use, which in turn contributes to the high relapse rate and poor longer term functioning reported in most patients, even with treatment. While the Diagnostic and Statistical Manual of Mental Disorders (DSM-5) defines AUD by a cluster of symptoms, parsing its heterogeneous phenotype by domains of behavior such as compulsivity may be a critical step to improve outcomes of this condition. Still, neurobiological underpinnings of compulsivity need to be fully elucidated in AUD in order to better design targeted treatment strategies. In this manuscript, we review and discuss findings supporting common mechanisms between AUD and OCD, dissecting the construct of compulsivity and focusing specifically on characteristic disruptions in habit learning and cognitive control in the two disorders. Finally, neuromodulatory interventions are proposed as a probe to test compulsivity as key pathophysiologic feature of AUD, and as a potential therapy for the subgroup of individuals with compulsive alcohol use, i.e., the more resistant stage of the disorder. This transdiagnostic approach may help to destigmatize the disorder, and suggest potential treatment targets across different conditions.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

Drunk decisions: Alcohol shifts choice from habitual towards goal-directed control in adolescent intermediate-risk drinkers

BACKGROUND

Studies in humans and animals suggest a shift from goal-directed to habitual decision-making in addiction. We therefore tested whether acute alcohol administration reduces goal-directed and promotes habitual decision-making, and whether these effects are moderated by self-reported drinking problems.

METHODS

Fifty-three socially drinking males completed the two-step task in a randomised crossover design while receiving an intravenous infusion of ethanol (blood alcohol level=80 mg%), or placebo. To minimise potential bias by long-standing heavy drinking and subsequent neuropsychological impairment, we tested 18- to 19-year-old adolescents.

RESULTS

Alcohol administration consistently reduced habitual, model-free decisions, while its effects on goal-directed, model-based behaviour varied as a function of drinking problems measured with the Alcohol Use Disorders Identification Test. While adolescents with low risk for drinking problems (scoring <8) exhibited an alcohol-induced numerical reduction in goal-directed choices, intermediate-risk drinkers showed a shift away from habitual towards goal-directed decision-making, such that alcohol possibly even improved their performance.

CONCLUSIONS

We assume that alcohol disrupted basic cognitive functions underlying habitual and goal-directed decisions in low-risk drinkers, thereby enhancing hasty choices. Further, we speculate that intermediate-risk drinkers benefited from alcohol as a negative reinforcer that reduced unpleasant emotional states, possibly displaying a novel risk factor for drinking in adolescence.

Acute drug effects on habitual and non-habitual responding in crossed high alcohol preferring mice

RATIONALE

Drug reward plays a central role in acquiring drug-seeking behavior. However, subjects may continue using drugs despite negative consequences because self-administration becomes habitual, and divorced from outcome values. Although a history of drug and alcohol use expedite habit acquisition, and in spite of the fact that self-administration leads to intoxication, the acute effects of drugs on habitual responding are not well understood.

OBJECTIVES

We sought to observe how acute ethanol and amphetamine affect the balance between habitual and goal-directed behavior, as measured by a fluid-reinforced operant conditioning task.

METHODS

Selectively bred crossed high-alcohol-preferring (cHAP) mice were trained on an operant conditioning task reinforced on a variable interval schedule with 1% banana solution, which was subsequently devalued via LiCl pairing in half the animals. Ethanol (1.0 g/kg), amphetamine (2.0 mg/kg), or saline was administered prior to a post-devaluation test.

RESULTS

Overall, mice showed habitual behavior, but when divided into high- or low-responding groups based on training response rates, saline-treated, low-responding animals devalued, while saline-treated high-responding animals did not. Furthermore, amphetamine elicited devaluation even in high-responding animals, while ethanol prevented devaluation even in low-responding animals.

CONCLUSIONS

These data show that ethanol shifts animals toward behaving habitually. This may illuminate why alcohol-intoxicated individuals display impaired judgment about the relative merits of drinking, and potentially serve as a mechanism by which intoxicated subjects resume previously devalued behaviors, such as comorbid drug use. These findings also show that high variable interval response rates facilitate a shift from goal-directed to habitual behavior.

Translational Assessments of Reward and Anhedonia: A Tribute to Athina Markou

Loss of pleasure (clinically referred to as anhedonia), impairments in other reward-related processes such as reward learning, motivation, and reward valuation, and blunted affect characterize several mood and other psychiatric disorders. Despite the availability of many therapeutic options for these disorders, reward-related impairments remain challenging to treat and often persist despite alleviation of other symptoms. Lack of animal models of reward-related impairments and affect that have high construct and predictive validity is a key obstacle to developing novel treatments. This review highlights 1) guidelines to consider when developing translatable animal models; and 2) recent efforts to develop new reward-related assessments in humans and nonhuman animals that have been translated or back-translated from one species to another. The procedures described in this review are used to assess aspects of reward learning, motivated behavior, reward valuation, and affect. In several cases, researchers have attempted to implement task parameters that are as identical as possible to the parallel parameters used in existing cross-species tasks, with the goal of improving the translation of preclinical drug discovery findings to the clinic. In this regard, Dr. Athina Markou, who worked tirelessly throughout her career to understand and treat reward-related impairments across several psychiatric disorders, had great influence on conceptualizing the development and use of translational animal models of reward-related processes.

No association of goal-directed and habitual control with alcohol consumption in young adults

Alcohol dependence is a mental disorder that has been associated with an imbalance in behavioral control favoring model-free habitual over model-based goal-directed strategies. It is as yet unknown, however, whether such an imbalance reflects a predisposing vulnerability or results as a consequence of repeated and/or excessive alcohol exposure. We, therefore, examined the association of alcohol consumption with model-based goal-directed and model-free habitual control in 188 18-year-old social drinkers in a two-step sequential decision-making task while undergoing functional magnetic resonance imaging before prolonged alcohol misuse could have led to severe neurobiological adaptations. Behaviorally, participants showed a mixture of model-free and model-based decision-making as observed previously. Measures of impulsivity were positively related to alcohol consumption. In contrast, neither model-free nor model-based decision weights nor the trade-off between them were associated with alcohol consumption. There were also no significant associations between alcohol consumption and neural correlates of model-free or model-based decision quantities in either ventral striatum or ventromedial prefrontal cortex. Exploratory whole-brain functional magnetic resonance imaging analyses with a lenient threshold revealed early onset of drinking to be associated with an enhanced representation of model-free reward prediction errors in the posterior putamen. These results suggest that an imbalance between model-based goal-directed and model-free habitual control might rather not be a trait marker of alcohol intake per se.

The propositional basis of cue-controlled reward seeking

Two experiments examined the role of propositional and automatic (ideomotor) processes in cue-elicited responding for rewarding outcomes (beer and chocolate). In a training phase, participants earned either chocolate or beer points by making one of two button-press responses. Rewards were indicated by the presentation of chocolate and beer pictures. On test, each trial began with a picture of beer or chocolate, or a blank screen, and choice of the beer versus chocolate response was assessed in the presence of these three pictures. Participants tended to choose the beer and chocolate response in the presence of the beer and chocolate pictures, respectively. In Experiment 1, instructions signalling that the pictures did not indicate which response would be rewarded significantly reduced the priming effect. In Experiment 2, instructions indicating that the pictures signified which response would not be rewarded resulted in a reversed priming effect. Finally, in both experiments, the priming effect correlated with self-reported beliefs that the cues signalled which response was more likely to be reinforced. These results suggest that cue-elicited response selection is mediated by a propositional belief regarding the efficacy of the response–outcome relationship, rather than an automatic ideomotor mechanism.

Habitual Alcohol Seeking: Neural Bases and Possible Relations to Alcohol Use Disorders

Loss of flexible control over alcohol use may contribute to the development of alcohol use disorders. An increased contribution of response habits to alcohol-related behaviors may help explain this loss of control. Focusing on data from outcome devaluation and Pavlovian-instrumental transfer procedures, we review evidence for loss of goal-directed control over alcohol seeking and consumption drawing from both preclinical findings and clinical data where they exist. Over the course of extended alcohol self-administration and exposure, the performance of alcohol-seeking responses becomes less sensitive to reduction in the value of alcohol and more vulnerable to the influences of alcohol-predictive stimuli. These behavioral changes are accompanied by a shift in the corticostriatal circuits that control responding from circuits centered on the dorsomedial to those centered on the dorsolateral striatum. These changes in behavioral and neural control could help explain failures to abstain from alcohol despite intention to do so. Understanding and ultimately ameliorating these changes will aid development of more effective treatment interventions.

The role of habit in compulsivity

Compulsivity has been recently characterized as a manifestation of an imbalance between the brain׳s goal-directed and habit-learning systems. Habits are perhaps the most fundamental building block of animal learning, and it is therefore unsurprising that there are multiple ways in which the development and execution of habits can be promoted/discouraged. Delineating these neurocognitive routes may be critical to understanding if and how habits contribute to the many faces of compulsivity observed across a range of psychiatric disorders. In this review, we distinguish the contribution of excessive stimulus-response habit learning from that of deficient goal-directed control over action and response inhibition, and discuss the role of stress and anxiety as likely contributors to the transition from goal-directed action to habit. To this end, behavioural, pharmacological, neurobiological and clinical evidence are synthesised and a hypothesis is formulated to capture how habits fit into a model of compulsivity as a trans-diagnostic psychiatric trait.

Risk Factors for Addiction and Their Association with Model-Based Behavioral Control

Addiction shows familial aggregation and previous endophenotype research suggests that healthy relatives of addicted individuals share altered behavioral and cognitive characteristics with individuals suffering from addiction. In this study we asked whether impairments in behavioral control proposed for addiction, namely a shift from goal-directed, model-based toward habitual, model-free control, extends toward an unaffected sample (n = 20) of adult children of alcohol-dependent fathers as compared to a sample without any personal or family history of alcohol addiction (n = 17). Using a sequential decision-making task designed to investigate model-free and model-based control combined with a computational modeling analysis, we did not find any evidence for altered behavioral control in individuals with a positive family history of alcohol addiction. Independent of family history of alcohol dependence, we however observed that the interaction of two different risk factors of addiction, namely impulsivity and cognitive capacities, predicts the balance of model-free and model-based behavioral control. Post-hoc tests showed a positive association of model-based behavior with cognitive capacity in the lower, but not in the higher impulsive group of the original sample. In an independent sample of particularly high- vs. low-impulsive individuals, we confirmed the interaction effect of cognitive capacities and high vs. low impulsivity on model-based control. In the confirmation sample, a positive association of omega with cognitive capacity was observed in highly impulsive individuals, but not in low impulsive individuals. Due to the moderate sample size of the study, further investigation of the association of risk factors for addiction with model-based behavior in larger sample sizes is warranted.

Addiction History Associates with the Propensity to Form Habits

Learned habitual responses to environmental stimuli allow efficient interaction with the environment, freeing cognitive resources for more demanding tasks. However, when the outcome of such actions is no longer a desired goal, established stimulus-response (S-R) associations or habits must be overcome. Among people with substance use disorders (SUDs), difficulty in overcoming habitual responses to stimuli associated with their addiction in favor of new, goal-directed behaviors contributes to relapse. Animal models of habit learning demonstrate that chronic self-administration of drugs of abuse promotes habitual responding beyond the domain of compulsive drug seeking. However, whether a similar propensity toward domain-general habitual responding occurs in humans with SUDs has remained unclear. To address this question, we used a visuomotor S-R learning and relearning task, the Hidden Association between Images Task, which employs abstract visual stimuli and manual responses. This task allows us to measure new S-R association learning and well-learned S-R association execution and includes a response contingency change manipulation to quantify the degree to which responding is habit-based, rather than goal-directed. We find that people with SUDs learn new S-R associations as well as healthy control participants do. Moreover, people with an SUD history slightly outperform controls in S-R execution. In contrast, people with SUDs are specifically impaired in overcoming well-learned S-R associations; those with SUDs make a significantly greater proportion of perseverative errors during well-learned S-R replacement, indicating the more habitual nature of their responses. Thus, with equivalent training and practice, people with SUDs appear to show enhanced domain-general habit formation.

The effects of stereotype threat and contextual cues on alcohol users' inhibitory control

AIM

Previous research indicates that users of illicit substances exhibit diminished cognitive function under stereotype threat. Advancing this research, the current study aimed to examine the effects of stereotype threat on alcohol users' inhibitory control. It also examined whether drinkers demonstrate a greater approach bias towards alcohol-related relative to neutral stimuli.

METHOD

Fifty-five participants were assigned randomly to a stereotype threat condition, in which they were primed with a negative stereotype linking drinking behavior to cognitive decline, or a non-threat control condition. All participants then completed a modified version of the Cued Go/No-Go Association Test that exposed participants to alcohol-related and neutral pictorial stimuli and sound cues.

RESULTS

Stereotype threatened participants demonstrated a speed-accuracy trade off, taking significantly longer to respond to go-trials with equivalent accuracy to the control condition. They also showed reduced response accuracy to both alcohol-related and neutral stimuli in reversed instruction trials. Participants in the control condition were both more accurate and quicker to respond to alcohol-related stimuli compared to neutral stimuli.

CONCLUSION

These results suggest that awareness of negative stereotypes pertaining to alcohol-related impulsivity may have a harmful effect on inhibitive cognitive performance. This may have implications for public health campaigns and for methodological designs with high levels of procedural signaling with respect to not inadvertently inducing stereotype threat and impacting impulsivity.

Translational Research on Habit and Alcohol

Habitual actions enable efficient daily living, but they can also contribute to pathological behaviors that resistant change, such as alcoholism. Habitual behaviors are learned actions that appear goal-directed but are in fact no longer under the control of the action's outcome. Instead, these actions are triggered by stimuli, which may be exogenous or interoceptive, discrete or contextual. A major hallmark characteristic of alcoholism is continued alcohol use despite serious negative consequences. In essence, although the outcome of alcohol seeking and drinking is dramatically devalued, these actions persist, often triggered by environmental cues associated with alcohol use. Thus, alcoholism meets the definition of an initially goal-directed behavior that converts to a habit-based process. Habit and alcohol have been well investigated in rodent models, with comparatively less research in non-human primates and people. This review focuses on translational research on habit and alcohol with an emphasis on cross-species methodology and neural circuitry.

Lateral prefrontal model-based signatures are reduced in healthy individuals with high trait impulsivity

High impulsivity is an important risk factor for addiction with evidence from endophenotype studies. In addiction, behavioral control is shifted toward the habitual end. Habitual control can be described by retrospective updating of reward expectations in 'model-free' temporal-difference algorithms. Goal-directed control relies on the prospective consideration of actions and their outcomes, which can be captured by forward-planning 'model-based' algorithms. So far, no studies have examined behavioral and neural signatures of model-free and model-based control in healthy high-impulsive individuals. Fifty healthy participants were drawn from the upper and lower ends of 452 individuals, completing the Barratt Impulsiveness Scale. All participants performed a sequential decision-making task during functional magnetic resonance imaging (fMRI) and underwent structural MRI. Behavioral and fMRI data were analyzed by means of computational algorithms reflecting model-free and model-based control. Both groups did not differ regarding the balance of model-free and model-based control, but high-impulsive individuals showed a subtle but significant accentuation of model-free control alone. Right lateral prefrontal model-based signatures were reduced in high-impulsive individuals. Effects of smoking, drinking, general cognition or gray matter density did not account for the findings. Irrespectively of impulsivity, gray matter density in the left dorsolateral prefrontal cortex was positively associated with model-based control. The present study supports the idea that high levels of impulsivity are accompanied by behavioral and neural signatures in favor of model-free behavioral control. Behavioral results in healthy high-impulsive individuals were qualitatively different to findings in patients with the same task. The predictive relevance of these results remains an important target for future longitudinal studies.

Negative mood reverses devaluation of goal-directed drug-seeking favouring an incentive learning account of drug dependence

BACKGROUND

Two theories explain how negative mood primes smoking behaviour. The stimulus-response (S-R) account argues that in the negative mood state, smoking is experienced as more reinforcing, establishing a direct (automatic) association between the negative mood state and smoking behaviour. By contrast, the incentive learning account argues that in the negative mood state smoking is expected to be more reinforcing, which integrates with instrumental knowledge of the response required to produce that outcome.

OBJECTIVES

One differential prediction is that whereas the incentive learning account anticipates that negative mood induction could augment a novel tobacco-seeking response in an extinction test, the S-R account could not explain this effect because the extinction test prevents S-R learning by omitting experience of the reinforcer.

METHODS

To test this, overnight-deprived daily smokers (n = 44) acquired two instrumental responses for tobacco and chocolate points, respectively, before smoking to satiety. Half then received negative mood induction to raise the expected value of tobacco, opposing satiety, whilst the remainder received positive mood induction. Finally, a choice between tobacco and chocolate was measured in extinction to test whether negative mood could augment tobacco choice, opposing satiety, in the absence of direct experience of tobacco reinforcement.

RESULTS

Negative mood induction not only abolished the devaluation of tobacco choice, but participants with a significant increase in negative mood increased their tobacco choice in extinction, despite satiety.

CONCLUSIONS

These findings suggest that negative mood augments drug-seeking by raising the expected value of the drug through incentive learning, rather than through automatic S-R control.

Model-based and model-free decisions in alcohol dependence

BACKGROUND

Human and animal work suggests a shift from goal-directed to habitual decision-making in addiction. However, the evidence for this in human alcohol dependence is as yet inconclusive.

METHODS

Twenty-six healthy controls and 26 recently detoxified alcohol-dependent patients underwent behavioral testing with a 2-step task designed to disentangle goal-directed and habitual response patterns.

RESULTS

Alcohol-dependent patients showed less evidence of goal-directed choices than healthy controls, particularly after losses. There was no difference in the strength of the habitual component. The group differences did not survive controlling for performance on the Digit Symbol Substitution Task.

CONCLUSION

Chronic alcohol use appears to selectively impair goal-directed function, rather than promoting habitual responding. It appears to do so particularly after nonrewards, and this may be mediated by the effects of alcohol on more general cognitive functions subserved by the prefrontal cortex.

A hierarchical instrumental decision theory of nicotine dependence

It is important to characterize the learning processes governing tobacco-seeking in order to understand how best to treat this behavior. Most drug learning theories have adopted a Pavlovian framework wherein the conditioned response is the main motivational process. We favor instead a hierarchical instrumental decision account, wherein expectations about the instrumental contingency between voluntary tobacco-seeking and the receipt of nicotine reward determines the probability of executing this behavior. To support this view, we review titration and nicotine discrimination research showing that internal signals for deprivation/satiation modulate expectations about the current incentive value of smoking, thereby modulating the propensity of this behavior. We also review research on cue-reactivity which has shown that external smoking cues modulate expectations about the probability of the tobacco-seeking response being effective, thereby modulating the propensity of this behavior. Economic decision theory is then considered to elucidate how expectations about the value and probability of response-nicotine contingency are integrated to form an overall utility estimate for that option for comparison with qualitatively different, nonsubstitute reinforcers, to determine response selection. As an applied test for this hierarchical instrumental decision framework, we consider how well it accounts for individual liability to smoking uptake and perseveration, pharmacotherapy, cue-extinction therapies, and plain packaging. We conclude that the hierarchical instrumental account is successful in reconciling this broad range of phenomenon precisely because it accepts that multiple diverse sources of internal and external information must be integrated to shape the decision to smoke.

Translational Assessment of Reward and Motivational Deficits in Psychiatric Disorders

Deficits in reward and motivation are common symptoms characterizing several psychiatric and neurological disorders. Such deficits may include anhedonia, defined as loss of pleasure, as well as impairments in anticipatory pleasure, reward valuation, motivation/effort, and reward learning. This chapter describes recent advances in the development of behavioral tasks used to assess different aspects of reward processing in both humans and non-human animals. While earlier tasks were generally developed independently with limited cross-species correspondence, a newer generation of translational tasks has emerged that are theoretically and procedurally analogous across species and allow parallel testing, data analyses, and interpretation between human and rodent behaviors. Such enhanced conformity between cross-species tasks will facilitate investigation of the neurobiological mechanisms underlying discrete reward and motivated behaviors and is expected to improve our understanding and treatment of neuropsychiatric disorders characterized by reward and motivation deficits.

Computer-aided diagnosis of alcoholism-related EEG signals

Alcoholism is a severe disorder that affects the functionality of neurons in the central nervous system (CNS) and alters the behavior of the affected person. Electroencephalogram (EEG) signals can be used as a diagnostic tool in the evaluation of subjects with alcoholism. The neurophysiological interpretation of EEG signals in persons with alcoholism (PWA) is based on observation and interpretation of the frequency and power in their EEGs compared to EEG signals from persons without alcoholism. This paper presents a review of the known features of EEGs obtained from PWA and proposes that the impact of alcoholism on the brain can be determined by computer-aided analysis of EEGs through extracting the minute variations in the EEG signals that can differentiate the EEGs of PWA from those of nonaffected persons. The authors advance the idea of automated computer-aided diagnosis (CAD) of alcoholism by employing the EEG signals. This is achieved through judicious combination of signal processing techniques such as wavelet, nonlinear dynamics, and chaos theory and pattern recognition and classification techniques. A CAD system is cost-effective and efficient and can be used as a decision support system by physicians in the diagnosis and treatment of alcoholism especially those who do not specialize in alcoholism or neurophysiology. It can also be of great value to rehabilitation centers to assess PWA over time and to monitor the impact of treatment aimed at minimizing or reversing the effects of the disease on the brain. A CAD system can be used to determine the extent of alcoholism-related changes in EEG signals (low, medium, high) and the effectiveness of therapeutic plans.

Ethanol exposure interacts with training conditions to influence behavioral adaptation to a negative instrumental contingency

We previously reported that, in male, Long Evans rats, instrumental lever pressing that had been reinforced during limited training under a variable interval (VI) schedule by oral self-administration of a 10% sucrose/10% ethanol (10S10E) solution was insensitive to devaluation of 10S10E. In contrast, lever pressing that had been reinforced under a variable ratio (VR) schedule, or by self-administration of 10% sucrose (10S) alone, was sensitive to outcome devaluation. The relative insensitivity to outcome devaluation indicated that seeking of 10S10E by the VI-trained rats had become an instrumental habit. In the present study we employed an alternative operational definition of an instrumental habit and compared the effect of reversing the action-outcome contingency on lever press performance by rats trained under the same experimental conditions. Male Long Evans rats received daily operant training, in which lever presses were reinforced by 10S10E or 10S, under VI or VR schedules. After nine sessions of VI or VR training, rats were tested over four sessions in which the instrumental contingency was changed so that a lever press would prevent reinforcer delivery for 120 s. We found that rats that had been trained to lever press for 10S10E under the VR schedule showed a greater change in lever pressing across testing sessions than those that had received 10S10E reinforcement under the VI schedule. There was no such interaction with reinforcement schedule for rats that had received only 10S reinforcement during training. These findings are consistent with those of our previous study, and provide further evidence that addition of ethanol to sucrose may promote habitual responding in an instrumental task.

Electrophysiological responses to alcohol cues are not associated with Pavlovian-to-instrumental transfer in social drinkers

Pavlovian to Instrumental Transfer (PIT) refers to the behavioral phenomenon of increased instrumental responding for a reinforcer when in the presence of Pavlovian conditioned stimuli that were separately paired with that reinforcer. PIT effects may play an important role in substance use disorders, but little is known about the brain mechanisms that underlie these effects in alcohol consumers. We report behavioral and electroencephalographic (EEG) data from a group of social drinkers (n = 31) who performed a PIT task in which they chose between two instrumental responses in pursuit of beer and chocolate reinforcers while their EEG reactivity to beer, chocolate and neutral pictorial cues was recorded. We examined two markers of the motivational salience of the pictures: the P300 and slow wave event-related potentials (ERPs). Results demonstrated a behavioral PIT effect: responding for beer was increased when a beer picture was presented. Analyses of ERP amplitudes demonstrated significantly larger slow potentials evoked by beer cues at various electrode clusters. Contrary to hypotheses, there were no significant correlations between behavioral PIT effects, electrophysiological reactivity to the cues, and individual differences in drinking behaviour. Our findings are the first to demonstrate a PIT effect for beer, accompanied by increased slow potentials in response to beer cues, in social drinkers. The lack of relationship between behavioral and EEG measures, and between these measures and individual differences in drinking behaviour may be attributed to methodological features of the PIT task and to characteristics of our sample.

Measuring reinforcement learning and motivation constructs in experimental animals: relevance to the negative symptoms of schizophrenia

The present review article summarizes and expands upon the discussions that were initiated during a meeting of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS; http://cntrics.ucdavis.edu) meeting. A major goal of the CNTRICS meeting was to identify experimental procedures and measures that can be used in laboratory animals to assess psychological constructs that are related to the psychopathology of schizophrenia. The issues discussed in this review reflect the deliberations of the Motivation Working Group of the CNTRICS meeting, which included most of the authors of this article as well as additional participants. After receiving task nominations from the general research community, this working group was asked to identify experimental procedures in laboratory animals that can assess aspects of reinforcement learning and motivation that may be relevant for research on the negative symptoms of schizophrenia, as well as other disorders characterized by deficits in reinforcement learning and motivation. The tasks described here that assess reinforcement learning are the Autoshaping Task, Probabilistic Reward Learning Tasks, and the Response Bias Probabilistic Reward Task. The tasks described here that assess motivation are Outcome Devaluation and Contingency Degradation Tasks and Effort-Based Tasks. In addition to describing such methods and procedures, the present article provides a working vocabulary for research and theory in this field, as well as an industry perspective about how such tasks may be used in drug discovery. It is hoped that this review can aid investigators who are conducting research in this complex area, promote translational studies by highlighting shared research goals and fostering a common vocabulary across basic and clinical fields, and facilitate the development of medications for the treatment of symptoms mediated by reinforcement learning and motivational deficits.

The contributions of value-based decision-making and attentional bias to alcohol-seeking following devaluation

AIMS

To investigate the mediating role of attentional bias for alcohol cues on alcohol-seeking following devaluation of alcohol.

DESIGN

Between subject.

SETTING

Eye-tracking laboratory at the University of Liverpool.

PARTICIPANTS

Student social drinkers (n = 64).

MEASUREMENTS

An operant choice task in which participants chose between simultaneously presented alcohol and non-alcohol drink rewards, while attentional bias for alcohol and non-alcohol drink cues was inferred from eye movements. Participants then consumed 30 mL of an alcoholic beverage, which was either presented alone (no devaluation: n = 32) or had been adulterated to taste unpleasant (devaluation: n = 32). Choice and attentional bias for the alcohol and non-alcohol drink pictures were then measured again.

FINDINGS

Alcohol devaluation reduced behavioural choice for alcohol (F = 32.64, P < 0.001) and attentional bias for the alcohol pictures indexed by dwell time (F = 22.68, P < 0.001), initial fixation (F = 7.08, P = 0.01) and final fixation (F = 22.44, P < 0.001). Mediation analysis revealed that attentional bias partially mediated the effect of devaluation on alcohol choice; however, the proportion of the variance accounted for by attentional bias is low to moderate (~30%).

CONCLUSIONS

Among student social drinkers, attentional bias is only a partial mediator of alcohol choice following devaluation of alcohol. Value-based decision-making may be a more important determinant of drinking behaviour among student social drinkers than attentional bias.

Phasic transition from goal-directed to habitual control over drug-seeking produced by conflicting reinforcer expectancy

The transition from goal-directed to habitual control over drug-seeking has been experimentally demonstrated in animals, but there have been no comparable reports in humans. Following a recent animal design, the current study employed an outcome-devaluation procedure to test whether goal-directed control over tobacco seeking would be abolished by alcohol expectancy. Eighty smokers first learned that two responses earned tobacco or chocolate points, respectively, before tobacco was devalued by health warnings and smoking satiety. Participants were then presented with either a glass of beer/wine or water with instructions that this item could be consumed after the task (alternative reward). Then choice between the tobacco and chocolate response was measured in extinction to assess goal-directed control of tobacco seeking, in a nominal Pavlovian to instrumental transfer (PIT) test to assess stimulus control of tobacco seeking, and in a reacquisition test to assess the impact of direct feedback from the outcomes. The results showed that alcohol expectancy selectively abolished goal-directed control of tobacco seeking but not stimulus control or the impact of feedback from outcomes. These data suggest that 'endogenous' retrieval of low drug value governing goal-directed regulation of drug seeking is disrupted by conflicting appraisal of an alternative reinforcer, promoting habitual control, which may play a role in relapse.

Effects of alcohol on disinhibition towards alcohol-related cues

BACKGROUND

We investigated (1) the effects of acute alcohol on inhibition of alcohol-related versus neutral cues, (2) the effects of drinking status on inhibition of alcohol-related versus neutral cues, and (3) the similarity of any effects of alcohol or drinking status across two different cue types (lexical versus pictorial).

METHODS

Participants received 0.0 g/kg, 0.4 g/kg or 0.6g/kg of alcohol in a between-subjects design. Healthy, heavy and light social alcohol users (n=96) completed both lexical and pictorial cue versions of an alcohol-shifting task. Participants were instructed to respond to target stimuli by pressing the spacebar, but to ignore distracter stimuli. Errors towards distracter stimuli were analysed using a series of mixed-model ANOVAs, with between-subjects factors of challenge and drinking status and within-subjects factors of distracter type (alcohol, neutral) and block (shift, non-shift).

RESULTS

Lexical commission error data indicated a main effect of distracter (F [1,90]=43.25, p<0.001, η(2)=0.33), which was qualified by a marginal interaction with challenge condition (F [2,90]=2.77, p=0.068, η(2)=0.06). Following an acute high dose of alcohol participants made more errors towards alcohol distracters. Pictorial commission error data indicated a significant main effect of distracter (F [1,90]=67.40, p<0.001, η(2)=0.43), such that all participants made more errors towards neutral image distracters versus alcohol distracter images.

CONCLUSIONS

Our results reveal acute alcohol's impairment of inhibitory control may be enhanced when a response towards alcohol-related lexical stimuli is required to be withheld.

BIS impulsivity and acute nicotine exposure are associated with discounting global consequences in the Harvard game

OBJECTIVE

The Barratt Impulsivity Scale (BIS) provides a transdiagnostic marker for a number of psychiatric conditions and drug abuse, but the precise psychological trait(s) tapped by this questionnaire remain obscure.

METHOD

To address this, 51 smokers completed in counterbalanced order the BIS, a delay discounting task and a Harvard game that measured choice between a response that yielded a high immediate monetary payoff but decreased opportunity to earn money overall (local choice) versus a response that yielded a lower immediate payoff but afforded a greater opportunity to earn overall (global choice).

RESULTS

Individual level of BIS impulsivity and self-elected smoking prior to the study were independently associated with increased preference for the local over the global choice in the Harvard game, but not delay discounting.

CONCLUSIONS

BIS impulsivity and acute nicotine exposure reflect a bias in the governance of choice by immediate reward contingencies over global consequences, consistent with contemporary dual-process instrumental learning theories.

Associative learning mechanisms underpinning the transition from recreational drug use to addiction

Learning theory proposes that drug seeking is a synthesis of multiple controllers. Whereas goal-directed drug seeking is determined by the anticipated incentive value of the drug, habitual drug seeking is elicited by stimuli that have formed a direct association with the response. Moreover, drug-paired stimuli can transfer control over separately trained drug seeking responses by retrieving an expectation of the drug's identity (specific transfer) or incentive value (general transfer). This review covers outcome devaluation and transfer of stimulus-control procedures in humans and animals, which isolate the differential governance of drug seeking by these four controllers following various degrees of contingent and noncontingent drug exposure. The neural mechanisms underpinning these four controllers are also reviewed. These studies suggest that although initial drug seeking is goal-directed, chronic drug exposure confers a progressive loss of control over action selection by specific outcome representations (impaired outcome devaluation and specific transfer), and a concomitant increase in control over action selection by antecedent stimuli (enhanced habit and general transfer). The prefrontal cortex and mediodorsal thalamus may play a role in this drug-induced transition to behavioral autonomy.