Neurobiological Basis of Individual Variation in Stimulus-Reward Learning

Cortico-striatal action control inherent of opponent cognitive-motivational styles

Turning on cue or stopping at a red light requires the detection of such cues to select action sequences, or suppress action, in accordance with cue-associated action rules. Cortico-striatal projections are an essential part of the brain's attention-motor interface. Glutamate-sensing microelectrode arrays were used to measure glutamate transients in the dorsomedial striatum (DMS) of male and female rats walking a treadmill and executing cued turns and stops. Prelimbic-DMS projections were chemogenetically inhibited to determine their behavioral necessity and the cortico-striatal origin of cue-evoked glutamate transients. Furthermore, we investigated rats exhibiting preferably goal-directed (goal trackers, GTs) versus cue-driven attention (sign trackers, STs), to determine the impact of such cognitive-motivational biases on cortico-striatal control. GTs executed more cued turns and initiated such turns more slowly than STs. During turns, but not missed turns or cued stops, cue-evoked glutamate concentrations were higher in GTs than in STs. In conjunction with turn cue-evoked glutamate spike levels, the presence of a single spike rendered GTs to be almost twice as likely to turn than STs. In contrast, multiple glutamate spikes predicted GTs to be less likely to turn than STs. In GTs, but not STs, inhibition of prelimbic-DMS projections attenuated turn rates, turn cue-evoked glutamate peaks, and increased the number of spikes. These findings suggest that turn cue-evoked glutamate release in GTs is tightly controlled by cortico-striatal neuronal activity. In contrast, in STs, glutamate release from DMS glutamatergic terminals may be regulated by other striatal circuitry, preferably mediating cued suppression of action and reward tracking.

Significance Statement

Adaptive behavior involves the selection of behaviorally significant cues and the capacity of selected cues to control behavioral action. Neuronal projections from cortex to striatum are essential for such an integration of attentional with motor functions. Here we demonstrated that glutamate release from cortico-striatal projections primarily influences cued turns but not cued suppression of actions (cued stops). Cortico-striatal control of cued turning was especially powerful in rats which, as a psychological trait, preferably deploy goal-directed attention. Together, our findings demonstrate the role of cortico-striatal input in cued action selection, and they emphasize the experimental and biopsychological significance of investigating the brain's attentional-motor interface in the context of broader cognitive-motivational styles.

Variants of the P3 event-related potential operate as indicators of distinct mechanisms contributing to problematic alcohol use

Considerable research has linked relative reduction in the amplitude of the P3 event-related potential (ERP) during cognitive task performance (i.e., Target-P3) with increased risk of alcohol-related problems. A separate literature indicates that a relative increase in the amplitude of the P3 elicited by cues signaling alcohol availability (i.e., ACR-P3) also is associated with alcohol use and problems. To date, no research has integrated these seemingly discrepant findings. Here, we aimed to demonstrate that P3 amplitudes elicited in different task contexts reflect distinct domains of functioning relevant to problematic alcohol involvement (PAI), and therefore can inform heterogeneity in the etiology of PAI. 156 emerging adults (61% women; 88% White/Non-Hispanic) completed a mental rotation task and a picture-viewing task while ERPs were recorded. Participants also completed questionnaire measures of trait disinhibition, alcohol use, and alcohol-related problems. Findings from regression analyses indicated that (a) Target-P3 was negatively associated and ACR-P3 was positively associated with a PAI latent variable; (b) the two P3s accounted for unique variance in PAI, beyond that accounted for by recent drinking; and (c) the association between Target-P3 and PAI-but not ACR-P3 and PAI-was statistically mediated by trait disinhibition. The present findings highlight the unique contributions of distinct functional domains associated with disinhibition and incentive salience in the etiology of PAI. Moreover, findings are consistent with a nuanced understanding of the P3 ERP, whereby its specific meaning varies according to the task context in which it is elicited.

Genomic Loci Influencing Cue-Reactivity in Heterogeneous Stock Rats

Addiction vulnerability is associated with the tendency to attribute incentive salience to reward predictive cues; both addiction and the attribution of incentive salience are influenced by environmental and genetic factors. To characterize the genetic contributions to incentive salience attribution, we performed a genome-wide association study (GWAS) in a cohort of 1,645 genetically diverse heterogeneous stock (HS) rats. We tested HS rats in a Pavlovian conditioned approach task, in which we characterized the individual responses to food-associated stimuli ("cues"). Rats exhibited either cue-directed "sign-tracking" behavior or food-cup directed "goal-tracking" behavior. We then used the conditioned reinforcement procedure to determine whether rats would perform a novel operant response for unrewarded presentations of the cue. We found that these measures were moderately heritable (SNP heritability, h2 = .189-.215). GWAS identified 14 quantitative trait loci (QTLs) for 11 of the 12 traits we examined. Interval sizes of these QTLs varied widely. 7 traits shared a QTL on chromosome 1 that contained a few genes (e.g. Tenm4, Mir708) that have been associated with substance use disorders and other mental health traits in humans. Other candidate genes (e.g. Wnt11, Pak1) in this region had coding variants and expression-QTLs in mesocorticolimbic regions of the brain. We also conducted a Phenome-Wide Association Study (PheWAS) on other behavioral measures in HS rats and found that regions containing QTLs on chromosome 1 were also associated with nicotine self-administration in a separate cohort of HS rats. These results provide a starting point for the molecular genetic dissection of incentive salience and provide further support for a relationship between attribution of incentive salience and drug abuse-related traits.

A translational rodent model of individual differences in sensitivity to the aversive properties of ethanol

BACKGROUND

A strong relationship exists between individual sensitivity to the aversive properties of ethanol and risk for alcohol use disorder (AUD). Despite this, our understanding of the neurobiological mechanisms underlying the subjective response to ethanol is limited. A major contributor to this lack of knowledge is the absence of preclinical models that enable exploration of this individual variability such as is possible in studies of humans.

METHODS

Adult male and female Long-Evans rats were trained to associate a novel tastant (saccharin) with acute exposure to either saline or ethanol (1.5 g/kg or 2.0 g/kg i.p.) over three conditioning days using a standard conditioned taste aversion (CTA) procedure. Variability in sensitivity to ethanol-induced CTA was phenotypically characterized using a median split across the populations studied.

RESULTS

When examining group averages, both male and female rats exposed to saccharin paired with either dose of ethanol exhibited lower saccharin intake relative to saline controls indicative of ethanol-induced CTA. Examination of individual data revealed a bimodal distribution of responses uncovering two distinct phenotypes present in both sexes. CTA-sensitive rats exhibited a rapid and progressive reduction in saccharin intake with each successive ethanol pairing. In contrast, saccharin intake was unchanged or maintained after an initial decrease from baseline levels in CTA-resistant rats. While CTA magnitude was similar between male and female CTA-sensitive rats, among CTA-resistant animals females were more resistant to the development of ethanol-induced CTA than males. Phenotypic differences were not driven by differences in baseline saccharin intake.

CONCLUSIONS

These data parallel work in humans by revealing individual differences in sensitivity to the aversive properties of ethanol that emerge immediately after initial exposure to ethanol in both sexes. This model can be used in future studies to investigate the neurobiological mechanisms that confer risk for AUD.

Role of nucleus accumbens D1-type medium spiny neurons in the expression and extinction of sign-tracking

While sign-tracking, also known as autoshaping, has been studied for many decades, only recently has the tendency to show sign-tracking behavior been linked to the development and persistence of addiction. Sign-tracking is dependent upon dopamine activity in the nucleus accumbens (NAc). The NAc is comprised predominantly of medium spiny projection neurons (MSN) that can be differentiated by their D1-like or D2-like dopamine receptor expression. Here we determined how reducing activity of D1-type MSNs in the NAc affects the expression and extinction of sign-tracking. To address this, we transfected the NAc of transgenic male and female rats that selectively express Cre recombinase in D1-type MSNs with a DIO viral vector expressing hM4Di. Cre- rats were given the same viral infusion but did not express the hM4Di receptor and therefore served as controls. Rats were then conditioned to associate lever presentations with pellet delivery. After sign-tracking was established, all rats were administered clozapine-n-oxide (CNO) prior to three additional conditioning sessions to assess the effects of NAc D1-MSNs inhibition on sign-tracking in the presence of reward. CNO treatment did not alter the expression of sign-tracking in Cre+ or Cre- rats. Next rats underwent extinction training where lever presentations occurred without pellet delivery and all rats received a CNO injection prior to each extinction session. In these extinction conditions, Cre+ rats exhibited robust extinction of sign-tracking across sessions, whereas Cre- rats did not. To determine if D1-MSN inhibition merely produced a temporary cessation of sign-tracking or instead had facilitated a persistent loss of sign-tracking, we evaluated the reemergence of sign-tracking in a test for reconditioning. During testing, reintroduction of the CS-US pairing did not promote the reemergence of sign-tracking in Cre+ rats, but restored sign-tracking in Cre- rats. Thus, chemogenetic inhibition of NAc D1-MSNs promoted extinction of sign-tracking. Collectively, these data suggest that D1-MSNs play an important role in resistance to extinction that typifies sign-tracking behavior.

Sensory Cues Potentiate VTA Dopamine Mediated Reinforcement

Sensory cues are critical for shaping decisions and invigorating actions during reward seeking. Dopamine neurons in the ventral tegmental area (VTA) are central in this process, supporting associative learning in Pavlovian and instrumental settings. Studies of intracranial self-stimulation (ICSS) behavior, which show that animals will work hard to receive stimulation of dopamine neurons, support the notion that dopamine transmits a reward or value signal to support learning. Recent studies have begun to question this, however, emphasizing dopamine's value-free functions, leaving its contribution to behavioral reinforcement somewhat muddled. Here, we investigated the role of sensory stimuli in dopamine-mediated reinforcement, using an optogenetic ICSS paradigm in tyrosine hydroxylase (TH)-Cre rats. We find that while VTA dopamine neuron activation in the absence of explicit external cues is sufficient to maintain robust self-stimulation, the presence of cues dramatically potentiates ICSS behavior. Our results support a framework where dopamine can have some base value as a reinforcer, but the impact of this signal is modulated heavily by the sensory learning context.

On the reliability of value-modulated attentional capture: An online replication and multiverse analysis

Stimuli predicting rewards are more likely to capture attention, even when they are not relevant to our current goals. Individual differences in value-modulated attentional capture (VMAC) have been associated with various psychopathological conditions in the scientific literature. However, the claim that this attentional bias can predict individual differences requires further exploration of the psychometric properties of the most common experimental paradigms. The current study replicated the VMAC effect in a large online sample (N = 182) and investigated the internal consistency, with a design that allowed us to measure the effect during learning (rewarded phase) and after acquisition, once feedback was omitted (unrewarded phase). Through the rewarded phase there was gradual increase of the VMAC effect, which did not decline significantly throughout the unrewarded phase. Furthermore, we conducted a reliability multiverse analysis for 288 different data preprocessing specifications across both phases. Specifications including more blocks in the analysis led to better reliability estimates in both phases, while specifications that removed more outliers also improved reliability, suggesting that specifications with more, but less noisy, trials led to better reliability estimates. Nevertheless, in most instances, especially those considering fewer blocks of trials, reliability estimates fell below the minimum recommended thresholds for research on individual differences. Given the present results, we encourage researchers working on VMAC to take into account reliability when designing studies aimed at capturing individual differences and provide recommendations to improve methodological practices.

Sensory cues potentiate VTA dopamine mediated reinforcement

Sensory cues are critical for shaping decisions and invigorating actions during reward seeking. Dopamine neurons in the ventral tegmental area (VTA) are critical in this process, supporting associative learning in Pavlovian and instrumental settings. Studies of intracranial self stimulation (ICSS) behavior, which show that animals will work hard to receive stimulation of dopamine neurons, support the notion that dopamine transmits a reward or value signal to support learning. Recent studies have begun to question this, however, emphasizing dopamine's value-free functions, leaving its contribution to behavioral reinforcement somewhat muddled. Here, we investigated the role of sensory stimuli in dopamine-mediated reinforcement, using an optogenetic ICSS paradigm in tyrosine hydroxylase (TH)-cre rats. We find that while VTA dopamine neuron activation in the absence of any external cueing stimulus is sufficient to maintain robust self stimulation, the presence of cues dramatically potentiates ICSS behavior. Our results support a framework where dopamine can have some base value as a reinforcer, but the impact of this signal is modulated heavily by the sensory learning context.

Separating desire from prediction of outcome value

Individuals typically want what they expect to like, often based on memories of previous positive experiences. However, in some situations desire can decouple completely from memories and from learned predictions of outcome value. The potential for desire to separate from prediction arises from independent operating rules that control motivational incentive salience. Incentive salience, or 'wanting', is a type of mesolimbic desire that evolved for adaptive goals, but can also generate maladaptive addictions. Two proof-of-principle examples are presented here to show how motivational 'wanting' can soar above memory-based predictions of outcome value: (i) 'wanting what is remembered to be disgusting', and (ii) 'wanting what is predicted to hurt'. Consequently, even outcomes remembered and predicted to be negatively aversive can become positively 'wanted'. Similarly, in human addictions, people may experience powerful cue-triggered cravings for outcomes that are not predicted to be enjoyable.

Nucleus accumbens core acetylcholine receptors modulate the balance of flexible and inflexible cue-directed motivation

Sign-tracking is a conditioned response where animals interact with reward-predictive cues due to the cues having motivational value, or incentive salience. The nucleus accumbens core (NAc) has been implicated in mediating the sign-tracking response. Additionally, acetylcholine (ACh) transmission throughout the striatum has been attributed to both incentive motivation and behavioral flexibility. Here, we demonstrate a role for NAc ACh receptors in the flexibility of sign-tracking. Sign-tracking animals were exposed to an omission contingency, in which vigorous sign-tracking was punished by reward omission. Animals rapidly adjusted their behavior, but they maintained sign-tracking in a less vigorous manner that did not cancel reward. Within this context of sign-tracking being persistent yet flexible in structure, blockade of NAc nicotinic receptors (nAChRs) led to a persistence in the initial sign-tracking response during omission followed by a period of change in the makeup of sign-tracking, whereas blockade of muscarinic receptors (mAChRs) oppositely enhanced the omission-related development of the new sign-tracking behaviors. Later, once omission learning had occurred, nAChR blockade uniquely led to reduced sign-tracking and elevated reward-directed behaviors instead. These results indicate that NAc ACh receptors have opposing roles in maintaining learned patterns of sign-tracking, with nAChRs having a special involvement in regulating the structure of the sign-tracking response.

A translational rodent model of individual differences in sensitivity to the aversive properties of ethanol

Background

A strong relationship exists between individual sensitivity to the aversive properties of ethanol and risk for alcohol use disorder (AUD). Despite this, our understanding of the neurobiological mechanisms underlying subjective response to ethanol is relatively poor. A major contributor to this is the absence of preclinical models that enable exploration of this individual variability similar to studies performed in humans.

Methods

Adult male and female Long-Evans rats were trained to associate a novel tastant (saccharin) with acute exposure to either saline or ethanol (1.5 g/kg or 2.0 g/kg i.p.) over three conditioning days using a standard conditioned taste aversion (CTA) procedure. Variability in sensitivity to ethanol-induced CTA was phenotypically characterized using a median split across the populations studied.

Results

When examining group averages, both male and female rats that had saccharin paired with either dose of ethanol exhibited reduced saccharin intake relative to saline controls of ethanol-induced CTA. Examination of individual data revealed a bimodal distribution of responses uncovering two distinct phenotypes present in both sexes. CTA-sensitive rats exhibited a rapid and progressive reduction in saccharin intake with each successive ethanol pairing. In contrast, saccharin intake was unchanged or maintained after an initial decrease from baseline levels in CTA-resistant rats. While CTA magnitude was similar between male and female CTA-sensitive rats, CTA-resistant females were more resistant to the development of ethanol-induced CTA than their male counterparts. Phenotypic differences were not driven by differences in baseline saccharin intake. CTA sensitivity correlated with behavioral signs of intoxication in only a subset of rats.

Conclusions

These data parallel work in humans by revealing individual differences in sensitivity to the aversive properties of ethanol that emerge immediately after initial exposure to ethanol in both sexes. This model can be leveraged in future studies to investigate the neurobiological mechanisms that confer risk for AUD.

The propensity to sign-track is associated with externalizing behavior and distinct patterns of reward-related brain activation in youth

Externalizing behaviors in childhood often predict impulse control disorders in adulthood; however, the underlying bio-behavioral risk factors are incompletely understood. In animals, the propensity to sign-track, or the degree to which incentive motivational value is attributed to reward cues, is associated with externalizing-type behaviors and deficits in executive control. Using a Pavlovian conditioned approach paradigm, we quantified sign-tracking in 40 healthy 9-12-year-olds. We also measured parent-reported externalizing behaviors and anticipatory neural activations to outcome-predicting cues using the monetary incentive delay fMRI task. Sign-tracking was associated with attentional and inhibitory control deficits and the degree of amygdala, but not cortical, activation during reward anticipation. These findings support the hypothesis that youth with a propensity to sign-track are prone to externalizing tendencies, with an over-reliance on subcortical cue-reactive brain systems. This research highlights sign-tracking as a promising experimental approach delineating the behavioral and neural circuitry of individuals at risk for externalizing disorders.

Ventral pallidal regulation of motivated behaviors and reinforcement

The interconnected nuclei of the ventral basal ganglia have long been identified as key regulators of motivated behavior, and dysfunction of this circuit is strongly implicated in mood and substance use disorders. The ventral pallidum (VP) is a central node of the ventral basal ganglia, and recent studies have revealed complex VP cellular heterogeneity and cell- and circuit-specific regulation of reward, aversion, motivation, and drug-seeking behaviors. Although the VP is canonically considered a relay and output structure for this circuit, emerging data indicate that the VP is a central hub in an extensive network for reward processing and the regulation of motivation that extends beyond classically defined basal ganglia borders. VP neurons respond temporally faster and show more advanced reward coding and prediction error processing than neurons in the upstream nucleus accumbens, and regulate the activity of the ventral mesencephalon dopamine system. This review will summarize recent findings in the literature and provide an update on the complex cellular heterogeneity and cell- and circuit-specific regulation of motivated behaviors and reinforcement by the VP with a specific focus on mood and substance use disorders. In addition, we will discuss mechanisms by which stress and drug exposure alter the functioning of the VP and produce susceptibility to neuropsychiatric disorders. Lastly, we will outline unanswered questions and identify future directions for studies necessary to further clarify the central role of VP neurons in the regulation of motivated behaviors. Significance: Research in the last decade has revealed a complex cell- and circuit-specific role for the VP in reward processing and the regulation of motivated behaviors. Novel insights obtained using cell- and circuit-specific interrogation strategies have led to a major shift in our understanding of this region. Here, we provide a comprehensive review of the VP in which we integrate novel findings with the existing literature and highlight the emerging role of the VP as a linchpin of the neural systems that regulate motivation, reward, and aversion. In addition, we discuss the dysfunction of the VP in animal models of neuropsychiatric disorders.

Sleep-mediated regulation of reward circuits: implications in substance use disorders

Our modern society suffers from both pervasive sleep loss and substance abuse-what may be the indications for sleep on substance use disorders (SUDs), and could sleep contribute to the individual variations in SUDs? Decades of research in sleep as well as in motivated behaviors have laid the foundation for us to begin to answer these questions. This review is intended to critically summarize the circuit, cellular, and molecular mechanisms by which sleep influences reward function, and to reveal critical challenges for future studies. The review also suggests that improving sleep quality may serve as complementary therapeutics for treating SUDs, and that formulating sleep metrics may be useful for predicting individual susceptibility to SUDs and other reward-associated psychiatric diseases.

The role of reinforcement learning and value-based decision-making frameworks in understanding food choice and eating behaviors

The obesogenic food environment includes easy access to highly-palatable, energy-dense, "ultra-processed" foods that are heavily marketed to consumers; therefore, it is critical to understand the neurocognitive processes the underlie overeating in response to environmental food-cues (e.g., food images, food branding/advertisements). Eating habits are learned through reinforcement, which is the process through which environmental food cues become valued and influence behavior. This process is supported by multiple behavioral control systems (e.g., Pavlovian, Habitual, Goal-Directed). Therefore, using neurocognitive frameworks for reinforcement learning and value-based decision-making can improve our understanding of food-choice and eating behaviors. Specifically, the role of reinforcement learning in eating behaviors was considered using the frameworks of (1) Sign-versus Goal-Tracking Phenotypes; (2) Model-Free versus Model-Based; and (3) the Utility or Value-Based Model. The sign-and goal-tracking phenotypes may contribute a mechanistic insight on the role of food-cue incentive salience in two prevailing models of overconsumption-the Extended Behavioral Susceptibility Theory and the Reactivity to Embedded Food Cues in Advertising Model. Similarly, the model-free versus model-based framework may contribute insight to the Extended Behavioral Susceptibility Theory and the Healthy Food Promotion Model. Finally, the value-based model provides a framework for understanding how all three learning systems are integrated to influence food choice. Together, these frameworks can provide mechanistic insight to existing models of food choice and overconsumption and may contribute to the development of future prevention and treatment efforts.

An automated feeding system for the African killifish reveals the impact of diet on lifespan and allows scalable assessment of associative learning

The African turquoise killifish is an exciting new vertebrate model for aging studies. A significant challenge for any model organism is the control over its diet in space and time. To address this challenge, we created an automated and networked fish feeding system. Our automated feeder is designed to be open-source, easily transferable, and built from widely available components. Compared to manual feeding, our automated system is highly precise and flexible. As a proof of concept for the feeding flexibility of these automated feeders, we define a favorable regimen for growth and fertility for the African killifish and a dietary restriction regimen where both feeding time and quantity are reduced. We show that this dietary restriction regimen extends lifespan in males (but not in females) and impacts the transcriptomes of killifish livers in a sex-specific manner. Moreover, combining our automated feeding system with a video camera, we establish a quantitative associative learning assay to provide an integrative measure of cognitive performance for the killifish. The ability to precisely control food delivery in the killifish opens new areas to assess lifespan and cognitive behavior dynamics and to screen for dietary interventions and drugs in a scalable manner previously impossible with traditional vertebrate model organisms.

Novelty-induced locomotor behavior predicts heroin addiction vulnerability in male, but not female, rats

RATIONALE

The ongoing rise in opioid use disorder (OUD) has made it imperative to better model the individual variation within the human population that contributes to OUD vulnerability. Using animal models that capture such variation can be a useful tool. Individual variation in novelty-induced locomotion is predictive of substance use disorder (SUD) propensity. In this model, rats are characterized as high-responders (HR) or low-responders (LR) using a median split based on distance travelled during a locomotor test, and HR rats are generally found to exhibit a more SUD vulnerable behavioral phenotype.

OBJECTIVES

The HR/LR model has commonly been used to assess behaviors in male rats using psychostimulants, with limited knowledge of the predictive efficacy of this model in females or the use of an opioid as the reward. In the current study, we assessed several behaviors across the different phases of drug addiction (heroin taking, refraining, and seeking) in over 500 male and female heterogeneous stock rats run at two geographically separate locations. Rats were characterized as HRs or LRs within each sex for analysis.

RESULTS

Overall, females exhibit a more OUD vulnerable phenotype relative to males. Additionally, the HR/LR model was predictive of OUD-like behaviors in male, but not female rats. Furthermore, phenotypes did not differ in anxiety-related behaviors, reacquisition of heroin-taking, or punished heroin-taking behavior in either sex.

CONCLUSIONS

These results emphasize the importance of assessing females in models of individual variation in SUD and highlight limitations in using the HR/LR model to assess OUD propensity.

A Videogame as a Tool for Clinical Screening of Possible Vulnerability to Impulsivity and Attention Disturbances in Children

An attention disturbance is a problem that affects many school-aged children. The assessment in children is usually report-based, and as a result, controversy surrounds the diagnosis. To solve this issue, the aim of this study was to develop a new tool to detect possible attention-related problems and impulsive behavior in 4- and 5-year-old children. This tool was developed as an Android app and could be used to provide an early indicator of possible future development problems. A sample of 103 children (48 girls and 55 boys) was randomly selected from primary schools and assessed by Pinky-Piggy, a videogame application based on a classical paradigm in experimental psychology. Data from this app were compared with a Child Neuropsychological Maturity Questionnaire. The subjects displayed different patterns of response to play a very simple game called Pinky-Piggy. The application discriminated between high-responders and low responders. The results showed a relationship between these two profiles and the levels of attention and neurodevelopment in each group. The tool could identify different types of profiles and demonstrated its potential to evaluate endophenotypes to predict attentional problems related to impulsive behavior. Additionally, it required less time and fewer tests to identify possible at-risk populations, thus assisting in clinical diagnosis.

Sign-tracking modulates reward-related neural activation to reward cues, but not reward feedback

Research shows cognitive and neurobiological overlap between sign-tracking [value-modulated attentional capture (VMAC) by response-irrelevant, discrete cues] and maladaptive behaviour (e.g. substance abuse). We investigated the neural correlates of sign-tracking in 20 adults using an additional singleton task (AST) and functional magnetic resonance imaging (fMRI). Participants responded to a target to win monetary reward, the amount of which was signalled by singleton type (reward cue: high value vs. low value). Singleton responses resulted in monetary deductions. Sign-tracking-greater distraction by high-value vs. low-value singletons (H > L)-was observed, with high-value singletons producing slower responses to the target than low-value singletons. Controlling for age and sex, analyses revealed no differential brain activity across H > L singletons. Including sign-tracking as a regressor of interest revealed increased activity (H > L singletons) in cortico-subcortical loops, regions associated with Pavlovian conditioning, reward processing, attention shifts and relative value coding. Further analyses investigated responses to reward feedback (H > L). Controlling for age and sex, increased activity (H > L reward feedback) was found in regions associated with reward anticipation, attentional control, success monitoring and emotion regulation. Including sign-tracking as a regressor of interest revealed increased activity in the temporal pole, a region related to value discrimination. Results suggest sign-tracking is associated with activation of the 'attention and salience network' in response to reward cues but not reward feedback, suggesting parcellation between the two at the level of the brain. Results add to the literature showing considerable overlap in neural systems implicated in reward processing, learning, habit formation, emotion regulation and substance craving.

Investigating individual differences in opioid-taking and opioid-seeking behavior in male rats

RATIONALE

Understanding the behavioral and neurobiological factors that render some individuals more susceptible than others to opioid addiction will be critical in combatting the opioid crisis.

OBJECTIVE

The purpose of the current study was to determine if behavioral traits associated with an increased likelihood to take and seek cocaine are the same traits that render one more susceptible to opioid-taking and opioid-seeking behavior. Individual differences in the acquisition of remifentanil self-administration and subsequent cue-induced reinstatement of remifentanil-seeking behavior were investigated using two animal models: the high-responder (HR)/low-responder (LR) and sign-tracker (ST)/goal-tracker (GT) models. Relative to LR rats, HR rats show increased novelty-induced locomotion or "sensation-seeking" behavior, and are more likely to acquire cocaine-taking behavior and do so at a faster rate. Relative to GT rats, ST rats attribute greater incentive motivational value to reward cues and are more likely to exhibit reinstatement of cocaine-seeking behavior.

RESULTS

In contrast to previous work using cocaine, we did not observe individual differences with respect to the acquisition of remifentanil self-administration- or cue-induced reinstatement of remifentanil-seeking behavior within the context of either the HR/LR or ST/GT model. Thus, neither the sensation-seeking trait nor the propensity to attribute incentive motivational value to reward cues predicts remifentanil-taking or remifentanil-seeking behavior.

CONCLUSIONS

These findings suggest that different traits may confer the initiation of opioid- vs. cocaine-taking behavior, and the propensity to relapse to opioid- vs. cocaine-seeking. Additional studies are needed to identify which neurobehavioral constructs confer liability to opioid use and relapse.

Behavioral response bias and event-related brain potentials implicate elevated incentive salience attribution to alcohol cues in emerging adults with lower sensitivity to alcohol

AIMS

This study used a behavioral approach-avoidance task including images of alcoholic beverages to test whether low sensitivity to alcohol (LS) is a phenotypical marker of a dispositional propensity to attribute bottom-up incentive value to naturally conditioned alcohol cues.

DESIGN, SETTING AND PARTICIPANTS

Experimental study with a measured individual difference variable at a university psychology laboratory in Missouri, MO, USA. Participants were 178 emerging adults (aged 18-20 years) varying in self-reported sensitivity to alcohol's acute effects.

MEASUREMENTS

Participants completed the alcohol approach-avoidance task while behavior (response time; RT) and the electroencephalogram (EEG) were recorded. Stimulus-locked event-related potentials (ERPs) provided indices of integrated (top-down and bottom-up) stimulus incentive value (P3 amplitude) and conflict between top-down task demands and bottom-up response propensities (N450 amplitude).

FINDINGS

Linear mixed models showed faster RT for 'alcohol-approach' relative to 'alcohol-avoid' trials for lower-sensitivity (LS) [mean_D  ± standard error_D (M_D  ± SE_D ) = 29.51 ± 9.74 ms, t₍₃₂₈₎  = 3.03, P = 0.003] but not higher-sensitivity (HS) individuals (M_D  ± SE_D  = 2.27 ± 9.33 ms, t₍₃₂₈₎  = 0.243, P = 0.808). There was enhanced N450 amplitude (response conflict) for alcohol-avoid relative to alcohol-approach trials for LS participants (M_D  ± SE_D  = 0.811 ± 0.198 μV, Z = 4.108, P < 0.001) and enhanced N450 amplitude for alcohol-approach relative to alcohol-avoid for HS participants (M_D  ± SE_D  = 0.419 ± 0.188 μV, Z = 2.235, P = 0.025). There was also enhanced P3 amplitude for alcohol-approach relative to alcohol-avoid for LS (M_D  ± SE_D  = 0.825 ± 0.204 μV, Z = 4.045, P < 0.001) but not HS (M_D  ± SE_D  = 0.013 ± 0.194 μV, Z = 0.068, P = 0.946).

CONCLUSIONS

Findings from a human laboratory study appear to support the notion that low sensitivity to alcohol indexes a propensity to attribute bottom-up incentive value to naturally conditioned alcohol cues.

Inhibition of a cortico-thalamic circuit attenuates cue-induced reinstatement of drug-seeking behavior in "relapse prone" male rats

RATIONALE

Relapse often occurs when individuals are exposed to stimuli or cues previously associated with the drug-taking experience. The ability of drug cues to trigger relapse is believed to be a consequence of incentive salience attribution, a process by which the incentive value of reward is transferred to the reward-paired cue. Sign-tracker (ST) rats that attribute enhanced incentive value to reward cues are more prone to relapse compared to goal-tracker (GT) rats that primarily attribute predictive value to such cues.

OBJECTIVES

The neurobiological mechanisms underlying this individual variation in relapse propensity remains largely unexplored. The paraventricular nucleus of the thalamus (PVT) has been identified as a critical node in the regulation of cue-elicited behaviors in STs and GTs, including cue-induced reinstatement of drug-seeking behavior. Here we used a chemogenetic approach to assess whether "top-down" cortical input from the prelimbic cortex (PrL) to the PVT plays a role in mediating individual differences in relapse propensity.

RESULTS

Chemogenetic inhibition of the PrL-PVT pathway selectively decreased cue-induced reinstatement of drug-seeking behavior in STs, without affecting behavior in GTs. In contrast, cocaine-primed drug-seeking behavior was not affected in either phenotype. Furthermore, when rats were characterized based on a different behavioral phenotype-locomotor response to novelty-inhibition of the PrL-PVT pathway had no effect on either cue- or drug-induced reinstatement.

CONCLUSIONS

These results highlight an important role for the PrL-PVT pathway in vulnerability to relapse that is consequent to individual differences in the propensity to attribute incentive salience to discrete reward cues.

The enduring behavioral and neurobiological effects of a flavor cue paired with alcohol drinking during adolescence on the incentive properties of the flavor cue in adulthood in female alcohol-preferring (P) rats

BACKGROUND

Alcohol use disorders (AUDs) affect 15 million people nationwide, 4% of which are adolescents (ages 12-17) and adolescents who binge drink significantly increase their likelihood of suffering from an AUD in adulthood. Research shows that cues (i.e. flavors) paired with alcohol (EtOH) produce significant cue-induced alcohol craving and contribute to relapse in adolescent and adult populations. However, there is a lack of research focused on how cues that accompany EtOH drinking during adolescence, affect EtOH craving later in life. The current study sought to examine the sex- and developmental-dependent effects of adolescent exposure to flavor cues associated with EtOH on operant-lick behavior and cue-induced dopamine (DA) levels within the nucleus accumbens shell (AcbSh; reward structure) in adulthood.

METHODS

Adolescent alcohol-preferring (P) rats were randomly assigned to one of 4 groups and received 24 hr. access to three bottles on their home cage: Paired: 0.1% blueberry flavor extract (BB) + 15% v/v EtOH and 2 water bottles; Unpaired: 0.1% BB, 15% v/v EtOH, and water; 15% EtOH alone, and 2 water bottles; BB alone and 2 water bottles. Home cage fluid consumption was measured for 2-weeks. On the third week bottles were removed and all animals underwent 9 days of operant training using an operant sipper paradigm. This consisted of two sipper spouts connected to the computer by a lickometer, which registered tongue contacts with the sipper tube (Paired: BB+EtOH or water; Unpaired BB or EtOH; EtOH alone: EtOH or water; BB alone: BB or water). When the fixed ratio (FR) requirement for number of licks/tongue contacts was met, a liquid delivery solenoid dispensed 0.05 ml of fluid into the sipper tube. Following the final operant session all rats remained in their home-cage for approximately 40 days until adulthood at which point they were returned to the operant chambers and tested for appetitive and consummatory behavior in response to the flavor cue (all rats: BB or water; NO EtOH). Two weeks after the final operant session all rats underwent microdialysis testing to examine cue-induced DA levels in the AcbSh.

RESULTS

Data indicated that animals in the paired group exhibited a significantly greater level of licking at the BB sipper and a significantly greater level of DA release in response to the flavor cue compared to the other groups.

CONCLUSIONS

Overall, the data suggest that cues paired with EtOH during adolescence may produce persistent changes to the behavioral and neurobiological mechanisms that contribute to an increased risk of developing an AUD later in life.

From skinner box to daily life: Sign-tracker phenotype co-segregates with impulsivity, compulsivity, and addiction tendencies in humans

Pavlovian conditioning holds the potential to incentivize environmental cues, leading to approach behavior toward them, even outside our awareness. Animal models suggest that this is particularly true for the so-called sign-tracker (ST) phenotype, which is considered to reflect a predisposition toward developing addiction-related behaviours. Despite its potential clinical relevance, few studies have demonstrated the translational validity of this model, likely due to difficulties in studying Pavlovian processes in humans. To fill this gap, we combined an ecological momentary assessment with ambulatory peripheral autonomic monitoring to test the hypothesis that traits associated with ST in preclinical studies would be associated with attribution of high incentive salience to reward-related cues. Several times for 2 days, participants were asked to rate the attractiveness of several preselected ecological rewards (e.g., coffee) and the preceding cues (the smell of coffee) while their electrocardiogram was recorded. While no absolute difference in subjective and physiological measures of motivational approach to daily cues compared with rewards emerged, individuals with high levels of impulsivity, obsessive-compulsive, and addiction-prone behaviors rated as more attractive and showed a greater increase in sympathetic arousal to cues versus rewards. The opposite pattern emerged for those with low levels in those dispositional traits, who responded more (both subjectively and physiologically) to rewards compared with their preceding cues. This study represents an attempt to answer the call to parcel complex behaviors into smaller constructs, improving the early detection of those who are vulnerable to develop psychopathological disorders, particularly in the domain of impulse control such as addiction.

Aberrant orbitofrontal cortex reactivity to erotic cues in Compulsive Sexual Behavior Disorder

BACKGROUND AND AIMS

Compulsive Sexual Behavior Disorder (CSBD) is characterized by increased reactivity to erotic reward cues. Cue-encoded reward parameters, such as type (e.g. erotic or monetary) or probability of anticipated reward, shape reward-related motivational processes, increase the attractiveness of cues and therefore might enhance maladaptive behavioral patterns in CSBD. Studies on the neural patterns of cue processing in individuals with CSBD have been limited mainly to ventral striatal responses. Therefore, here we aimed to examine the cue reactivity of multiple key structures in the brain's reward system, taking into account not only the type of predicted reward but also its probability.

METHODS

Twenty Nine men seeking professional help due to CSBD and 24 healthy volunteers took part in an fMRI study with a modified Incentive Delay Task with erotic and monetary rewards preceded by cues indicating a 25%, 50%, or 75% chance of reward. Analyses of functional patterns of activity related to cue type and probability were conducted on the whole-brain and ROI levels.

RESULTS

Increased anticipatory response to cues predictive of erotic rewards was observed among CSBD participants when compared to controls, in the ventral striatum and anterior orbitofrontal cortex (aOFC). The activity in aOFC was modulated by reward probability.

DISCUSSION AND CONCLUSIONS

Type of anticipated reward (erotic vs monetary) affects reward-related behavioral motivation in CSBD more strongly than reward probability. We present evidence of abnormal aOFC function in CSBD by demonstrating the recruitment of additional subsections of this region by erotic reward cues.

Sign- and goal-tracking score does not correlate with addiction-like behavior following prolonged cocaine self-administration

RATIONALE

In classical conditioning, sign-tracking reflects behavior directed toward a conditioned stimulus (CS) in expectation of a reward (unconditioned stimulus, US); in contrast, goal-tracking describes behavior directed toward the location of delivery of a US. As cues previously paired with drugs of abuse promote drug-seeking and drug-taking behavior in both animals and humans and thus contribute to the severity of substance abuse, sign-tracking may represent a maladaptive cue-focused behavior that may increase addiction vulnerability as compared to goal-tracking. Recent studies do, in fact, support this possibility. Previous work in this area has focused primarily on paradigms using relatively limited exposure to drug rather than extended drug intake.

OBJECTIVES

Here, we used the DSM-IV-based 3-criteria (3-CRIT) model and examined whether a relationship exists between sign- or goal-tracking phenotypes and the prevalence of criteria associated with addiction-like behavior following extended cocaine self-administration as measured in this model.

METHODS

Forty-six male Sprague Dawley rats underwent a Pavlovian conditioned approach (PCA) procedure and were characterized along a continuum as goal-trackers (GTs), intermediates (INTs), or sign-trackers (STs). The animals were subsequently trained to intravenous self-administer cocaine during 45 self-administration (SA) sessions and characterized for the 3 criteria outlined in the model: persistence of drug-seeking, motivation for cocaine-taking, and resistance to punishment.

RESULTS

We performed correlational analyses on the traits measured, finding no relationships between PCA score and addiction-like characteristics measured using the 3-CRIT model of addiction. However, STs showed significantly greater resistance to punishment than GTs.

CONCLUSIONS

Phenotyping along a continuum of PCA scores may not be a valid predictor for identifying vulnerability to the addiction-like behaviors examined using the 3-CRIT model. However, PCA phenotype may predict a single feature of the 3-CRIT model, resistance to punishment, among those rats classified as either STs or GTs.

The Paraventricular Thalamus as a Critical Node of Motivated Behavior via the Hypothalamic-Thalamic-Striatal Circuit

In this review, we highlight evidence that supports a role for the paraventricular nucleus of the thalamus (PVT) in motivated behavior. We include a neuroanatomical and neurochemical overview, outlining what is known of the cellular makeup of the region and its most prominent afferent and efferent connections. We discuss how these connections and distinctions across the anterior-posterior axis correspond to the perceived function of the PVT. We then focus on the hypothalamic-thalamic-striatal circuit and the neuroanatomical and functional placement of the PVT within this circuit. In this regard, the PVT is ideally positioned to integrate information regarding internal states and the external environment and translate it into motivated actions. Based on data that has emerged in recent years, including that from our laboratory, we posit that orexinergic (OX) innervation from the lateral hypothalamus (LH) to the PVT encodes the incentive motivational value of reward cues and thereby alters the signaling of the glutamatergic neurons projecting from the PVT to the shell of the nucleus accumbens (NAcSh). The PVT-NAcSh pathway then modulates dopamine activity and resultant cue-motivated behaviors. As we and others apply novel tools and approaches to studying the PVT we will continue to refine the anatomical, cellular, and functional definitions currently ascribed to this nucleus and further elucidate its role in motivated behaviors.

Male Goal-Tracker and Sign-Tracker Rats Do Not Differ in Neuroendocrine or Behavioral Measures of Stress Reactivity

Environmental cues attain the ability to guide behavior via learned associations. As predictors, cues can elicit adaptive behavior and lead to valuable resources (e.g., food). For some individuals, however, cues are transformed into incentive stimuli and elicit motivational states that can be maladaptive. The goal-tracker (GT)/sign-tracker (ST) animal model captures individual differences in cue-motivated behaviors, with reward-associated cues serving as predictors of reward for both phenotypes but becoming incentive stimuli to a greater degree for STs. While these distinct phenotypes are characterized based on Pavlovian conditioned approach (PavCA) behavior, they exhibit differences on a number of behaviors relevant to psychopathology. To further characterize the neurobehavioral endophenotype associated with individual differences in cue-reward learning, neuroendocrine and behavioral profiles associated with stress and anxiety were investigated in male GT, ST, and intermediate responder (IR) rats. It was revealed that baseline corticosterone (CORT) increases with Pavlovian learning, but to the same degree, regardless of phenotype. No significant differences in behavior were observed between GTs and STs during an elevated plus maze (EPM) or open field test (OFT), nor were there differences in CORT response to the OFT or physiological restraint. Upon examination of central markers associated with stress reactivity, we found that STs have greater glucocorticoid receptor (GR) mRNA expression in the ventral hippocampus, with no phenotypic differences in the dorsal hippocampus or prelimbic cortex (PrL). These findings demonstrate that GTs and STs do not differ on stress-related and anxiety-related behaviors, and suggest that differences in neuroendocrine measures between these phenotypes can be attributed to distinct cue-reward learning styles.

Prelimbic prefrontal cortical encoding of reward predictive cues

Animals appoint incentive value and learn to approach otherwise behaviorally inert stimuli if these stimuli come to predict the delivery of reward. Interestingly, this adaptive Pavlovian learning process has been implicated in behavioral control disorders, such as drug addiction. One brain region implicated in directing conditioned approach behavior is the prelimbic region of the prefrontal cortex. The present study employed in vivo electrophysiology in the prelimbic cortex to characterize the distribution of neural responses to the presence of a cue that had acquired incentive value after being associated with a primary reward. Male rats were trained in a Pavlovian autoshaping task in which a lever was presented prior to reward delivery. Following repeated pairings of lever availability and reward delivery, rats pressed the lever even though reward delivery was not contingent on any interaction with the lever. Neurons in the prelimbic cortex selectively encoded the presentation of the reward-predicting lever. Although the response was heterogeneous, most responsive neurons decreased their firing rate in response to the presence of the lever. These findings characterize the varied responses of prelimbic cortical neurons to reward cues and are consistent with evidence that the role of the prelimbic cortex in reward learning depends on the downstream target.

A proposed role for glucocorticoids in mediating dopamine-dependent cue-reward learning

Learning to respond appropriately to one's surrounding environment is fundamental to survival. Importantly, however, individuals vary in how they respond to cues in the environment and this variation may be a key determinant of psychopathology. The ability of seemingly neutral cues to promote maladaptive behavior is a hallmark of several psychiatric disorders including, substance use disorder, post-traumatic stress disorder, eating disorders and obsessive-compulsive disorder. Thus, it is important to uncover the neural mechanisms by which such cues are able to attain inordinate control and promote psychopathological behavior. Here, we suggest that glucocorticoids play a critical role in this process. Glucocorticoids are primarily recognized as the main hormone secreted in response to stress but are known to exert their effects across the body and the brain, and to affect learning and memory, cognition and reward-related behaviors, among other things. Here we speculate that glucocorticoids act to facilitate a dopamine-dependent form of cue-reward learning that appears to be relevant to a number of psychiatric conditions. Specifically, we propose to utilize the sign-tracker/goal-tracker animal model as a means to capture individual variation in stimulus-reward learning and to isolate the role of glucocorticoid-dopamine interactions in mediating these individual differences. It is hoped that this framework will lead to the discovery of novel mechanisms that contribute to complex neuropsychiatric disorders and their comorbidity.

Instrumental and Pavlovian Mechanisms in Alcohol Use Disorder

Purpose of Review

Current theories of alcohol use disorders (AUD) highlight the importance of Pavlovian and instrumental learning processes mainly based on preclinical animal studies. Here, we summarize available evidence for alterations of those processes in human participants with AUD with a focus on habitual versus goal-directed instrumental learning, Pavlovian conditioning, and Pavlovian-to-instrumental transfer (PIT) paradigms.

Recent Findings

The balance between habitual and goal-directed control in AUD participants has been studied using outcome devaluation or sequential decision-making procedures, which have found some evidence of reduced goal-directed/model-based control, but little evidence for stronger habitual responding. The employed Pavlovian learning and PIT paradigms have shown considerable differences regarding experimental procedures, e.g., alcohol-related or conventional reinforcers or stimuli.

Summary

While studies of basic learning processes in human participants with AUD support a role of Pavlovian and instrumental learning mechanisms in the development and maintenance of drug addiction, current studies are characterized by large variability regarding methodology, sample characteristics, and results, and translation from animal paradigms to human research remains challenging. Longitudinal approaches with reliable and ecologically valid paradigms of Pavlovian and instrumental processes, including alcohol-related cues and outcomes, are warranted and should be combined with state-of-the-art imaging techniques, computational approaches, and ecological momentary assessment methods.

Motivation and reward mechanisms in health behavior change processes

With increasing prevalence of lifestyle-related chronic diseases worldwide, understanding health behavior change and the development of successful interventions to support lifestyle modification is gaining increasing interest among politicians, scientists, therapists and patients alike. A number of health behavior change theories have been developed aiming at explaining health behavior change and understanding the domains that make change more likely. Until now, only few studies have taken into account automatic, implicit or non-cognitive aspects of behavior, including emotion and positive affect. Recent progress in the neuroscience of motivation and reward systems can provide further insights into the relevance of such domains. In this integrative review, we present a description of the possible motivation and reward systems (approach/wanting = pleasure; aversion/avoiding = relief; assertion/non-wanting = quiescence) involved in behavior change. Therefore, based on established theories encompassing both initiation and maintenance of behavior change, we create a flexible seven-stage behavior change process with three engagement phases (non-engagement, motivational engagement, executive engagement) and relate the motivation and reward systems to each of these stages. We propose that either appetitive (preferably) or aversive motivational salience is activated during motivational engagement, that learning leads to continued behavior and that assertive salience prevails when the new behavior has become habitual. We discuss under which circumstances these mechanisms and reward-motivation pathways are likely to occur and address potential shortcomings of our proposed theoretical framework. We highlight implications for future interventions aiming at lifestyle modification.

A thalamic bridge from sensory perception to cognition

The ability to adapt to dynamic environments requires tracking multiple signals with variable sensory salience and fluctuating behavioral relevance. This complex process requires integrative crosstalk between sensory and cognitive brain circuits. Functional interactions between cortical and thalamic regions are now considered essential for both sensory perception and cognition but a clear account of the functional link between sensory and cognitive circuits is currently lacking. This review aims to document how thalamic nuclei may effectively act as a bridge allowing to fuse perceptual and cognitive events into meaningful experiences. After highlighting key aspects of thalamocortical circuits such as the classic first-order/higher-order dichotomy, we consider the role of the thalamic reticular nucleus from directed attention to cognition. We next summarize research relying on Pavlovian learning paradigms, showing that both first-order and higher-order thalamic nuclei contribute to associative learning. Finally, we propose that modulator inputs reaching all thalamic nuclei may be critical for integrative purposes when environmental signals are computed. Altogether, the thalamus appears as the bridge linking perception, cognition and possibly affect.

Toward Precision Medicine for Smoking Cessation: Developing a Neuroimaging-Based Classification Algorithm to Identify Smokers at Higher Risk for Relapse

INTRODUCTION

By improving our understanding of the neurobiological mechanisms underlying addiction, neuroimaging research is helping to identify new targets for personalized treatment interventions. When trying to quit, smokers with larger electrophysiological responses to cigarette-related, compared with pleasant, stimuli ("C > P") are more likely to relapse than smokers with the opposite brain reactivity profile ("P > C").

AIM AND METHOD

The goal was to (1) build a classification algorithm to identify smokers characterized by P > C or C > P neuroaffective profiles and (2) validate the algorithm's classification outcomes in an independent data set where we assessed both smokers' electrophysiological responses at baseline and smoking abstinence during a quit attempt. We built the classification algorithm applying discriminant function analysis on the event-related potentials evoked by emotional images in 180 smokers.

RESULTS

The predictive validity of the classifier showed promise in an independent data set that included new data from 177 smokers interested in quitting; the algorithm classified 111 smokers as P > C and 66 as C > P. The overall abstinence rate was low; 15 individuals (8.5% of the sample) achieved CO-verified 12-month abstinence. Although individuals classified as P > C were nearly 2.5 times more likely to be abstinent than smokers classified as C > P (12 vs. 3, or 11% vs. 4.5%), this result was nonsignificant, preliminary, and in need of confirmation in larger trials.

CONCLUSION

These results suggest that psychophysiological techniques have the potential to advance our knowledge of the neurobiological underpinnings of nicotine addiction and improve clinical applications. However, larger sample sizes are necessary to reliably assess the predictive ability of our algorithm.

IMPLICATIONS

We assessed the clinical relevance of a neuroimaging-based classification algorithm on an independent sample of smokers enrolled in a smoking cessation trial and found those with the tendency to attribute more relevance to rewards than cues were nearly 2.5 times more likely to be abstinent than smokers with the opposite brain reactivity profile (11% vs. 4.5%). Although this result was not statistically significant, it suggests our neuroimaging-based classification algorithm can potentially contribute to the development of new precision medicine interventions aimed at treating substance use disorders. Regardless, these findings are still preliminary and in need of confirmation in larger trials.

Morphine-Conditioned Placebo Analgesia in Female and Male Rats with Chronic Neuropathic Pain: c-Fos Expression in the Rostral Ventromedial Medulla

Placebo analgesia has great potential to overcome the inadequacies of current drug therapies to treat conditions of chronic pain. The rostral ventromedial medulla (RVM) has been implicated as a critical relay in the antinociceptive pathway underpinning placebo analgesia in humans. We developed a model of opiate-conditioned placebo analgesia in rats with neuropathic injury to identify medullary nuclei active during placebo analgesia. Using female and male rats the degree of thermal allodynia was first determined following nerve injury, and a pharmacological conditioning procedure, pairing contextual cues with the experience of morphine-induced analgesia, was used to elicit placebo analgesic reactions. This protocol revealed clear subpopulations of placebo reactors (36% of males, 25% of females) and non-reactors in proportions similar to those reported in human studies. We detected injury-specific c-Fos expression in the gracile nucleus and morphine-specific c-Fos expression in the serotonergic midline raphe nuclei and the caudal nuclei of the solitary tract. However, c-Fos expression did not differ between placebo reactors and non-reactors in either serotonergic or non-serotonergic neurons of the RVM. Despite a subpopulation of rats demonstrating placebo reactions, we found no evidence for enhanced activity in the nuclei from which the classical RVM → spinal cord descending analgesic pathways emerge.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Sign tracking predicts cue-induced but not drug-primed reinstatement to methamphetamine seeking in rats: Effects of oxytocin treatment

BACKGROUND

The incentive sensitisation theory of addiction posits that drug-associated stimuli become imbued with incentive motivational properties, driving pathological drug seeking. However, pre-existing variability in the incentive salience to non-drug reward cues ('sign trackers' (STs); 'goal trackers' (GTs)) is also predictive of the desire for and relapse to cocaine and opioids. Here, we asked whether variation in propensity to attribute incentive salience to a food cue is predictive of reinstatement to the highly addictive psychostimulant methamphetamine (METH), and whether treatment with the promising anti-addiction therapy oxytocin differentially reduces METH behaviour between STs and GTs.

METHODS

Rats were trained to associate a Pavlovian cue with delivery of a sucrose pellet over 8 days. They then received jugular vein catheters for intravenous METH self-administration, followed by behavioural extinction, and cue-induced and METH-primed reinstatement to METH-seeking behaviours. Oxytocin was administered prior to self-administration and reinstatement tests.

RESULTS

Despite the self-administration of similar amounts of METH, STs reinstated more to METH cues than did GTs, yet METH-priming reinstated STs and GTs similarly. Furthermore, oxytocin attenuated cue-induced reinstatement more so in STs than in GTs, and reduced METH-primed reinstatement to a greater extent in the top quartile of reinstaters, indicating that oxytocin treatment may be most effective for those at highest risk of addiction.

CONCLUSIONS

This pre-existing bias towards reward cues presents a possible tool to screen for METH addiction susceptibility and may be useful for understanding the neurobiology of addiction and for pharmacotherapeutic discovery.

Traumatic brain injury substantially reduces the conditioned reinforcing effects of environmental cues in rats

Traumatic brain injury affects millions of people each year and is an established risk factor for addiction. Recent animal studies have causally demonstrated that injuries can increase drug self-administration across a variety of substances. One potential behavioral mediator for this finding is an increased responsivity to drug-associated cues. This endophenotype can be identified by profiling non-drug-related behaviors. The current study evaluated several paradigms (conditioned approach, conditioned reinforcement, extinction from variable interval responding, conditioned facilitation) to determine how rats with a frontal TBI differed in their response to Pavlovian conditioning in response to food-paired cues. Surprisingly, rats with a TBI demonstrated increased goal-tracking in a conditioned approach paradigm and exerted less effort for a conditioned reinforcer. Moreover, they had slightly facilitated extinction (as demonstrated by significantly larger interresponse times) in the face of reinforcer-associated cues. Despite these effects, TBI rats still demonstrated conditioned facilitation to an auditory stimulus. Together, these effects suggest a phenotype in the opposite direction of what might be anticipated. Cues still served a strong discriminative function and altered behavior; however, they did not function as strong conditioned reinforcers for TBI animals. One potential reason for this is that substantial changes to the dopamine system after TBI may reduce the conditioned reinforcing effects of cues, but sensitize the brain to potent drugs of abuse. More research will be needed to determine whether this is the case.

Modeling neuroaffective biomarkers of drug addiction: A Bayesian nonparametric approach using dirichlet process mixtures

BACKGROUND

The properties of neurophysiological processes related to addiction have received much attention in the literature. However, empirical evidence of meaningful and useful characterization of these processes is limited. Recent studies have found that electrophysiological responses to emotional and drug-related cues can be used to create profiles that reliably predict smoking relapse.

NEW METHOD

This paper evaluates the validity of classifying electrophysiological responses into distinct profiles using a Bayesian dirichlet process mixture (DPM) model. The DPM is a Bayesian nonparametric (BNP) method to modeling unknown number of profiles characterized by uncertainty in cluster membership and in cluster number.

RESULTS

The DPM model confirmed previously identified neuroaffective reactivity profiles, but also revealed a finer level of granularity in the clustering. Specifically, in addition to the two clusters previously identified in the literature, the BNP methods identified a cluster of individuals showing similar responses to smoking, pleasant, neutral and unpleasant cues.

COMPARISON WITH EXISTING METHODS

BNP models provide an alternative to the k-mean clustering approach to modeling EEG-based neuroaffective profiles. Unlike k-means clustering, BNP models compute the probability that a subject belongs to a cluster while taking into consideration uncertainty in the number of clusters.

CONCLUSIONS

Our results confirm the reliability of the two clusters previously identified in these data, but also provide new insights by revealing a cluster that presented similar responses to stimuli with different contents. This finding may be related to the uncertainty in classification or overlapping brain-reactivity profiles.

Sex-dependent associations between addiction-related behaviors and the microbiome in outbred rats

BACKGROUND

Multiple factors contribute to the etiology of addiction, including genetics, sex, and a number of addiction-related behavioral traits. One behavioral trait where individuals assign incentive salience to food stimuli ("sign-trackers", ST) are more impulsive compared to those that do not ("goal-trackers", GT), as well as more sensitive to drugs and drug stimuli. Furthermore, this GT/ST phenotype predicts differences in other behavioral measures. Recent studies have implicated the gut microbiota as a key regulator of brain and behavior, and have shown that many microbiota-associated changes occur in a sex-dependent manner. However, few studies have examined how the microbiome might influence addiction-related behaviors. To this end, we sought to determine if gut microbiome composition was correlated with addiction-related behaviors determined by the GT/ST phenotype.

METHODS

Outbred male (N=101) and female (N=101) heterogeneous stock rats underwent a series of behavioral tests measuring impulsivity, attention, reward-learning, incentive salience, and locomotor response. Cecal microbiome composition was estimated using 16S rRNA gene amplicon sequencing. Behavior and microbiome were characterized and correlated with behavioral phenotypes. Robust sex differences were observed in both behavior and microbiome; further analyses were conducted within sex using the pre-established goal/sign-tracking (GT/ST) phenotype and partial least squares differential analysis (PLS-DA) clustered behavioral phenotype.

RESULTS

Overall microbiome composition was not associated to the GT/ST phenotype. However, microbial alpha diversity was significantly decreased in female STs. On the other hand, a measure of impulsivity had many significant correlations to microbiome in both males and females. Several measures of impulsivity were correlated with the genus Barnesiella in females. Female STs had notable correlations between microbiome and attentional deficient. In both males and females, many measures were correlated with the bacterial families Ruminocococcaceae and Lachnospiraceae.

CONCLUSIONS

These data demonstrate correlations between several addiction-related behaviors and the microbiome specific to sex.

Mapping sign-tracking and goal-tracking onto human behaviors

As evidenced through classic Pavlovian learning mechanisms, environmental cues can become incentivized and influence behavior. These stimulus-outcome associations are relevant in everyday life but may be particularly important for the development of impulse control disorders including addiction. Rodent studies have elucidated specific learning profiles termed 'sign-tracking' and 'goal-tracking' which map onto individual differences in impulsivity and other behaviors associated with impulse control disorders' etiology, course, and relapse. Whereas goal-trackers are biased toward the outcome, sign-trackers fixate on features that are associated with but not necessary for achieving an outcome; a pattern of behavior that often leads to escalation of reward-seeking that can be maladaptive. The vast majority of the sign- and goal-tracking research has been conducted using rodent models and very few have bridged this concept into the domain of human behavior. In this review, we discuss the attributes of sign- and goal-tracking profiles, how these are manifested neurobiologically, and how these distinct learning styles could be an important tool for clinical interventions in human addiction.

Effects of Limited and Extended Pavlovian Training on Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Individual differences in Pavlovian approach predict differences in devaluation sensitivity. Recent studies indicate goal-tracking (GT) rats are sensitive to outcome devaluation while sign-tracking (ST) rats are not. With extended training in Pavlovian lever autoshaping (PLA), GT rats display more lever-directed behavior, typical of ST rats, suggesting they may become insensitive to devaluation with more Pavlovian training experience. Here, we use a within-subject satiety-induced outcome devaluation procedure to test devaluation sensitivity after limited and extended PLA training in GT and ST rats. We trained rats in PLA to determine GT and ST groups. Then, we sated rats on either the training pellets (devalued condition) or homecage chow (valued condition) prior to brief non-reinforced test sessions after limited (sessions 5/6) and extended (sessions 17/18) PLA training. GT rats decreased conditioned responding under devalued relative to valued conditions after both limited and extended training, demonstrating they are sensitive to satiety devaluation regardless of the amount of PLA training. While ST rats were insensitive to satiety devaluation after limited training, their lever directed behavior became devaluation sensitive after extended training. To determine whether sign-tracking rats also displayed sensitivity to illness-induced outcome devaluation after extended training, we trained a separate cohort of rats in extended PLA and devalued the outcome with lithium chloride injections after pellet consumption in the homecage. ST rats failed to decrease conditioned responding after illness-induced outcome devaluation, while Non-ST rats (GT and intermediates) were sensitive to illness-induced outcome devaluation after extended training. Together, our results confirm devaluation sensitivity is stable in GT rats across training and devaluation approaches. Extended training unmasks devaluation sensitivity in ST rats after satiety, but not illness-induced devaluation, suggesting ST rats respond appropriately by decreasing responding to cues during state-dependent but not inference-based devaluation. The differences in behavioral flexibility across tracking groups and devaluation paradigms have translational relevance for the understanding state- vs. inference-based reward devaluation as it pertains to drug addiction vulnerability.

Addiction vulnerability and the processing of significant cues: Sign-, but not goal-, tracker perceptual sensitivity relies on cue salience

The identification of broadly defined psychological traits that bestow vulnerability for the manifestation of addiction-like behaviors can guide the discovery of the neuronal mechanisms underlying the propensity for drug taking. Sign-tracking behavior in rats (STs) signifies the presence of a trait that predicts a relatively greater behavioral control of Pavlovian drug and reward cues than in rats that exhibit goal-tracking behavior (GTs). We previously demonstrated that relatively poor cholinergic-attentional control in STs is an essential component of the trait indexed by sign-tracking and that this trait aspect contributes to the relatively greater power of drug cues to control the behavior of STs. Here we addressed the possibility that STs and GTs employ fundamentally different psychological mechanisms for the detection of cues in attention-demanding contexts. Rats were trained to perform an operant Sustained Attention Task. As task training advanced to the stage that taxed attentional control, the relative brightness of visual target signals significantly influenced detection performance in STs but not GTs. This finding suggests that STs, but not GTs, rely on bottom-up, cue salience-driven mechanisms to detect cues. GTs may be able to resist behavioral control by Pavlovian drug cues by utilizing goal-directed decisional processes that minimize the influence of the salience of drug cues. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

Dissociable dopaminergic and pavlovian influences in goal-trackers and sign-trackers on a model of compulsive checking in OCD

RATIONALE

Checking is a functional behaviour that provides information to guide behaviour. However, in obsessive-compulsive disorder (OCD), checking may escalate to dysfunctional levels. The processes underpinning the transition from functional to dysfunctional checking are unclear but may be associated with individual differences that support the development of maladaptive behaviour. We examined one such predisposition, sign-tracking to a pavlovian conditioned stimulus, which we previously found associated with dysfunctional checking. How sign-tracking interacts with another treatment with emerging translational validity for OCD-like checking, chronic administration of the dopamine D₂ receptor agonist quinpirole, is unknown.

OBJECTIVES

We tested how functional and dysfunctional checking in the rat observing response task (ORT) was affected by chronic quinpirole administration in non-autoshaped controls and autoshaped animals classified as sign-trackers or goal-trackers.

METHODS

Sign-trackers or goal-trackers were trained on the ORT before the effects of chronic quinpirole administration on checking were assessed. Subsequently, the effects on checking of different behavioural challenges, including reward omission and the use of unpredictable reinforcement schedules, were tested.

RESULTS

Prior autoshaping increased checking. Sign-trackers and goal-trackers responded differently to quinpirole sensitization, reward omission and reinforcement uncertainty. Sign-trackers showed greater elevations in dysfunctional checking, particularly during uncertainty. By contrast, goal-trackers predominantly increased functional checking responses, possibly in response to reduced discrimination accuracy in the absence of cues signalling which lever was currently active.

CONCLUSIONS

The results are discussed in terms of how pavlovian associations influence behaviour that becomes compulsive in OCD and how this may be dependent on striatal dopamine D₂ receptors.

Lipopolysaccharide Increases Cortical Kynurenic Acid and Deficits in Reference Memory in Mice

Kynurenic acid (KYNA), a glial-derived metabolite of tryptophan metabolism, is an antagonist of the alpha 7 nicotinic acetylcholine receptor and the glycine-binding site of N-methyl-d-aspartate (NMDA) receptors. Kynurenic acid levels are increased in both the brain and cerebrospinal fluid of several psychiatric disorders including bipolar disorder, schizophrenia, and Alzheimer disease. In addition, pro-inflammatory cytokines have been found to be elevated in the blood of schizophrenic patients suggesting inflammation may play a role in psychiatric illness. As both pro-inflammatory cytokines and KYNA can be elevated in the brain by peripheral lipopolysaccharide (LPS) injection, we therefore sought to characterize the role of neuroinflammation on learning and memory using a well-described dual-LPS injection model. Mice were injected with an initial injection (0.25 mg/kg LPS, 0.50 mg/kg, or saline) of LPS and then administrated a second injection 16 hours later. Our results indicate both 0.25 and 0.50 mg/kg dual-LPS treatment increased l-kynurenine and KYNA levels in the medial pre-frontal cortex (mPFC). Mice exhibited impaired acquisition of CS+ (conditioned stimulus) Pavlovian conditioning. Notably, mice showed impairment in reference memory while working memory was normal in an 8-arm maze. Taken together, our findings suggest that neuroinflammation induced by peripheral LPS administration contributes to cognitive dysfunction.

Affective valence in the brain: modules or modes?

How do brain systems evaluate the affective valence of a stimulus - that is, its quality of being good or bad? One possibility is that a neural subsystem, or 'module' (such as a subregion of the brain, a projection pathway, a neuronal population or an individual neuron), is permanently dedicated to mediate only one affective function, or at least only one specific valence - an idea that is termed here the 'affective modules' hypothesis. An alternative possibility is that a given neural module can exist in multiple neurobiological states that give it different affective functions - an idea termed here the 'affective modes' hypothesis. This suggests that the affective function or valence mediated by a neural module need not remain permanently stable but rather can change dynamically across different situations. An evaluation of evidence for the 'affective modules' versus 'affective modes' hypotheses may be useful for advancing understanding of the affective organization of limbic circuitry.

Attentional capture by Pavlovian reward-signalling distractors in visual search persists when rewards are removed

Existing research indicates that learning about the Pavlovian ‘signal value’ of stimuli can induce attentional biases: findings suggest that our attentional system prioritises detection of stimuli that have previously signalled availability of high reward. These findings potentially provide a human analogue of sign-tracking behaviour previously reported in studies of non-human animals. Here we examine a visual search task that has been developed to demonstrate the Pavlovian influence of reward on attention, in which the critical reward-signalling stimuli are never explicit targets of search. This procedure has previously yielded robust effects of reward on attention; however it remains unclear whether this pattern reflects a persistent and automatic bias in attentional capture based on prior experience of stimulus–reward pairings, or whether it results from participants strategically attending to reward-signalling distractors because they provide useful information about reward magnitude. To investigate this issue, in the current study participants initially completed a rewarded visual search task, in which colours of distractor stimuli signalled availability of high or low reward. Participants then completed a test phase in which rewards were no longer available, such that distractor colours no longer provided useful information on reward availability. Performance during the initial rewarded phase was impaired by the presence of a distractor signalling availability of high relative to low reward. Crucially, the magnitude of this reward-related distraction effect did not reduce in the subsequent unrewarded test phase. This suggests that participants’ experience of differences in reward value signalled by distractor stimuli in this task can induce persistent biases in the extent to which these stimuli involuntarily capture attention, even when they are entirely task-irrelevant.

Evidence for incentive salience sensitization as a pathway to alcohol use disorder

The incentive salience sensitization (ISS) theory of addiction holds that addictive behavior stems from the ability of drugs to progressively sensitize the brain circuitry that mediates attribution of incentive salience (IS) to reward-predictive cues and its behavioral manifestations. In this article, we establish the plausibility of ISS as an etiological pathway to alcohol use disorder (AUD). We provide a comprehensive and critical review of evidence for: (1) the ability of alcohol to sensitize the brain circuitry of IS attribution and expression; and (2) attribution of IS to alcohol-predictive cues and its sensitization in humans and non-human animals. We point out gaps in the literature and how these might be addressed. We also highlight how individuals with different alcohol subjective response phenotypes may differ in susceptibility to ISS as a pathway to AUD. Finally, we discuss important implications of this neuropsychological mechanism in AUD for psychological and pharmacological interventions attempting to attenuate alcohol craving and cue reactivity.

Understanding Addiction Using Animal Models

Drug addiction is a neuropsychiatric disorder with grave personal consequences that has an extraordinary global economic impact. Despite decades of research, the options available to treat addiction are often ineffective because our rudimentary understanding of drug-induced pathology in brain circuits and synaptic physiology inhibits the rational design of successful therapies. This understanding will arise first from animal models of addiction where experimentation at the level of circuits and molecular biology is possible. We will review the most common preclinical models of addictive behavior and discuss the advantages and disadvantages of each. This includes non-contingent models in which animals are passively exposed to rewarding substances, as well as widely used contingent models such as drug self-administration and relapse. For the latter, we elaborate on the different ways of mimicking craving and relapse, which include using acute stress, drug administration or exposure to cues and contexts previously paired with drug self-administration. We further describe paradigms where drug-taking is challenged by alternative rewards, such as appetitive foods or social interaction. In an attempt to better model the individual vulnerability to drug abuse that characterizes human addiction, the field has also established preclinical paradigms in which drug-induced behaviors are ranked by various criteria of drug use in the presence of negative consequences. Separation of more vulnerable animals according to these criteria, along with other innate predispositions including goal- or sign-tracking, sensation-seeking behavior or impulsivity, has established individual genetic susceptibilities to developing drug addiction and relapse vulnerability. We further examine current models of behavioral addictions such as gambling, a disorder included in the DSM-5, and exercise, mentioned in the DSM-5 but not included yet due to insufficient peer-reviewed evidence. Finally, after reviewing the face validity of the aforementioned models, we consider the most common standardized tests used by pharmaceutical companies to assess the addictive potential of a drug during clinical trials.

Neural circuits linking sleep and addiction: Animal models to understand why select individuals are more vulnerable to substance use disorders after sleep deprivation

Individuals differ widely in their drug-craving behaviors. One reason for these differences involves sleep. Sleep disturbances lead to an increased risk of substance use disorders and relapse in only some individuals. While animal studies have examined the impact of sleep on reward circuitry, few have addressed the role of individual differences in the effects of altered sleep. There does, however, exist a rodent model of individual differences in reward-seeking behavior: the sign/goal-tracker model of Pavlovian conditioned approach. In this model, only some rats show the key behavioral traits associated with addiction, including impulsivity and poor attentional control, making this an ideal model system to examine individually distinct sleep-reward interactions. Here, we describe how the limbic neural circuits responsible for individual differences in incentive motivation overlap with those involved in sleep-wake regulation, and how this model can elucidate the common underlying mechanisms. Consideration of individual differences in preclinical models would improve our understanding of how sleep interacts with motivational systems, and why sleep deprivation contributes to addiction in only select individuals.