Effects of outcome revaluation on attentional prioritisation of reward-related stimuli

Stimuli associated with rewards can acquire the ability to capture our attention independently of our goals and intentions. Here, we examined whether attentional prioritisation of reward-related cues is sensitive to changes in the value of the reward itself. To this end, we incorporated an instructed outcome devaluation (Experiment 1a), "super-valuation" (Experiment 1b), or value switch (Experiment 2) into a visual search task, using eye-tracking to examine attentional prioritisation of stimuli signalling high- and low-value rewards. In Experiments 1a and 1b, we found that prioritisation of high- and low-value stimuli was insensitive to devaluation of a previously high-value outcome, and super-valuation of a previously low-value outcome, even when participants were provided with further experience of receiving that outcome. In Experiment 2, following a value-switch manipulation, we found that prioritisation of a high-value stimulus could not be overcome with knowledge of the new values of outcomes alone. Only when provided with further experience of receiving the outcomes did patterns of attentional prioritisation of high- and low-value stimuli switch, in line with the updated values of the outcomes they signalled. To reconcile these findings, we suggest that participants were motivated to engage in effortful updating of attentional control settings when there was a relative difference between reward values at test (Experiment 2) but that previous settings were allowed to persist when both outcomes had the same value at test (Experiments 1a and 1b). These findings provide a novel framework to further understand the role of cognitive control in driving reward-modulated attention and behaviour.

Prediction error in models of adaptive behavior

Pavlovian conditioning is evident in every species in which it has been assessed, and there is a consensus about its interpretation across behavioral,1,2 brain,3,4,5,6 and computational analyses7,8,9,10,11: conditioned behavior reflects the formation of a directional associative link from the memory of one stimulus (e.g., a visual stimulus) to another (e.g., food), with learning stopping when there is no error between the prediction generated by the visual stimulus and what happens next (e.g., food). This consensus fails to anticipate the results that we report here. In our experiments with rats, we find that arranging predictive (visual stimulus→food) and nonpredictive (food→visual stimulus) relationships produces marked and sustained changes in conditioned behaviors when the visual stimulus is presented alone. Moreover, the type of relationship affects (1) the distribution of conditioned behaviors related to the properties of both food (called goal-tracking) and the visual stimulus (called sign-tracking) and (2) when in the visual stimulus, these two behaviors are evident. These results represent an impetus for a fundamental shift in how Pavlovian conditioning is interpreted: animals learn about the relationship between two stimuli irrespective of the order in which they are presented, but they exhibit this knowledge in different ways. This interpretation and our new results are captured by a recent model of Pavlovian conditioning,12,13 HeiDI, and both are consistent with the need for animals to represent the fact that the impact of a cause (e.g., the ingestion of nutrients or the bite of a predator) can be felt before or after the cause has been perceived.

Effects of Three Serotonin Reuptake Inhibitors on Sign-Tracking in Male Sprague-Dawley Rats

Sign-tracking is a behavior with relevance to cue-triggered relapse addiction, a Pavlovian conditioned approach behavior directed at the conditioned stimulus. The study examined one strategy for reducing the magnetic pull of drug-associated conditioned stimuli, using selective serotonin reuptake inhibitors (SSRIs) citalopram (0, 10, and 20 mg/kg), escitalopram (0, 10, and 20 mg/kg) and fluoxetine (0, 5, and 10 mg/kg). Male Sprague-Dawley rats were first trained in a standard sign-tracking task and then acutely administered these drugs in a series of three experiments. In each study, it was found that measures of sign-tracking were reduced, although effects on goal-tracking were different between drugs. This study provides evidence that administration of serotonergic antidepressants is effective in reducing sign-tracking and may have some efficacy in preventing cue-triggered relapse.

Basal Forebrain Chemogenetic Inhibition Converts the Attentional Control Mode of Goal-Trackers to That of Sign-Trackers

Sign tracking versus goal tracking in rats indicate vulnerability and resistance, respectively, to Pavlovian cue-evoked addictive drug taking and relapse. Here, we tested hypotheses predicting that the opponent cognitive-behavioral styles indexed by sign tracking versus goal tracking include variations in attentional performance which differentially depend on basal forebrain projection systems. Pavlovian Conditioned Approach (PCA) testing was used to identify male and female sign-trackers (STs) and goal-trackers (GTs), as well as rats with an intermediate phenotype (INTs). Upon reaching asymptotic performance in an operant task requiring the detection of visual signals (hits) as well as the reporting of signal absence for 40 min per session, GTs scored more hits than STs, and hit rates across all phenotypes correlated with PCA scores. STs missed relatively more signals than GTs specifically during the last 15 min of a session. Chemogenetic inhibition of the basal forebrain decreased hit rates in GTs but was without effect in STs. Moreover, the decrease in hits in GTs manifested solely during the last 15 min of a session. Transfection efficacy in the horizontal limb of the diagonal band (HDB), but not substantia innominate (SI) or nucleus basalis of Meynert (nbM), predicted the behavioral efficacy of chemogenetic inhibition in GTs. Furthermore, the total subregional transfection space, not transfection of just cholinergic neurons, correlated with performance effects. These results indicate that the cognitive-behavioral phenotype indexed by goal tracking, but not sign tracking, depends on activation of the basal forebrain-frontal cortical projection system and associated biases toward top-down or model-based performance.

Dextromethorphan Reduces Sign-Tracking but Not Goal-Tracking in Male Sprague-Dawley Rats

Sign-tracking is a well-known phenomenon in appetitive Pavlovian conditioning in which subjects approach the site of a conditioned stimulus (CS) associated with an appetitive unconditioned stimulus (US) even when the two are located separately. Control of sign-tracking may be important in rehabilitation from drug dependence to help ward off relapse. Recent studies have found success in using ketamine to reduce sign-tracking. In this study, we employed a similar but unscheduled drug, dextromethorphan (DXM), which affects many of the same molecular targets as ketamine, in an attempt to reduce sign-tracking in a standard paradigm. DXM was found to reduce sign-tracking at the doses examined in this study, while goal-tracking (approaching the site of the US rather than CS) was relatively unaffected. DXM offers advantages over ketamine in terms of use with patients and may have some utility in rehabilitation.

Transfer of incentive salience from a first-order alcohol cue to a novel second-order alcohol cue among individuals at risk for alcohol use disorder: electrophysiological evidence

BACKGROUND AND AIMS

In susceptible individuals, cues associated with drug use are theorized to take on incentive-motivational properties, including the ability to reinforce higher-order, drug-related associative learning. This study aimed to test this prediction among people varying in risk for alcohol use disorder.

DESIGN, SETTING AND PARTICIPANTS

Repeated-measures experiment with a measured individual difference variable at a University psychology laboratory in Missouri, USA. One hundred and six young adults (96 contributed complete data) were pre-selected to represent the upper and lower quartiles of self-reported sensitivity to alcohol's acute effects.

MEASUREMENTS

Participants completed a second-order Pavlovian conditioning paradigm in which an initially neutral visual cue (second-order conditional stimulus; CS₂ ) predicted onset of an olfactory cue (first-order conditional stimulus; CS₁ ). Olfactory cues were isolated from alcoholic beverages, sweets and non-comestible substances, each presumed to have a natural history of first-order conditioning. Event-related potential responses to the CS₂ across its conditioning and extinction, and to the CS₁ , provided neurophysiological indices of incentive salience (IS).

FINDINGS

The IS of the alcohol CS₁ was higher among participants low in alcohol sensitivity (LS), relative to their higher-sensitivity (HS) peers. The IS of the CS₂ paired with the alcohol CS₁ increased across the CS₂ conditioning phase among LS but not HS participants. Also, LS (but not HS) individuals also experienced increases in alcohol craving following alcohol CS₁ exposure, and this change was correlated with increases in the IS of the CS₂ paired with the alcohol CS₁ .

CONCLUSIONS

Alcoholic beverage odor, a proximal cue for alcohol consumption, appears to reinforce conditioning of neurophysiological responses to a novel cue among low alcohol sensitivity (LS) individuals but not high alcohol sensitivity individuals, providing the first evidence that the LS phenotype may be associated with differences in the conditioned reinforcing properties of alcohol-related cues. These findings support the idea that the LS phenotype may increase alcohol use disorder risk via susceptibility to incentive salience sensitization.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Effects of Limited and Extended Pavlovian Training on Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Individual differences in Pavlovian approach predict differences in devaluation sensitivity. Recent studies indicate goal-tracking (GT) rats are sensitive to outcome devaluation while sign-tracking (ST) rats are not. With extended training in Pavlovian lever autoshaping (PLA), GT rats display more lever-directed behavior, typical of ST rats, suggesting they may become insensitive to devaluation with more Pavlovian training experience. Here, we use a within-subject satiety-induced outcome devaluation procedure to test devaluation sensitivity after limited and extended PLA training in GT and ST rats. We trained rats in PLA to determine GT and ST groups. Then, we sated rats on either the training pellets (devalued condition) or homecage chow (valued condition) prior to brief non-reinforced test sessions after limited (sessions 5/6) and extended (sessions 17/18) PLA training. GT rats decreased conditioned responding under devalued relative to valued conditions after both limited and extended training, demonstrating they are sensitive to satiety devaluation regardless of the amount of PLA training. While ST rats were insensitive to satiety devaluation after limited training, their lever directed behavior became devaluation sensitive after extended training. To determine whether sign-tracking rats also displayed sensitivity to illness-induced outcome devaluation after extended training, we trained a separate cohort of rats in extended PLA and devalued the outcome with lithium chloride injections after pellet consumption in the homecage. ST rats failed to decrease conditioned responding after illness-induced outcome devaluation, while Non-ST rats (GT and intermediates) were sensitive to illness-induced outcome devaluation after extended training. Together, our results confirm devaluation sensitivity is stable in GT rats across training and devaluation approaches. Extended training unmasks devaluation sensitivity in ST rats after satiety, but not illness-induced devaluation, suggesting ST rats respond appropriately by decreasing responding to cues during state-dependent but not inference-based devaluation. The differences in behavioral flexibility across tracking groups and devaluation paradigms have translational relevance for the understanding state- vs. inference-based reward devaluation as it pertains to drug addiction vulnerability.

Repeated subcutaneous administration of PT150 has dose-dependent effects on sign tracking in male Japanese quail

A devastating feature of drug dependence is the susceptibility of relapse (40-60%) after stretches of abstinence. In both animal and human research, it has been demonstrated that cues (e.g., levers, paraphernalia) associated with drug reward can instigate renewed drug taking. Research has shown animals that attend to a cue that predicts reward more than the location of reward delivery when the cue is present (sign trackers) have an increase in corticosterone (CORT), a primary stress hormone when compared with animals that do not sign track. This interaction of sign tracking and CORT implicate CORT's effects as a possible pharmacological target for cue-induced relapse behaviors. PT150 is a novel glucocorticoid receptor antagonist that reduces the effects of CORT. Previous research has shown that oral administration of 40 mg/kg PT150 reduced sign tracking. To better understand dose-dependent effects and to control for more accurate doses, the current experiment hypothesized that PT150 (20/40/60 mg/kg) given by subcutaneous (SC) injection to male quail would reduce sign tracking to a keylight conditional stimulus that predicts a grain unconditioned stimulus dose dependently. Results showed that SC injection of 20 mg/kg PT150 reduced sign tracking, but 40 or 60 mg/kg did not. The main findings from the current study are that the glucocorticoid receptor antagonist PT150 reduces sign tracking behavior dose dependently, and SC administration may provide better bioavailability compared with our previous study that used an oral route of administration. The current findings support previous literature by suggesting that the glucocorticoid receptor may be a potential pharmacological target for reducing relapse-like behaviors. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

The paraventricular thalamus is a critical mediator of top-down control of cue-motivated behavior in rats

Cues in the environment can elicit complex emotional states, and thereby maladaptive behavior, as a function of their ascribed value. Here we capture individual variation in the propensity to attribute motivational value to reward-cues using the sign-tracker/goal-tracker animal model. Goal-trackers attribute predictive value to reward-cues, and sign-trackers attribute both predictive and incentive value. Using chemogenetics and microdialysis, we show that, in sign-trackers, stimulation of the neuronal pathway from the prelimbic cortex (PrL) to the paraventricular nucleus of the thalamus (PVT) decreases the incentive value of a reward-cue. In contrast, in goal-trackers, inhibition of the PrL-PVT pathway increases both the incentive value and dopamine levels in the nucleus accumbens shell. The PrL-PVT pathway, therefore, exerts top-down control over the dopamine-dependent process of incentive salience attribution. These results highlight PrL-PVT pathway as a potential target for treating psychopathologies associated with the attribution of excessive incentive value to reward-cues, including addiction.

The nature of phenotypic variation in Pavlovian conditioning

Pavlovian conditioning procedures result in dramatic individual differences in the topography of learnt behaviors in rats: When the temporary insertion of a lever into an operant chamber is paired with food pellets, some rats (known as sign-trackers) predominantly interact with the lever, while others (known as goal-trackers) predominantly approach the food well. Two experiments examined the sensitivity of these two behaviors to changing reinforcement contingencies in groups of male and female rats exhibiting the different phenotypes (i.e., sign-trackers and goal-trackers). In both phenotypes, behavior oriented to the food well was more sensitive to contingency changes (e.g., a reversal in which of two levers was reinforced) than was lever-oriented behavior. That is, the nature of the two behaviors differed independently of the rats in which they were manifest. These results indicate that the behavioral phenotypes reflect the parallel operation of a stimulus-stimulus associative process that gives rise to food-well activity and a stimulus-response process that gives rise to lever-oriented activity, rather than the operation of a single process (e.g., stimulus-stimulus) that generates both behaviors. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

The role of glutamate signaling in incentive salience: second-by-second glutamate recordings in awake Sprague-Dawley rats

The attribution of incentive salience to reward-predictive stimuli has been shown to be associated with substance abuse-like behavior such as increased drug taking. Evidence suggests that glutamate neurotransmission and sequential N-methyl-D-aspartate (NMDA) activation are involved in the attribution of incentive salience. Here, we further explore the role of second-by-second glutamate neurotransmission in the attribution of incentive salience to reward-predictive stimuli by measuring sign-tracking behavior during a Pavlovian conditioned approach procedure using ceramic-based microelectrode arrays configured for sensitive measures of extracellular glutamate in awake behaving Sprague-Dawley rats. Specifically, we show that there is an increase in extracellular glutamate levels in the prelimbic cortex (PrL) and the nucleus accumbens core (NAcC) during sign-tracking behavior to a food-predictive conditioned stimulus (CS+) compared to the presentation of a non-predictive conditioned stimulus (CS-). Furthermore, the results indicate greater increases in extracellular glutamate levels in the PrL compared to NAcC in response to the CS+, including differences in glutamate release and signal decay. Taken together, the present research suggests that there is differential glutamate signaling in the NAcC and PrL during sign-tracking behavior to a food-predictive CS+.

Unresponsive Choline Transporter as a Trait Neuromarker and a Causal Mediator of Bottom-Up Attentional Biases

Some rats [sign-trackers (STs)] are prone to attribute incentive salience to reward cues, which can manifest as a propensity to approach and contact pavlovian cues, and for addiction-like behavior. STs also exhibit poor attentional performance, relative to goal-trackers (GTs), which is associated with attenuated acetylcholine (ACh) levels in prefrontal cortex (Paolone et al., 2013). Here, we demonstrate a cellular mechanism, linked to ACh synthesis, that accounts for attenuated cholinergic capacity in STs. First, we found that electrical stimulation of the basal forebrain increased cortical choline transporter (CHT)-mediated choline transport in GTs, paralleled by a redistribution of CHTs to the synaptic plasma membrane. Neither increases in choline uptake nor translocation of CHTs occurred in STs. Second, and consistent with uptake/translocation alterations, STs demonstrated a reduced ability to support cortical ACh release in vivo compared with GTs after reverse-dialysis to elevate extracellular potassium levels. Third, rats were significantly more likely to develop sign-tracking behavior if treated systemically before pavlovian conditioned approach training with the CHT inhibitor VU6001221. Consistent with its proposed mechanisms, administration of VU6001221 attenuated potassium-evoked ACh levels in prefrontal cortex measured with in vivo microdialysis. We propose that loss of CHT-dependent activation of cortical cholinergic activity in STs degrades top-down executive control over behavior, producing a bias for bottom-up or stimulus-driven attention. Such an attentional bias contributes to nonadaptive reward processing and thus identifies a novel mechanism that can support psychopathology, including addiction.SIGNIFICANCE STATEMENT The vulnerability for addiction-like behavior has been associated with psychological traits, such as the propensity to attribute incentive salience to reward cues that is modeled in rats by sign-tracking behavior. Sign-trackers tend to approach and contact cues associated with reward, whereas their counterparts, the goal-trackers, have a preference for approaching the location of the reward. Here, we show that the capacity of presynaptic cholinergic synapses to respond to stimulation by elevating presynaptic choline uptake and releasing acetylcholine is attenuated in sign-trackers. Furthermore, pharmacological inhibition of choline transport induced sign-tracking behavior. Our findings suggest that reduced levels of cholinergic neuromodulation can mediate an attentional bias toward reward-related cues, thereby allowing such cues to exert relatively greater control over behavior.

Subanesthetic ketamine decreases the incentive-motivational value of reward-related cues

The attribution of incentive-motivational value to reward-related cues contributes to cue-induced craving and relapse in addicted patients. Recently, it was demonstrated that subanesthetic ketamine increases motivation to quit and decreases cue-induced craving in cocaine-dependent individuals. Although the underlying mechanism of this effect is currently unknown, one possibility is that subanesthetic ketamine decreases the incentive-motivational value of reward-related cues. In the present study, we used a Pavlovian conditioned approach procedure to identify sign-trackers, rats that attribute incentive-motivational value to reward-related cues, and goal-trackers, rats that assign only predictive value to reward-related cues. This model is of interest because sign-trackers are more vulnerable to cue-induced reinstatement of drug-seeking behavior and will persist in this drug-seeking behavior despite adverse consequences. We tested the effect of subanesthetic ketamine on the expression of Pavlovian conditioned approach behavior and the conditioned reinforcing properties of a reward-related cue in sign- and goal-trackers. We found that subanesthetic ketamine decreased sign-tracking and increased goal-tracking behavior in sign-trackers, though it had no effect on conditioned reinforcement. These results suggest that subanesthetic ketamine may be a promising pharmacotherapy for addiction that acts by decreasing the incentive-motivational value of reward-related cues.

Individual Differences in the Attribution of Incentive Salience to a Pavlovian Alcohol Cue

Individual differences exist in the attribution of incentive salience to conditioned stimuli associated with food. Here, we investigated whether individual differences also manifested with a Pavlovian alcohol conditioned stimulus (CS). We compiled data from five experiments that used a Pavlovian autoshaping paradigm and tests of conditioned reinforcement. In all experiments, male, Long-Evans rats with unrestricted access to food and water were acclimated to 15% ethanol. Next, rats received Pavlovian autoshaping training, in which a 10 s presentation of a retractable lever served as the CS and 0.2 mL of 15% ethanol served as the unconditioned stimulus (US). Finally, rats underwent conditioned reinforcement tests in which nose-pokes to an active aperture led to brief presentations of the lever-CS, but nose-pokes to an inactive aperture had no consequence. Rats were categorized as sign-trackers, goal-trackers and intermediates based on a response bias score that reflected their tendencies to sign-track or goal-track at different times during training. We found that distinct groups of rats either consistently interacted with the lever-CS (“sign-trackers”) or routinely approached the port during the lever-CS (“goal-trackers”) across a majority of the training sessions. However, some individuals (“shifted sign-trackers”) with an early tendency to goal-track later shifted to comparable asymptotic levels of sign-tracking as the group identified as sign-trackers. The lever-CS functioned as a conditioned reinforcer for sign-trackers and shifted sign-trackers, but not for goal-trackers. These results provide evidence of robust individual differences in the extent to which a Pavlovian alcohol cue gains incentive salience and functions as a conditioned reinforcer.

A food-predictive cue attributed with incentive salience engages subcortical afferents and efferents of the paraventricular nucleus of the thalamus

The paraventricular nucleus of the thalamus (PVT) has been implicated in behavioral responses to reward-associated cues. However, the precise role of the PVT in these behaviors has been difficult to ascertain since Pavlovian-conditioned cues can act as both predictive and incentive stimuli. The "sign-tracker/goal-tracker" rat model has allowed us to further elucidate the role of the PVT in cue-motivated behaviors, identifying this structure as a critical component of the neural circuitry underlying individual variation in the propensity to attribute incentive salience to reward cues. The current study assessed differences in the engagement of specific PVT afferents and efferents in response to presentation of a food-cue that had been attributed with only predictive value or with both predictive and incentive value. The retrograde tracer fluorogold (FG) was injected into the PVT or the nucleus accumbens (NAc) of rats, and cue-induced c-Fos in FG-labeled cells was quantified. Presentation of a predictive stimulus that had been attributed with incentive value elicited c-Fos in PVT afferents from the lateral hypothalamus, medial amygdala (MeA), and the prelimbic cortex (PrL), as well as posterior PVT efferents to the NAc. PVT afferents from the PrL also showed elevated c-Fos levels following presentation of a predictive stimulus alone. Thus, presentation of an incentive stimulus results in engagement of subcortical brain regions; supporting a role for the hypothalamic-thalamic-striatal axis, as well as the MeA, in mediating responses to incentive stimuli; whereas activity in the PrL to PVT pathway appears to play a role in processing the predictive qualities of reward-paired stimuli.

Chemogenetic manipulation of ventral pallidal neurons impairs acquisition of sign-tracking in rats

Cues associated with rewarding events acquire value themselves as a result of the incentive value of the reward being transferred to the cue. Consequently, presentation of a reward-paired cue can trigger reward-seeking behaviours towards the cue itself (i.e. sign-tracking). The ventral pallidum (VP) has been demonstrated to be involved in a number of motivated behaviours, both conditioned and unconditioned. However, its contribution to the acquisition of incentive value is unknown. Using a discriminative autoshaping procedure with levers, the effects of disrupting VP activity in rats on the emergence of sign-tracking was investigated using chemogenetics, i.e. Designer Receptors Exclusively Activated by Designer Drugs (DREADDs). Transient disruption of VP neurons [activation of the inhibitory hM4D(Gi) DREADD through systemic injections of clozapine N-oxide (CNO) prior to each autoshaping session] impaired acquisition of sign-tracking (lever press rate) without having any effect on approach to the site of reward delivery (i.e. goal-tracking) or on the expression of sign-tracking after it was acquired. In addition, electrophysiological recordings were conducted in freely behaving rats following VP DREADD activation. The majority of VP units that were responsive to CNO injections exhibited rapid inhibition relative to baseline, a subset of CNO-responsive units showed delayed excitation, and a smaller subset displayed a mixed response of inhibition and excitation following CNO injections. It is argued that disruption of VP during autoshaping specifically disrupted the transfer of incentive value that was attributed to the lever cue, suggesting a surprisingly fundamental role for the VP in acquiring, compared with expressing, Pavlovian incentive values.

Sign-trackers have elevated myo-inositol in the nucleus accumbens and ventral hippocampus following Pavlovian conditioned approach

Pavlovian conditioned approach (PCA) is a behavioral procedure that can be used to assess individual differences in the addiction vulnerability of drug-naïve rats and identify addiction vulnerability factors. Using proton magnetic resonance spectroscopy (¹ H-MRS) ex vivo, we simultaneously analyzed concentrations of multiple neurochemicals throughout the mesocorticolimbic system 2 weeks after PCA training in order to identify potential vulnerability factors to addiction in drug-naïve rats for future investigations. Levels of myo-inositol (Ins), a ¹ H-MRS-detectable marker of glial activity/proliferation, were increased in the nucleus accumbens (NAc) and ventral hippocampus, but not dorsal hippocampus or medial prefrontal cortex, of sign-trackers compared to goal-trackers or intermediate responders. In addition, Ins levels positively correlated with PCA behavior in the NAc and ventral hippocampus. Because the sign-tracker phenotype is associated with increased drug-seeking behavior, these results observed in drug-naïve rats suggest that alterations in glial activity/proliferation within these regions may represent an addiction vulnerability factor. Sign-tracking rats preferentially approach reward cues during Pavlovian conditioning, while goal-trackers instead approach the location of impending reward. Sign-trackers are also more prone to cue-induced drug-seeking behavior. We used magnetic resonance spectroscopy to show that myo-inositol levels are higher in the ventral hippocampus and nucleus accumbens of sign-trackers relative to goal-trackers. Thus, elevated myo-inositol may be a vulnerability factor for addiction.

Individual variation in incentive salience attribution and accumbens dopamine transporter expression and function

Cues (conditioned stimuli; CSs) associated with rewards can come to motivate behavior, but there is considerable individual variation in their ability to do so. For example, a lever-CS that predicts food reward becomes attractive and wanted, and elicits reward-seeking behavior, to a greater extent in some rats ('sign-trackers'; STs) than others ('goal-trackers'; GTs). Variation in dopamine (DA) neurotransmission in the nucleus accumbens (NAc) core is thought to contribute to such individual variation. Given that the DA transporter (DAT) exerts powerful regulation over DA signaling, we characterized the expression and function of the DAT in the accumbens of STs and GTs. STs showed greater DAT surface expression in ventral striatal synaptosomes than GTs, and ex vivo fast-scan cyclic voltammetry recordings of electrically evoked DA release confirmed enhanced DAT function in STs, as indicated by faster DA uptake, specifically in the NAc core. Consistent with this, systemic amphetamine (AMPH) produced greater inhibition of DA uptake in STs than in GTs. Furthermore, injection of AMPH directly into the NAc core enhanced lever-directed approach in STs, presumably by amplifying the incentive value of the CS, but had no effect on goal-tracking behavior. On the other hand, there were no differences between STs and GTs in electrically-evoked DA release in slices, or in total ventral striatal DA content. We conclude that greater DAT surface expression may facilitate the attribution of incentive salience to discrete reward cues. Investigating this variability in animal sub-populations may help explain why some people abuse drugs while others do not.

Individual variability in behavioral flexibility predicts sign-tracking tendency

Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS-) and second-order (SOCS+, SOCS-) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.

Individual variation in the motivational properties of a nicotine cue: sign-trackers vs. goal-trackers

RATIONALE

Individuals vary in the extent to which they attribute incentive salience to reward cues. Discrete food and drug (cocaine and opioid) cues become more attractive, eliciting approach toward them, and more "wanted," in that they serve as more effective conditioned reinforcers, in some rats (sign-trackers, STs), than in others (goal-trackers, GTs).

OBJECTIVES

We asked whether there is similar variation in the extent to which a cue associated with a drug from another class, nicotine, acquires incentive motivational properties.

METHODS

First, a Pavlovian conditioned approach procedure was used to identify rats that attribute incentive salience to a food cue (i.e., STs and GTs). We then measured the extent to which a cue (a light) paired with intravenous nicotine injections acquired two properties of an incentive stimulus: (1) the ability to elicit approach toward it, and (2) the ability to act as a conditioned reinforcer.

RESULTS

In contrast to previous findings with food, cocaine, and opioid cues, we found that the nicotine cue was equally attractive in STs and GTs, eliciting dose-dependent approach behavior in both. However, the nicotine cue was a more effective conditioned reinforcer in STs than in GTs.

CONCLUSIONS

We suggest the dissociation between these two measures of incentive salience attribution may be related to the fact that when present (as in the test of Pavlovian approach), nicotine can act as a potent "incentive amplifier," and by this action, nicotine may render cues especially salient for all animals.

Drug versus sweet reward: greater attraction to and preference for sweet versus drug cues

Despite the unique ability of addictive drugs to directly activate brain reward circuits, recent evidence suggests that drugs induce reinforcing and incentive effects that are comparable to, or even lower than some nondrug rewards. In particular, when rats have a choice between pressing a lever associated with intravenous cocaine or heroin delivery and another lever associated with sweet water delivery, most respond on the latter. This outcome suggests that sweet water is more reinforcing and attractive than either drug. However, this outcome may also be due to the differential ability of sweet versus drug levers to elicit Pavlovian feeding-like conditioned responses that can cause involuntary lever pressing, such as pawing and biting the lever. To test this hypothesis, rats first underwent Pavlovian conditioning to associate one lever with sweet water (0.2% saccharin) and a different lever with intravenous cocaine (0.25 mg) or heroin (0.01 mg). Choice between these two levers was then assessed under two operant choice procedures: one that permitted the expression of Pavlovian-conditioned lever press responses during choice, the other not. During conditioning, Pavlovian-conditioned lever press responses were considerably higher on the sweet lever than on either drug lever, and slightly greater on the heroin lever than on the cocaine lever. Importantly, though these differences in Pavlovian-conditioned behavior predicted subsequent preference for sweet water during choice, they were not required for its expression. Overall, this study confirms that rats prefer the sweet lever because sweet water is more reinforcing and attractive than cocaine or heroin.

Blocking NMDA-receptors in the pigeon's "prefrontal" caudal nidopallium impairs appetitive extinction learning in a sign-tracking paradigm

Extinction learning provides the ability to flexibly adapt to new contingencies by learning to inhibit previously acquired associations in a context-dependent manner. The neural networks underlying extinction learning were mostly studied in rodents using fear extinction paradigms. To uncover invariant properties of the neural basis of extinction learning, we employ pigeons as a model system. Since the prefrontal cortex (PFC) of mammals is a key structure for extinction learning, we assessed the role of N-methyl-D-aspartate receptors (NMDARs) in the nidopallium caudolaterale (NCL), the avian functional equivalent of mammalian PFC. Since NMDARs in PFC have been shown to be relevant for extinction learning, we locally antagonized NMDARs through 2-Amino-5-phosphonovalerianacid (APV) during extinction learning in a within-subject sign-tracking ABA-renewal paradigm. APV-injection slowed down extinction learning and in addition also caused a disinhibition of responding to a continuously reinforced control stimulus. In subsequent retrieval sessions, spontaneous recovery was increased while ABA renewal was unaffected. The effect of APV resembles that observed in studies of fear extinction with rodents, suggesting common neural substrates of extinction under both appetitive and aversive conditions and highlighting the similarity of mammalian PFC and the avian caudal nidopallium despite 300 million years of independent evolution.

A potential role for the paraventricular nucleus of the thalamus in mediating individual variation in Pavlovian conditioned responses

There is ample evidence to suggest that the paraventricular nucleus of the thalamus (PVT) mediates cue-reward learning, especially as it relates to drug-seeking behavior. However, its exact role in these complex processes remains unknown. Here we will present and discuss data from our own laboratory which suggests that the PVT plays a role in multiple forms of stimulus-reward learning, and does so via distinct neurobiological systems. Using an animal model that captures individual variation in response to reward-associated cues, we are able to parse the incentive from the predictive properties of reward cues and to elucidate the neural circuitry underlying these different forms of cue-reward learning. When rats are exposed to a classical Pavlovian conditioning paradigm, wherein a cue predicts food reward, some rats, termed sign-trackers, approach and manipulate the cue upon its presentation. This behavior is indicative of attributing incentive salience to the cue. That is, the cue gains excessive control over behavior for sign-trackers. In contrast, other rats, termed goal-trackers, treat the cue as a mere predictor, and upon its presentation go to the location of reward delivery. Based on our own data utilizing this model, we hypothesize that the PVT represents a common node, but differentially regulates the sign- vs. goal-tracking response. We postulate that the PVT regulates sign-tracking behavior, or the attribution of incentive salience, via subcortical, dopamine-dependent mechanisms. In contrast, we propose that goal-tracking behavior, or the attribution of predictive value, is the product of “top-down” glutamatergic processing between the prelimbic cortex (PrL) and the PVT. Together, data from our laboratory and others support a role for the PVT in cue-motivated behaviors and suggest that it may be an important locus within the neural circuitry that goes awry in addiction and related disorders.

The habenula governs the attribution of incentive salience to reward predictive cues

The attribution of incentive salience to reward associated cues is critical for motivation and the pursuit of rewards. Disruptions in the integrity of the neural systems controlling these processes can lead to avolition and anhedonia, symptoms that cross the diagnostic boundaries of many neuropsychiatric illnesses. Here, we consider whether the habenula (Hb), a region recently demonstrated to encode negatively valenced events, also modulates the attribution of incentive salience to a neutral cue predicting a food reward. The Pavlovian autoshaping paradigm was used in the rat as an investigative tool to dissociate Pavlovian learning processes imparting strictly predictive value from learning that attributes incentive motivational value. Electrolytic lesions of the fasciculus retroflexus (fr), the sole pathway through which descending Hb efferents are conveyed, significantly increased incentive salience as measured by conditioned approaches to a cue light predictive of reward. Conversely, generation of a fictive Hb signal via fr stimulation during CS+ presentation significantly decreased the incentive salience of the predictive cue. Neither manipulation altered the reward predictive value of the cue as measured by conditioned approach to the food. Our results provide new evidence supporting a significant role for the Hb in governing the attribution of incentive motivational salience to reward predictive cues and further imply that pathological changes in Hb activity could contribute to the aberrant pursuit of debilitating goals or avolition and depression-like symptoms.

Predicting extinction and reinstatement of alcohol and sucrose self-administration in outbred rats

Preventing relapse to drinking or escalation to excessive drinking could be aided by identifying factors that predict these behaviors. Animal models, particularly those that utilize operant self-administration techniques, can be useful. In a prior operant study, we noted a good deal of variability in behaviors during training and test sessions. We utilized data obtained from that study of two groups of rats, trained and tested identically except one responded for alcohol and the other for sucrose, to explore for associations related to relapse (reinstatement) or to excessive drinking (maintenance). Data were obtained from sessions conducted under fixed- and progressive-ratio schedules as well as from extinction and reinstatement sessions. Variables assessed included active and inactive presses, head entries into the dipper trough, and automated recordings of body movements during these sessions as well as alcohol preference before training. First, using multiple regression, we examined whether alcohol preference before training associated with any response variable among alcohol-responding rats. Second, using factor analysis, we identified a training variable, body movements, that associated with responses during tests. Using this measure, rats were divided into low- and high-response groups and compared on active lever presses and head entries across test sessions. Results show that among alcohol-responding rats, alcohol preference predicted head entries during extinction. High-body-movement rats emitted significantly fewer active lever presses and had fewer head entries across test sessions, particularly during reinstatement, compared with low-body-movement rats. Results from this exploratory study provide clues for future experimental studies.

A classically conditioned cocaine cue acquires greater control over motivated behavior in rats prone to attribute incentive salience to a food cue

RATIONALE

Cues associated with rewards bias attention towards them and can motivate drug-seeking and drug-taking behavior. There is, however, considerable individual variation in the extent to which cues associated with rewards acquire motivational properties. For example, only in some rats does a localizable food cue become attractive, eliciting approach towards it, and "wanted", in the sense that it serves as an effective conditioned reinforcer.

OBJECTIVES

We asked whether the propensity of animals to attribute incentive salience to a food cue predicts the extent to which a classically conditioned cocaine cue acquires incentive motivational properties.

METHODS

First, a Pavlovian conditioned approach procedure was used to identify rats prone to attribute incentive salience to a food cue. We then measured the extent to which a classically conditioned cocaine cue acquired two properties of an incentive stimulus: (1) the ability to elicit approach towards it, and (2) the ability to reinstate drug-seeking behavior, using an extinction-reinstatement procedure (i.e., to act as a conditioned reinforcer).

RESULTS

We found that a classically conditioned cocaine cue became more attractive, in that it elicited greater approach toward it, and more desired, in that it supported more drug-seeking behavior under extinction conditions, in individuals prone to attribute incentive salience to a food cue.

CONCLUSIONS

We conclude that rats vary in their propensity to attribute incentive salience to both food and cocaine cues, and it is possible to predict, prior to any drug experience, in which rats a cocaine cue will acquire the strongest motivational control over behavior.

A cocaine cue is more preferred and evokes more frequency-modulated 50-kHz ultrasonic vocalizations in rats prone to attribute incentive salience to a food cue

RATIONALE

Individuals vary considerably in the extent to which they attribute incentive salience to food-associated cues.

OBJECTIVES

We asked whether individuals prone to attribute incentive salience to a food cue are also prone to attribute incentive properties to a stimulus associated with a drug of abuse-cocaine.

METHODS

We first identified those rats that attributed incentive salience to a food cue by quantifying the extent to which they came to approach and engage a food cue. We then used a conditioned place preference procedure to pair an injection of 10 mg/kg cocaine (i.p.) with one distinct floor texture (grid or holes) and saline with another. Following 8 days of conditioning, each rat was given a saline injection and placed into a chamber that had both floors present. We measured the time spent on each floor, and also 50-kHz ultrasonic vocalizations, which have been associated with positive affective states.

RESULTS

Rats that vigorously engaged the food cue ("sign trackers") expressed a preference for the cocaine-paired floor compared to those that did not ("goal trackers"). In addition, sign trackers made substantially more frequency-modulated 50-kHz vocalizations when injected with cocaine and when later exposed to the cocaine cue.

CONCLUSIONS

Rats prone to attribute incentive salience to a food cue are also prone to attribute incentive motivational properties to a tactile cue associated with cocaine. We suggest that individuals prone to attribute incentive salience to reward cues will have difficulty resisting them and, therefore, may be especially vulnerable to develop impulse control disorders, including addiction.

A cocaine cue acts as an incentive stimulus in some but not others: implications for addiction

BACKGROUND

In addicts drug cues attract attention, elicit approach, and motivate drug-seeking and drug-taking behavior, and addicts find it difficult to resist such cues. In preclinical studies we have found, however, that food cues acquire incentive motivational properties only in a subset of individuals. For example, a food cue becomes attractive, eliciting approach and engagement with it, and acts as an effective conditional reinforcer in some rats but not others. We asked, therefore, whether rats that have a propensity to attribute incentive salience to a food cue are the same ones that attribute incentive value to a drug (cocaine) cue.

METHODS

We first used a Pavlovian conditioned approach procedure to determine which individual rats attributed incentive salience to a food cue. A second cue was then associated with the IV self-administration of cocaine. Later, the ability of the cocaine cue to maintain self-administration behavior and to reinstate self-administration after extinction was assessed.

RESULTS

We report that in individuals that had a propensity to attribute incentive salience to a food cue, a cocaine cue spurred motivation to take drugs (its removal greatly diminished self-administration) and reinstated robust drug-seeking after extinction. However, in those individuals that failed to attribute incentive salience to a food cue, the cocaine cue was relatively devoid of incentive motivational properties.

CONCLUSIONS

We conclude that it is possible to determine, before any drug experience, which individuals will most likely have difficulty resisting drug cues, a trait that might confer susceptibility to addiction.

Dissociating the predictive and incentive motivational properties of reward-related cues through the study of individual differences

BACKGROUND

If presentation of a stimulus (conditional stimulus, CS) reliably predicts delivery of a reward, the CS will come to evoke a conditional response (CR) through Pavlovian learning, and the CS may also acquire incentive motivational properties. Thus, CSs can have both predictive and incentive properties. We ask here whether it is possible to dissociate the predictive versus incentive properties of a CS in rats by considering individual differences in the nature of the CR.

METHODS

We used Pavlovian procedures to study the ability of a localizable CS (an illuminated lever) to acquire two properties of an incentive stimulus-the ability to attract and the ability to act as a conditional reinforcer.

RESULTS

For some rats, the CS evoked a "sign-tracking" CR, consisting of approach toward and engagement with the CS itself. For other rats, the CS instead produced a "goal-tracking" CR: approach was directed away from the CS toward the site of food delivery. For sign-trackers (but not goal-trackers) the CS also acted as an effective conditional reinforcer.

CONCLUSIONS

The predictive and incentive properties of a CS can be dissociated by considering individual differences in the CR. In a given animal, a cue that is predictive of reward, supporting Pavlovian learning, may or may not be attributed with incentive salience. This procedure may provide a powerful means to test hypotheses regarding the role of neural systems in learning versus incentive motivational functions and to study individual variation in the extent to which reward-associated stimuli act as incentive stimuli.

Effects of prior amphetamine exposure on approach strategy in appetitive Pavlovian conditioning in rats

RATIONALE

Pavlovian conditioning with a discrete reward-predictive visual cue can elicit two classes of behaviors: "sign-tracking" (approach toward and contact with the cue) and "goal-tracking" (approach toward the site of reward delivery). Sign-tracking has been proposed to be linked to behavioral disorders involving compulsive reward-seeking, such as addiction. Prior exposure to psychostimulant drugs of abuse can facilitate reward-seeking behaviors through enhancements in incentive salience attribution. Thus, it was predicted that a sensitizing regimen of amphetamine exposure would increase sign-tracking behavior.

OBJECTIVE

The purpose of these experiments was to determine how a regimen of exposure to amphetamine affects subsequent sign-tracking behavior.

MATERIALS AND METHODS

Male Long-Evans rats were given daily injections of d-amphetamine (2.0 mg/kg) or saline for 5 days, then given a 7-day drug-free period followed by testing in a Pavlovian conditioning task. In experiment 1, rats were presented with a visual cue (simultaneous illumination of a light and extension of a lever) located either to the left or right of a centrally located food trough. One cue (CS+) was always followed by food delivery, whereas the other (CS-) was not. In experiment 2, rats were tested in a nondiscriminative (CS+ only) version of the task.

RESULTS

In both experiments, amphetamine-exposed rats showed less sign-tracking and more goal-tracking compared to saline controls.

CONCLUSIONS

Contrary to predictions, prior amphetamine exposure decreased sign-tracking and increased goal-tracking behavior. However, these results do support the hypothesis that psychostimulant exposure and incentive sensitization enhance behavior directed toward reward-proximal cues at the expense of reward-distal cues.

Individual differences in the attribution of incentive salience to reward-related cues: Implications for addiction

Drugs of abuse acquire different degrees of control over thoughts and actions based not only on the effects of drugs themselves, but also on predispositions of the individual. Those individuals who become addicted are unable to shift their thoughts and actions away from drugs and drug-associated stimuli. Thus in addicts, exposure to places or things (cues) that has been previously associated with drug-taking often instigates renewed drug-taking. We and others have postulated that drug-associated cues acquire the ability to maintain and instigate drug-taking behavior in part because they acquire incentive motivational properties through Pavlovian (stimulus-stimulus) learning. In the case of compulsive behavioral disorders, including addiction, such cues may be attributed with pathological incentive value ("incentive salience"). For this reason, we have recently begun to explore individual differences in the tendency to attribute incentive salience to cues that predict rewards. When discrete cues are associated with the non-contingent delivery of food or drug rewards some animals come to quickly approach and engage the cue even if it is located at a distance from where the reward will be delivered. In these animals the reward-predictive cue itself becomes attractive, eliciting approach towards it, presumably because it is attributed with incentive salience. Animals that develop this type of conditional response are called "sign-trackers". Other animals, "goal-trackers", do not approach the reward-predictive cue, but upon cue presentation they immediately go to the location where food will be delivered (the "goal"). For goal-trackers the reward-predictive cue is not attractive, presumably because it is not attributed with incentive salience. We review here preliminary data suggesting that these individual differences in the tendency to attribute incentive salience to cues predictive of reward may confer vulnerability or resistance to compulsive behavioral disorders, including addiction. It will be important, therefore, to study how environmental, neurobiological and genetic interactions determine the extent to which individuals attribute incentive value to reward-predictive stimuli.

Behavioral characteristics and neurobiological substrates shared by Pavlovian sign-tracking and drug abuse

Drug abuse researchers have noted striking similarities between behaviors elicited by Pavlovian sign-tracking procedures and prominent symptoms of drug abuse. In Pavlovian sign-tracking procedures, repeated paired presentations of a small object (conditioned stimulus, CS) with a reward (unconditioned stimulus, US) elicits a conditioned response (CR) that typically consists of approaching the CS, contacting the CS, and expressing consummatory responses at the CS. Sign-tracking CR performance is poorly controlled and exhibits spontaneous recovery and long-term retention, effects that resemble relapse. Sign-tracking resembles psychomotor activation, a syndrome of behavioral responses evoked by addictive drugs, and the effects of sign-tracking on corticosterone levels and activation of dopamine pathways resemble the neurobiological effects of abused drugs. Finally, the neurobiological profile of individuals susceptible to sign-tracking resembles the pathophysiological profile of vulnerability to drug abuse, and vulnerability to sign-tracking predicts vulnerability to impulsive responding and alcohol self-administration. Implications of sign-tracking for models of drug addiction are considered.

Individual differences in the attribution of incentive salience to a reward-related cue: influence on cocaine sensitization

When a discrete cue (a "sign") is presented repeatedly in anticipation of a food reward the cue can become imbued with incentive salience, leading some animals to approach and engage it, a phenomenon known as "sign-tracking" (the animals are sign-trackers; STs). In contrast, other animals do not approach the cue, but upon cue presentation go to the location where food will be delivered (the goal). These animals are known as goal-trackers (GTs). It has been hypothesized that individuals who attribute excessive incentive salience to reward-related cues may be especially vulnerable to develop compulsive behavioral disorders, including addiction. We were interested, therefore, in whether individual differences in the propensity to sign-track are associated with differences in responsivity to cocaine. Using an autoshaping procedure in which lever (conditioned stimulus) presentation was immediately followed by the response-independent delivery of a food pellet (unconditioned stimulus), rats were first characterized as STs or GTs and subsequently studied for the acute psychomotor response to cocaine and the propensity for cocaine-induced psychomotor sensitization. We found that GTs were more sensitive than STs to the acute locomotor activating effects of cocaine, but STs showed a greater propensity for psychomotor sensitization upon repeated treatment. These data suggest that individual differences in the tendency to attribute incentive salience to a discrete reward-related cue, and to approach and engage it, are associated with susceptibility to a form of cocaine-induced plasticity that may contribute to the development of addiction.