Nucleus accumbens dopamine depletion impairs both acquisition and performance of appetitive Pavlovian approach behaviour: implications for mesoaccumbens dopamine function

Role of nucleus accumbens D1-type medium spiny neurons in the expression and extinction of sign-tracking

While sign-tracking, also known as autoshaping, has been studied for many decades, only recently has the tendency to show sign-tracking behavior been linked to the development and persistence of addiction. Sign-tracking is dependent upon dopamine activity in the nucleus accumbens (NAc). The NAc is comprised predominantly of medium spiny projection neurons (MSN) that can be differentiated by their D1-like or D2-like dopamine receptor expression. Here we determined how reducing activity of D1-type MSNs in the NAc affects the expression and extinction of sign-tracking. To address this, we transfected the NAc of transgenic male and female rats that selectively express Cre recombinase in D1-type MSNs with a DIO viral vector expressing hM4Di. Cre- rats were given the same viral infusion but did not express the hM4Di receptor and therefore served as controls. Rats were then conditioned to associate lever presentations with pellet delivery. After sign-tracking was established, all rats were administered clozapine-n-oxide (CNO) prior to three additional conditioning sessions to assess the effects of NAc D1-MSNs inhibition on sign-tracking in the presence of reward. CNO treatment did not alter the expression of sign-tracking in Cre+ or Cre- rats. Next rats underwent extinction training where lever presentations occurred without pellet delivery and all rats received a CNO injection prior to each extinction session. In these extinction conditions, Cre+ rats exhibited robust extinction of sign-tracking across sessions, whereas Cre- rats did not. To determine if D1-MSN inhibition merely produced a temporary cessation of sign-tracking or instead had facilitated a persistent loss of sign-tracking, we evaluated the reemergence of sign-tracking in a test for reconditioning. During testing, reintroduction of the CS-US pairing did not promote the reemergence of sign-tracking in Cre+ rats, but restored sign-tracking in Cre- rats. Thus, chemogenetic inhibition of NAc D1-MSNs promoted extinction of sign-tracking. Collectively, these data suggest that D1-MSNs play an important role in resistance to extinction that typifies sign-tracking behavior.

From safety to frustration: The neural substrates of inhibitory learning in aversive and appetitive conditioning procedures

Inhibitory associative learning counters the effects of excitatory learning, whether appetitively or aversively motivated. Moreover, the affective responses accompanying the inhibitory associations are of opponent valence to the excitatory conditioned responses. Inhibitors for negative aversive outcomes (e.g. shock) signal safety, while inhibitors for appetitive outcomes (e.g. food reward) elicit frustration and/or disappointment. This raises the question as to whether studies using appetitive and aversive conditioning procedures should demonstrate the same neural substrates for inhibitory learning. We review the neural substrates of appetitive and aversive inhibitory learning as measured in different procedural variants and in the context of the underpinning excitatory conditioning on which it depends. The mesocorticolimbic dopamine pathways, retrosplenial cortex and hippocampus are consistently implicated in inhibitory learning. Further neural substrates identified in some procedural variants may be related to the specific motivation of the learning task and modalities of the learning cues. Finally, we consider the translational implications of our understanding of the neural substrates of inhibitory learning, for obesity and addictions as well as for anxiety disorders.

Chronic methamphetamine exposure exerts few effects on the iTat mouse model of HIV, but blocks Tat expression-induced slowed reward retrieval

Human immunodeficiency virus (HIV) continues to infect millions worldwide, negatively impacting neurobehavioral function. Further understanding of the combined effects of HIV and methamphetamine use is crucial, as methamphetamine use is prevalent in people with HIV. The HIV-associated protein Tat may contribute to cognitive dysfunction, modeled preclinically in mice using doxycycline (DOX)-inducible Tat expression (iTat). Tat may exert its effects on cognitive function via disruption of the dopamine transporter, similar to the action of methamphetamine. Additionally, Tat and methamphetamine both decrease interneuron populations, including those expressing calbindin. It is important to understand the combined effects of Tat and methamphetamine in preclinical models of HIV infection. Here, we used iTat transgenic mice and a chronic binge regimen of methamphetamine exposure to determine their combined impact on reward learning and motivation. We also measured calbindin expression in behavior-relevant brain regions. Before induction with DOX, iTat mice exhibited no differences in behavior. Chronic methamphetamine exposure before Tat induction impaired initial reward learning but did not affect motivation. Furthermore, DOX-induced Tat expression did not alter behavior, but slowed latencies to retrieve rewards. This effect of Tat, however, was not observed in methamphetamine-treated mice, indicative of a potential protective effect. Finally, Tat expression was associated with an increase in calbindin-expressing cells in the VTA, while methamphetamine exposure did not alter calbindin numbers. These findings may indicate a protective role of methamphetamine in HIV neuropathology, which in turn may help in our understanding of why people with HIV use methamphetamine at disproportionately higher rates.

Continuous cholinergic-dopaminergic updating in the nucleus accumbens underlies approaches to reward-predicting cues

The ability to learn Pavlovian associations from environmental cues predicting positive outcomes is critical for survival, motivating adaptive behaviours. This cued-motivated behaviour depends on the nucleus accumbens (NAc). NAc output activity mediated by spiny projecting neurons (SPNs) is regulated by dopamine, but also by cholinergic interneurons (CINs), which can release acetylcholine and glutamate via the activity of the vesicular acetylcholine transporter (VAChT) or the vesicular glutamate transporter (VGLUT3), respectively. Here we investigated behavioural and neurochemical changes in mice performing a touchscreen Pavlovian approach task by recording dopamine, acetylcholine, and calcium dynamics from D1- and D2-SPNs using fibre photometry in control, VAChT or VGLUT3 mutant mice to understand how these signals cooperate in the service of approach behaviours toward reward-predicting cues. We reveal that NAc acetylcholine-dopaminergic signalling is continuously updated to regulate striatal output underlying the acquisition of Pavlovian approach learning toward reward-predicting cues.

Anticipation: An Essential Feature of Anhedonia

The following essay addresses the evolution of the term "anhedonia" as a key construct in biological psychiatry, especially as it pertains to positive emotional and motivational states central to mental health and well-being. In its strictest definition, anhedonia was intended to convey an inability to experience "pleasure" derived from ingestion of sweet tastes or the experience of pleasant odors and tactile sensations, among a host of positive sensations. However, this definition has proved to be too restrictive to capture the complexity of key psychological factors linked to major depression, schizophrenia, and substance use disorders it was originally intended to address. Despite the appeal of the elegant simplicity of the term anhedonia, its limitations soon became apparent when used to explain psychological constructs including aspects of learning, memory, and incentive motivation that are major determinants of success in securing the necessities of life. Accordingly, the definition of anhedonia has morphed into a much broader term that includes key roles in the disturbance of motivation in the form of anergia, impaired incentive motivation, along with deficits in associative learning and key aspects of memory, on which the ability to predict the consequences of one's actions are based. Here we argue that it is this latter capacity, namely predicting the likely consequences of motivated behavior, which can be termed "anticipation," that is especially important in the key deficits implied by the general term anhedonia in the context of neuropsychiatric conditions.

Reinforcement-learning in fronto-striatal circuits

We review the current state of knowledge on the computational and neural mechanisms of reinforcement-learning with a particular focus on fronto-striatal circuits. We divide the literature in this area into five broad research themes: the target of the learning-whether it be learning about the value of stimuli or about the value of actions; the nature and complexity of the algorithm used to drive the learning and inference process; how learned values get converted into choices and associated actions; the nature of state representations, and of other cognitive machinery that support the implementation of various reinforcement-learning operations. An emerging fifth area focuses on how the brain allocates or arbitrates control over different reinforcement-learning sub-systems or "experts". We will outline what is known about the role of the prefrontal cortex and striatum in implementing each of these functions. We then conclude by arguing that it will be necessary to build bridges from algorithmic level descriptions of computational reinforcement-learning to implementational level models to better understand how reinforcement-learning emerges from multiple distributed neural networks in the brain.

Adolescent Dopamine Neurons Represent Reward Differently during Action and State Guided Learning

Neuronal underpinning of learning cause-and-effect associations in the adolescent brain remains poorly understood. Two fundamental forms of associative learning are Pavlovian (classical) conditioning, where a stimulus is followed by an outcome, and operant (instrumental) conditioning, where outcome is contingent on action execution. Both forms of learning, when associated with a rewarding outcome, rely on midbrain dopamine neurons in the ventral tegmental area (VTA) and substantia nigra (SN). We find that, in adolescent male rats, reward-guided associative learning is encoded differently by midbrain dopamine neurons in each conditioning paradigm. Whereas simultaneously recorded VTA and SN adult neurons have a similar phasic response to reward delivery during both forms of conditioning, adolescent neurons display a muted reward response during operant but a profoundly larger reward response during Pavlovian conditioning. These results suggest that adolescent neurons assign a different value to reward when it is not gated by action. The learning rate of adolescents and adults during both forms of conditioning was similar, supporting the notion that differences in reward response in each paradigm may be because of differences in motivation and independent of state versus action value learning. Static characteristics of dopamine neurons, such as dopamine cell number and size, were similar in the VTA and SN of both ages, but there were age-related differences in stimulated dopamine release and correlated spike activity, suggesting that differences in reward responsiveness by adolescent dopamine neurons are not because of differences in intrinsic properties of these neurons but engagement of different dopaminergic networks.SIGNIFICANCE STATEMENT Reckless behavior and impulsive decision-making by adolescents suggest that motivated behavioral states are encoded differently by the adolescent brain. Motivated behavior, which is dependent on the function of the dopamine system, follows learning of cause-and-effect associations in the environment. We find that dopamine neurons in adolescents encode reward differently depending on the cause-and-effect relationship of the means to receive that reward. Compared with adults, reward contingent on action led to a muted response, whereas reward that followed a cue but was not gated by action produced an augmented phasic response. These data demonstrate an age-related difference in dopamine neuron response to reward that is not uniform and is guided by processes that differentiate between state and action values.

Endocannabinoid Modulation of Nucleus Accumbens Microcircuitry and Terminal Dopamine Release

The nucleus accumbens (NAc) is located in the ventromedial portion of the striatum and is vital to valence-based predictions and motivated action. The neural architecture of the NAc allows for complex interactions between various cell types that filter incoming and outgoing information. Dopamine (DA) input serves a crucial role in modulating NAc function, but the mechanisms that control terminal DA release and its effect on NAc neurons continues to be elucidated. The endocannabinoid (eCB) system has emerged as an important filter of neural circuitry within the NAc that locally shapes terminal DA release through various cell type- and site-specific actions. Here, we will discuss how eCB signaling modulates terminal DA release by shaping the activity patterns of NAc neurons and their afferent inputs. We then discuss recent technological advancements that are capable of dissecting how distinct cell types, their afferent projections, and local neuromodulators influence valence-based actions.

Sign tracking predicts suboptimal behavior in a rodent gambling task

RATIONALE

Reward-associated cues can promote maladaptive behavior, including risky decision-making in a gambling setting. A propensity for sign tracking over goal tracking-i.e., interaction with a reward-predictive cue rather than the site of reward-demonstrates an individual's tendency to transfer motivational value to a cue. However, the relationship of sign tracking to risky decision-making remains unclear.

OBJECTIVES

To determine whether sign tracking predicts risky choice, we used a Pavlovian conditioned approach task to evaluate the tendency of male rats to sign track to a lever cue and then trained rats on a rodent gambling task (rGT) with win-associated cues. We also tested the effects of D-amphetamine, quinpirole (a D₂/D₃ receptor agonist), and PD128907 (a D₃ receptor agonist) on gambling behavior in sign tracker and goal tracker individuals.

RESULTS

Increased sign tracking relative to goal tracking was associated with suboptimal performance on the rGT, including decreased selection of the optimal choice, increased selection of a high-risk/high-reward option, and increased impulsive premature choices. Amphetamine increased choices of a low-risk/low-reward option at the expense of optimal and high-risk choices, whereas quinpirole and PD128907 had little effect on choice allocation, but reduced impulsivity. Drug effects were similar across sign tracker and goal tracker individuals.

CONCLUSIONS

Cue reactivity, as measured by sign tracking, is predictive and may be an important driver of risky and impulsive choices in a gambling setting laden with salient audiovisual cues. Evaluating an individual's sign tracking behavior may be an avenue to predict vulnerability to pathological gambling and the efficacy of treatments.

Acute depletion of dopamine precursors in the human brain: effects on functional connectivity and alcohol attentional bias

Individuals who abuse alcohol often show exaggerated attentional bias (AB) towards alcohol-related cues, which is thought to reflect reward conditioning processes. Rodent studies indicate that dopaminergic pathways play a key role in conditioned responses to reward- and alcohol-associated cues. However, investigation of the dopaminergic circuitry mediating this process in humans remains limited. We hypothesized that depletion of central dopamine levels in adult alcohol drinkers would attenuate AB and that these effects would be mediated by altered function in frontolimbic circuitry. Thirty-four male participants (22-38 years, including both social and heavy drinkers) underwent a two-session, placebo-controlled, double-blind dopamine precursor depletion procedure. At each visit, participants consumed either a balanced amino acid (control) beverage or an amino acid beverage lacking dopamine precursors (order counterbalanced), underwent resting-state fMRI, and completed behavioral testing on three AB tasks: an alcohol dot-probe task, an alcohol attentional blink task, and a task measuring AB to a reward-conditioned cue. Dopamine depletion significantly diminished AB in each behavioral task, with larger effects among subjects reporting higher levels of binge drinking. The depletion procedure significantly decreased resting-state functional connectivity among ventral tegmental area, striatum, amygdala, and prefrontal regions. Beverage-related AB decreases were mediated by decreases in functional connectivity between the fronto-insular cortex and striatum and, for alcohol AB only, between anterior cingulate cortex and amygdala. The results support a substantial role for dopamine in AB, and suggest specific dopamine-modulated functional connections between frontal, limbic, striatal, and brainstem regions mediate general reward AB versus alcohol AB.

Cue and Reward Evoked Dopamine Activity Is Necessary for Maintaining Learned Pavlovian Associations

Associating natural rewards with predictive environmental cues is crucial for survival. Dopamine (DA) neurons of the ventral tegmental area (VTA) are thought to play a crucial role in this process by encoding reward prediction errors (RPEs) that have been hypothesized to play a role in associative learning. However, it is unclear whether this signal is still necessary after animals have acquired a cue-reward association. In order to investigate this, we trained mice to learn a Pavlovian cue-reward association. After learning, mice show robust anticipatory and consummatory licking behavior. As expected, calcium activity of VTA DA neurons goes up for cue presentation as well as reward delivery. Optogenetic inhibition during the moment of reward delivery disrupts learned behavior, even in the continued presence of reward. This effect is more pronounced over trials and persists on the next training day. Moreover, outside of the task licking behavior and locomotion are unaffected. Similarly to inhibitions during the reward period, we find that inhibiting cue-induced dopamine (DA) signals robustly decreases learned licking behavior, indicating that cue-related DA signals are a potent driver for learned behavior. Overall, we show that inhibition of either of these DA signals directly impairs the expression of learned associative behavior. Thus, continued DA signaling in a learned state is necessary for consolidating Pavlovian associations.SIGNIFICANCE STATEMENT Dopamine (DA) neurons of the ventral tegmental area (VTA) have long been suggested to be necessary for animals to associate environmental cues with rewards that they predict. Here, we use time-locked optogenetic inhibition of these neurons to show that the activity of these neurons is directly necessary for performance on a Pavlovian conditioning task, without affecting locomotor per se These findings provide further support for the direct importance of second-by-second DA neuron activity in associative learning.

Computational phenotyping of brain-behavior dynamics underlying approach-avoidance conflict in major depressive disorder

Adaptive behavior requires balancing approach and avoidance based on the rewarding and aversive consequences of actions. Imbalances in this evaluation are thought to characterize mood disorders such as major depressive disorder (MDD). We present a novel application of the drift diffusion model (DDM) suited to quantify how offers of reward and aversiveness, and neural correlates thereof, are dynamically integrated to form decisions, and how such processes are altered in MDD. Hierarchical parameter estimation from the DDM demonstrated that the MDD group differed in three distinct reward-related parameters driving approach-based decision making. First, MDD was associated with reduced reward sensitivity, measured as the impact of offered reward on evidence accumulation. Notably, this effect was replicated in a follow-up study. Second, the MDD group showed lower starting point bias towards approaching offers. Third, this starting point was influenced in opposite directions by Pavlovian effects and by nucleus accumbens activity across the groups: greater accumbens activity was related to approach bias in controls but avoid bias in MDD. Cross-validation revealed that the combination of these computational biomarkers were diagnostic of patient status, with accumbens influences being particularly diagnostic. Finally, within the MDD group, reward sensitivity and nucleus accumbens parameters were differentially related to symptoms of perceived stress and depression. Collectively, these findings establish the promise of computational psychiatry approaches to dissecting approach-avoidance decision dynamics relevant for affective disorders.

Primate ventral striatum maintains neural representations of the value of previously rewarded objects for habitual seeking

The ventral striatum (VS) is considered a key region that flexibly updates recent changes in reward values for habit learning. However, this update process may not serve to maintain learned habitual behaviors, which are insensitive to value changes. Here, using fMRI in humans and single-unit electrophysiology in macaque monkeys we report another role of the primate VS: that the value memory subserving habitual seeking is stably maintained in the VS. Days after object-value associative learning, human and monkey VS continue to show increased responses to previously rewarded objects, even when no immediate reward outcomes are expected. The similarity of neural response patterns to each rewarded object increases after learning among participants who display habitual seeking. Our data show that long-term memory of high-valued objects is retained as a single representation in the VS and may be utilized to evaluate visual stimuli automatically to guide habitual behavior.

Prenatal Alcohol Exposure in Rats Diminishes Postnatal Cxcl16 Chemokine Ligand Brain Expression

Maternal ethanol consumption during pregnancy is one of the main causes of Neurodevelopmental disorders (NDD). Prenatal alcohol exposure (PAE) produces several adverse manifestations. Even low or moderate intake has been associated with long-lasting behavioral and cognitive impairment in offspring. In this study we examined the gene expression profile in the rat nucleus accumbens using microarrays, comparing animals exposed prenatally to ethanol and controls. Microarray gene expression showed an overall downward regulatory effect of PAE. Gene cluster analysis reveals that the gene groups most affected are related to transcription regulation, transcription factors and homeobox genes. We focus on the expression of the C-X-C motif chemokine ligand 16 (Cxcl16) which was differentially expressed. There is a significant reduction in the expression of this chemokine throughout the brain under PAE conditions, evidenced here by quantitative polymerase chain reaction qPCR and immunohistochemistry. Chemokines are involved in neuroprotection and implicated in alcohol-induced brain damage and neuroinflammation in the developing central nervous system (CNS), therefore, the significance of the overall decrease in Cxcl16 expression in the brain as a consequence of PAE may reflect a reduced ability in neuroprotection against subsequent conditions, such as excitotoxic damage, inflammatory processes or even hypoxic-ischemic insult.

Shifting motivational states: The effects of nucleus accumbens dopamine and opioid receptor activation on a modified effort-based choice task

The nucleus accumbens (NAc) is critical for regulating the appetitive and consummatory phases of motivated behavior. These experiments examined the effects of dopamine and opioid receptor manipulations within the NAc during an effort-based choice task that allowed for simultaneous assessment of both phases of motivation. Male Sprague-Dawley rats received bilateral guide cannulas targeting the NAc core and were tested in 1-hr sessions with free access to rat chow and the choice to work for sugar pellets on a progressive ratio 2 (PR2) reinforcement schedule. Individual groups of rats were tested following stimulation or blockade of NAc D1-like or D2-like receptors, stimulation of μ-, δ-, or κ-opioid receptors, or antagonism of opioid receptors. Behavior was examined under ad libitum conditions and following 23-h food restriction. NAc blockade of the D1-like receptors or stimulation of the D2 receptor reduced break point for earning sugar pellets; D2 receptor stimulation also modestly lowered chow intake. NAc μ-opioid receptor stimulation increased intake of the freely-available chow while simultaneously reducing break point for the sugar pellets. In non-restricted conditions, δ-opioid receptor stimulation increased both food intake and breakpoint. There were no effects of stimulating NAc D1 or κ receptors, nor did blocking D2 or opioid receptors affect task behavior. These data support prior literature linking dopamine to appetitive motivational processes, and suggest that μ- and δ-opioid receptors affect food-directed motivation differentially. Specifically, μ-opioid receptors shifted behavior towards consumption, and δ-opioid receptor enhanced both sugar-seeking and consumption of the pabulum chow when animals were not food restricted.

Accumbal Dopamine Release Tracks the Expectation of Dopamine Neuron-Mediated Reinforcement

Dopamine (DA) transmission in the nucleus accumbens (NAc) facilitates cue-reward associations and appetitive action. Reward-related accumbal DA release dynamics are traditionally ascribed to ventral tegmental area (VTA) DA neurons. Activation of VTA to NAc DA signaling is thought to reinforce action and transfer reward-related information to predictive cues, allowing cues to guide behavior and elicit dopaminergic activity. Here, we use optogenetics to control DA neuron activity and voltammetry to simultaneously record accumbal DA release in order to quantify how reinforcer-evoked dopaminergic activity shapes conditioned mesolimbic DA transmission. We find that cues predicting access to DA neuron self-stimulation elicit conditioned responding and NAc DA release. However, cue-evoked DA release does not reflect the cost or magnitude of DA neuron activation. Accordingly, conditioned accumbal DA release selectively tracks the expected availability of DA-neuron-mediated reinforcement. This work provides insight into how mesolimbic DA transmission drives and encodes appetitive action.

Testosterone-related behavioral and neural mechanisms associated with location preferences: A model for territorial establishment

Territoriality is an adaptive behavioral trait that is important for animal's fitness and there still remains much to learn about the proximate mechanisms underlying the development of territoriality. We speculate that the formation of a conditioned place preference (CPP), an increased time allocation to the environment where a rewarding experience occurred, contributes to territoriality. Testosterone (T) plays an important role in modulating territorial behaviors and T pulses can induce a CPP. We confirmed previous findings in California mice (Peromyscus californicus) that T pulses can induce a CPP in singly-housed, but not group-housed males. Housing singly may be similar enough to dispersal in nature to initiate similar hormonal and neuroanatomical changes needed for the development of territoriality. We further revealed that T pulses interact with the single housing experience and appear to enhance the motivation to be aggressive towards a stimulus male. On a neural level, being singly housed upregulated levels of androgen receptors in the preoptic area, which positively correlated with the strength of the CPP. We speculate that this change in androgen sensitivity in the preoptic area is characteristic of males that have dispersed, making them more sensitive to T pulses. Also, single housing increased markers of synaptic plasticity in the nucleus accumbens, ventral and dorsal hippocampus, neural changes that may be associated with dispersal, reproduction and territory establishment. These behavioral and neural changes may reflect the life history transition from residing in the natal territory to dispersing and establishing a new territory.

Dopaminergic signaling of uncertainty and the aetiology of gambling addiction

Although there is increasing clinical recognition of behavioral addictions, of which gambling disorder is the prototype example, there is a limited understanding of the psychological properties of (non-substance-related) behaviors that enable them to become 'addictive' in a way that is comparable to drugs of abuse. According to an influential application of reinforcement learning to substance addictions, the direct effects of drugs to release dopamine can create a perpetual escalation of incentive salience. This article focusses on reward uncertainty, which is proposed to be the core feature of gambling that creates the capacity for addiction. We describe the neuro-dynamics of the dopamine response to uncertainty that may allow a similar escalation of incentive salience, and its relevance to behavioral addictions. We review translational evidence from both preclinical animal models and human clinical research, including studies in people with gambling disorder. Further, we describe the evidence for 1) the effects of the omission of expected reward as a stressor and to promote sensitization, 2) the effect of the resolution of reward uncertainty as a source of value, 3) structural characteristics of modern Electronic Gaming Machines (EGMs) in leveraging these mechanisms, 4) analogies to the aberrant salience hypothesis of psychosis for creating and maintaining gambling-related cognitive distortions. This neurobiologically-inspired model has implications for harm profiling of other putative behavioral addictions, as well as offering avenues for enhancing neurological, pharmacological and psychological treatments for gambling disorder, and harm reduction strategies for EGM design.

Caudate reactivity to smoking cues is associated with increased responding to monetary reward in nicotine-dependent individuals

Quitting smoking is challenging in part because environmental smoking cues can trigger the desire to smoke. Neurobiological responses to smoking cues are often observed in reward-related brain regions such as the caudate and nucleus accumbens (NAc). While reward plays a well-established role in the formation of cue reactivity, whether general reward responsiveness contributes to individual differences in cue-reactivity among chronic smokers is unclear; establishing such link could provide insight into the mechanisms maintaining cue reactivity. The current study explored this relationship by assessing smoking cue reactivity during functional magnetic imaging followed by an out-of-scanner probabilistic reward task (PRT) in 24 nicotine-dependent smokers (14 women). In addition, owing to sex differences in cue reactivity and reward function, this same relationship was examined as a function of sex. Following recent smoking, greater reward responsiveness on the PRT was associated with enhanced left caudate reactivity to smoking cues. No relationship was found in any other striatal subregion. The positive relationship between reward responsiveness and caudate smoking cue reactivity was significant only in male smokers, fitting with the idea that males and females respond to the reinforcing elements of smoking cues differently. These findings are clinically relevant as they show that, following recent smoking, nicotine-dependent individuals who are more cue reactive are also more likely to be responsive to non-drug rewards, which may be useful for making individualized treatment decisions that involve behavioral reward contingencies.

Amphetamine disrupts haemodynamic correlates of prediction errors in nucleus accumbens and orbitofrontal cortex

In an uncertain world, the ability to predict and update the relationships between environmental cues and outcomes is a fundamental element of adaptive behaviour. This type of learning is typically thought to depend on prediction error, the difference between expected and experienced events and in the reward domain that has been closely linked to mesolimbic dopamine. There is also increasing behavioural and neuroimaging evidence that disruption to this process may be a cross-diagnostic feature of several neuropsychiatric and neurological disorders in which dopamine is dysregulated. However, the precise relationship between haemodynamic measures, dopamine and reward-guided learning remains unclear. To help address this issue, we used a translational technique, oxygen amperometry, to record haemodynamic signals in the nucleus accumbens (NAc) and orbitofrontal cortex (OFC), while freely moving rats performed a probabilistic Pavlovian learning task. Using a model-based analysis approach to account for individual variations in learning, we found that the oxygen signal in the NAc correlated with a reward prediction error, whereas in the OFC it correlated with an unsigned prediction error or salience signal. Furthermore, an acute dose of amphetamine, creating a hyperdopaminergic state, disrupted rats' ability to discriminate between cues associated with either a high or a low probability of reward and concomitantly corrupted prediction error signalling. These results demonstrate parallel but distinct prediction error signals in NAc and OFC during learning, both of which are affected by psychostimulant administration. Furthermore, they establish the viability of tracking and manipulating haemodynamic signatures of reward-guided learning observed in human fMRI studies by using a proxy signal for BOLD in a freely behaving rodent.

Dissociable dopaminergic and pavlovian influences in goal-trackers and sign-trackers on a model of compulsive checking in OCD

RATIONALE

Checking is a functional behaviour that provides information to guide behaviour. However, in obsessive-compulsive disorder (OCD), checking may escalate to dysfunctional levels. The processes underpinning the transition from functional to dysfunctional checking are unclear but may be associated with individual differences that support the development of maladaptive behaviour. We examined one such predisposition, sign-tracking to a pavlovian conditioned stimulus, which we previously found associated with dysfunctional checking. How sign-tracking interacts with another treatment with emerging translational validity for OCD-like checking, chronic administration of the dopamine D₂ receptor agonist quinpirole, is unknown.

OBJECTIVES

We tested how functional and dysfunctional checking in the rat observing response task (ORT) was affected by chronic quinpirole administration in non-autoshaped controls and autoshaped animals classified as sign-trackers or goal-trackers.

METHODS

Sign-trackers or goal-trackers were trained on the ORT before the effects of chronic quinpirole administration on checking were assessed. Subsequently, the effects on checking of different behavioural challenges, including reward omission and the use of unpredictable reinforcement schedules, were tested.

RESULTS

Prior autoshaping increased checking. Sign-trackers and goal-trackers responded differently to quinpirole sensitization, reward omission and reinforcement uncertainty. Sign-trackers showed greater elevations in dysfunctional checking, particularly during uncertainty. By contrast, goal-trackers predominantly increased functional checking responses, possibly in response to reduced discrimination accuracy in the absence of cues signalling which lever was currently active.

CONCLUSIONS

The results are discussed in terms of how pavlovian associations influence behaviour that becomes compulsive in OCD and how this may be dependent on striatal dopamine D₂ receptors.

Striatopallidal neurons control avoidance behavior in exploratory tasks

The dorsal striatum has been linked to decision-making under conflict, but the mechanism by which striatal neurons contribute to approach-avoidance conflicts remains unclear. We hypothesized that striatopallidal dopamine D2 receptor (D2R)-expressing neurons promote avoidance, and tested this hypothesis in two exploratory approach-avoidance conflict paradigms in mice: the elevated zero maze and open field. Genetic elimination of D2Rs on striatopallidal neurons (iMSNs), but not other neural populations, increased avoidance of the open areas in both tasks, in a manner that was dissociable from global changes in movement. Population calcium activity of dorsomedial iMSNs was disrupted in mice lacking D2Rs on iMSNs, suggesting that disrupted output of iMSNs contributes to heightened avoidance behavior. Consistently, artificial disruption of iMSN output with optogenetic stimulation heightened avoidance of open areas of these tasks, while inhibition of iMSN output reduced avoidance. We conclude that dorsomedial striatal iMSNs control approach-avoidance conflicts in exploratory tasks, and highlight this neural population as a potential target for reducing avoidance in anxiety disorders.

The role of the nucleus accumbens in learned approach behavior diminishes with training

Nucleus accumbens dopamine plays a key role in reward-directed approach. Past findings suggest that dopamine's role in the expression of learned behavior diminishes with extended training. However, little is known about the central substrates that mediate the shift to dopamine-independent reward approach. In the present study, rats approached and inserted the head into a reward compartment in response to a cue signaling food delivery. On days 4 and 5 of 28-trial-per-day sessions, D1 receptor antagonist R(+)-7-chloro-8-hydroxy-3-methyl-1-phenyl-2,3,4,5-tetrahydro-1H-3-benzazepine hydrochloride (SCH23390) infused to the NAc core reduced the probability and speed of cued approach. The disruptive effect of D1 receptor blockade was specific to the nucleus accumbens core and not seen with drug infusions to nearby dopamine target regions. In rats that received drug infusions after extended training (days 10 or 11), accumbens core D1 receptor blockade produced little effect on the expression of the same behavior. These results could have been due to a continued accumbens mediation of cued approach even after the behavior had become independent of accumbens D1 receptors. However, accumbens core ionotropic glutamate receptor blockade disrupted cued approach during early but not late stages of training, similar to the effects of D1 antagonist infusions. The results suggest that with extended training, a nucleus accumbens D1-dependent behavior becomes less dependent not only on nucleus accumbens D1 transmission but also on excitatory transmission in the nucleus accumbens. These findings fill an important gap in a growing literature on reorganization of striatal function over the course of training.

Buprenorphine prevents stress-induced blunting of nucleus accumbens dopamine response and approach behavior to food reward in mice

Alterations to the mesolimbic dopamine (DA) system are thought to underlie dysfunctional reward processing in stress-related psychiatric disorders. Using in vivio microdialysis in awake freely moving mice, we assessed the effects of stress on the motivational and neurochemical correlates underlying conditioned approach behavior for palatable food in the non-deprived mouse. Mice trained to approach and consume food in a familiar environment exhibited a 30% increase in nucleus accumbens shell (AcbSh) extracellular dopamine levels coincident with approach towards and consumption of the food reward. This effect was not observed in mice that were presented with the food in an unfamiliar environment or were exposed for the first time and were region specific. The addition of an acute environmental stressor (bright light and novel scent) during food exposure decreased DA release and delayed approach to the food. The disruptive impact of acute novelty stress on DA levels and approach behavior was reversed in animals pretreated with buprenorphine, an opioid drug with antidepressant-like and anxiolytic effects. Together, these data indicate that exposure to mild stress reduces incentive drive to approach palatable food via alterations in AcbSh dopamine responsiveness to food reward. Moreover, they implicate the brain opioid system as a potential pharmacological target for counteracting behavioral and neurochemical elements associated with stress.

Opposing roles for amygdala and vmPFC in the return of appetitive conditioned responses in humans

Learning accounts of addiction and obesity emphasize the persistent power of Pavlovian reward cues to trigger craving and increase relapse risk. While extinction can reduce conditioned responding, Pavlovian relapse phenomena-the return of conditioned responding following successful extinction-challenge the long-term success of extinction-based treatments. Translational laboratory models of Pavlovian relapse could therefore represent a valuable tool to investigate the mechanisms mediating relapse, although so far human research has mostly focused on return of fear phenomena. To this end we developed an appetitive conditioning paradigm with liquid food rewards in combination with a 3-day design to investigate the return of appetitive Pavlovian responses and the involved neural structures in healthy subjects. Pavlovian conditioning (day 1) was assessed in 62 participants, and a subsample (n = 33) further completed extinction (day 2) and a reinstatement test (day 3). Conditioned responding was assessed on explicit (pleasantness ratings) and implicit measures (reaction time, skin conductance, heart rate, startle response) and reinstatement effects were further evaluated using fMRI. We observed a return of conditioned responding during the reinstatement test, evident by enhanced skin conductance responses, accompanied by enhanced BOLD responses in the amygdala. On an individual level, psychophysiological reinstatement intensity was significantly anticorrelated with ventromedial prefrontal cortex (vmPFC) activation, and marginally anticorrelated with enhanced amygdala-vmPFC connectivity during late reinstatement. Our results extend evidence from return of fear phenomena to the appetitive domain, and highlight the role of the vmPFC and its functional connection with the amygdala in regulating appetitive Pavlovian relapse.

Reward prediction error does not explain movement selectivity in DMS-projecting dopamine neurons

Although midbrain dopamine (DA) neurons have been thought to primarily encode reward prediction error (RPE), recent studies have also found movement-related DAergic signals. For example, we recently reported that DA neurons in mice projecting to dorsomedial striatum are modulated by choices contralateral to the recording side. Here, we introduce, and ultimately reject, a candidate resolution for the puzzling RPE vs movement dichotomy, by showing how seemingly movement-related activity might be explained by an action-specific RPE. By considering both choice and RPE on a trial-by-trial basis, we find that DA signals are modulated by contralateral choice in a manner that is distinct from RPE, implying that choice encoding is better explained by movement direction. This fundamental separation between RPE and movement encoding may help shed light on the diversity of functions and dysfunctions of the DA system.

Behavioral flexibility in a mouse model for obsessive-compulsive disorder: Impaired Pavlovian reversal learning in SAPAP3 mutants

Obsessive-compulsive disorder (OCD) is characterized by obsessive thinking, compulsive behavior and anxiety, and is often accompanied by cognitive deficits. The neuropathology of OCD involves dysregulation of cortical-striatal circuits. Similar to OCD patients, SAPAP3 knockout mice 3 (SAPAP3-/- ) exhibit compulsive behavior (grooming), anxiety and dysregulated cortical-striatal function. However, it is unknown whether SAPAP3-/- display cognitive deficits and how these different behavioral traits relate to one another. SAPAP3-/- and wild-type (WT) littermates were trained in a Pavlovian conditioning task pairing visual cues with the delivery of sucrose solution. After mice learned to discriminate between a reward-predicting conditioned stimulus (CS+) and a non-reward stimulus (CS-), contingencies were reversed (CS+ became CS- and vice versa). Additionally, we assessed grooming, anxiety and general activity. SAPAP3-/- acquired Pavlovian approach behavior similarly to WT, albeit less vigorously and with a different strategy. However, unlike WT, SAPAP3-/- were unable to adapt their behavior after contingency reversal, exemplified by a lack of re-establishing CS+ approach behavior (sign tracking). Surprisingly, such behavioral inflexibility, decreased vigor, compulsive grooming and anxiety were unrelated. This study shows that SAPAP3-/- are capable of Pavlovian learning, but lack flexibility to adapt associated conditioned approach behavior. Thus, SAPAP3-/- not only display compulsive-like behavior and anxiety, but also cognitive deficits, confirming and extending the validity of SAPAP3-/- as a suitable model for the study of OCD. The observation that compulsive-like behavior, anxiety and behavioral inflexibility were unrelated suggests a non-causal relationship between these traits and may be of clinical relevance for the treatment of OCD.

Sign Tracking and Goal Tracking Are Characterized by Distinct Patterns of Nucleus Accumbens Activity

During Pavlovian conditioning, if a cue (e.g., lever extension) predicts reward delivery in a different location (e.g., a food magazine), some individuals will come to approach and interact with the cue, a behavior known as sign tracking (ST), and others will approach the site of reward, a behavior known as goal tracking (GT). In rats, the acquisition of ST versus GT behavior is associated with distinct profiles of dopamine release in the nucleus accumbens (NAc), but it is unknown whether it is associated with different patterns of accumbens neural activity. Therefore, we recorded from individual neurons in the NAc core during the acquisition, maintenance, and extinction of ST and GT behavior. Even though NAc dopamine is specifically important for the acquisition and expression of ST, we found that cue-evoked excitatory responses encode the vigor of both ST and GT behavior. In contrast, among sign trackers only, there was a prominent decrease in reward-related activity over the course of training, which may reflect the decreasing reward prediction error encoded by phasic dopamine. Finally, both behavior and cue-evoked activity were relatively resistant to extinction in sign trackers, as compared with goal trackers, although a subset of neurons in both groups retained their cue-evoked responses. Overall, the results point to the convergence of multiple forms of reward learning in the NAc.

The Psychopharmacology of Effort-Related Decision Making: Dopamine, Adenosine, and Insights into the Neurochemistry of Motivation

Effort-based decision making is studied using tasks that offer choices between high-effort options leading to more highly valued reinforcers versus low-effort/low-reward options. These tasks have been used to study the involvement of neural systems, including mesolimbic dopamine and related circuits, in effort-related aspects of motivation. Moreover, such tasks are useful as animal models of some of the motivational symptoms that are seen in people with depression, schizophrenia, Parkinson's disease, and other disorders. The present review will discuss the pharmacology of effort-related decision making and will focus on the use of these tasks for the development of drug treatments for motivational dysfunction. Research has identified pharmacological conditions that can alter effort-based choice and serve as models for depression-related symptoms (e.g., the vesicular monoamine transport-2 inhibitor tetrabenazine and proinflammatory cytokines). Furthermore, tests of effort-based choice have identified compounds that are particularly useful for stimulating high-effort work output and reversing the deficits induced by tetrabenazine and cytokines. These studies indicate that drugs that act by facilitating dopamine transmission, as well as adenosine A_2A antagonists, are relatively effective at reversing effort-related impairments. Studies of effort-based choice may lead to the identification of drug targets that could be useful for treating motivational treatments that are resistant to commonly used antidepressants such as serotonin transport inhibitors.

Cocaine, nicotine, and their conditioned contexts enhance consolidation of object memory in rats

To test the hypothesis that drugs of abuse and their conditioned stimuli (CSs) enhance memory consolidation, the effects of post-training exposure to cocaine and nicotine were compared to the effects of post-training exposure to contextual stimuli that were paired with the effects of these drugs. Using the object recognition (OR) task, it was first demonstrated that both 10 and 20 mg/kg cocaine, and 0.2 and 0.4 mg/kg nicotine, enhanced recognition memory when administered immediately after, but not 6 h after the sample phase. To establish the drug CSs, rats were confined for 2 h in a chamber (the CS+) after injections of 20 mg/kg cocaine, or 0.4 mg/kg nicotine, and in another chamber (the CS−) after injections of vehicle. This was repeated over 10 d (5 drug/CS+ and 5 vehicle/CS− pairings in total). At the end of this conditioning period, when tested in a drug-free state, rats displayed conditioned hyperactivity in the CS+ relative to the CS−. More important, immediate, but not delayed, post-sample exposure to the cocaine CS+, or nicotine CS+, enhanced OR memory. Therefore, this study reports for the first time that contextual stimuli paired with cocaine and nicotine, like the drugs themselves, have the ability to enhance memory consolidation.

Differential involvement of dopamine receptor subtypes in the acquisition of Pavlovian sign-tracking and goal-tracking responses

RATIONALE

Previous work has identified that different forms of Pavlovian conditioned approach, sign-tracking and goal-tracking, are governed by distinct neurochemical mechanisms when compared in animals predisposed to learning one form vs. the other.

OBJECTIVES

The present study aimed to investigate whether these are also neurochemically distinct processes in a population of animals capable of developing either response when this is manipulated via the use of distinct conditioned stimuli (CS).

METHODS

Rats were trained on one of two Pavlovian conditioning procedures in which the CS was either a lever, which elicits sign-tracking, or an auditory click, which elicits goal-tracking. The differential involvement of dopamine D1- and D2-receptors (D1R; D2R) in the acquisition of approach types was investigated via systemic administration of antagonists selective to one or both receptor subtypes during Pavlovian training.

RESULTS

Results indicate that dopaminergic signalling is important for the acquisition of both sign-tracking and goal-tracking responses. However, whilst development of sign-tracking to a lever depends on activity at both D1R and D2R, development of goal-tracking in response to a click was shown to depend only on activity at D1R.

CONCLUSIONS

We suggest that the importance of D1R activity in both sign- and goal-tracking acquisition reflects a general role in learning Pavlovian associations, which aligns with data implicating dopamine in prediction error processes. In contrast, the selective involvement of D2R activity in sign-tracking acquisition may reflect its importance in motivational processes such as incentive salience attribution.

Pavlovian-instrumental interactions in active avoidance: The bark of neutral trials

In active avoidance tasks, subjects have to learn to execute particular actions in order to avoid an aversive stimulus, such as a shock. Such paradigms pose a number of psychological and neural enigmas, and so have attracted substantial computational interest. However, the ratio of conjecture to confirmation remains high. Here, we perform a theoretical inquiry into a recent experiment by Gentry, Lee, and Roesch ('Phasic dopamine release in the rat nucleus accumbens predicts approach and avoidance performance', Nat. Commun., 7:13154) who measured phasic dopamine concentrations in the nucleus accumbens core of rats whilst they avoided shocks, acquired food, or acted to gain no programmed outcome. These last, neutral, trials turned out to be a perfect probe for the workings of avoidance, partly because of the substantial differences between subjects and sessions revealed in the experiment. We suggest a way to interpret this probe, gaining support for opponency-, safety-, and Pavlovian-influenced treatments of avoidance.

The hot 'n' cold of cue-induced drug relapse

Environmental cues associated with rewards can acquire motivational properties. However, there is considerable variation in the extent to which a reward cue gains motivational control over behavior, depending on the individual and the form of the cue. When a discrete cue is paired with food reward, it acquires greater control over motivated behavior in some rats (sign-trackers, STs) than others (goal-trackers, GTs) as indicated by the propensity to approach the cue, the willingness to work to obtain it, and its ability to reinstate reward-seeking behavior. Here, we review studies that employ this ST/GT animal model to investigate characteristics of individuals that are especially susceptible to reward cue-elicited behavior and the involvement of dopamine and acetylcholine neuromodulator systems in the susceptibility to cue-induced drug relapse. First, we discuss individual differences in the attribution of incentive salience to different forms of reward cues and the involvement of the mesolimbic dopamine system. We then discuss individual differences in cognitive/attentional control and the contributions of the cholinergic system in processing reward cues. It is suggested that in STs a propensity to attribute motivational properties to a drug cue is combined with poor attentional control in the face of these cues, making them particularly vulnerable to transition from casual/experimental patterns of drug use to addiction and to cue-induced relapse.

Nucleus Accumbens Dopamine D1-Receptor-Expressing Neurons Control the Acquisition of Sign-Tracking to Conditioned Cues in Mice

Following repeated pairings, the reinforcing and motivational properties (incentive salience) of a reward can be transferred onto an environmental stimulus which can then elicit conditioned responses, including Pavlovian approach behavior to the stimulus (a sign-tracking response). In rodents, acquisition of sign-tracking in autoshaping paradigms is sensitive to lesions and dopamine D1 receptor antagonism of the nucleus accumbens (NAc) of the ventral striatum. However, currently, the possible roles of dorsal striatal subregions, as well as of the two major striatal neuron types, dopamine D1-/D2-expressing medium spiny neurons (MSNs), in controlling the development of conditioned responses is still unclear and warrants further study. Here, for the first time, we used a transgenic mouse line combined with striatal subregion-specific AAV virus injections to separately express tetanus toxin in D1-/D2- MSNs in the NAc, dorsomedial striatum, and dorsolateral striatum, to permanently block neurotransmission in these neurons during acquisition of an autoshaping task. Neurotransmission blocking of NAc D1-MSNs inhibited the acquisition of sign-tracking responses when the initial conditioned response for each conditioned stimulus presentation was examined, confirming our initial hypothesis. These findings suggest that activity in NAc D1-MSNs contributes to the attribution of incentive salience to conditioned stimuli.

Dopamine receptors mediate strategy abandoning via modulation of a specific prelimbic cortex-nucleus accumbens pathway in mice

The ability to abandon old strategies and adopt new ones is essential for survival in a constantly changing environment. While previous studies suggest the importance of the prefrontal cortex and some subcortical areas in the generation of strategy-switching flexibility, the fine neural circuitry and receptor mechanisms involved are not fully understood. In this study, we showed that optogenetic excitation and inhibition of the prelimbic cortex-nucleus accumbens (NAc) pathway in the mouse respectively enhances and suppresses strategy-switching ability in a cross-modal spatial-egocentric task. This ability is dependent on an intact dopaminergic tone in the NAc, as local dopamine denervation impaired the performance of the animal in the switching of tasks. In addition, based on a brain-slice preparation obtained from Drd2-EGFP BAC transgenic mice, we demonstrated direct innervation of D2 receptor-expressing medium spiny neurons (D2-MSNs) in the NAc by prelimbic cortical neurons, which is under the regulation by presynaptic dopamine receptors. While presynaptic D1-type receptor activation enhances the glutamatergic transmission from the prelimbic cortex to D2-MSNs, D2-type receptor activation suppresses this synaptic connection. Furthermore, manipulation of this pathway by optogenetic activation or administration of a D1-type agonist or a D2-type antagonist could restore impaired task-switching flexibility in mice with local NAc dopamine depletion; this restoration is consistent with the effects of knocking down the expression of specific dopamine receptors in the pathway. Our results point to a critical role of a specific prelimbic cortex-NAc subpathway in mediating strategy abandoning, allowing the switching from one strategy to another in problem solving.

Maternal immune activation alters sensitivity to action-outcome contingency in adult rat offspring

Epidemiological studies have provided convincing evidence for a role of maternal immune activation in the pathogenesis of neurodevelopmental disorders such as autism and schizophrenia. In recent years, several research groups have capitalised on this discovery and developed animal models such as the maternal immune activation (MIA) model that emulates many phenotypes characteristic of disorders such as schizophrenia. In the present series of experiments we used the MIA model to examine motivation, a core component of the negative symptomatology in schizophrenia. Contrary to what we expected, in the progressive ratio task, which assesses an animals' willingness to work for a reward under increasing effort requirements, we found that MIA rats appeared more motivated than controls. Subsequent tests showed that this seemingly enhanced motivation was not due to an overall increase in responding, nor due to enhanced attribution of incentive salience to reward associated responses. Instead, we found that the increased willingness to work exhibited by MIA animals was due to an inability to detect changes in the contingency between their behaviour and the resulting rewarding outcome. With regard to motivation, the experiments reported here are the first to subject the MIA model to a rigorous experimental analysis of behaviour by parsing underlying processes that give rise to the overt symptoms in psychiatric disease.

Partial lesion of dopamine neurons of rat substantia nigra impairs conditioned place aversion but spares conditioned place preference

Midbrain dopamine neurons play critical roles in reward- and aversion-driven associative learning. However, it is not clear whether they do this by a common mechanism or by separate mechanisms that can be dissociated. In the present study we addressed this question by testing whether a partial lesion of the dopamine neurons of the rat SNc has comparable effects on conditioned place preference (CPP) learning and conditioned place aversion (CPA) learning. Partial lesions of dopamine neurons in the rat substantia nigra pars compacta (SNc) induced by bilateral intranigral infusion of 6-hydroxydopamine (6-OHDA, 3μg/side) or 1-methyl-4-phenyl-1,2,3,6-tetrahydropyridine (MPTP, 200μg/side) impaired learning of conditioned place aversion (CPA) without affecting conditioned place preference (CPP) learning. Control experiments demonstrated that these lesions did not impair motor performance and did not alter the hedonic value of the sucrose and quinine. The number of dopamine neurons in the caudal part of the SNc positively correlated with the CPP scores of the 6-OHDA rats and negatively correlated with CPA scores of the SHAM rats. In addition, the CPA scores of the 6-OHDA rats positively correlated with the tissue content of striatal dopamine. Insomuch as reward-driven learning depends on an increase in dopamine release by nigral neurons, these findings show that this mechanism is functional even in rats with a partial lesion of the SNc. On the other hand, if aversion-driven learning depends on a reduction of extracellular dopamine in the striatum, the present study suggests that this mechanism is no longer functional after the partial SNc lesion.

Enduring, Sexually Dimorphic Impact of In Utero Exposure to Elevated Levels of Glucocorticoids on Midbrain Dopaminergic Populations

Glucocorticoid hormones (GCs) released from the fetal/maternal glands during late gestation are required for normal development of mammalian organs and tissues. Accordingly, synthetic glucocorticoids have proven to be invaluable in perinatal medicine where they are widely used to accelerate fetal lung maturation when there is risk of pre-term birth and to promote infant survival. However, clinical and pre-clinical studies have demonstrated that inappropriate exposure of the developing brain to elevated levels of GCs, either as a result of clinical over-use or after stress-induced activation of the fetal/maternal adrenal cortex, is linked with significant effects on brain structure, neurological function and behaviour in later life. In order to understand the underlying neural processes, particular interest has focused on the midbrain dopaminergic systems, which are critical regulators of normal adaptive behaviours, cognitive and sensorimotor functions. Specifically, using a rodent model of GC exposure in late gestation (approximating human brain development at late second/early third trimester), we demonstrated enduring effects on the shape and volume of the ventral tegmental area (VTA) and substantia nigra pars compacta (SNc) (origins of the mesocorticolimbic and nigrostriatal dopaminergic pathways) on the topographical organisation and size of the dopaminergic neuronal populations and astrocytes within these nuclei and on target innervation density and neurochemical markers of dopaminergic transmission (receptors, transporters, basal and amphetamine-stimulated dopamine release at striatal and prefrontal cortical sites) that impact on the adult brain. The effects of antenatal GC treatment (AGT) were both profound and sexually-dimorphic, not only in terms of quantitative change but also qualitatively, with several parameters affected in the opposite direction in males and females. Although such substantial neurobiological changes might presage marked behavioural effects, in utero GC exposure had only a modest or no effect, depending on sex, on a range of conditioned and unconditioned behaviours known to depend on midbrain dopaminergic transmission. Collectively, these findings suggest that apparent behavioural normality in certain tests, but not others, arises from AGT-induced adaptations or compensatory mechanisms within the midbrain dopaminergic systems, which preserve some, but not all functions. Furthermore, the capacities for molecular adaptations to early environmental challenge are different, even opponent, in males and females, which may account for their differential resilience or failure to perform adequately in behavioural tests. Behavioural "normality" is thus achieved by the midbrain dopaminergic network operating outside its normal limits (in a state of allostasis), rendering it at greater risk to malfunction when challenged in later life. Sex-specific neurobiological programming of midbrain dopaminergic systems may, therefore, have psychopathological relevance for the sex bias commonly found in brain disorders associated with these systems, and which have a neurodevelopmental component, including schizophrenia, ADHD (attention/deficit hyperactivity disorders), autism, depression and substance abuse.

Reassessing wanting and liking in the study of mesolimbic influence on food intake

Humans and animals such as rats and mice tend to overconsume calorie-dense foods, a phenomenon that likely contributes to obesity. One often-advanced explanation for why we preferentially consume sweet and fatty foods is that they are more "rewarding" than low-calorie foods. "Reward" has been subdivided into three interdependent psychological processes: hedonia (liking a food), reinforcement (formation of associations among stimuli, actions, and/or the food), and motivation (wanting the food). Research into these processes has focused on the mesolimbic system, which comprises both dopamine neurons in the ventral tegmental area and neurons in their major projection target, the nucleus accumbens. The mesolimbic system and closely connected structures are commonly referred to as the brain's "reward circuit." Implicit in this title is the assumption that "rewarding" experiences are generally the result of activity in this circuit. In this review, I argue that food intake and the preference for calorie-dense foods can be explained without reference to subjective emotions. Furthermore, the contribution of mesolimbic dopamine to food intake and preference may not be a general one of promoting or coordinating behaviors that result in the most reward or caloric intake but may instead be limited to the facilitation of a specific form of neural computation that results in conditioned approach behavior. Studies on the neural mechanisms of caloric intake regulation must address how sensory information about calorie intake affects not just the mesolimbic system but also many other forms of computation that govern other types of food-seeking and food-oriented behaviors.

Cognitive Judgment Bias Interacts with Risk Based Decision Making and Sensitivity to Dopaminergic Challenge in Male Rats

Although the cognitive theory has implicated judgment bias in various psychopathologies, its role in decision making under risk remains relatively unexplored. In the present study, we assessed the effects of cognitive judgment bias on risky choices in rats. First, we trained and tested the animals on the rat version of the probability-discounting (PD) task. During discrete trials, the rats chose between two levers; a press on the “small/certain” lever always resulted in the delivery of one reward pellet, whereas a press on the “large/risky” lever resulted in the delivery of four pellets. However, the probability of receiving a reward from the “large/risky” lever gradually decreased over the four trial blocks. Subsequently, the rats were re-trained and evaluated on a series of ambiguous-cue interpretation (ACI) tests, which permitted their classification according to the display of “optimistic” or “pessimistic” traits. Because dopamine (DA) has been implicated in both: risky choices and optimism, in the last experiment, we compared the reactivity of the dopaminergic system in the “optimistic” and “pessimistic” animals using the apomorphine (APO; 2 mg/kg s.c.) sensitivity test. We demonstrated that as risk increased, the proportion of risky lever choices decreased significantly slower in “optimists” compared with “pessimists” and that these differences between the two groups of rats were associated with different levels of dopaminergic system reactivity. Our findings suggest that cognitive judgment bias, risky decision-making and DA are linked, and they provide a foundation for further investigation of the behavioral traits and cognitive processes that influence risky choices in animal models.

Toward isolating the role of dopamine in the acquisition of incentive salience attribution

Stimulus-reward learning has been heavily linked to the reward-prediction error learning hypothesis and dopaminergic function. However, some evidence suggests dopaminergic function may not strictly underlie reward-prediction error learning, but may be specific to incentive salience attribution. Utilizing a Pavlovian conditioned approach procedure consisting of two stimuli that were equally reward-predictive (both undergoing reward-prediction error learning) but functionally distinct in regard to incentive salience (levers that elicited sign-tracking and tones that elicited goal-tracking), we tested the differential role of D1 and D2 dopamine receptors and nucleus accumbens dopamine in the acquisition of sign- and goal-tracking behavior and their associated conditioned reinforcing value within individuals. Overall, the results revealed that both D1 and D2 inhibition disrupted performance of sign- and goal-tracking. However, D1 inhibition specifically prevented the acquisition of sign-tracking to a lever, instead promoting goal-tracking and decreasing its conditioned reinforcing value, while neither D1 nor D2 signaling was required for goal-tracking in response to a tone. Likewise, nucleus accumbens dopaminergic lesions disrupted acquisition of sign-tracking to a lever, while leaving goal-tracking in response to a tone unaffected. Collectively, these results are the first evidence of an intraindividual dissociation of dopaminergic function in incentive salience attribution from reward-prediction error learning, indicating that incentive salience, reward-prediction error, and their associated dopaminergic signaling exist within individuals and are stimulus-specific. Thus, individual differences in incentive salience attribution may be reflective of a differential balance in dopaminergic function that may bias toward the attribution of incentive salience, relative to reward-prediction error learning only.

Safety out of control: dopamine and defence

We enjoy a sophisticated understanding of how animals learn to predict appetitive outcomes and direct their behaviour accordingly. This encompasses well-defined learning algorithms and details of how these might be implemented in the brain. Dopamine has played an important part in this unfolding story, appearing to embody a learning signal for predicting rewards and stamping in useful actions, while also being a modulator of behavioural vigour. By contrast, although choosing correct actions and executing them vigorously in the face of adversity is at least as important, our understanding of learning and behaviour in aversive settings is less well developed. We examine aversive processing through the medium of the role of dopamine and targets such as D2 receptors in the striatum. We consider critical factors such as the degree of control that an animal believes it exerts over key aspects of its environment, the distinction between 'better' and 'good' actual or predicted future states, and the potential requirement for a particular form of opponent to dopamine to ensure proper calibration of state values.