Inhibiting Mesolimbic Dopamine Neurons Reduces the Initiation and Maintenance of Instrumental Responding

Dopamine builds and reveals reward-associated latent behavioral attractors

Phasic variations in dopamine levels are interpreted as a teaching signal reinforcing rewarded behaviors. However, behavior also depends on the motivational, neuromodulatory effect of phasic dopamine. In this study, we reveal a neurodynamical principle that unifies these roles in a recurrent network-based decision architecture embodied through an action-perception loop with the task space, the MAGNet model. Dopamine optogenetic conditioning in mice was accounted for by an embodied network model in which attractors encode internal goals. Dopamine-dependent synaptic plasticity created "latent" attractors, to which dynamics converged, but only locally. Attractor basins were widened by dopamine-modulated synaptic excitability, rendering goals accessible globally, i.e. from distal positions. We validated these predictions optogenetically in mice: dopamine neuromodulation suddenly and specifically attracted animals toward rewarded locations, without off-target motor effects. We thus propose that motivational dopamine reveals dopamine-built attractors representing potential goals in a behavioral landscape.

Exploring mechanisms of anhedonia in depression through neuroimaging and data-driven approaches

BACKGROUND

Anhedonia is a core symptom of depression that is closely related to prognosis and treatment outcomes. However, accurate and efficient treatments for anhedonia are lacking, mandating a deeper understanding of the underlying mechanisms.

METHODS

A total of 303 patients diagnosed with depression and anhedonia were assessed by the Snaith-Hamilton Pleasure Scale (SHAPS) and magnetic resonance imaging (MRI). The patients were categorized into a low-anhedonia group and a high-anhedonia group using the K-means algorithm. A data-driven approach was used to explore the differences in brain structure and function with different degrees of anhedonia based on MATLAB. A random forest model was used exploratorily to test the predictive ability of differences in brain structure and function on anhedonia in depression.

RESULTS

Structural and functional differences were apparent in several brain regions of patients with depression and high-level anhedonia, including in the temporal lobe, paracingulate gyrus, superior frontal gyrus, inferior occipital gyrus, right insular gyrus, and superior parietal lobule. And changes in these brain regions were significantly correlated with scores of SHAPS.

CONCLUSIONS

These brain regions may be useful as biomarkers that provide a more objective assessment of anhedonia in depression, laying the foundation for precision medicine in this treatment-resistant, relatively poor prognosis group.

Enhanced cognitive flexibility and phasic striatal dopamine dynamics in a mouse model of low striatal tonic dopamine

The catecholamine neuromodulators dopamine and norepinephrine are implicated in motor function, motivation, and cognition. Although roles for striatal dopamine in these aspects of behavior are well established, the specific roles for cortical catecholamines in regulating striatal dopamine dynamics and behavior are less clear. We recently showed that elevating cortical dopamine but not norepinephrine suppresses hyperactivity in dopamine transporter knockout (DAT-KO) mice, which have elevated striatal dopamine levels. In contrast, norepinephrine transporter knockout (NET-KO) mice have a phenotype distinct from DAT-KO mice, as they show elevated extracellular cortical catecholamines but reduced baseline striatal dopamine levels. Here we evaluated the consequences of altered catecholamine levels in NET-KO mice on cognitive flexibility and striatal dopamine dynamics. In a probabilistic reversal learning task, NET-KO mice showed enhanced reversal learning, which was consistent with larger phasic dopamine transients (dLight) in the dorsomedial striatum (DMS) during reward delivery and reward omission, compared to WT controls. Selective depletion of dorsal medial prefrontal cortex (mPFC) norepinephrine in WT mice did not alter performance on the reversal learning task but reduced nestlet shredding. Surprisingly, NET-KO mice did not show altered breakpoints in a progressive ratio task, suggesting intact food motivation. Collectively, these studies show novel roles of cortical catecholamines in the regulation of tonic and phasic striatal dopamine dynamics and cognitive flexibility, updating our current views on dopamine regulation and informing future therapeutic strategies to counter multiple psychiatric disorders.

Kv7 channel opener retigabine reduces self-administration of cocaine but not sucrose in rats

The increasing rates of drug misuse highlight the urgency of identifying improved therapeutics for treatment. Most drug-seeking behaviours that can be modelled in rodents utilize the repeated intravenous self-administration (SA) of drugs. Recent studies examining the mesolimbic pathway suggest that Kv7/KCNQ channels may contribute to the transition from recreational to chronic drug use. However, to date, all such studies used noncontingent, experimenter-delivered drug model systems, and the extent to which this effect generalizes to rats trained to self-administer drugs is not known. Here, we tested the ability of retigabine (ezogabine), a Kv7 channel opener, to regulate instrumental behaviour in male Sprague Dawley rats. We first validated the ability of retigabine to target experimenter-delivered cocaine in a conditioned place preference (CPP) assay and found that retigabine reduced the acquisition of place preference. Next, we trained rats for cocaine-SA under a fixed-ratio or progressive-ratio reinforcement schedule and found that retigabine pretreatment attenuated the SA of low to moderate doses of cocaine. This was not observed in parallel experiments, with rats self-administering sucrose, a natural reward. Compared with sucrose-SA, cocaine-SA was associated with reductions in the expression of the Kv7.5 subunit in the nucleus accumbens, without alterations in Kv7.2 and Kv7.3. Therefore, these studies reveal a reward-specific reduction in SA behaviour and support the notion that Kv7 is a potential therapeutic target for human psychiatric diseases with dysfunctional reward circuitry.

Rethinking dopamine-guided action sequence learning

As opposed to those requiring a single action for reward acquisition, tasks necessitating action sequences demand that animals learn action elements and their sequential order and sustain the behaviour until the sequence is completed. With repeated learning, animals not only exhibit precise execution of these sequences but also demonstrate enhanced smoothness and efficiency. Previous research has demonstrated that midbrain dopamine and its major projection target, the striatum, play crucial roles in these processes. Recent studies have shown that dopamine from the substantia nigra pars compacta (SNc) and the ventral tegmental area (VTA) serve distinct functions in action sequence learning. The distinct contributions of dopamine also depend on the striatal subregions, namely the ventral, dorsomedial and dorsolateral striatum. Here, we have reviewed recent findings on the role of striatal dopamine in action sequence learning, with a focus on recent rodent studies.

Ventral pallidum neurons projecting to the ventral tegmental area reinforce but do not invigorate reward-seeking behavior

Reward-predictive cues acquire motivating and reinforcing properties that contribute to the escalation and relapse of drug use in addiction. The ventral pallidum (VP) and ventral tegmental area (VTA) are two key nodes in brain reward circuitry implicated in addiction and cue-driven behavior. In the current study, we use in vivo fiber photometry and optogenetics to record from and manipulate VP→VTA in rats performing a discriminative stimulus task to determine the role these neurons play in invigoration and reinforcement by reward cues. We find that VP→VTA neurons are active during reward consumption and that optogenetic stimulation of these neurons biases choice behavior and is reinforcing. Critically, we find no encoding of reward-seeking vigor, and optogenetic stimulation does not enhance the probability or vigor of reward seeking in response to cues. Our results suggest that VP→VTA activity is more important for reinforcement than for invigoration of reward seeking by cues.

Nucleus accumbens and dorsal medial striatal dopamine and neural activity are essential for action sequence performance

Separable striatal circuits have unique functions in Pavlovian and instrumental behaviors but how these roles relate to performance of sequences of actions with and without associated cues are less clear. Here, we tested whether dopamine transmission and neural activity more generally in three striatal subdomains are necessary for performance of an action chain leading to reward delivery. Male and female Long-Evans rats were trained to press a series of three spatially distinct levers to receive reward. We assessed the contribution of neural activity or dopamine transmission within each striatal subdomain when progression through the action sequence was explicitly cued and in the absence of cues. Behavior in both task variations was substantially impacted following microinfusion of the dopamine antagonist, flupenthixol, into nucleus accumbens core (NAc) or dorsomedial striatum (DMS), with impairments in sequence timing and numbers of rewards earned after NAc flupenthixol. In contrast, after pharmacological inactivation to suppress overall activity, there was minimal impact on total rewards earned. Instead, inactivation of both NAc and DMS impaired sequence timing and led to sequence errors in the uncued, but not cued task. There was no impact of dopamine antagonism or reversible inactivation of dorsolateral striatum on either cued or uncued action sequence completion. These results highlight an essential contribution of NAc and DMS dopamine systems in motivational and performance aspects of chains of actions, whether cued or internally generated, as well as the impact of intact NAc and DMS function for correct sequence performance.

Apathy in Parkinson's Disease: Clinical Patterns and Neurobiological Basis

Apathy is commonly defined as a loss of motivation leading to a reduction in goal-directed behaviors. This multidimensional syndrome, which includes cognitive, emotional and behavioral components, is one of the most prevalent neuropsychiatric features of Parkinson's disease (PD). It has been established that the prevalence of apathy increases as PD progresses. However, the pathophysiology and anatomic substrate of this syndrome remain unclear. Apathy seems to be underpinned by impaired anatomical structures that link the prefrontal cortex with the limbic system. It can be encountered in the prodromal stage of the disease and in fluctuating PD patients receiving bilateral chronic subthalamic nucleus stimulation. In these stages, apathy may be considered as a disorder of motivation that embodies amotivational behavioral syndrome, is underpinned by combined dopaminergic and serotonergic denervation and is dopa-responsive. In contrast, in advanced PD patients, apathy may be considered as cognitive apathy that announces cognitive decline and PD dementia, is underpinned by diffuse neurotransmitter system dysfunction and Lewy pathology spreading and is no longer dopa-responsive. In this review, we discuss the clinical patterns of apathy and their treatment, the neurobiological basis of apathy, the potential role of the anatomical structures involved and the pathways in motivational and cognitive apathy.

K v 7 Channel Opener Retigabine Reduces Self-Administration of Cocaine but Not Sucrose in Rats

The increasing rates of drug misuse highlight the urgency of identifying improved therapeutics for treatment. Most drug-seeking behaviors that can be modeled in rodents utilize the repeated intravenous self-administration (SA) of drugs. Recent studies examining the mesolimbic pathway suggest that K v 7/KCNQ channels may contribute in the transition from recreational to chronic drug use. However, to date, all such studies used noncontingent, experimenter-delivered drug model systems, and the extent to which this effect generalizes to rats trained to self-administer drug is not known. Here, we tested the ability of retigabine (ezogabine), a K v 7 channel opener, to regulate instrumental behavior in male Sprague Dawley rats. We first validated the ability of retigabine to target experimenter-delivered cocaine in a CPP assay and found that retigabine reduced the acquisition of place preference. Next, we trained rats for cocaine-SA under a fixed-ratio or progressive-ratio reinforcement schedule and found that retigabine-pretreatment attenuated the self-administration of low to moderate doses of cocaine. This was not observed in parallel experiments, with rats self-administering sucrose, a natural reward. Compared to sucrose-SA, cocaine-SA was associated with reductions in the expression of the K v 7.5 subunit in the nucleus accumbens, without alterations in K v 7.2 and K v 7.3. Therefore, these studies reveal a reward specific reduction in SA behavior considered relevant for the study of long-term compulsive-like behavior and supports the notion that K v 7 is a potential therapeutic target for human psychiatric diseases with dysfunctional reward circuitry.

Dopaminergic and prefrontal dynamics co-determine mouse decisions in a spatial gambling task

The neural mechanisms by which animals initiate goal-directed actions, choose between options, or explore opportunities remain unknown. Here, we develop a spatial gambling task in which mice, to obtain intracranial self-stimulation rewards, self-determine the initiation, direction, vigor, and pace of their actions based on their knowledge of the outcomes. Using electrophysiological recordings, pharmacology, and optogenetics, we identify a sequence of oscillations and firings in the ventral tegmental area (VTA), orbitofrontal cortex (OFC), and prefrontal cortex (PFC) that co-encodes and co-determines self-initiation and choices. This sequence appeared with learning as an uncued realignment of spontaneous dynamics. Interactions between the structures varied with the reward context, particularly the uncertainty associated with the different options. We suggest that self-generated choices arise from a distributed circuit based on an OFC-VTA core determining whether to wait for or initiate actions, while the PFC is specifically engaged by reward uncertainty in action selection and pace.

Critical review of RDoC approaches to the study of motivation with animal models: effort valuation/willingness to work

The NIMH research domain criteria (RDoC) approach was instigated to refocus mental health research on the neural circuits that mediate psychological functions, with the idea that this would foster an understanding of the neural basis of specific psychiatric dysfunctions (i.e. 'symptoms and circuits') and ultimately facilitate treatment. As a general idea, this attempt to go beyond traditional diagnostic categories and focus on neural circuit dysfunctions related to specific symptoms spanning multiple disorders has many advantages. For example, motivational dysfunctions are present in multiple disorders, including depression, schizophrenia, Parkinson's disease, and other conditions. A critical aspect of motivation is effort valuation/willingness to work, and several clinical studies have identified alterations in effort-based decision making in various patient groups. In parallel, formal animal models focusing on the exertion of effort and effort-based decision making have been developed. This paper reviews the literature on models of effort-based motivational function in the context of a discussion of the RDoC approach, with an emphasis on the dissociable nature of distinct aspects of motivation. For example, conditions associated with depression and schizophrenia blunt the selection of high-effort activities as measured by several tasks in animal models (e.g. lever pressing, barrier climbing, wheel running). Nevertheless, these manipulations also leave fundamental aspects of hedonic reactivity, food motivation, and reinforcement intact. This pattern of effects demonstrates that the general emphasis of the RDoC on the specificity of the neural circuits mediating behavioral pathologies, and the dissociative nature of these dysfunctions, is a valid concept. Nevertheless, the specific placement of effort-related processes as simply a 'sub-construct' of 'reward processing' is empirically and conceptually problematic. Thus, while the RDoC is an excellent general framework for new ways to approach research and therapeutics, it still needs further refinement.

The Arousal-motor Hypothesis of Dopamine Function: Evidence that Dopamine Facilitates Reward Seeking in Part by Maintaining Arousal

Dopamine facilitates approach to reward via its actions on dopamine receptors in the nucleus accumbens. For example, blocking either D1 or D2 dopamine receptors in the accumbens reduces the proportion of reward-predictive cues to which rats respond with cued approach. Recent evidence indicates that accumbens dopamine also promotes wakefulness and arousal, but the relationship between dopamine's roles in arousal and reward seeking remains unexplored. Here, we show that the ability of systemic or intra-accumbens injections of the D1 antagonist SCH23390 to reduce cued approach to reward depends on the animal's state of arousal. Handling the animal, a manipulation known to increase arousal, was sufficient to reverse the behavioral effects of the antagonist. In addition, SCH23390 reduced spontaneous locomotion and increased time spent in sleep postures, both consistent with reduced arousal, but also increased time spent immobile in postures inconsistent with sleep. In contrast, the ability of the D2 antagonist haloperidol to reduce cued approach was not reversible by handling. Haloperidol reduced spontaneous locomotion but did not increase sleep postures, instead increasing immobility in non-sleep postures. We place these results in the context of the extensive literature on dopamine's contributions to behavior, and propose the arousal-motor hypothesis. This novel synthesis, which proposes that two main functions of dopamine are to promote arousal and facilitate motor behavior, accounts both for our findings and many previous behavioral observations that have led to disparate and conflicting conclusions.

Deconstructing and reconstructing behaviour relevant to mental health disorders: The benefits of a psychological approach, with a focus on addiction

RUTHERFORD, L.G. and Milton, A.L. Deconstructing and reconstructing behaviour relevant to mental health disorders: what can psychology offer? NEUROSCI BIOBEHAV REV XX(X)XXX-XXX, 2021. - Current treatments for mental health disorders are successful only for some patients, and there is an unmet clinical need for new treatment development. One challenge for treatment development has been how best to model complex human conditions in animals, where mechanism can be more readily studied with a range of neuroscientific techniques. We suggest that an approach to modelling based on associative animal learning theory provides a good framework for deconstructing complex mental health disorders such that they can be studied in animals. These individual simple models can subsequently be used in combination to 'reconstruct' a more complex model of the mental health disorder of interest. Using examples primarily from the field of drug addiction, we explore the 'psychological approach' and suggest that in addition to facilitating translation and backtranslation of tasks between animal models and patients, it is also highly concordant with the concept of triangulation.

Anhedonia as a central factor in depression: Neural mechanisms revealed from preclinical to clinical evidence

Anhedonia is one of the core symptoms of major depressive disorder (MDD), which is often inadequately treated by traditional antidepressants. The modern framework of anhedonia extends the definition from impaired consummatory pleasure or interest in rewards to a broad spectrum of deficits that impact functions such as reward anticipation, approach motivation, effort expenditure, reward valuation, expectation, and reward-cue association learning. Substantial preclinical and clinical research has explored the neural basis of reward deficits in the context of depression, and has implicated mesocorticolimbic reward circuitry comprising the nucleus accumbens, ventral pallidum, ventral tegmental area, amygdala, hippocampus, anterior cingulate, insula, orbitofrontal cortex, and other prefrontal cortex regions. Dopamine modulates several reward facets including anticipation, motivation, effort, and learning. As well, serotonin, norepinephrine, opioids, glutamate, Gamma aminobutyric acid (GABA), and acetylcholine are also involved in anhedonia, and medications targeting these systems may also potentially normalize reward processing in depression. Unfortunately, whereas reward anticipation and reward outcome are extensively explored by both preclinical and clinical studies, translational gaps remain in reward motivation, effort, valuation, and learning, where clinical neuroimaging studies are in the early stages. This review aims to synthesize the neurobiological mechanisms underlying anhedonia in MDD uncovered by preclinical and clinical research. The translational difficulties in studying the neural basis of reward are also discussed.

Acute and chronic bupropion treatment does not prevent morphine-induced conditioned place preference in mice

A substantial barrier to the treatment of Opioid Use Disorder (OUD) is the elevated relapse rates in affected patients, and a significant contributor to these events of relapse is exposure to cues and contexts that are intensely associated with prior drug abuse. The neurotransmitter dopamine plays a key role in reward-related behaviors, and previous studies have illustrated that dopamine hypofunction in periods of abstinence serves to prompt drug craving and seeking. We hypothesized that restoration of dopaminergic signaling could attenuate drug-seeking behaviors. Therefore, we investigated whether use of an FDA-approved drug, bupropion, an inhibitor of the dopamine transporter (DAT), or a dopamine uptake inhibitor with high affinity for DAT, JHW 007, was able to decrease preference for a drug-paired context. In these experiments, mice underwent 5 days of non-contingent morphine (10 mg/kg) exposure in a conditioned place preference (CPP) paradigm. We found that systemic injection of bupropion (20 mg/kg, i. p.) or intracranial injection of JHW 007 into the nucleus accumbens shell did not prevent the expression of morphine CPP. We then investigated whether chronic bupropion treatment (via implanted osmotic pumps) would influence morphine CPP. We observed that chronic bupropion treatment for 21 days following morphine conditioning did not attenuate the prolonged preference for morphine-paired contexts. Overall, with our dose and paradigm, neither acute nor chronic bupropion diminishes morphine CPP. Continued studies should address FDA-approved medications and their potential for recovery in OUD patients.

Effects of brief inhibition of the ventral tegmental area dopamine neurons on the cocaine seeking during abstinence

Preclinical studies strongly suggest that cocaine seeking depends on the neuronal activity of the ventral tegmental area (VTA) and phasic dopaminergic (DA) signaling. Notably, VTA pharmacological inactivation or dopamine receptor blockade in the forebrain may induce behavioral inhibition in general and acute aversive states in particular, thus reducing cocaine seeking indirectly. Such artifacts hinder successful translation of these findings in clinical studies and practice. Here, we aimed to evaluate if dynamic VTA manipulations effectively reduce cocaine seeking. We used male tyrosine hydroxylase (TH) IRES-Cre⁺ rats along with optogenetic tools to inhibit directly and briefly VTA DA neurons during conditioned stimulus (CS)-induced cocaine seeking under extinction conditions. The behavioral effects of optogenetic inhibition were also assessed in the real-time dynamic place aversion, conditioned place aversion, and CS-induced food-seeking tests. We found that brief and nondysphoric/nonsedative pulses of VTA photo-inhibition (1 s every 9 s, ie, for 10% of time) attenuated CS-induced cocaine seeking under extinction conditions in rats expressing archaerhodopsin selectively on the TH⁺ neurons. Furthermore, direct inhibition of the VTA DA activity reduced CS-induced cocaine seeking 24 hours after photo-modulation. Importantly, such effect appears to be selective for cocaine seeking as similar inhibition of the VTA DA activity had no effect on CS-induced food seeking. Thus, briefly inhibiting VTA DA activity during CS-induced cocaine seeking drastically and selectively reduces seeking without behavioral artifacts such as sedation or dysphoria. Our results point to the therapeutic possibilities of coupling nonpharmacologic treatments with extinction training in reducing cocaine addiction.

Cue-Evoked Dopamine Neuron Activity Helps Maintain but Does Not Encode Expected Value

Cue-evoked midbrain dopamine (DA) neuron activity reflects expected value, but its influence on reward assessment is unclear. In mice performing a trial-based operant task, we test if bidirectional manipulations of cue or operant-associated DA neuron activity drive learning as a result of under- or overexpectation of reward value. We target optogenetic manipulations to different components of forced trials, when only one lever is presented, and assess lever biases on choice trials in the absence of photomanipulation. Although lever biases are demonstrated to be flexible and sensitive to changes in expected value, augmentation of cue or operant-associated DA signaling does not significantly alter choice behavior, and blunting DA signaling during any component of the forced trials reduces choice trial responses on the associated lever. These data suggest cue-evoked DA helps maintain cue-value associations but does not encode expected value as to set the benchmark against which received reward is judged.

Heterogeneity in striatal dopamine circuits: Form and function in dynamic reward seeking

The striatal dopamine system has long been studied in the context of reward learning, motivation, and movement. Given the prominent role dopamine plays in a variety of adaptive behavioral states, as well as diseases like addiction, it is essential to understand the full complexity of dopamine neurons and the striatal systems they target. A growing number of studies are uncovering details of the heterogeneity in dopamine neuron subpopulations. Here, we review that work to synthesize current understanding of dopamine system heterogeneity across three levels, anatomical organization, functions in behavior, and modes of action, wherein we focus on signaling profiles and local mechanisms for modulation of dopamine release. Together, these studies reveal new and emerging dimensions of the striatal dopamine system, informing its contribution to dynamic motivational and decision-making processes.

Biotechnical development of genetic addiction risk score (GARS) and selective evidence for inclusion of polymorphic allelic risk in substance use disorder (SUD)

Research into the neurogenetic basis of addiction identified and characterized by Reward Deficiency Syndrome (RDS) includes all drug and non-drug addictive, obsessive and compulsive behaviors. We are proposing herein that a new model for the prevention and treatment of Substance Use Disorder (SUD) a subset of RDS behaviors, based on objective biologic evidence, should be given serious consideration in the face of a drug epidemic. The development of the Genetic Addiction Risk Score (GARS) followed seminal research in 1990, whereby, Blum’s group identified the first genetic association with severe alcoholism published in JAMA. While it is true that no one to date has provided adequate RDS free controls there have been many studies using case –controls whereby SUD has been eliminated. We argue that this deficiency needs to be addressed in the field and if adopted appropriately many spurious results would be eliminated reducing confusion regarding the role of genetics in addiction. However, an estimation, based on these previous literature results provided herein, while not representative of all association studies known to date, this sampling of case- control studies displays significant associations between alcohol and drug risk. In fact, we present a total of 110,241 cases and 122,525 controls derived from the current literature. We strongly suggest that while we may take argument concerning many of these so-called controls (e.g. blood donors) it is quite remarkable that there are a plethora of case –control studies indicating selective association of these risk alleles ( measured in GARS) for the most part indicating a hypodopaminergia. The paper presents the detailed methodology of the GARS. Data collection procedures, instrumentation, and the analytical approach used to obtain GARS and subsequent research objectives are described. Can we combat SUD through early genetic risk screening in the addiction field enabling early intervention by the induction of dopamine homeostasis? It is envisaged that GARS type of screening will provide a novel opportunity to help identify causal pathways and associated mechanisms of genetic factors, psychological characteristics, and addictions awaiting additional scientific evidence including a future meta- analysis of all available data –a work in progress.

Phasic dopamine responses to a food-predictive cue are suppressed by the glucagon-like peptide-1 receptor agonist Exendin-4

Phasic dopamine activity is evoked by reliable predictors of food reward and plays a role in cue-triggered, goal-directed behavior. While this important signal is modulated by physiological state (e.g. hunger, satiety), the mechanisms by which physiological state is integrated by dopamine neurons is only beginning to be elucidated. Activation of central receptors for glucagon-like peptide-1 (GLP-1R) via long-acting agonists (e.g., Exendin-4) suppresses food intake and food-directed motivated behavior, in part, through action in regions with dopamine cell bodies, terminals, and/or neural populations that directly target the mesolimbic dopamine system. However, the effects of GLP-1R activation on cue-evoked, phasic dopamine signaling remain unknown. Here, in vivo fiber photometry was used to capture real-time signaling dynamics selectively from dopamine neurons in the ventral tegmental area of male and female transgenic (tyrosine hydroxylase-Cre; TH:Cre+) rats trained to associate an audio cue with the brief availability of a sucrose solution. Cue presentation evoked a brief spike in dopamine activity. Administration of Exendin-4 (Ex4; 0, 0.05, 0.1 μg) to the lateral ventricle both dose-dependently suppressed sucrose-directed behaviors and the magnitude of cue-evoked dopamine activity. Moreover, the amplitude of cue evoked dopamine activity was significantly correlated with subsequent sucrose-directed behaviors. While female rats exhibited overall reduced dopamine responses to the sucrose-paired cue relative to males, there was no significant interaction with Ex4. Together, these findings support a role for central GLP-1Rs in modulating a form of dopamine signaling that influences approach behavior and provide a potential mechanism whereby GLP-1 suppresses food-directed behaviors.

Time-dependent changes in cognitive flexibility performance during intermittent social stress: Relevance for motivation and reward-seeking behavior

Repeated exposure to stress produces cognitive impairments that can lead to psychiatric disorders. How and when these cognitive impairments occur during repeated exposure to stress is not well understood. The present study investigates the time course effects of the exposure to intermittent episodes of social stress on cognitive flexibility performance. Male rats were trained to perform a cognitive flexibility (set-shifting) task. Then they were submitted to intermittent social defeat stress, which consisted of exposing animals to social defeat once every three days for ten days (four stress episodes). Set-shifting performance was evaluated before and in between social stress episodes, and up to ten days after the end of the stress protocol. Plasma levels of corticosterone after tail-pinch were also measured after the last set-shifting session. Intermittent exposure to social stress did not impair cognitive flexibility but produced short-and long-term changes in set-shifting performance. After the third social defeat episode, stressed animals required significantly more time to respond to cues (instrumental action) and to poke in the food-trough when no food pellet was delivered. These effects were reversed ten days after stress and suggest a decrease in motivation to pursue rewards. In contrast, stressed animals responded more accurately to light cues that predicted rewards. This effect, which emerged ten days after stress, suggests an increase in the reactivity to salient cues in the long-term. These time-dependent behavioral changes might help to understand the transition from social stress to stress-related disorders including drug abuse.

Mesolimbic dopamine projections mediate cue-motivated reward seeking but not reward retrieval in rats

Efficient foraging requires an ability to coordinate discrete reward-seeking and reward-retrieval behaviors. We used pathway-specific chemogenetic inhibition to investigate how rats’ mesolimbic and mesocortical dopamine circuits contribute to the expression and modulation of reward seeking and retrieval. Inhibiting ventral tegmental area dopamine neurons disrupted the tendency for reward-paired cues to motivate reward seeking, but spared their ability to increase attempts to retrieve reward. Similar effects were produced by inhibiting dopamine inputs to nucleus accumbens, but not medial prefrontal cortex. Inhibiting dopamine neurons spared the suppressive effect of reward devaluation on reward seeking, an assay of goal-directed behavior. Attempts to retrieve reward persisted after devaluation, indicating they were habitually performed as part of a fixed action sequence. Our findings show that complete bouts of reward seeking and retrieval are behaviorally and neurally dissociable from bouts of reward seeking without retrieval. This dichotomy may prove useful for uncovering mechanisms of maladaptive behavior.

Quantity versus quality: Convergent findings in effort-based choice tasks

Organisms must frequently make cost-benefit decisions based on time, risk, and effort in choosing rewards to pursue. Various tasks have been developed to assess effort-based choice in rats, and experimenters have found largely similar results across tasks and brain regions. In this review, we focus primarily on the convergence of different effort-based choice tasks where quality or quantity of reward are manipulated. In the former, the rat is typically presented with the option to work for a preferred reward or select a less preferred, but freely-available reward. In such paradigms, the rewards are of different identities but are confirmed to differ qualitatively in value by a food preference task when both are freely-available. In the latter task type, rats are required to select between higher magnitude versus lower magnitudes of the same reward, but each with a similar effort requirement. We discuss the strengths/limitations of these paradigms, and describe brain regions that have been probed that result in converging or equivocal findings. Results are also reviewed with reference to a need for future work, and the broader impacts and implications of studies probing the mechanisms of effort.

Decreases in Cued Reward Seeking After Reward-Paired Inhibition of Mesolimbic Dopamine

Reward-paired optogenetic manipulation of dopamine neurons can increase or decrease behavioral responding to antecedent cues when subjects have the opportunity for new learning, in accordance with a dopamine-mediated error learning signal. Here we examined the impact of reward-paired dopamine neuron inhibition on behavioral responding to reward-predictive cues after subjects had learned. We trained male TH-IRES-Cre mice to lever press for food reward in a progressive ratio procedure, a 2-cue choice procedure, or when continuously reinforced; in all procedures, completion of the response requirement was signaled by an auditory cue presented prior to food delivery. After training, mice underwent successive sessions in which optogenetic inhibition of dopamine neurons was triggered during food receipt. Rather than mimic brief inhibitions associated with negative reward prediction errors, we applied inhibition throughout the ingestion period on each trial. We found in all procedures that optogenetic inhibition of dopamine neurons during reward receipt decreased behavioral responding to the preceding reward-predictive cue over days, a behavioral change observed during time periods without optogenetic neuronal inhibition. Extinction-like behavioral responding was selective for learned associations: it was observed in the 2-cue choice procedure in which each subject was trained on two associations and inhibition was paired with reward for only one of the associations. Thus, inhibition during reward receipt can decrease responding to reward-predictive cues, sharing some features of behavioral extinction. These findings suggest changes in mesolimbic dopaminergic transmission at the time of experienced reward impacts subsequent responding to cues in well-trained subjects as predicted for a learning signal.

The timing of action determines reward prediction signals in identified midbrain dopamine neurons

Animals adapt behavior in response to informative sensory cues using multiple brain circuits. The activity of midbrain dopamine (mDA) neurons is thought to convey a critical teaching signal: reward prediction error (RPE). Although RPE signals are thought to be essential to learning, little is known about the dynamic changes in mDA neuron activity as animals learn about novel sensory cues and appetitive rewards. Here we describe a large dataset of cell-attached recordings of identified dopaminergic neurons as naïve mice learned a novel cue-reward association. During learning mDA neuron activity results from summation of sensory cue-related and movement initiation-related response components. These components are both a function of reward expectation yet dissociable. Learning produces an increasingly precise coordination of action initiation following sensory cues that results in apparent RPE correlates. Our data thus provide new insights into circuit mechanisms underlying a critical computation in a highly conserved learning circuit.

The Psychopharmacology of Effort-Related Decision Making: Dopamine, Adenosine, and Insights into the Neurochemistry of Motivation

Effort-based decision making is studied using tasks that offer choices between high-effort options leading to more highly valued reinforcers versus low-effort/low-reward options. These tasks have been used to study the involvement of neural systems, including mesolimbic dopamine and related circuits, in effort-related aspects of motivation. Moreover, such tasks are useful as animal models of some of the motivational symptoms that are seen in people with depression, schizophrenia, Parkinson's disease, and other disorders. The present review will discuss the pharmacology of effort-related decision making and will focus on the use of these tasks for the development of drug treatments for motivational dysfunction. Research has identified pharmacological conditions that can alter effort-based choice and serve as models for depression-related symptoms (e.g., the vesicular monoamine transport-2 inhibitor tetrabenazine and proinflammatory cytokines). Furthermore, tests of effort-based choice have identified compounds that are particularly useful for stimulating high-effort work output and reversing the deficits induced by tetrabenazine and cytokines. These studies indicate that drugs that act by facilitating dopamine transmission, as well as adenosine A2A antagonists, are relatively effective at reversing effort-related impairments. Studies of effort-based choice may lead to the identification of drug targets that could be useful for treating motivational treatments that are resistant to commonly used antidepressants such as serotonin transport inhibitors.

Oxygen responses within the nucleus accumbens are associated with individual differences in effort exertion in rats

Goal‐directed motivated behaviour is crucial for everyday life. Such behaviour is often measured, in rodents, under a progressive ratio (PR) schedule of reinforcement. Previous studies have identified a few brain structures critical for supporting PR performance. However, the association between neural activity within these regions and individual differences in effort‐related behaviour is not known. Presently, we used constant potential in vivo oxygen amperometry, a surrogate for functional resonance imaging in rodents, to assess changes in tissue oxygen levels within the nucleus accumbens (NAc) and orbitofrontal cortex (OFC) in male Wistar rats performing a PR task. Within both regions, oxygen responses to rewards increased as the effort exerted to obtain the rewards was larger. Furthermore, higher individual breakpoints were associated with greater magnitude NAc oxygen responses. This association could not be explained by temporal confounds and remained significant when controlling for the different number of completed trials. Animals with higher breakpoints also showed greater magnitude NAc oxygen responses to rewards delivered independently of any behaviour. In contrast, OFC oxygen responses were not associated with individual differences in behavioural performance. The present results suggest that greater NAc oxygen responses following rewards, through a process of incentive motivation, may allow organisms to remain on task for longer and to overcome greater effort costs.

Challenges to Body Fluid Homeostasis Differentially Recruit Phasic Dopamine Signaling in a Taste-Selective Manner

The internal environment of an organism must remain stable to ensure optimal performance and ultimately survival. The generation of motivated behaviors is an adaptive mechanism for defending homeostasis. Although physiological state modulates motivated behaviors, the influence of physiological state on phasic dopamine signaling, an underlying neurobiological substrate of reward-driven behavior, is underexplored. Here, we use sodium depletion and water restriction, manipulations of body fluid homeostasis, to determine the flexibility and specificity of dopamine responses. Changes in dopamine concentration were measured using fast-scan cyclic voltammetry in the nucleus accumbens shell of male rats in response to intraoral infusions of fluids that either satisfied or did not satisfy homeostatic need. Increases in dopamine concentration during intraoral infusions were observed only under conditions of physiological deficit. Furthermore, dopamine increases were selective and limited to those that satisfied the need state of the animal. Thus, dopamine neurons track fluid balance and respond to salt and water stimuli in a state- and taste-dependent manner. Using Fluoro-Gold tracing and immunohistochemistry for c-Fos and Foxp2, a marker of sodium-deprivation responsive neurons, we revealed brainstem populations of neurons that are activated by sodium depletion and project directly to the ventral tegmental area. The identified projections may modulate dopamine neuron excitability and consequently the state-specific dopamine release observed in our experiments. This work illustrates the impact of physiological state on mesolimbic dopamine signaling and a potential circuit by which homeostatic disruptions are communicated to mesolimbic circuitry to drive the selective reinforcement of biologically-required stimuli under conditions of physiological need.SIGNIFICANCE STATEMENT Motivated behaviors arise during physiological need and are highly selective for homeostasis-restoring stimuli. Although phasic dopamine signaling has been shown to contribute to the generation of motivated behaviors, the state and stimulus specificity of phasic dopamine signaling is less clear. These studies use thirst and sodium appetite to show that dopamine neurons dynamically track body fluid homeostasis and respond to water and salt stimuli in a state- and taste-dependent manner. We also identify hindbrain sodium deprivation-responsive neurons that project directly to the ventral tegmental area, where dopamine neuron cell bodies reside. This work demonstrates command of homeostasis over dopamine signaling and proposes a circuit by which physiological need drives motivated behavior by state- and taste-selective recruitment of phasic dopamine signaling.

Dopamine, Effort-Based Choice, and Behavioral Economics: Basic and Translational Research

Operant behavior is not only regulated by factors related to the quality or quantity of reinforcement, but also by the work requirements inherent in performing instrumental actions. Moreover, organisms often make effort-related decisions involving economic choices such as cost/benefit analyses. Effort-based decision making is studied using behavioral procedures that offer choices between high-effort options leading to relatively preferred reinforcers vs. low effort/low reward choices. Several neural systems, including the mesolimbic dopamine (DA) system and other brain circuits, are involved in regulating effort-related aspects of motivation. Considerable evidence indicates that mesolimbic DA transmission exerts a bi-directional control over exertion of effort on instrumental behavior tasks. Interference with DA transmission produces a low-effort bias in animals tested on effort-based choice tasks, while increasing DA transmission with drugs such as DA transport blockers tends to enhance selection of high-effort options. The results from these pharmacology studies are corroborated by the findings from recent articles using optogenetic, chemogenetic and physiological techniques. In addition to providing important information about the neural regulation of motivated behavior, effort-based choice tasks are useful for developing animal models of some of the motivational symptoms that are seen in people with various psychiatric and neurological disorders (e.g., depression, schizophrenia, Parkinson’s disease). Studies of effort-based decision making may ultimately contribute to the development of novel drug treatments for motivational dysfunction.

Ventral pallidal encoding of reward-seeking behavior depends on the underlying associative structure

Despite its being historically conceptualized as a motor expression site, emerging evidence suggests the ventral pallidum (VP) plays a more active role in integrating information to generate motivation. Here, we investigated whether rat VP cue responses would encode and contribute similarly to the vigor of reward-seeking behaviors trained under Pavlovian versus instrumental contingencies, when these behavioral responses consist of superficially similar locomotor response patterns but may reflect distinct underlying decision-making processes. We find that cue-elicited activity in many VP neurons predicts the latency of instrumental reward seeking, but not of Pavlovian response latency. Further, disruption of VP signaling increases the latency of instrumental but not Pavlovian reward seeking. This suggests that VP encoding of and contributions to response vigor are specific to the ability of incentive cues to invigorate reward-seeking behaviors upon which reward delivery is contingent.

Sounds or other cues associated with receiving a reward can have a powerful effect on an individual’s behavior or emotions. For example, the sound of an ice cream truck might cause salivation and motivate an individual to stand in a long line. Cues may prompt specific actions necessary to receive a reward, for example, approaching the ice cream truck and paying to get an ice cream. This is called instrumental conditioning. Some cues predict reward delivery, without requiring a specific action. This is called Pavlovian conditioning. Pavlovian cues can still prompt actions, such as approaching the truck, even though the action is not required. But exactly what happens in the brain to generate these actions during the two types of learning, is unclear.

Learning more about these reward-driven brain mechanisms might help scientists to develop better treatments for people with addiction or other conditions that involve compulsive reward-seeking behavior. Currently, scientists do not know enough about how the brain triggers this kind of behavior or how these processes lead to relapse in individuals who have been abstinent. Basic studies on the brain mechanisms that trigger reward-seeking behavior are needed.

Now, Richard et al. show that a greater activity in neurons, or brain cells, in a part of the brain called the ventral pallidum predicts a faster response to a reward cue. In the experiments, some rats were trained to approach a certain location when they heard a particular sound in order to receive sugar water, a form of instrumental conditioning. Another group of rats underwent Pavlovian training and learned to expect sugar water every time they heard sound even if they did nothing. Both groups learned to approach the sugar water location when they heard the cue, despite the different training requirements.

Richard et al. measured the activity of neurons in the ventral pallidum when the rats in the two groups heard the reward-associated sound. The experiments showed that the amount of activity in the brain cells in this area predicted whether a rat would approach the sugar-water delivery area and how quickly they would approach the reward after hearing the cue. The predictions were most reliable for rats that had to do something to get the sugar water. When Richard et al. reduced the activity in these cells they found the rats took longer to approach the reward source, but only when this action was required to receive sugar water. The experiments show that the ventral pallidum may provide the motivation to undertake reward-seeking behavior.