D1 or D2 antagonism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consumption

From valence encoding to motivated behavior: A focus on the nucleus accumbens circuitry

How do our brains determine whether something is good or bad? The brain's ability to evaluate stimuli as positive or negative - by attributing valence - is fundamental to survival and decision-making. Different brain regions have been associated with valence encoding, including the nucleus accumbens (NAc). The NAc is predominantly composed of GABAergic medium spiny neurons (MSNs), which segregate into two distinct populations based on their dopamine receptor expression: D1-receptor-expressing (D1-MSNs) and D2-receptor-expressing neurons (D2-MSNs). Classical models propose a binary functional role, where D1-MSNs exclusively mediated reward and positive valence, while D2-MSNs processed aversion and negative valence. However, we now recognize that NAc MSN subpopulations operate in a more complex manner than previously thought, often working cooperatively rather than antagonistically in valence-related behaviors. This review synthesizes our current knowledge of valence-encoding neurocircuitry, with emphasis on the NAc. We examine electrophysiological, calcium imaging, optogenetic, chemogenetic and pharmacological studies detailing the contribution of NAc medium spiny neurons for rewarding and aversive responses. Finally, we explore emerging technical innovations that promise to advance our understanding of how the mammalian brain encodes valence and translates it into behavior.

A2A-Positive Neurons in the Nucleus Accumbens Core Regulate Effort Exertion

Previous work has implicated the nucleus accumbens (NAc) in the regulation of effort, defined as the amount of work an animal is willing to perform for a given reward, but little is known about the specific contributions of neuronal populations within the NAc to effort regulation. In this study, using male and female mice, we examined the contributions of direct pathway and indirect pathway neurons in the NAc core using an operant effort regulation task, in which the effort requirement is the number of lever presses needed for earning a food reward. Using optogenetics, we manipulated the activity of direct pathway spiny projection neurons (SPNs; dopamine D1-like, D1+) and indirect pathway SPNs (iSPNs; adenosine 2A receptor, A2A+). Activating dSPNs reduced lever pressing regardless of the effort requirement, as it elicited gnawing, a competing consummatory behavior. On the other hand, activating iSPNs in the NAc core (but not in the shell) reduced lever pressing in an effort-dependent manner: stimulation-induced reduction in performance was greater at higher press-to-reward ratio requirements. In contrast, optogenetically inhibiting NAc core iSPN output resulted in increased levels of effort exertion. Our results show that the indirect pathway output from the NAc core can bidirectionally regulate effort exertion.

Reduced sensitivity to cocaine effects and changes in mesocorticolimbic dopamine receptors in adolescent sexually active female rats

RATIONALE

The sexual behavior of the female rat is highly motivated, and the mesocorticolimbic dopaminergic system -involved in psychostimulants effects- has been implicated in its regulation. Female rats begin to express sexual behavior during adolescence, a period during which this system is not yet mature.

OBJECTIVE

To examine the impact of cocaine on sexual motivation and behavior of adolescent and adult female rats, and to determine the dopamine receptors binding in mesocorticolimbic areas of these females.

METHODS

The effect of acute administration of cocaine (0.0, 10.0, and 20.0 mg/kg, intraperitoneally) on the male´s incentive value for females and on their sexual behavior in late adolescent (45-55 days old) and adult (100-120 days old) rats was assessed during late proestrus. The binding of D1-like and D2-like receptors in the striatum and medial prefrontal cortex (mPFC) of adolescent and adult rats were determined by autoradiography.

RESULTS

Cocaine did not affect females´ preference for the male. However, 10 mg/kg of cocaine reduced the expression of sexual motivated responses and 20 mg/kg also diminished sexual receptivity exclusively in adult subjects. Moreover, cocaine-induced a more pronounced hyper-locomotion in adult than in late adolescent rats. Late adolescent females exhibited higher dopamine receptors binding in the mPFC and reduced D2-like receptors binding in the Nucleus Accumbens shell when compared to adults.

CONCLUSIONS

Late adolescent females are less sensitive than adults to the detrimental effects of cocaine on sexual behavior and locomotion. This phenomenon is accompanied by variation in dopamine receptors in mesocorticolimbic areas affected by this psychostimulant.

Mild forced exercise in young mice prevents anergia induced by dopamine depletion in late adulthood: Relation to CDNF and DARPP-32 phosphorylation patterns in nucleus accumbens

Mesolimbic dopamine (DA) plays a critical role in behavioral activation and exertion of effort in motivated behaviors. DA antagonism and depletion in nucleus accumbens (Nacb) induces anergia in effort-based decision-making tasks. Exercise improves motor function in Parkinson's disease (PD). However, the beneficial effects of physical exercise on anergia, a symptom present in many psychiatric and neurological pathologies needs to be studied. During 9 weeks, young CD1 male mice were trained to run at a moderate speed in automatically turning running wheels (RW) (forced exercise group) or locked in static RWs (control group) in 1 h daily sessions. Both groups were tested in a 3-choice-T-maze task developed for the assessment of preference between active (RW) vs. sedentary reinforcers, and vulnerability to DA depletion-induced anergia was studied after tetrabenazine administration (TBZ; VMAT-2 blocker). Exercise did not change spontaneous preferences, did not affect body weight, plasma corticosterone levels or measures of anxiety, but it increased the cerebral DA neurotrophic factor (CDNF) in Nacb core, suggesting a neuroprotective effect in this nucleus. After TBZ administration, only the non-trained group showed a shift in relative preferences from active to sedentary options, reducing time running but increasing consumption of pellets, thus showing a typical anergic but not anhedonic effect. Moreover, only in the non-trained group, phosphorylation of DARPP-32(Thr34) increased after TBZ administration. These results are the first to show that mild forced exercise carried out from a young age to adulthood could act on Nacb-related functions, and prevent the anergia-inducing effects of DA depletion.

Incentive motivation for palatable food blocked by intra-accumbens melanin-concentrating hormone (MCH) receptor-1 antagonist in female rats

Melanin-concentrating hormone (MCH) activity in the nucleus accumbens (Acb) has been shown to influence feeding behavior, yet this has not been characterized in terms of homeostatic vs. hedonic feeding processes. Hedonic feeding, driven by palatability rather than energy deficit, can be modeled through intra-Acb administration of the selective μ-opioid receptor agonist d-Ala2, NMe-Phe4, Glyol5-enkephalin (DAMGO), which preferentially increases consumption and incentive motivation to obtain preferred palatable food. Pharmacological activation of MCH 1 receptors (MCHR1) within Acb has been shown to promote general feeding of chow in males, but not females. However, the effects of MCH on the incentive motivation to obtain preferred palatable food have not been explored. Here, we investigated the role of MCHR1 within the Acb in DAMGO-induced incentive motivation to obtain a sucrose pellet reward. Female Sprague Dawley rats were trained and tested for operant responding under a progressive ratio (PR) breakpoint in response to concurrent intra-Acb administration of DAMGO (0 μg and 0.025 μg/.5 μl/side) immediately following intra-Acb administration of the MCHR1 antagonist (N-(3-{1-[4-(3,4-difluoro-phenoxy)-benzyl]-piperdin-4-yl}-4-methyl-phenyl)-isobutyramide (SNAP-94847; 0 μg, 1.5 μg, and 15 μg/.5 μl/side), in a counterbalanced fashion. As expected, DAMGO significantly increased PR breakpoint and overall active lever presses. SNAP-94847 did not influence PR breakpoint by itself, compared to vehicle; however, both 1.5 and 15 μg doses of SNAP-94847 significantly blocked the increased PR breakpoint produced by intra-Acb DAMGO. The results of the study demonstrate that Acb MCHR1 may play a specific role in the hedonically-driven motivation for palatable food in females.

VTA dopamine neurons are hyperexcitable in 3xTg-AD mice due to casein kinase 2-dependent SK channel dysfunction

Alzheimer's disease (AD) patients exhibit neuropsychiatric symptoms that extend beyond classical cognitive deficits, suggesting involvement of subcortical areas. Here, we investigated the role of midbrain dopamine (DA) neurons in AD using the amyloid + tau-driven 3xTg-AD mouse model. We found deficits in reward-based operant learning in AD mice, suggesting possible VTA DA neuron dysregulation. Physiological assessment revealed hyperexcitability and disrupted firing in DA neurons caused by reduced activity of small-conductance calcium-activated potassium (SK) channels. RNA sequencing from contents of single patch-clamped DA neurons (Patch-seq) identified up-regulation of the SK channel modulator casein kinase 2 (CK2), which we corroborated by immunohistochemical protein analysis. Pharmacological inhibition of CK2 restored SK channel activity and normal firing patterns in 3xTg-AD mice. These findings identify a mechanism of ion channel dysregulation in VTA DA neurons that could contribute to behavioral abnormalities in AD, paving the way for novel treatment strategies.

Pharmacological characterization of sex differences in the effects of dopaminergic drugs on effort-based decision making in rats

RATIONALE

Motivational dysfunctions related to effort exertion are common in psychiatric disorders. Dopamine systems regulate exertion of effort and effort-based choice in humans and rodents.

OBJECTIVES

Previous rodent studies mainly employed male rats, and it is imperative to conduct studies in male and female rats.

METHODS

The present studies compared the effort-related effects of IP injections of the dopamine antagonists ecopipam and haloperidol, and the vesicular monoamine transport-2 inhibitor tetrabenazine (TBZ), in male and female rats using the fixed ratio 5/chow feeding choice task.

RESULTS

Ecopipam (0.05-0.2 mg/kg) and haloperidol (0.05-0.15 mg/kg) induced a low-effort bias, decreasing lever pressing and increasing chow intake in males and females in the same dose range. With lever pressing, there was a modest but significant dose x sex interaction after ecopipam injection, but there was no significant interaction after administration of haloperidol. In the first study with TBZ (0.25-1.0 mg/kg), there was a robust sex difference. TBZ shifted choice from lever pressing to chow intake in male rats, but was ineffective in females. In a second experiment, 2.0 mg/kg affected choice behavior in both males and females. TBZ increased accumbens c-Fos immunoreactivity in a sex-dependent manner, with males significantly increasing at 1.0 mg/kg, while females showed augmented immunoreactivity at 2.0 mg/kg.

CONCLUSIONS

The neural and behavioral effects of TBZ differed across sexes, emphasizing the importance of conducting studies in male and female rats. This research has implications for understanding the effort-related motivational dysfunctions seen in psychopathology.

Potential therapeutics for effort-related motivational dysfunction: assessing novel atypical dopamine transport inhibitors

People with depression and other neuropsychiatric disorders can experience motivational dysfunctions such as fatigue and anergia, which involve reduced exertion of effort in goal-directed activity. To model effort-related motivational dysfunction, effort-based choice tasks can be used, in which rats can select between obtaining a preferred reinforcer by high exertion of effort vs. a low effort/less preferred option. Preclinical data indicate that dopamine transport (DAT) inhibitors can reverse pharmacologically-induced low-effort biases and increase selection of high-effort options in effort-based choice tasks. Although classical DAT blockers like cocaine can produce undesirable effects such as liability for misuse and psychotic reactions, not all DAT inhibitors have the same neurochemical profile. The current study characterized the effort-related effects of novel DAT inhibitors that are modafinil analogs and have a range of binding profiles and neurochemical actions (JJC8-088, JJC8-089, RDS3-094, and JJC8-091) by using two different effort-related choice behavior tasks in male Sprague-Dawley rats. JJC8-088, JJC8-089, and RDS3-094 significantly reversed the low-effort bias induced by the VMAT-2 inhibitor tetrabenazine, increasing selection of high-effort fixed ratio 5 lever pressing vs. chow intake. In addition, JJC8-089 reversed the effects of tetrabenazine in female rats. JJC8-088 and JJC8-089 also increased selection of high-effort progressive ratio responding in a choice task. However, JJC8-091 failed to produce these outcomes, potentially due to its unique pharmacological profile (i.e., binding to an occluded conformation of DAT). Assessment of a broad range of DAT inhibitors with different neurochemical characteristics may lead to the identification of compounds that are useful for treating motivational dysfunction in humans.

Short-active gestational photoperiod reduces effortful choice behavior in mice, partial normalization by d-amphetamine

RATIONALE

Seasonal birth patterns consistently implicate winter gestation as a risk factor for several psychiatric conditions. We recently demonstrated that short-active (SA; 19:5 light:dark)-i.e., "winter-like"-photoperiod exposure across gestation and early life (E0-P28) induces psychiatrically relevant behavioral abnormalities in adult mice, including reduced immobility in the forced swim test (FST) and effortful amotivation. It is unknown, however, whether these effects were driven primarily by prenatal or postnatal mechanisms, and whether perinatal SA photoperiod would similarly reduce effort expenditure in a task relevant to everyday decision-making.

OBJECTIVES AND METHODS

We first tested male and female mice exposed to either gestational (E0-P0) or postnatal (E0-P28) SA photoperiod in the FST to determine whether the previously observed alteration was driven primarily by prenatal versus postnatal photoperiod. We then assessed whether SA gestational photoperiod reduces effortful choice behavior in the cross-species effort-based decision-making task (EBDMT) and whether any such deficit could be remediated by d-amphetamine (0.1 and 0.3 mg/kg, i.p.).

RESULTS

Mice exposed to prenatal, but not postnatal, SA photoperiod exhibited reduced FST immobility relative to controls and also demonstrated condition-dependently reduced preference for high-effort/high-reward versus low-effort/low-reward contingencies in the EBDMT. This effortful choice deficit was normalized by 0.1 mg/kg amphetamine.

CONCLUSIONS

These data: (1) suggest a greater contribution of gestational versus postnatal light conditions to the behavioral effects of perinatal SA photoperiod; and (2) implicate altered dopamine signaling in the behavioral phenotype of the SA-born mouse and possibly in the etiology of winter gestation-associated cases of psychiatric disease.

Not all discounts are created equal: Regional activity and brain networks in temporal and effort discounting

Reward outcomes associated with costs like time delay and effort investment are generally discounted in decision-making. Standard economic models predict rewards associated with different types of costs are devalued in a similar manner. However, our review of rodent lesion studies indicated partial dissociations between brain regions supporting temporal- and effort-based decision-making. Another debate is whether options involving low and high costs are processed in different brain substrates (dual-system) or in the same regions (single-system). This research addressed these issues using coordinate-based, connectivity-based, and activation network-based meta-analyses to identify overlapping and separable neural systems supporting temporal (39 studies) and effort (20 studies) discounting. Coordinate-based activation likelihood estimation and resting-state connectivity analyses showed immediate-small reward and delayed-large reward choices engaged distinct regions with unique connectivity profiles, but their activation network mapping was found to engage the default mode network. For effort discounting, salience and sensorimotor networks supported low-effort choices, while the frontoparietal network supported high-effort choices. There was little overlap between the temporal and effort networks. Our findings underscore the importance of differentiating different types of costs in decision-making and understanding discounting at both regional and network levels.

Neurosteroids and the mesocorticolimbic system

The mesocorticolimbic system coordinates executive functions, such as working memory and behavioral flexibility. This circuit includes dopaminergic projections from the ventral tegmental area to the nucleus accumbens and medial prefrontal cortex. In this review, we summarize evidence that cells in multiple nodes of the mesocorticolimbic system produce neurosteroids (steroids synthesized in the nervous system) and express steroid receptors. Here, we focus on neuroandrogens (androgens synthesized in the nervous system), neuroestrogens (estrogens synthesized in the nervous system), and androgen and estrogen receptors. We also summarize how (neuro)androgens and (neuro)estrogens affect dopamine signaling in the mesocorticolimbic system and regulate executive functions. Taken together, the data suggest that steroids produced in the gonads and locally in the brain modulate higher-order cognition and executive functions.

Administration of neuropeptide Y into the rat nucleus accumbens shell, but not core, attenuates the motivational impairment from systemic dopamine receptor antagonism by α-flupenthixol

Previous research has demonstrated that dopamine and Neuropeptide Y (NPY) promote motivated behavior, and there is evidence to suggest that they interact within neural circuitry involved in motivation. NPY and dopamine both modulate appetitive motivation towards food through direct actions in the nucleus accumbens (NAc), although how they interact in this region to promote motivation is presently unclear. In this study, we sought to further elucidate the relationship between NAc NPY and dopamine and their effects on motivated behavior. Specifically, we examined whether NAc injections of NPY might reverse behavioral deficits caused by reduced dopamine signaling due to systemic dopamine receptor antagonism. Appetitive motivation was measured using a progressive ratio-2 paradigm. Male Sprague Dawley rats were treated with systemic injections of the dopamine antagonist, α-flupenthixol or a saline vehicle. Two hours following injections, they were administered infusions of NPY (at 0, 156, or 235 pmol) into either the NAc shell (n = 12) or the NAc core (n = 10) and were placed in operant chambers. In both groups, α-flupenthixol impaired performance on the PR-2 task. NPY receptor stimulation of the NAc shell significantly increased both breakpoint and active lever presses during the PR-2 task, and dose-dependently increased responding following systemic dopamine receptor blockade. NPY did not affect appetitive motivation when injected into the NAc core. These data demonstrate that NPY in the NAc shell can improve motivational impairments that result from dopamine antagonism, and that these effects are site specific. These results also suggest that upregulation of NPY in neurodegenerative diseases may possibly buffer early motivational deficits caused by dopamine depletion in Parkinson's and Huntington's disease patients, both of which show increased NPY expression after disease onset.

Critical review of RDoC approaches to the study of motivation with animal models: effort valuation/willingness to work

The NIMH research domain criteria (RDoC) approach was instigated to refocus mental health research on the neural circuits that mediate psychological functions, with the idea that this would foster an understanding of the neural basis of specific psychiatric dysfunctions (i.e. 'symptoms and circuits') and ultimately facilitate treatment. As a general idea, this attempt to go beyond traditional diagnostic categories and focus on neural circuit dysfunctions related to specific symptoms spanning multiple disorders has many advantages. For example, motivational dysfunctions are present in multiple disorders, including depression, schizophrenia, Parkinson's disease, and other conditions. A critical aspect of motivation is effort valuation/willingness to work, and several clinical studies have identified alterations in effort-based decision making in various patient groups. In parallel, formal animal models focusing on the exertion of effort and effort-based decision making have been developed. This paper reviews the literature on models of effort-based motivational function in the context of a discussion of the RDoC approach, with an emphasis on the dissociable nature of distinct aspects of motivation. For example, conditions associated with depression and schizophrenia blunt the selection of high-effort activities as measured by several tasks in animal models (e.g. lever pressing, barrier climbing, wheel running). Nevertheless, these manipulations also leave fundamental aspects of hedonic reactivity, food motivation, and reinforcement intact. This pattern of effects demonstrates that the general emphasis of the RDoC on the specificity of the neural circuits mediating behavioral pathologies, and the dissociative nature of these dysfunctions, is a valid concept. Nevertheless, the specific placement of effort-related processes as simply a 'sub-construct' of 'reward processing' is empirically and conceptually problematic. Thus, while the RDoC is an excellent general framework for new ways to approach research and therapeutics, it still needs further refinement.

Effort-based decision making in response to high-dose androgens: role of dopamine receptors

INTRODUCTION

Anabolic-androgenic steroids (AAS) are performance-enhancing drugs used by both world-class and rank-and-file athletes. AAS abuse has been linked with risky decision-making, ranging from drunk driving to abusing multiple drugs. Our lab uses operant behavior in rats to test the effects of AAS (testosterone) on decision making. In our previous study, testosterone caused rats to work harder for food reward during an effort discounting (ED) task. ED is sensitive to dopamine in the nucleus accumbens, and AAS alter accumbens dopamine receptor expression. Accordingly, we determined if testosterone increases response to dopamine receptor antagonists during ED.

METHODS

Rats were treated chronically with high-dose testosterone (7.5 mg/kg; n = 9) or vehicle (n = 9). We measured baseline preference for the large reward in an ED task, where rats choose between a small easy reward (one lever press for one sugar pellet) and a large difficult reward (2, 5, 10, or 15 presses for three pellets). Preference for the large reward was measured after administration of D1-like (SCH23390, 0.01 mg/kg) or D2-like (eticlopride, 0.06 mg/kg) receptor antagonists.

RESULTS

At baseline, testosterone- and vehicle-treated rats showed similar preference for the large reward lever (FR5, testosterone: 68.6 ± 9.7% and vehicle: 85.7 ± 2.5%). SCH23390 reduced large reward preference significantly in both groups (FR5, testosterone: 41.3 ± 9.2%; vehicle: 49.1 ± 8.2%; F(1,16) = 17.7; P < 0.05). Eticlopride decreased large reward preference in both groups, but more strongly in testosterone-treated rats (FR5: testosterone: 37.0 ± 9.7%; vehicle: 56.3 ± 7.8%; F(1,16) = 35.3; P < 0.05).

CONCLUSION

Testosterone increases response to dopamine D2-like receptor blockade, and this contributes to previously observed changes in decision-making behaviors.

Vigor, Effort-Related Aspects of Motivation and Anhedonia

In this chapter we provide an overview of the pharmacological and circuit mechanisms that determine the willingness to expend effort in pursuit of rewards. A particular focus will be on the role of the mesolimbic dopamine system, as well the contributing roles of limbic and cortical brains areas involved in the evaluation, selection, and invigoration of goal-directed actions. We begin with a review of preclinical studies, which have provided key insights into the brain systems that are necessary and sufficient for effort-based decision-making and have characterized novel compounds that enhance selection of high-effort activities. Next, we summarize translational studies identifying and expanding this circuitry in humans. Finally, we discuss the relevance of this work for understanding common motivational impairments as part of the broader anhedonia symptom domain associated with mental illness, and the identification of new treatment targets within this circuitry to improve motivation and effort-expenditure.

Systemic D1-R and D2-R antagonists in Non-Human Primates Differentially Impact Learning and Memory While Impairing Motivation and Motor Performance

Dopamine (DA) modulates cognition in part via differential activation of D1 and D2 receptors within the striatum and prefrontal cortex, yet evidence for cognitive impairments stemming from DA blockade or deficiency is inconsistent. Given the predominance of D1 over D2 receptors (R) in the prefrontal cortex of primates, D1-R blockade should more strongly influence frontal executive function (including working memory), while D2-R blockade should impair processes more strongly associated with the dorsal striatum (including cognitive flexibility, and learning). To test how systemic DA blockade disrupts cognition, we administered D1-R and D2-R like antagonists to healthy monkeys while they performed a series of cognitive tasks. Two selective DA receptor antagonist drugs (SCH-23390 hydrochloride: D1/D5-R antagonist; or Eticlopride hydrochloride: D2/D3-R antagonist) or placebo (0.9% saline) were systemically administered. Four tasks were used: (1) 'visually guided reaching', to test response time and accuracy, (2) 'reversal learning', to test association learning and attention, (3) 'self-ordered sequential search' to test spatial working memory, and (4) 'delayed match to sample' to test object working memory. Increased reach response times and decreased motivation to work for liquid reward was observed with both the D1/D5-R and D2/D3-R antagonists at the maximum dosages that still enabled task performance. The D2/D3-R antagonist impaired performance in the reversal learning task, while object and spatial working memory performance was not consistently affected in the tested tasks for either drug. These results are consistent with the theory that systemic D2/D3-R antagonists preferentially influence striatum processes (cognitive flexibility) while systemic D1/D5-R administration is less detrimental to frontal executive function.

Towards a Unifying Account of Dopamine’s Role in Cost-Benefit Decision Making

Dopamine is thought to play a crucial role in cost-benefit decision making, but so far there is no consensus on the precise role of dopamine in decision making. Here, we review the literature on dopaminergic manipulations of cost-benefit decision making in humans and evaluate how well different theoretical accounts explain the existing body of evidence. Reduced D₂ stimulation tends to increase the willingness to bear delay and risk costs (i.e., wait for later rewards, take riskier options), while increased D₁ and D₂ receptor stimulation increases willingness to bear effort costs. We argue that the empirical findings can best be explained by combining the strengths of two theoretical accounts: in cost-benefit decision making, dopamine may play a dual role both in promoting the pursuit of psychologically close options (e.g., sooner and safer rewards) and in computing which costs are acceptable for a reward at stake. Moreover, we identify several limiting factors in the study designs of previous investigations that prevented a fuller understanding of dopamine's role in value-based choice. Together, the proposed theoretical framework and the methodological suggestions for future studies may bring us closer to a unifying account of dopamine in healthy and impaired cost-benefit decision making.

Adolescent high-fructose corn syrup consumption leads to dysfunction in adult affective behaviors and mesolimbic proteins in male Sprague-Dawley rats

Adolescence is a critical period of development, during which the brain undergoes rapid maturation. Problematically, adolescents are the top consumers of high fructose corn syrup (HFCS) sweetened beverages and snacks, which may have neurodevelopmental consequences. While HFCS consumption has been linked to an increased likelihood of obesity and other physical health impairments, the link between HFCS and persistent behavioral changes is not yet fully established. The present study aimed to assess whether adolescent HFCS consumption could lead to alterations in adult behaviors and protein expression, following cessation. Adolescent HFCS-exposure contributed to deficits in learning and motivation on an effort-related T-Maze procedure, as well as increased immobility time in the forced swim paradigm during adulthood. Molecularly, protracted decreases in accumbal dopamine D1 and D2 receptors and protein kinase G (PKG), as well as increases in tyrosine hydroxylase and GluA2 receptor subunits, were observed following HFCS-exposure. Taken together, these data suggest that adolescent HFCS-consumption leads to protracted dysfunction in affective behaviors and alterations in accumbal proteins which persist following cessation of HFCS-consumption.

The role of the nucleus accumbens and ventral pallidum in feeding and obesity

Obesity is a growing global epidemic that stems from the increasing availability of highly-palatable foods and the consequent enhanced calorie consumption. Extensive research has shown that brain regions that are central to reward seeking modulate feeding and evidence linking obesity to pathology in such regions have recently started to accumulate. In this review we focus on the contribution of two major interconnected structures central to reward processing, the nucleus accumbens and the ventral pallidum, to obesity. We first review the known literature linking these structures to feeding behavior, then discuss recent advances connecting pathology in the nucleus accumbens and ventral pallidum to obesity, and finally examine the similarities and differences between drug addiction and obesity in the context of these two structures. The understanding of how pathology in brain regions involved in reward seeking and consumption may drive obesity and how mechanistically similar obesity and addiction are, is only now starting to be revealed. We hope that future research will advance knowledge in the field and open new avenues to studying and treating obesity.

Preference for vigorous exercise versus sedentary sucrose drinking: an animal model of anergia induced by dopamine receptor antagonism

Motivation has activational and directional components. Mesolimbic dopamine is critical for the regulation of behavioral activation and effort-related processes in motivated behaviors. Impairing mesolimbic dopamine function leads to fatigue and anergia, but leaves intact other aspects of reinforce seeking behaviors, such as the consummatory or hedonic component. In male Swiss mice, we characterized the impact of dopamine antagonism on the selection of concurrently presented stimuli that have different vigor requirements. We analyzed running wheel activity versus sucrose solution intake, typically used as a measure of anhedonia. Results are compared with data from nonconcurrent presentation to those stimuli. In the concurrent presentation experiment, control mice preferred to spend time running compared to sucrose intake. Dopamine antagonism shifted relative reinforcer preference, reducing time spent on the running wheel, but actually increasing time-consuming sucrose. Mice increased frequency of bouts for both reinforcers, suggesting that there was fatigue in the running wheel rather than aversion. Moreover, satiation or habituation by preexposing animals to both reinforcers did not shift preferences. In the nonconcurrent experiments, haloperidol reduced running wheel but had no impact on sucrose consumption. Dopamine antagonism did not change preference for sucrose or total volume consumed. Additional correlational analyses indicated that baseline differences in sucrose consumption were independent of baseline running or novelty exploration. Thus, dopamine antagonism seems to have anergic rather than anhedonic effects, and the concurrent presentation in this setting could be useful for assessing preferences based on effort requirements.

Disruptions in effort-based decision-making following acute optogenetic stimulation of ventral tegmental area dopamine cells

The dopamine system has been implicated in decision-making particularly when associated with effortful behavior. We examined acute optogenetic stimulation of dopamine cells in the ventral tegmental area (VTA) as mice engaged in an effort-based decision-making task. Tyrosine hydroxylase-Cre mice were injected with Cre-dependent ChR2 or eYFP control virus in the VTA. While eYFP control mice showed effortful discounting, stimulation of dopamine cells in ChR2 mice disrupted effort-based decision-making by reducing choice toward the lever associated with a preferred outcome and greater effort. Surprisingly, disruptions in effortful discounting were observed in subsequent test sessions conducted in the absence of optogenetic stimulation, however during these sessions ChR2 mice displayed enhanced high choice responding across trial blocks. These findings suggest increases in VTA dopamine cell activity can disrupt effort-based decision-making in distinct ways dependent on the timing of optogenetic stimulation.

Shifting motivational states: The effects of nucleus accumbens dopamine and opioid receptor activation on a modified effort-based choice task

The nucleus accumbens (NAc) is critical for regulating the appetitive and consummatory phases of motivated behavior. These experiments examined the effects of dopamine and opioid receptor manipulations within the NAc during an effort-based choice task that allowed for simultaneous assessment of both phases of motivation. Male Sprague-Dawley rats received bilateral guide cannulas targeting the NAc core and were tested in 1-hr sessions with free access to rat chow and the choice to work for sugar pellets on a progressive ratio 2 (PR2) reinforcement schedule. Individual groups of rats were tested following stimulation or blockade of NAc D1-like or D2-like receptors, stimulation of μ-, δ-, or κ-opioid receptors, or antagonism of opioid receptors. Behavior was examined under ad libitum conditions and following 23-h food restriction. NAc blockade of the D1-like receptors or stimulation of the D2 receptor reduced break point for earning sugar pellets; D2 receptor stimulation also modestly lowered chow intake. NAc μ-opioid receptor stimulation increased intake of the freely-available chow while simultaneously reducing break point for the sugar pellets. In non-restricted conditions, δ-opioid receptor stimulation increased both food intake and breakpoint. There were no effects of stimulating NAc D1 or κ receptors, nor did blocking D2 or opioid receptors affect task behavior. These data support prior literature linking dopamine to appetitive motivational processes, and suggest that μ- and δ-opioid receptors affect food-directed motivation differentially. Specifically, μ-opioid receptors shifted behavior towards consumption, and δ-opioid receptor enhanced both sugar-seeking and consumption of the pabulum chow when animals were not food restricted.

The non-selective adenosine antagonist theophylline reverses the effects of dopamine antagonism on tremor, motor activity and effort-based decision-making

Considerable evidence indicates that adenosine and dopamine systems interact in the regulation of basal ganglia function. Nonselective adenosine antagonists such as the methylxanthine caffeine as well as selective adenosine A_2A antagonists have been shown to produce antiparkinsonian and antidepressant effects in animal models. The present studies were conducted to assess if another methylxantine, theophylline, can reverse motor and motivational impairments induced by dopamine antagonism in rats. RESULTS: Theophylline (3.75-30.0 mg/kg, IP) reversed tremulous jaw movements (TJMs), catalepsy, and locomotor suppression induced by the dopamine D2 antagonist pimozide. It also reversed TJMs induced by the muscarinic receptor agonist pilocarpine, which is a well-known tremorogenic agent. Parallel studies assessed the ability of theophylline (5.0-20.0 mg/kg, IP) to reverse the changes in effort-related choice behavior induced by the dopamine D1 antagonist ecopipam (0.2 mg/kg, IP) and the D2 antagonist haloperidol (0.1 mg/kg, IP). Rats were tested on two different operant choice tasks which assess the tendency to work for a preferred reinforcer by lever pressing (for palatable pellets or a high 5% sucrose solution) vs. approaching and consuming a less preferred reinforcer (freely available lab chow or a less concentrated 0.3% sucrose solution). Theophylline restored food and sucrose-reinforced lever pressing in animals treated with the D2 antagonist. However, it was unable to reverse the effects of the D1 antagonist. Overall, the effects of theophylline resembled those previously reported for adenosine A_2A antagonists, and suggest that theophylline could be clinically useful for the treatment of motor and motivational symptoms in humans.

Double dissociation between actions of dopamine D1 and D2 receptors of the ventral and dorsolateral striatum to produce reinstatement of cocaine seeking behavior

One of the hallmarks of addiction is the enduring vulnerability to relapse. Following repeated use, cocaine (COC) induces neuroadaptations within the dopamine (DA) system, arguably underlying several aspects of COC-seeking behavior. Peripheral stimulation of D2, but not D1, receptors induces relapse. However, where in the brain these effects occur is still matter of debate. The D1 and D2 receptors (D1R; D2R) are highly expressed in the nucleus accumbens (NAcc) and the dorsolateral striatum (DLS), but their specific involvement in the reinstatement of COC-seeking remains elusive. We assessed the reinstating effects of intracerebral infusions of agonists of D1R (SKF82958) or D2R (quinelorane) within the NAcc or DLS of rats after extinction of COC self-administration (COC SA). To assess whether we could block peripheral D2 agonist (quinelorane) induced reinstatement, we simultaneously infused either a D1R (SCH23390) or a D2R (raclopride) antagonist within the NAcc or DLS. When infused into the NAcc, but not into the DLS, SKF82958 induced reinstatement of COC-seeking; conversely, quinelorane had no effect when injected into the NAcc, but induced reinstatement when infused into the DLS while the D1R agonist has no effect. While administration of raclopride into the NAcc or DLS impedes the reinstating effect of a systemic quinelorane injection, the infusion of SCH23390 into the NAcc or DLS surprisingly, blocks the reinstatement induced by the peripheral D2R stimulation. Our results point to a double dissociation between D1R and D2R of the NAcc and DLS, highlighting their complex interactions within both structures, in the reinstatement of COC-seeking behavior.

Chemogenetic Modulation and Single-Photon Calcium Imaging in Anterior Cingulate Cortex Reveal a Mechanism for Effort-Based Decisions

The ACC is implicated in effort exertion and choices based on effort cost, but it is still unclear how it mediates this cost-benefit evaluation. Here, male rats were trained to exert effort for a high-value reward (sucrose pellets) in a progressive ratio lever-pressing task. Trained rats were then tested in two conditions: a no-choice condition where lever-pressing for sucrose was the only available food option, and a choice condition where a low-value reward (lab chow) was freely available as an alternative to pressing for sucrose. Disruption of ACC, via either chemogenetic inhibition or excitation, reduced lever-pressing in the choice, but not in the no-choice, condition. We next looked for value coding cells in ACC during effortful behavior and reward consumption phases during choice and no-choice conditions. For this, we used in vivo miniaturized fluorescence microscopy to reliably track responses of the same cells and compare how ACC neurons respond during the same effortful behavior where there was a choice versus when there was no-choice. We found that lever-press and sucrose-evoked responses were significantly weaker during choice compared with no-choice sessions, which may have rendered them more susceptible to chemogenetic disruption. Together, findings from our interference experiments and neural recordings suggest that a mechanism by which ACC mediates effortful decisions is in the discrimination of the utility of available options. ACC regulates these choices by providing a stable population code for the relative value of different options.

SIGNIFICANCE STATEMENT The ACC is implicated in effort-based decision-making. Here, we used chemogenetics and in vivo calcium imaging to explore its mechanism. Rats were trained to lever press for a high-value reward and tested in two conditions: a no-choice condition where lever-pressing for the high-value reward was the only option, and a choice condition where a low-value reward was also available. Inhibition or excitation of ACC reduced effort toward the high-value option, but only in the choice condition. Neural responses in ACC were weaker in the choice compared with the no-choice condition. A mechanism by which ACC regulates effortful decisions is in providing a stable population code for the discrimination of the utility of available options.

Inhibition of Catechol-O-methyltransferase Does Not Alter Effort-Related Choice Behavior in a Fixed Ratio/Concurrent Chow Task in Male Mice

Effort-related choice (ERC) tasks allow animals to choose between high-value reinforcers that require high effort to obtain and low-value/low-effort reinforcers. Dopaminergic neuromodulation regulates ERC behavior. The enzyme catechol-O-methyltransferase (COMT) metabolizes synaptically-released dopamine. COMT is the predominant regulator of dopamine turnover in regions of the brain with low levels of dopamine transporters (DATs), including the prefrontal cortex (PFC). Here, we evaluated the effects of the COMT inhibitor tolcapone on ERC performance in a touchscreen-based fixed-ratio/concurrent chow task in male mice. In this task, mice were given the choice between engaging in a fixed number of instrumental responses to acquire a strawberry milk reward and consuming standard lab chow concurrently available on the chamber floor. We found no significant effects of tolcapone treatment on either strawberry milk earned or chow consumed compared to vehicle treatment. In contrast, we found that haloperidol decreased instrumental responding for strawberry milk and increased chow consumption as seen in previously published studies. These data suggest that COMT inhibition does not significantly affect effort-related decision making in a fixed-ratio/concurrent chow task in male mice.

Dopamine D1 receptors in the medial orbitofrontal cortex support effort-related responding in rats

Rodent studies on effort-related responding provide a tool to analyze basal aspects of motivation and to model psychiatric motivational dysfunctions reflecting low exertion of effort or reduced behavioral activation. It turned out that dopamine (DA) signaling in brain areas such as nucleus accumbens are essential in regulating effort-related motivational function and could play a major role in motivational dysfunction in psychiatric disorders. Recent rodent studies revealed that the medial orbitofrontal cortex (mOFC) is another key component of the neural circuitry regulating effort-related motivational function. The mOFC receives prominent DA input, however, the behavioral role of mOFC DA signaling is unknown. Here, we investigated whether DA signaling in the mOFC supports effort-related responding in rats. Results demonstrate that an intra-mOFC D1 receptor blockade markedly reduced effort-related responding in a progressive ratio task. Notably, the magnitude of this effect was comparable to the one caused by a systemic DA depletion induced by the VMAT-2 inhibitor tetrabenazine or by a satiety-induced motivational downshift. Collectively, our data show for the first time that D1 receptor activity in the mOFC plays a critical role in high effort responding. These results support findings in humans pointing to a role of the mOFC in effort-related responding. It is well known that the mOFC becomes dysfunctional in depression and schizophrenia. Our data point to the possibility that reduced mOFC DA activity could contribute to effort-related motivational symptoms in these disorders and support the notion that the DA system may be a drug target to treat effort-related motivational symptoms.

The Triple Combination Phentermine Plus 5-HTP/Carbidopa Leads to Greater Weight Loss, With Fewer Psychomotor Side Effects Than Each Drug Alone

Obesity has become a serious public health problem. Although diet, surgery, and exercise are the primary treatments for obesity, these activities are often supplemented using appetite suppressants. A previous study reported that obesity specialists frequently prescribed a new drug combination for its treatment that includes phentermine (Phen; dopaminergic appetite suppressant), a serotonin (5-HT) precursor 5-hydroxytryptophan (5-HTP; an appetite suppressant that increases the 5-HT concentration), and carbidopa (CB; peripheral blocker of conversion of 5-HTP to 5-HT). Despite its widespread use, there is neither a preclinical study confirming the drug efficacy nor studies of its effects on the brain. To fill this gap, in rats for seven consecutive days, we administered Phen intraperitoneally at different doses either alone or in combination with a fixed dose of 5-HTP/CB. In a different group, we infused drugs via an intraperitoneal catheter while extracellular-recordings were performed in the nucleus accumbens shell (NAcSh), a brain region with dopamine-releasing effects that is involved in the action of appetite suppressants. We found that the triple-drug combination leads to greater weight-loss than each drug alone. Moreover, and as the treatment progresses, the triple drug combination partially reversed psychomotor side-effects induced by Phen. Electrophysiological results revealed that Phen alone evoked a net inhibitory imbalance in NAcSh population activity that correlated with the onset of psychomotor effects. In addition, and unlike the greater weight loss, the addition of 5-HTP/CB did not alter the Phen-evoked inhibitory imbalance in NAcSh responses. Subsequent experiments shed light on the underlying mechanism. That is the majority of NAcSh neurons modulated by 5-HTP/CB were suppressed by Phen. Notably, and despite acting via a different mechanism of action (DA for Phen vs. 5-HT for 5-HTP/CB), both drugs recruited largely overlapping NAcSh neuronal ensembles. These data suggest that the neural correlates of the greater weight loss could be located outside the NAcSh, in other brain circuits. Furthermore, we conclude that Phen + 5-HTP/CB is a potential treatment for overweight and obesity.

Alterations in effort-related decision-making induced by stimulation of dopamine D1, D2, D3, and corticotropin-releasing factor receptors in nucleus accumbens subregions

RATIONALE

Nucleus accumbens (NAc) dopamine (DA) plays an integral role in overcoming effort costs, as blockade of D₁ and D₂ receptors reduces the choice of larger, more-costly rewards. Similarly, the stress neuropeptide corticotropin-releasing factor (CRF) modulates DA transmission and mediates stress-induced alterations in effort-related choice.

OBJECTIVES

The current study explored how excessive stimulation of different DA receptors within the NAc core and shell alters effort-related decision-making and compared these effects to those induced by CRF stimulation.

METHODS

Male Long Evans rats were well-trained on an effort-discounting task wherein they choose between a low-effort/low-reward and a high-effort/high-reward lever where the effort requirement increased over blocks (2-20 presses). Dopamine D₁ (SKF 81297, 0.2-2 μg), D_2/3 (quinpirole, 1-10 μg), or D₃ (PD 128,907, 1.5-3 μg) receptor agonists, or CRF (0.5 μg), were infused into the NAc core or shell prior to testing.

RESULTS

Stimulation of D_2/3 receptors with quinpirole in the NAc core or shell markedly reduced the choice of high-effort option and increase choice latencies, without altering preference for larger vs smaller rewards. Stimulation of D₁ or D₃ receptors did not alter choice, although SKF 81297 infusions into the shell reduced response vigor. In comparison, core infusions of CRF flattened the discounting curve, reducing effortful choice when costs were low and increasing it when costs were high.

CONCLUSIONS

Excessive stimulation of NAc D₂ receptors has detrimental effects on effort-related decision-making. Furthermore, CRF stimulation induces dissociable effects on decision-making compared with those induced the effects of stimulation of different DA receptors.

Anabolic-androgenic steroids and cognitive effort discounting in male rats

Anabolic-androgenic steroids (AAS) are drugs of abuse that impair behavior and cognition. In a rodent model of AAS abuse, testosterone-treated male rats expend more physical effort, by repeatedly pressing a lever for a large reward in an operant discounting task. However, since modern society prioritizes cognitive over physical effort, it is important to determine if AAS limit cognitive effort. Here we tested the effects of AAS on a novel cognitive-effort discounting task. Each operant chamber had 3 nose-pokes, opposite 2 levers and a pellet dispenser. Rats pressed a lever to illuminate 1 nose-poke; they responded in the illuminated nose-poke to receive sugar pellets. For the 'easy' lever, the light remained on for 1 s, and a correct response earned 1 pellet. For the 'hard' lever, the light duration decreased from 1 s to 0.1 s across 5 blocks of trials, and a correct response earned 4 pellets. As the duration of the nose-poke light decreased, all rats decreased their choice of the hard lever in a modest discounting curve. Task accuracy also decreased significantly across the 5 blocks of trials. However, there was no effect of testosterone on choice of the hard lever or task accuracy. Antagonism of dopamine D1 or D2 receptors had no effect on lever choice or task accuracy. However, serotonin depletion significantly decreased preference for the hard lever, and impaired task accuracy. Thus, physical effort discounting depends on dopamine activity, while cognitive effort discounting task is sensitive to serotonin. AAS impair physical effort discounting, but not cognitive effort discounting.

Corticotropin-Releasing Factor (CRF) circuit modulation of cognition and motivation

The neuropeptide, corticotropin-releasing factor (CRF), is a key modulator of physiological, endocrine, and behavioral responses during stress. Dysfunction of the CRF system has been observed in stress-related affective disorders including post-traumatic stress disorder, depression, and anxiety. Beyond affective symptoms, these disorders are also characterized by impaired cognition, for which current pharmacological treatments are lacking. Thus, there is a need for pro-cognitive treatments to improve quality of life for individuals suffering from mental illness. In this review, we highlight research demonstrating that CRF elicits potent modulatory effects on higher-order cognition via actions within the prefrontal cortex and subcortical monoaminergic and cholinergic systems. Additionally, we identify questions for future preclinical research on this topic, such as the need to investigate sex differences in the cognitive and microcircuit actions of CRF, and whether CRF may represent a pharmacological target to treat cognitive dysfunction. Addressing these questions will provide new insight into pathophysiology underlying cognitive dysfunction and may lead to improved treatments for neuropsychiatric disorders.

Dopamine-glutamate neuron projections to the nucleus accumbens medial shell and behavioral switching

Dopamine (DA) neuron projections to the striatum are functionally heterogeneous with diverse behavioral roles. We focus here on DA neuron projections to the nucleus accumbens (NAc) medial Shell, their distinct anatomical and functional connections, and discuss their role in motivated behavior. We first review rodent studies showing that a subpopulation of DA neurons in the medial ventral tegmental area (VTA) project to the NAc medial Shell. Using a combinatorial strategy, we show that the majority of DA neurons projecting to the NAc Shell express vesicular glutamate transporter 2 (VGLUT2) making them capable of glutamate co-transmission (DA-GLU neurons). In the NAc dorsal medial Shell, all of the DA neuron terminals arise from DA-GLU neurons, while in the lateral NAc Shell, DA neuron terminals arise from both DA-GLU neurons and DA-only neurons, without VGLUT2. DA-GLU neurons make excitatory connections to the three major cells types, spiny projection neurons, fast-spiking interneuron and cholinergic interneurons (ChIs). The strongest DA-GLU neuron excitatory connections are to ChIs. Photostimulation of DA-GLU neuron terminals in the slice drives ChIs to burst fire. Finally, we review studies that address specially the behavioral function of this subpopulation of DA neurons in extinction learning and latent inhibition. Taking into account findings from anatomical and functional connectome studies, we propose that DA-GLU neuron connections to ChIs in the medial Shell play a crucial role in switching behavioral responses under circumstances of altered cue-reinforcer contingencies.

Quantity versus quality: Convergent findings in effort-based choice tasks

Organisms must frequently make cost-benefit decisions based on time, risk, and effort in choosing rewards to pursue. Various tasks have been developed to assess effort-based choice in rats, and experimenters have found largely similar results across tasks and brain regions. In this review, we focus primarily on the convergence of different effort-based choice tasks where quality or quantity of reward are manipulated. In the former, the rat is typically presented with the option to work for a preferred reward or select a less preferred, but freely-available reward. In such paradigms, the rewards are of different identities but are confirmed to differ qualitatively in value by a food preference task when both are freely-available. In the latter task type, rats are required to select between higher magnitude versus lower magnitudes of the same reward, but each with a similar effort requirement. We discuss the strengths/limitations of these paradigms, and describe brain regions that have been probed that result in converging or equivocal findings. Results are also reviewed with reference to a need for future work, and the broader impacts and implications of studies probing the mechanisms of effort.

Rodent Models of Adaptive Value Learning and Decision-Making

Real-world decisions are rarely as straightforward as choosing between clearly "good" vs. "bad" options. More often, options must be evaluated carefully because they differ in relative value. For example, we typically learn about (and make decisions between) options in comparison, where one outcome may be more costly or risky than the other. Several neuropsychiatric conditions are characterized by atypical evaluation of effort and risk costs, including major depression, schizophrenia, autism, obsessive-compulsive disorder, and substance use disorders. Aberrant value learning and decision-making have long been considered a cognitive-behavioral endophenotype of these disorders and can be modeled in rodents. This chapter presents two general methodological domains that the experimenter can manipulate in animal decision-making tasks: risk and effort. Here, we present detailed methods of rodent tasks frequently employed within these domains: probabilistic reversal learning (PRL) and effort choice. These tasks recruit regions within rodent frontal cortex, the amygdala, and the striatum, and performance is heavily modulated by dopamine, making these assays highly valid measures in the study of behavioral and substance addictions, in particular.

Differential effects of D1 and D2 dopamine agonists on memory, motivation, learning and response time in non-human primates

Dopamine (DA) plays a critical role in cognition, motivation and information processing. DA action has been shown to both improve and/or impair cognition across different receptor types, species, subjects and tasks. This complex relationship has been described as an inverted U-shaped function and may be due to the differential effects of DA receptor activation in the striatum and prefrontal cortex. We have investigated the effects of selective DA agonists on cognitive performance in healthy monkeys using a touch screen running tasks from the CAmbridge Neuropsychological Test Automated Battery (CANTAB). One of two DA agonist drugs or placebo was administered prior to each daily CANTAB session: Dihydrexidine hydrochloride (selective D1 agonist, 0.4-0.9 mg/kg), or sumanirole maleate (selective D2 agonist 0.05-0.3 mg/kg). Three CANTAB tasks were tested: (a) "self-ordered sequential search task" which tested spatial working memory, (b) "reversal learning task," which tested association learning, cognitive flexibility and attention and (c) "visually guided reaching task," which tested reaction time and accuracy. At high dosages, the D2 agonist improved spatial working memory performance, while impairing reversal learning and slowing reach response latency. No consistent cognitive effects were observed with the D1 agonist across the dosages tested. A significant decrease in trial completion rate was observed at the higher dosages of both the D1 and D2 agonists which were consistent with decreased motivation. These results are consistent with task-specific effects of a D2 agonist as well as dose specific insensitivities of a D1 agonist on cognitive and motor behaviors in a healthy monkey.

Striatal dopamine D2 receptors regulate effort but not value-based decision making and alter the dopaminergic encoding of cost

Deficits in goal-directed motivation represent a debilitating symptom for many patients with schizophrenia. Impairments in motivation can arise from deficits in processing information about effort and or value, disrupting effective cost-benefit decision making. We have previously shown that upregulated dopamine D2 receptor expression within the striatum (D2R-OE mice) decreases goal-directed motivation. Here, we determine the behavioral and neurochemical mechanisms behind this deficit. Female D2R-OE mice were tested in several behavioral paradigms including recently developed tasks that independently assess the impact of Value or Effort manipulations on cost-benefit decision making. In vivo microdialysis was used to measure extracellular dopamine in the striatum during behavior. In a value-based choice task, D2R-OE mice show normal sensitivity to changes in reward value and used reward value to guide their actions. In an effort-based choice task, D2R-OE mice evaluate the cost of increasing the number of responses greater relative to the effort cost of longer duration responses compared to controls. This shift away from choosing to repeatedly execute a response is accompanied by a dampening of extracellular dopamine in the striatum during goal-directed behavior. In the ventral striatum, extracellular dopamine level negatively correlates with response cost in controls, but this relationship is lost in D2R-OE mice. These results show that D2R signaling in the striatum, as observed in some patients with schizophrenia, alters the relationship between effort expenditure and extracellular dopamine. This dysregulation produces motivation deficits that are specific to effort but not value-based decision making, paralleling the effort-based motivational deficits observed in schizophrenia.

The Psychopharmacology of Effort-Related Decision Making: Dopamine, Adenosine, and Insights into the Neurochemistry of Motivation

Effort-based decision making is studied using tasks that offer choices between high-effort options leading to more highly valued reinforcers versus low-effort/low-reward options. These tasks have been used to study the involvement of neural systems, including mesolimbic dopamine and related circuits, in effort-related aspects of motivation. Moreover, such tasks are useful as animal models of some of the motivational symptoms that are seen in people with depression, schizophrenia, Parkinson's disease, and other disorders. The present review will discuss the pharmacology of effort-related decision making and will focus on the use of these tasks for the development of drug treatments for motivational dysfunction. Research has identified pharmacological conditions that can alter effort-based choice and serve as models for depression-related symptoms (e.g., the vesicular monoamine transport-2 inhibitor tetrabenazine and proinflammatory cytokines). Furthermore, tests of effort-based choice have identified compounds that are particularly useful for stimulating high-effort work output and reversing the deficits induced by tetrabenazine and cytokines. These studies indicate that drugs that act by facilitating dopamine transmission, as well as adenosine A2A antagonists, are relatively effective at reversing effort-related impairments. Studies of effort-based choice may lead to the identification of drug targets that could be useful for treating motivational treatments that are resistant to commonly used antidepressants such as serotonin transport inhibitors.

Dissociable roles of the nucleus accumbens D1 and D2 receptors in regulating cue-elicited approach-avoidance conflict decision-making

RATIONALE

Approach and avoidance decisions are made when an animal experiences a state of motivational conflict inflicted by stimuli imbued with both positive and negative valences. The nucleus accumbens (NAc), a site where valenced information and action selection converge, has recently been found to be critically involved in the resolution of approach-avoidance conflict. However, the individual roles of the region's dopamine receptor D1 (D1R)- and D2 (D2R)-expressing medium spiny neurons (MSNs) in regulating conflict resolution have not been well established.

OBJECTIVES

Here, we examined the roles of NAc D1R and D2R in cue-elicited approach-avoidance decision-making.

METHODS

Using a conditioned mixed-valence conflict paradigm, rats were initially trained in a radial maze to associate separate visuotactile cues with sucrose reward, foot shock punishment, and no outcome. Following acquisition of the cue-outcome associations, rats were subjected to a conditioned approach-avoidance conflict scenario, in which they were presented with a maze arm containing a superimposition of the reward and punishment cues, and another arm containing neutral cues.

RESULTS

Post-training intra-NAc D1R antagonism (SCH23390) led to an avoidance of the arm containing the mixed-valence cue over the neutral arm, whereas intra-NAc D2R antagonism (sulpiride) resulted in rats exhibiting a preference for the mixed-valence arm.

CONCLUSION

Our results suggest that NAc D1R and D2R exert differential control over decision-making involving cue-elicited approach-avoidance conflict resolution.

Phasic Dopamine Signals in the Nucleus Accumbens that Cause Active Avoidance Require Endocannabinoid Mobilization in the Midbrain

Phasic dopamine (DA) release accompanies approach toward appetitive cues. However, a role for DA in the active avoidance of negative events remains undetermined. Warning signals informing footshock avoidance are associated with accumbal DA release, whereas depression of DA is observed with unavoidable footshock. Here, we reveal a causal role of phasic DA in active avoidance learning; specifically, optogenetic activation of DA neurons facilitates avoidance, whereas optical inhibition of these cells attenuates it. Furthermore, stimulation of DA neurons during presentation of a fear-conditioned cue accelerates the extinction of a passive defensive behavior (i.e., freezing). Dopaminergic control of avoidance requires endocannabinoids (eCBs), as perturbing eCB signaling in the midbrain disrupts avoidance, which is rescued by optical stimulation of DA neurons. Interestingly, once the avoidance task is learned, neither DA nor eCB manipulations affect performance, suggesting that once acquisition occurs, expression of this behavior is subserved by other anatomical frameworks. Our findings establish an instrumental role for DA release in learning active responses to aversive stimuli and its control by eCB signaling.

Dopamine, Effort-Based Choice, and Behavioral Economics: Basic and Translational Research

Operant behavior is not only regulated by factors related to the quality or quantity of reinforcement, but also by the work requirements inherent in performing instrumental actions. Moreover, organisms often make effort-related decisions involving economic choices such as cost/benefit analyses. Effort-based decision making is studied using behavioral procedures that offer choices between high-effort options leading to relatively preferred reinforcers vs. low effort/low reward choices. Several neural systems, including the mesolimbic dopamine (DA) system and other brain circuits, are involved in regulating effort-related aspects of motivation. Considerable evidence indicates that mesolimbic DA transmission exerts a bi-directional control over exertion of effort on instrumental behavior tasks. Interference with DA transmission produces a low-effort bias in animals tested on effort-based choice tasks, while increasing DA transmission with drugs such as DA transport blockers tends to enhance selection of high-effort options. The results from these pharmacology studies are corroborated by the findings from recent articles using optogenetic, chemogenetic and physiological techniques. In addition to providing important information about the neural regulation of motivated behavior, effort-based choice tasks are useful for developing animal models of some of the motivational symptoms that are seen in people with various psychiatric and neurological disorders (e.g., depression, schizophrenia, Parkinson’s disease). Studies of effort-based decision making may ultimately contribute to the development of novel drug treatments for motivational dysfunction.

A Neural Circuit Mechanism for the Involvements of Dopamine in Effort-Related Choices: Decay of Learned Values, Secondary Effects of Depletion, and Calculation of Temporal Difference Error

Dopamine has been suggested to be crucially involved in effort-related choices. Key findings are that dopamine depletion (i) changed preference for a high-cost, large-reward option to a low-cost, small-reward option, (ii) but not when the large-reward option was also low-cost or the small-reward option gave no reward, (iii) while increasing the latency in all the cases but only transiently, and (iv) that antagonism of either dopamine D1 or D2 receptors also specifically impaired selection of the high-cost, large-reward option. The underlying neural circuit mechanisms remain unclear. Here we show that findings i–iii can be explained by the dopaminergic representation of temporal-difference reward-prediction error (TD-RPE), whose mechanisms have now become clarified, if (1) the synaptic strengths storing the values of actions mildly decay in time and (2) the obtained-reward-representing excitatory input to dopamine neurons increases after dopamine depletion. The former is potentially caused by background neural activity–induced weak synaptic plasticity, and the latter is assumed to occur through post-depletion increase of neural activity in the pedunculopontine nucleus, where neurons representing obtained reward exist and presumably send excitatory projections to dopamine neurons. We further show that finding iv, which is nontrivial given the suggested distinct functions of the D1 and D2 corticostriatal pathways, can also be explained if we additionally assume a proposed mechanism of TD-RPE calculation, in which the D1 and D2 pathways encode the values of actions with a temporal difference. These results suggest a possible circuit mechanism for the involvements of dopamine in effort-related choices and, simultaneously, provide implications for the mechanisms of TD-RPE calculation.

To Do or Not to Do: Dopamine, Affordability and the Economics of Opportunity

Five years ago, we introduced the thrift hypothesis of dopamine (DA), suggesting that the primary role of DA in adaptive behavior is regulating behavioral energy expenditure to match the prevailing economic conditions of the environment. Here we elaborate that hypothesis with several new ideas. First, we introduce the concept of affordability, suggesting that costs must necessarily be evaluated with respect to the availability of resources to the organism, which computes a value not only for the potential reward opportunity, but also the value of resources expended. Placing both costs and benefits within the context of the larger economy in which the animal is functioning requires consideration of the different timescales against which to compute resource availability, or average reward rate. Appropriate windows of computation for tracking resources requires corresponding neural substrates that operate on these different timescales. In discussing temporal patterns of DA signaling, we focus on a neglected form of DA plasticity and adaptation, changes in the physical substrate of the DA system itself, such as up- and down-regulation of receptors or release probability. We argue that changes in the DA substrate itself fundamentally alter its computational function, which we propose mediates adaptations to longer temporal horizons and economic conditions. In developing our hypothesis, we focus on DA D2 receptors (D2R), arguing that D2R implements a form of “cost control” in response to the environmental economy, serving as the “brain’s comptroller”. We propose that the balance between the direct and indirect pathway, regulated by relative expression of D1 and D2 DA receptors, implements affordability. Finally, as we review data, we discuss limitations in current approaches that impede fully investigating the proposed hypothesis and highlight alternative, more semi-naturalistic strategies more conducive to neuroeconomic investigations on the role of DA in adaptive behavior.

The pathophysiological mechanisms of motivational deficits in Parkinson's disease

Parkinson's disease (PD) is a progressive degenerative disorder that leads to disabling motor symptoms and a wide variety of neuropsychiatric symptoms. Apathy is the most common psychiatric disorder in the early stages of untreated PD and can be defined as a hypodopaminergic syndrome, which also includes anxiety and depression. Apathy is also considered the core feature of the parkinsonian triad (apathy, anxiety and depression) of behavioural non-motor signs, including a motivational deficit. Moreover, apathy is recognised as a distinct chronic neuropsychiatric behavioural disorder based on specific diagnostic criteria. Given the prevalence of apathy in approximately 40% of the general Parkinson's disease population, this appears to be a contributing factor to dementia in PD; also, apathy symptoms are factors that potentially contribute to morbidity, leading to a major impairment of health-related quality of life, thus stressing the importance of understanding the pathophysiology of this disease. Several studies have clearly established a prominent role for DA-mediated signals in PD apathy. However, synergistic interaction between dopaminergic impairment resulting from the neurodegenerative process and deep brain stimulation of the subthalamic nucleus may cause or exacerbate apathy. Furthermore, serotoninergic mechanism signalling is also likely to be of importance in this pathophysiology.

Androgen Regulation of the Mesocorticolimbic System and Executive Function

Multiple lines of evidence indicate that androgens, such as testosterone, modulate the mesocorticolimbic system and executive function. This review integrates neuroanatomical, molecular biological, neurochemical, and behavioral studies to highlight how endogenous and exogenous androgens alter behaviors, such as behavioral flexibility, decision making, and risk taking. First, we briefly review the neuroanatomy of the mesocorticolimbic system, which mediates executive function, with a focus on the ventral tegmental area (VTA), nucleus accumbens (NAc), medial prefrontal cortex (mPFC), and orbitofrontal cortex (OFC). Second, we present evidence that androgen receptors (AR) and other steroid receptors are expressed in the mesocorticolimbic system. Using sensitive immunohistochemistry and quantitative polymerase chain reaction (qPCR) techniques, ARs are detected in the VTA, NAc, mPFC, and OFC. Third, we describe recent evidence for local androgens ("neuroandrogens") in the mesocorticolimbic system. Steroidogenic enzymes are expressed in mesocorticolimbic regions. Furthermore, following long-term gonadectomy, testosterone is nondetectable in the blood but detectable in the mesocorticolimbic system, using liquid chromatography tandem mass spectrometry. However, the physiological relevance of neuroandrogens remains unknown. Fourth, we review how anabolic-androgenic steroids (AAS) influence the mesocorticolimbic system. Fifth, we describe how androgens modulate the neurochemistry and structure of the mesocorticolimbic system, particularly with regard to dopaminergic signaling. Finally, we discuss evidence that androgens influence executive functions, including the effects of androgen deprivation therapy and AAS. Taken together, the evidence indicates that androgens are critical modulators of executive function. Similar to dopamine signaling, there might be optimal levels of androgen signaling within the mesocorticolimbic system for executive functioning. Future studies should examine the regulation and functions of neurosteroids in the mesocorticolimbic system, as well as the potential deleterious and enduring effects of AAS use.

Endocannabinoid Actions on Cortical Terminals Orchestrate Local Modulation of Dopamine Release in the Nucleus Accumbens

Dopamine (DA) transmission mediates numerous aspects of behavior. Although DA release is strongly linked to firing of DA neurons, recent developments indicate the importance of presynaptic modulation at striatal dopaminergic terminals. The endocannabinoid (eCB) system regulates DA release and is a canonical gatekeeper of goal-directed behavior. Here we report that extracellular DA increases induced by selective optogenetic activation of cholinergic neurons in the nucleus accumbens (NAc) are inhibited by CB1 agonists and eCBs. This modulation requires CB1 receptors on cortical glutamatergic afferents. Dopamine increases driven by optogenetic activation of prefrontal cortex (PFC) terminals in the NAc are similarly modulated by activation of these CB1 receptors. We further demonstrate that this same population of CB1 receptors modulates optical self-stimulation sustained by activation of PFC afferents in the NAc. These results establish local eCB actions on PFC terminals within the NAc that inhibit mesolimbic DA release and constrain reward-driven behavior.

Deciphering Decision Making: Variation in Animal Models of Effort- and Uncertainty-Based Choice Reveals Distinct Neural Circuitries Underlying Core Cognitive Processes

Maladaptive decision-making is increasingly recognized to play a significant role in numerous psychiatric disorders, such that therapeutics capable of ameliorating core impairments in judgment may be beneficial in a range of patient populations. The field of "decision neuroscience" is therefore in its ascendancy, with researchers from diverse fields bringing their expertise to bear on this complex and fascinating problem. In addition to the advances in neuroimaging and computational neuroscience that contribute enormously to this area, an increase in the complexity and sophistication of behavioral paradigms designed for nonhuman laboratory animals has also had a significant impact on researchers' ability to test the causal nature of hypotheses pertaining to the neural circuitry underlying the choice process. Multiple such decision-making assays have been developed to investigate the neural and neurochemical bases of different types of cost/benefit decisions. However, what may seem like relatively trivial variation in behavioral methodologies can actually result in recruitment of distinct cognitive mechanisms, and alter the neurobiological processes that regulate choice. Here we focus on two areas of particular interest, namely, decisions that involve an assessment of uncertainty or effort, and compare some of the most prominent behavioral paradigms that have been used to investigate these processes in laboratory rodents. We illustrate how an appreciation of the diversity in the nature of these tasks can lead to important insights into the circumstances under which different neural regions make critical contributions to decision making.

Decreasing Striatopallidal Pathway Function Enhances Motivation by Energizing the Initiation of Goal-Directed Action

Altered dopamine D2 receptor (D2R) binding in the striatum has been associated with abnormal motivation in neuropsychiatric disorders, including schizophrenia. Here, we tested whether motivational deficits observed in mice with upregulated D2Rs (D2R-OEdev mice) are reversed by decreasing function of the striatopallidal "no-go" pathway. To this end, we expressed the Gαi-coupled designer receptor hM4D in adult striatopallidal neurons and activated the receptor with clozapine-N-oxide (CNO). Using a head-mounted miniature microscope we confirmed with calcium imaging in awake mice that hM4D activation by CNO inhibits striatopallidal function measured as disinhibited downstream activity in the globus pallidus. Mice were then tested in three operant tasks that address motivated behavior, the progressive ratio task, the progressive hold-down task, and outcome devaluation. Decreasing striatopallidal function in the dorsomedial striatum or nucleus accumbens core enhanced motivation in D2R-OEdev mice and control littermates. This effect was due to increased response initiation but came at the cost of goal-directed efficiency. Moreover, response vigor and the sensitivity to changes in reward value were not altered. Chronic activation of hM4D by administering CNO for 2 weeks in drinking water did not affect motivation due to a tolerance effect. However, the acute effect of CNO on motivation was reinstated after discontinuing chronic treatment for 48 h. Used as a therapeutic approach, striatopallidal inhibition should consider the risk of impairing goal-directed efficiency and behavioral desensitization.

SIGNIFICANCE STATEMENT

Motivation involves a directional component that allows subjects to efficiently select the behavior that will lead to an optimal outcome and an activational component that initiates and maintains the vigor and persistence of actions. Striatal output pathways modulate motivated behavior, but it remains unknown how these pathways regulate specific components of motivation. Here, we found that the indirect pathway controls response initiation without affecting response vigor or the sensitivity to changes in the reward outcome. A specific enhancement in the activational component of motivation, however, can come at the cost of goal-directed efficiency when a sustained response is required to obtain the goal. These data should inform treatment strategies for brain disorders with impaired motivation such as schizophrenia and Parkinson's disease.

Anterior cingulate cortex supports effort allocation towards a qualitatively preferred option

The anterior cingulate cortex (ACC) is known to be involved in effortful choice, yet its role in cost-benefit evaluation of qualitatively different rewards (more/less preferred), beyond magnitude differences (larger/smaller), is poorly understood. Selecting between qualitatively different options is a decision type commonly faced by humans. Here, we assessed the role of ACC on a task that has primarily been used to probe striatal function in motivation. Rats were trained to stable performance on a progressive ratio schedule for sucrose pellets and were then given sham surgeries (control) or excitotoxic NMDA lesions of ACC. Subsequently, a choice was introduced: chow was concurrently available while animals could work for the preferred sucrose pellets. ACC lesions produced a significant decrease in lever presses for sucrose pellets compared to control, whereas chow consumption was unaffected. Lesions had no effect on sucrose pellet preference when both options were freely available. When laboratory chow was not concurrently available, ACC-lesioned rats exhibited similar lever pressing as controls. During a test under specific satiety for sucrose pellets, ACC-lesioned rats also showed intact devaluation effects. The effects of ACC lesions in our task are not mediated by decreased appetite, a change in food preference, a failure to update value or a learning deficit. Taken together, we found that ACC lesions decreased effort for a qualitatively preferred option. These results are discussed with reference to effects of striatal manipulations and our recent report of a role for basolateral amygdala in effortful choice.

Feeding-modulatory effects of mu-opioids in the medial prefrontal cortex: a review of recent findings and comparison to opioid actions in the nucleus accumbens

RATIONALE

Whereas reward-modulatory opioid actions have been intensively studied in subcortical sites such as the nucleus accumbens (Acb), the role of cortical opioid transmission has received comparatively little attention.

OBJECTIVES

The objective of this study is to describe recent findings on the motivational actions of opioids in the prefrontal cortex (PFC), emphasizing studies of food motivation and ingestion. PFC-based opioid effects will be compared/contrasted to those elicited from the Acb, to glean possible common functional principles. Finally, the motivational effects of opioids will be placed within a network context involving the PFC, Acb, and hypothalamus.

RESULTS

Mu-opioid receptor (μ-OR) stimulation in both the Acb and PFC induces eating and enhances food-seeking instrumental behaviors; μ-OR signaling also enhances taste reactivity within a highly circumscribed zone of medial Acb shell. In both the Acb and PFC, opioid-sensitive zones are aligned topographically with the sectors that project to feeding-modulatory zones of the hypothalamus and intact glutamate transmission in the lateral/perifornical (LH-PeF) hypothalamic areas is required for both Acb- and PFC-driven feeding. Conversely, opioid-mediated feeding responses elicited from the PFC are negatively modulated by AMPA signaling in the Acb shell.

CONCLUSIONS

Opioid signaling in the PFC engages functionally opposed PFC➔hypothalamus and PFC➔Acb circuits, which, respectively, drive and limit non-homeostatic feeding, producing a disorganized and "fragmented" pattern of impulsive food-seeking behaviors and hyperactivity. In addition, opioids act directly in the Acb to facilitate food motivation and taste hedonics. Further study of this cortico-striato-hypothalamic circuit, and incorporation of additional opioid-responsive telencephalic structures, could yield insights with translational relevance for eating disorders and obesity.