The role of accumbens dopamine in lever pressing and response allocation: effects of 6-OHDA injected into core and dorsomedial shell

A2A-Positive Neurons in the Nucleus Accumbens Core Regulate Effort Exertion

Previous work has implicated the nucleus accumbens (NAc) in the regulation of effort, defined as the amount of work an animal is willing to perform for a given reward, but little is known about the specific contributions of neuronal populations within the NAc to effort regulation. In this study, using male and female mice, we examined the contributions of direct pathway and indirect pathway neurons in the NAc core using an operant effort regulation task, in which the effort requirement is the number of lever presses needed for earning a food reward. Using optogenetics, we manipulated the activity of direct pathway spiny projection neurons (SPNs; dopamine D1-like, D1+) and indirect pathway SPNs (iSPNs; adenosine 2A receptor, A2A+). Activating dSPNs reduced lever pressing regardless of the effort requirement, as it elicited gnawing, a competing consummatory behavior. On the other hand, activating iSPNs in the NAc core (but not in the shell) reduced lever pressing in an effort-dependent manner: stimulation-induced reduction in performance was greater at higher press-to-reward ratio requirements. In contrast, optogenetically inhibiting NAc core iSPN output resulted in increased levels of effort exertion. Our results show that the indirect pathway output from the NAc core can bidirectionally regulate effort exertion.

Mild forced exercise in young mice prevents anergia induced by dopamine depletion in late adulthood: Relation to CDNF and DARPP-32 phosphorylation patterns in nucleus accumbens

Mesolimbic dopamine (DA) plays a critical role in behavioral activation and exertion of effort in motivated behaviors. DA antagonism and depletion in nucleus accumbens (Nacb) induces anergia in effort-based decision-making tasks. Exercise improves motor function in Parkinson's disease (PD). However, the beneficial effects of physical exercise on anergia, a symptom present in many psychiatric and neurological pathologies needs to be studied. During 9 weeks, young CD1 male mice were trained to run at a moderate speed in automatically turning running wheels (RW) (forced exercise group) or locked in static RWs (control group) in 1 h daily sessions. Both groups were tested in a 3-choice-T-maze task developed for the assessment of preference between active (RW) vs. sedentary reinforcers, and vulnerability to DA depletion-induced anergia was studied after tetrabenazine administration (TBZ; VMAT-2 blocker). Exercise did not change spontaneous preferences, did not affect body weight, plasma corticosterone levels or measures of anxiety, but it increased the cerebral DA neurotrophic factor (CDNF) in Nacb core, suggesting a neuroprotective effect in this nucleus. After TBZ administration, only the non-trained group showed a shift in relative preferences from active to sedentary options, reducing time running but increasing consumption of pellets, thus showing a typical anergic but not anhedonic effect. Moreover, only in the non-trained group, phosphorylation of DARPP-32(Thr34) increased after TBZ administration. These results are the first to show that mild forced exercise carried out from a young age to adulthood could act on Nacb-related functions, and prevent the anergia-inducing effects of DA depletion.

VTA dopamine neurons are hyperexcitable in 3xTg-AD mice due to casein kinase 2-dependent SK channel dysfunction

Alzheimer's disease (AD) patients exhibit neuropsychiatric symptoms that extend beyond classical cognitive deficits, suggesting involvement of subcortical areas. Here, we investigated the role of midbrain dopamine (DA) neurons in AD using the amyloid + tau-driven 3xTg-AD mouse model. We found deficits in reward-based operant learning in AD mice, suggesting possible VTA DA neuron dysregulation. Physiological assessment revealed hyperexcitability and disrupted firing in DA neurons caused by reduced activity of small-conductance calcium-activated potassium (SK) channels. RNA sequencing from contents of single patch-clamped DA neurons (Patch-seq) identified up-regulation of the SK channel modulator casein kinase 2 (CK2), which we corroborated by immunohistochemical protein analysis. Pharmacological inhibition of CK2 restored SK channel activity and normal firing patterns in 3xTg-AD mice. These findings identify a mechanism of ion channel dysregulation in VTA DA neurons that could contribute to behavioral abnormalities in AD, paving the way for novel treatment strategies.

Chronically dysregulated corticosterone impairs dopaminergic transmission in the dorsomedial striatum by sex-divergent mechanisms

Major depressive disorder (MDD) is a leading cause of disability worldwide. Individuals with MDD exhibit decreased motivation and deficits in reward processing. In a subset of MDD patients, chronic dysregulation of the hypothalamic-pituitary-adrenal (HPA) axis occurs, resulting in increased levels of the 'stress hormone' cortisol during the normal rest period (i.e., evening and night). However, the mechanistic relationship between chronically elevated resting cortisol and behavioral deficits in motivation and reward processing remains unclear. Given that women are diagnosed with MDD at twice the rate of men, it is important to understand whether the mechanisms linking cortisol to the symptoms of MDD differ by sex. In this study, we used subcutaneous implants to chronically elevate free plasma corticosterone (the rodent homolog of cortisol; 'CORT') during the rest period in male and female mice and examined changes in behavior and dopamine system function. We found that chronic CORT treatment impaired motivated reward-seeking in both sexes. In female but not male mice, CORT treatment reduced dopamine content in the dorsomedial striatum (DMS). In male but not female mice, CORT treatment impaired the function of the dopamine transporter (DAT) in DMS. From these studies, we conclude that chronic CORT dysregulation impairs motivation by impairing dopaminergic transmission in the DMS, but via different mechanisms in male and female mice. A better understanding of these sex-specific mechanisms could lead to new directions in MDD diagnosis and treatment.

What Can Reinforcement Learning Models of Dopamine and Serotonin Tell Us about the Action of Antidepressants?

Although evidence suggests that antidepressants are effective at treating depression, the mechanisms behind antidepressant action remain unclear, especially at the cognitive/computational level. In recent years, reinforcement learning (RL) models have increasingly been used to characterise the roles of neurotransmitters and to probe the computations that might be altered in psychiatric disorders like depression. Hence, RL models might present an opportunity for us to better understand the computational mechanisms underlying antidepressant effects. Moreover, RL models may also help us shed light on how these computations may be implemented in the brain (e.g., in midbrain, striatal, and prefrontal regions) and how these neural mechanisms may be altered in depression and remediated by antidepressant treatments. In this paper, we evaluate the ability of RL models to help us understand the processes underlying antidepressant action. To do this, we review the preclinical literature on the roles of dopamine and serotonin in RL, draw links between these findings and clinical work investigating computations altered in depression, and appraise the evidence linking modification of RL processes to antidepressant function. Overall, while there is no shortage of promising ideas about the computational mechanisms underlying antidepressant effects, there is insufficient evidence directly implicating these mechanisms in the response of depressed patients to antidepressant treatment. Consequently, future studies should investigate these mechanisms in samples of depressed patients and assess whether modifications in RL processes mediate the clinical effect of antidepressant treatments.

Adolescent high-fructose corn syrup consumption leads to dysfunction in adult affective behaviors and mesolimbic proteins in male Sprague-Dawley rats

Adolescence is a critical period of development, during which the brain undergoes rapid maturation. Problematically, adolescents are the top consumers of high fructose corn syrup (HFCS) sweetened beverages and snacks, which may have neurodevelopmental consequences. While HFCS consumption has been linked to an increased likelihood of obesity and other physical health impairments, the link between HFCS and persistent behavioral changes is not yet fully established. The present study aimed to assess whether adolescent HFCS consumption could lead to alterations in adult behaviors and protein expression, following cessation. Adolescent HFCS-exposure contributed to deficits in learning and motivation on an effort-related T-Maze procedure, as well as increased immobility time in the forced swim paradigm during adulthood. Molecularly, protracted decreases in accumbal dopamine D1 and D2 receptors and protein kinase G (PKG), as well as increases in tyrosine hydroxylase and GluA2 receptor subunits, were observed following HFCS-exposure. Taken together, these data suggest that adolescent HFCS-consumption leads to protracted dysfunction in affective behaviors and alterations in accumbal proteins which persist following cessation of HFCS-consumption.

Effects of average reward rate on vigor as a function of individual variation in striatal dopamine

RATIONALE

We constantly need to decide not only which actions to perform, but also how vigorously to perform them. In agreement with an earlier theoretical model, it has been shown that a significant portion of the variance in our action vigor can be explained by the average rate of rewards received for that action. Moreover, this invigorating effect of average reward rate was shown to vary with within-subject changes in dopamine, both in human individuals and experimental rodents.

OBJECTIVES

Here, we assessed whether individual differences in the effect of average reward rate on vigor are related to individual variation in a stable measure of striatal dopamine function in healthy, unmedicated participants.

METHODS

Forty-four participants performed a discrimination task to test the effect of average reward rate on response times to index vigor and completed an [¹⁸F]-DOPA PET scan to index striatal dopamine synthesis capacity.

RESULTS

We did not find an interaction between dopamine synthesis capacity and average reward rate across the entire group. However, a post hoc analysis revealed that participants with higher striatal dopamine synthesis capacity, particularly in the nucleus accumbens, exhibited a stronger invigorating effect of average reward rate among the 30 slowest participants.

CONCLUSIONS

Our findings provide converging evidence for a role of striatal dopamine in average reward rate signaling, thereby extending the current literature on the mechanistic link between average reward rate, vigor, and dopamine.

How changes in dopamine D2 receptor levels alter striatal circuit function and motivation

It was first posited, more than five decades ago, that the etiology of schizophrenia involves overstimulation of dopamine receptors. Since then, advanced clinical research methods, including brain imaging, have refined our understanding of the relationship between striatal dopamine and clinical phenotypes as well as disease trajectory. These studies point to striatal dopamine D2 receptors, the main target for all current antipsychotic medications, as being involved in both positive and negative symptoms. Simultaneously, animal models have been central to investigating causal relationships between striatal dopamine D2 receptors and behavioral phenotypes relevant to schizophrenia. We begin this article by reviewing the circuit, cell-type and subcellular locations of dopamine D2 receptors and their downstream signaling pathways. We then summarize results from several mouse models in which D2 receptor levels were altered in various brain regions, cell-types and developmental periods. Behavioral, electrophysiological and anatomical consequences of these D2 receptor perturbations are reviewed with a selective focus on striatal circuit function and alterations in motivated behavior, a core negative symptom of schizophrenia. These studies show that D2 receptors serve distinct physiological roles in different cell types and at different developmental time points, regulating motivated behaviors in sometimes opposing ways. We conclude by considering the clinical implications of this complex regulation of striatal circuit function by D2 receptors.

Obese Subjects without Eating Disorders Experience Binge Episodes Also Independently of Emotional Eating and Personality Traits among University Students of Southern Italy

It is widely acknowledged that obesity is a growing public clinical issue involving both physical and psychological well-being. Nevertheless, the relationship between psychological features and weight gain is still unclear. Although emotional eating (EE) and personality traits are considered significant predictors of eating disorders, their role in obesity without eating disorders (OB-wed) is far from proven. The present study aimed at investigating the cumulative effect of EE and personality traits on overeating behavior in a sample of 266 university students (169 female; mean age = 21.85, SD = 2.39) stratified based on their body mass index (BMI; normal weight, overweight, obese). They were enrolled during free screening days promoted by the Human Dietetic and Sport Service of a Southern Italian university. The results show a psychological pattern of increasing overeating behavior and lower Self-Directedness combined with higher Sadness and Anger. However, OB-wed subjects overate regardless of this emotional/personological configuration.

Rational inattention and tonic dopamine

Slow-timescale (tonic) changes in dopamine (DA) contribute to a wide variety of processes in reinforcement learning, interval timing, and other domains. Furthermore, changes in tonic DA exert distinct effects depending on when they occur (e.g., during learning vs. performance) and what task the subject is performing (e.g., operant vs. classical conditioning). Two influential theories of tonic DA-the average reward theory and the Bayesian theory in which DA controls precision-have each been successful at explaining a subset of empirical findings. But how the same DA signal performs two seemingly distinct functions without creating crosstalk is not well understood. Here we reconcile the two theories under the unifying framework of 'rational inattention,' which (1) conceptually links average reward and precision, (2) outlines how DA manipulations affect this relationship, and in so doing, (3) captures new empirical phenomena. In brief, rational inattention asserts that agents can increase their precision in a task (and thus improve their performance) by paying a cognitive cost. Crucially, whether this cost is worth paying depends on average reward availability, reported by DA. The monotonic relationship between average reward and precision means that the DA signal contains the information necessary to retrieve the precision. When this information is needed after the task is performed, as presumed by Bayesian inference, acute manipulations of DA will bias behavior in predictable ways. We show how this framework reconciles a remarkably large collection of experimental findings. In reinforcement learning, the rational inattention framework predicts that learning from positive and negative feedback should be enhanced in high and low DA states, respectively, and that DA should tip the exploration-exploitation balance toward exploitation. In interval timing, this framework predicts that DA should increase the speed of the internal clock and decrease the extent of interference by other temporal stimuli during temporal reproduction (the central tendency effect). Finally, rational inattention makes the new predictions that these effects should be critically dependent on the controllability of rewards, that post-reward delays in intertemporal choice tasks should be underestimated, and that average reward manipulations should affect the speed of the clock-thus capturing empirical findings that are unexplained by either theory alone. Our results suggest that a common computational repertoire may underlie the seemingly heterogeneous roles of DA.

'Liking' and 'wanting' in eating and food reward: Brain mechanisms and clinical implications

It is becoming clearer how neurobiological mechanisms generate 'liking' and 'wanting' components of food reward. Mesocorticolimbic mechanisms that enhance 'liking' include brain hedonic hotspots, which are specialized subregions that are uniquely able to causally amplify the hedonic impact of palatable tastes. Hedonic hotspots are found in nucleus accumbens medial shell, ventral pallidum, orbitofrontal cortex, insula cortex, and brainstem. In turn, a much larger mesocorticolimbic circuitry generates 'wanting' or incentive motivation to obtain and consume food rewards. Hedonic and motivational circuitry interact together and with hypothalamic homeostatic circuitry, allowing relevant physiological hunger and satiety states to modulate 'liking' and 'wanting' for food rewards. In some conditions such as drug addiction, 'wanting' is known to dramatically detach from 'liking' for the same reward, and this may also occur in over-eating disorders. Via incentive sensitization, 'wanting' selectively becomes higher, especially when triggered by reward cues when encountered in vulnerable states of stress, etc. Emerging evidence suggests that some cases of obesity and binge eating disorders may reflect an incentive-sensitization brain signature of cue hyper-reactivity, causing excessive 'wanting' to eat. Future findings on the neurobiological bases of 'liking' and 'wanting' can continue to improve understanding of both normal food reward and causes of clinical eating disorders.

A high-resolution fMRI approach to characterize functionally distinct neural pathways within dopaminergic midbrain and nucleus accumbens during reward and salience processing

Processing of reward and salience without reward association are known to critically rely on the dopamine system. A growing body of evidence from animal studies suggests that both functions may be subserved by distinct subregions in midbrain and ventral striatum, specifically nucleus accumbens (NAcc). Yet in vivo investigation of these brain structures in humans has been rare. Here we examined blood oxygen level dependent signals in response to frequently presented rewarding events and infrequently presented neutral events in 20 healthy subjects using high-resolution functional magnetic resonance imaging (fMRI) for imaging the human midbrain and NAcc. The present findings revealed distinct activation patterns in brain regions of interest, namely increased activation in substantia nigra pars compacta (SNc) and dorsolateral NAcc in response to neutral events, while the VTA and both the ventromedial and dorsolateral NAcc were significantly activated due to rewarding events. Moreover, psychophysiological interaction analyses demonstrated regionally specialized processing pathways, such as a dorsolateral pathway when processing salience per se, i.e. increased functional interactions between SNc, dorsolateral NAcc and dorsolateral and medial prefrontal cortex (PFC); and a ventromedial pathway during reward processing, i.e. increased functional coupling between VTA and ventromedial NAcc. Thus, these findings may not only accelerate the integration of animal models of brain function with human neuroscience but may also improve diagnosis and treatment in patients with neuropsychiatric disorders such as schizophrenia and depression in which dopaminergic dysfunction and aberrant attribution of salience have been implicated.

Heterogeneity in striatal dopamine circuits: Form and function in dynamic reward seeking

The striatal dopamine system has long been studied in the context of reward learning, motivation, and movement. Given the prominent role dopamine plays in a variety of adaptive behavioral states, as well as diseases like addiction, it is essential to understand the full complexity of dopamine neurons and the striatal systems they target. A growing number of studies are uncovering details of the heterogeneity in dopamine neuron subpopulations. Here, we review that work to synthesize current understanding of dopamine system heterogeneity across three levels, anatomical organization, functions in behavior, and modes of action, wherein we focus on signaling profiles and local mechanisms for modulation of dopamine release. Together, these studies reveal new and emerging dimensions of the striatal dopamine system, informing its contribution to dynamic motivational and decision-making processes.

Alterations in effort-related decision-making induced by stimulation of dopamine D1, D2, D3, and corticotropin-releasing factor receptors in nucleus accumbens subregions

RATIONALE

Nucleus accumbens (NAc) dopamine (DA) plays an integral role in overcoming effort costs, as blockade of D₁ and D₂ receptors reduces the choice of larger, more-costly rewards. Similarly, the stress neuropeptide corticotropin-releasing factor (CRF) modulates DA transmission and mediates stress-induced alterations in effort-related choice.

OBJECTIVES

The current study explored how excessive stimulation of different DA receptors within the NAc core and shell alters effort-related decision-making and compared these effects to those induced by CRF stimulation.

METHODS

Male Long Evans rats were well-trained on an effort-discounting task wherein they choose between a low-effort/low-reward and a high-effort/high-reward lever where the effort requirement increased over blocks (2-20 presses). Dopamine D₁ (SKF 81297, 0.2-2 μg), D_2/3 (quinpirole, 1-10 μg), or D₃ (PD 128,907, 1.5-3 μg) receptor agonists, or CRF (0.5 μg), were infused into the NAc core or shell prior to testing.

RESULTS

Stimulation of D_2/3 receptors with quinpirole in the NAc core or shell markedly reduced the choice of high-effort option and increase choice latencies, without altering preference for larger vs smaller rewards. Stimulation of D₁ or D₃ receptors did not alter choice, although SKF 81297 infusions into the shell reduced response vigor. In comparison, core infusions of CRF flattened the discounting curve, reducing effortful choice when costs were low and increasing it when costs were high.

CONCLUSIONS

Excessive stimulation of NAc D₂ receptors has detrimental effects on effort-related decision-making. Furthermore, CRF stimulation induces dissociable effects on decision-making compared with those induced the effects of stimulation of different DA receptors.

Altered monoamine levels in the dorsal striatum of the rat are associated with alterations in behavioural selection and motivation following peripheral nerve injury and acute stress

Chronic neuropathic pain and psychological stress interact to compromise goal-directed control over behaviour following mild psychological stress. The dorsomedial (DMS) and dorsolateral (DLS) striatum in the rat are crucial for the expression of goal-directed and habitual behaviours, respectively. This study investigated whether changes in monoamine levels in the DMS and DLS following nerve injury and psychological stress reflect these behavioural differences. Neuropathic pain was induced by a chronic constriction injury (CCI) of the sciatic nerve in Sprague-Dawley rats. Acute stress was induced using a 15-min restraint. Behavioural flexibility was assessed using the outcome devaluation paradigm. Noradrenaline, serotonin, dopamine and associated metabolites were measured bilaterally from the DLS and DMS. In uninjured rats, restraint increased dopaminergic markers in the left and serotonergic markers in the right of both the DMS and DLS, indicating a possible left hemisphere-mediated dominance. CCI led to a slightly different lateralised effect, with a larger effect in the DMS than in the DLS. Individual differences in behavioural flexibility following CCI negatively correlated with dopaminergic markers in the right DLS, but positively correlated with these markers in the left DMS. A combination of CCI and restraint reduced behavioural flexibility, which was associated with the loss of the left/DMS dominance. These data suggest that behavioural flexibility following psychological stress or pain is associated with a left hemisphere dominance within the dorsal striatum. The loss of behavioural flexibility following the combined stressors is then associated with a transition from left to right, and DMS to DLS dominance.

Effort Displayed During Appetitive Phase of Feeding Behavior Requires Infralimbic Cortex Activity and Histamine H1 Receptor Signaling

The chances to succeed in goal-directed behaviors, such as food or water-seeking, improve when the subject is in an increased arousal state. The appetitive phase of these motivated behaviors is characterized by high levels of behavioral and vegetative excitation. The key decision of engaging in those particular behaviors depends primarily on prefrontal cortical areas, such as the ventromedial prefrontal cortex. We propose that the infralimbic cortex (ILC) located in the medial prefrontal cortex induces an increase in arousal during the appetitive phase of motivated behavior, and that this increase in arousal is, in turn, mediated by the activation of the brain histaminergic system, resulting in higher motivation for getting food rewards. To test this hypothesis, we conduct a progressive ratio operant conditioning to test the degree of motivation for food, while simultaneously manipulating the histaminergic system through pharmacologic interventions. We found that the behavioral responses to obtain food in hungry rats were disrupted when the ILC was inhibited through muscimol infusion, blocking brain H1 histamine receptors by intracerebroventricular infusion of pyrilamine or by satiety. In contrast, the consummatory behavior was not affected by ILC inhibition. The extracellular histamine levels in the ILC were increased in direct correlation with the degree of motivation measured in the progressive ratio test. ILC inhibition also prevented this increase in histamine levels. The rise in extracellular histamine levels during the progressive ratio test was similar (ca. 200%) during the active or the resting period of the day. However, different basal levels are observed for these two periods. Our findings suggest that increased histamine levels during this behavior are not simply explained by the awaked state, but instead, there is a motivation-related release of histamine, suggestive of a specific form of brain activation. Serotonin (another critical component of the ascending arousal system) was also tested. Interestingly, changes in levels of this neuromodulator were not detected during the progressive ratio test. In conclusion, our results suggest that ILC activation and subsequent increase in brain histamine release are both necessary for the normal performance of a motivated behavior such as feeding.

The Novel Atypical Dopamine Uptake Inhibitor (S)-CE-123 Partially Reverses the Effort-Related Effects of the Dopamine Depleting Agent Tetrabenazine and Increases Progressive Ratio Responding

Animal studies of effort-based choice behavior are being used to model effort-related motivational dysfunctions in humans. With these procedures, animals are offered a choice between high-effort instrumental actions leading to highly valued reinforcers vs. low effort/low reward options. Several previous studies have shown that dopamine (DA) uptake inhibitors, including GBR12909, lisdexamfetamine, methylphenidate, and PRX-14040, can reverse the effort-related effects of the vesicular monoamine transport blocker tetrabenazine, which inhibits DA storage. Because many drugs that block DA transport act as major stimulants that also release DA, and produce a number of undesirable side effects, there is a need to develop and characterize novel atypical DA transport inhibitors. (S)-CE-123 ((S)-5-((benzhydrylsulfinyl) methyl)thiazole) is a recently developed analog of modafinil with the biochemical characteristics of an atypical DA transport blocker. The present paper describes the enantioselective synthesis and initial chemical characterization of (S)-CE-123, as well as behavioral experiments involving effort-based choice and microdialysis studies of extracellular DA. Rats were assessed using the fixed ratio 5/chow feeding choice test. Tetrabenazine (1.0 mg/kg) shifted choice behavior, decreasing lever pressing and increasing chow intake. (S)-CE-123 was coadministered at doses ranging from 6.0 to 24.0 mg/kg, and the highest dose partially but significantly reversed the effects of tetrabenazine, although this dose had no effect on fixed ratio responding when administered alone. Additional experiments showed that (S)-CE-123 significantly increased lever pressing on a progressive ratio/chow feeding choice task and that the effective dose (24.0 mg/kg) increased extracellular DA in nucleus accumbens core. In summary, (S)-CE-123 has the behavioral and neurochemical profile of a compound that can block DA transport, reverse the effort-related effects of tetrabenazine, and increase selection of high-effort progressive ratio responding. This suggests that (S)-CE-123 or a similar compound could be useful as a treatment for effort-related motivational dysfunction in humans.

Quantity versus quality: Convergent findings in effort-based choice tasks

Organisms must frequently make cost-benefit decisions based on time, risk, and effort in choosing rewards to pursue. Various tasks have been developed to assess effort-based choice in rats, and experimenters have found largely similar results across tasks and brain regions. In this review, we focus primarily on the convergence of different effort-based choice tasks where quality or quantity of reward are manipulated. In the former, the rat is typically presented with the option to work for a preferred reward or select a less preferred, but freely-available reward. In such paradigms, the rewards are of different identities but are confirmed to differ qualitatively in value by a food preference task when both are freely-available. In the latter task type, rats are required to select between higher magnitude versus lower magnitudes of the same reward, but each with a similar effort requirement. We discuss the strengths/limitations of these paradigms, and describe brain regions that have been probed that result in converging or equivocal findings. Results are also reviewed with reference to a need for future work, and the broader impacts and implications of studies probing the mechanisms of effort.

Blockade of muscarinic acetylcholine receptors facilitates motivated behaviour and rescues a model of antipsychotic-induced amotivation

Disruptions to motivated behaviour are a highly prevalent and severe symptom in a number of neuropsychiatric and neurodegenerative disorders. Current treatment options for these disorders have little or no effect upon motivational impairments. We assessed the contribution of muscarinic acetylcholine receptors to motivated behaviour in mice, as a novel pharmacological target for motivational impairments. Touchscreen progressive ratio (PR) performance was facilitated by the nonselective muscarinic receptor antagonist scopolamine as well as the more subtype-selective antagonists biperiden (M1) and tropicamide (M4). However, scopolamine and tropicamide also produced increases in non-specific activity levels, whereas biperiden did not. A series of control tests suggests the effects of the mAChR antagonists were sensitive to changes in reward value and not driven by changes in satiety, motor fatigue, appetite or perseveration. Subsequently, a sub-effective dose of biperiden was able to facilitate the effects of amphetamine upon PR performance, suggesting an ability to enhance dopaminergic function. Both biperiden and scopolamine were also able to reverse a haloperidol-induced deficit in PR performance, however only biperiden was able to rescue the deficit in effort-related choice (ERC) performance. Taken together, these data suggest that the M1 mAChR may be a novel target for the pharmacological enhancement of effort exertion and consequent rescue of motivational impairments. Conversely, M4 receptors may inadvertently modulate effort exertion through regulation of general locomotor activity levels.

Dorsal raphe serotonin neurons inhibit operant responding for reward via inputs to the ventral tegmental area but not the nucleus accumbens: evidence from studies combining optogenetic stimulation and serotonin reuptake inhibition

The monoamine neurotransmitter serotonin (5-hydroxytryptamine; 5-HT) exerts an inhibitory influence over motivation, but the circuits mediating this are unknown. Here, we used an optogenetic approach to isolate the contribution of dorsal raphe nucleus (DRN) 5-HT neurons and 5-HT innervation of the mesolimbic dopamine (DA) system to motivated behavior in mice. We found that optogenetic stimulation of DRN 5-HT neurons enhanced downstream 5-HT release, but this was not sufficient to inhibit operant responding for saccharin, a measure of motivated behavior. However, combining optogenetic stimulation of DRN 5-HT neurons with a low dose of the selective serotonin reuptake inhibitor (SSRI) citalopram synergistically reduced operant responding. We then examined whether these effects could be recapitulated if optogenetic stimulation specifically targeted 5-HT terminals in the ventral tegmental area (VTA) or nucleus accumbens (NAc) of the mesolimbic DA system. Optogenetic stimulation of 5-HT input to the VTA combined with citalopram treatment produced a synergistic decrease in responding for saccharin, resembling the changes produced by targeting 5-HT neurons in the DRN. However, this effect was not observed when optogenetic stimulation targeted 5-HT terminals in the NAc. Taken together, these results suggest that DRN 5-HT neurons exert an inhibitory influence over operant responding for reward through a direct interaction with the mesolimbic DA system at the level of the VTA. These studies support an oppositional interaction between 5-HT and DA systems in controlling motivation and goal-directed behavior, and have important implications for the development and refinement of treatment strategies for psychiatric disorders such as depression and addiction.

The Psychopharmacology of Effort-Related Decision Making: Dopamine, Adenosine, and Insights into the Neurochemistry of Motivation

Effort-based decision making is studied using tasks that offer choices between high-effort options leading to more highly valued reinforcers versus low-effort/low-reward options. These tasks have been used to study the involvement of neural systems, including mesolimbic dopamine and related circuits, in effort-related aspects of motivation. Moreover, such tasks are useful as animal models of some of the motivational symptoms that are seen in people with depression, schizophrenia, Parkinson's disease, and other disorders. The present review will discuss the pharmacology of effort-related decision making and will focus on the use of these tasks for the development of drug treatments for motivational dysfunction. Research has identified pharmacological conditions that can alter effort-based choice and serve as models for depression-related symptoms (e.g., the vesicular monoamine transport-2 inhibitor tetrabenazine and proinflammatory cytokines). Furthermore, tests of effort-based choice have identified compounds that are particularly useful for stimulating high-effort work output and reversing the deficits induced by tetrabenazine and cytokines. These studies indicate that drugs that act by facilitating dopamine transmission, as well as adenosine A2A antagonists, are relatively effective at reversing effort-related impairments. Studies of effort-based choice may lead to the identification of drug targets that could be useful for treating motivational treatments that are resistant to commonly used antidepressants such as serotonin transport inhibitors.

Oxygen responses within the nucleus accumbens are associated with individual differences in effort exertion in rats

Goal‐directed motivated behaviour is crucial for everyday life. Such behaviour is often measured, in rodents, under a progressive ratio (PR) schedule of reinforcement. Previous studies have identified a few brain structures critical for supporting PR performance. However, the association between neural activity within these regions and individual differences in effort‐related behaviour is not known. Presently, we used constant potential in vivo oxygen amperometry, a surrogate for functional resonance imaging in rodents, to assess changes in tissue oxygen levels within the nucleus accumbens (NAc) and orbitofrontal cortex (OFC) in male Wistar rats performing a PR task. Within both regions, oxygen responses to rewards increased as the effort exerted to obtain the rewards was larger. Furthermore, higher individual breakpoints were associated with greater magnitude NAc oxygen responses. This association could not be explained by temporal confounds and remained significant when controlling for the different number of completed trials. Animals with higher breakpoints also showed greater magnitude NAc oxygen responses to rewards delivered independently of any behaviour. In contrast, OFC oxygen responses were not associated with individual differences in behavioural performance. The present results suggest that greater NAc oxygen responses following rewards, through a process of incentive motivation, may allow organisms to remain on task for longer and to overcome greater effort costs.

Chronic light exposure alters serotonergic and orexinergic systems in the rat brain and reverses maternal separation-induced increase in orexin receptors in the prefrontal cortex

Maternal separation (MS) is a well-established rodent model of depression. Chronic constant light (CCL) treatment during adolescence has been shown to reverse the depression-like behaviour induced by MS. We aimed to further delineate the antidepressant effect of light by investigating the involvement of the dopaminergic, serotonergic and orexinergic systems. MS was used to induce changes in adult male Sprague-Dawley rats, some of whom were also treated with CCL for 3 weeks during adolescence. At P80, rats were decapitated and brain tissue collected for analysis of glutamate- and potassium-stimulated dopamine release in the nucleus accumbens (NAc) using an in vitro superfusion technique. Enzyme-linked immunosorbent assays were employed to measure 5-hydroxytryptamine (5-HT) levels in the hypothalamus and prefrontal cortex (PFC). Western blotting was used to measure orexin receptor 1 (OXR-1) and 2 (OXR-2) in the PFC. MS did not affect 5-HT levels in these rats. However, CCL increased hypothalamic 5-HT and reduced 5-HT levels in the PFC. CCL had opposite effects on OXR levels in the PFC of maternally separated and non-separated rats. MS increased OXR-1 and OXR-2 levels in the PFC, an effect that was normalized by CCL treatment. MS reduced glutamate-stimulated dopamine release in the NAc, an effect that was not reversed by CCL. The present results suggest that CCL treatment affects 5-HT and orexinergic systems in the MS model while not affecting the MS-induced decrease in dopamine release in the NAc. The reversal of changes in the orexinergic system may be of particular relevance to the antidepressant effect of CCL in depression.

Neural substrates underlying effort, time, and risk-based decision making in motivated behavior

All mobile organisms rely on adaptive motivated behavior to overcome the challenges of living in an environment in which essential resources may be limited. A variety of influences ranging from an organism's environment, experiential history, and physiological state all influence a cost-benefit analysis which allows motivation to energize behavior and direct it toward specific goals. Here we review the substantial amount of research aimed at discovering the interconnected neural circuits which allow organisms to carry-out the cost-benefit computations which allow them to behave in adaptive ways. We specifically focus on how the brain deals with different types of costs, including effort requirements, delays to reward and payoff riskiness. An examination of this broad literature highlights the importance of the extended neural circuits which enable organisms to make decisions about these different types of costs. This involves Cortical Structures, including the Anterior Cingulate Cortex (ACC), the Orbital Frontal Cortex (OFC), the Infralimbic Cortex (IL), and prelimbic Cortex (PL), as well as the Baso-Lateral Amygdala (BLA), the Nucleus Accumbens (NAcc), the Ventral Pallidal (VP), the Sub Thalamic Nucleus (STN) among others. Some regions are involved in multiple aspects of cost-benefit computations while the involvement of other regions is restricted to information relating to specific types of costs.

Behavioral representation of cost and benefit balance in rats

Decision making is dependent upon individual motivation. Previous studies showed that animals with higher levels of motivation are more likely to invest more time to acquire larger rewards rather than acquiring smaller rewards with less time to wait. However, little is known about how this motivation mediates the cognitive effort animals devote upon making said decisions in detail. In the present study, we investigated the behavioral response in a goal-directed action under a differential reward schedule by training rats to perform a "Do more, get more" (DM-GM) task using a nosepoke operandum when longer nosepoke durations resulted in correspondingly larger rewards. In general, the subjects learned this DM-GM rule and reached a steady behavioral state within 15days. During the training stage, the rats found the most cost-effective action choice and behaved according to that guideline more frequently than other possible actions. In addition, when the cost-benefit ratio changed, the rats again found a new most cost-effective choice to obtain maximum rewards. Our results demonstrate that there is a "balance point" of cost and benefit in rat valuation system and that this "balance point" not only guides the rats to make the appropriate decision, but that this point can be modified upon new situations to choose a newer optimum action plan.

Perturbations in Effort-Related Decision-Making Driven by Acute Stress and Corticotropin-Releasing Factor

Acute stress activates numerous systems in a coordinated effort to promote homeostasis, and can exert differential effects on mnemonic and cognitive functions depending on a myriad of factors. Stress can alter different forms of cost/benefit decision-making, yet the mechanisms that drive these effects, remain unclear. In the present study, we probed how corticotropin-releasing factor (CRF) may contribute to stress-induced alterations in cost/benefit decision-making, using an task where well-trained rats chose between a low effort/low reward lever (LR; two pellets) and a high effort/high reward lever (HR; four pellets), with the effort requirement increasing over a session (2, 5, 10, and 20 presses). One-hour restraint stress markedly reduced preference for the HR option, but this effect was attenuated by infusions of the CRF antagonist, alpha-helical CRF. Conversely, central CRF infusion mimicked the effect of stress on decision-making, as well as increased decision latencies and reduced response vigor. CRF infusions did not alter preference for larger vs smaller rewards, but did reduce responding for food delivered on a progressive ratio, suggesting that these treatments may amplify perceived effort costs that may be required to obtain rewards. CRF infusions into the ventral tegmental area recapitulated the effect of central CRF treatment and restraint on choice behavior, suggesting that these effects may be mediated by perturbations in dopamine transmission. These findings highlight the involvement of CRF in regulating effort-related decisions and suggest that increased CRF activity may contribute to motivational impairments and abnormal decision-making associated with stress-related psychiatric disorders such as depression.

Functionally Distinct Dopamine Signals in Nucleus Accumbens Core and Shell in the Freely Moving Rat

Dynamic signaling of mesolimbic dopamine (DA) neurons has been implicated in reward learning, drug abuse, and motivation. However, this system is complex because firing patterns of these neurons are heterogeneous; subpopulations receive distinct synaptic inputs, and project to anatomically and functionally distinct downstream targets, including the nucleus accumbens (NAc) shell and core. The functional roles of these cell populations and their real-time signaling properties in freely moving animals are unknown. Resolving the real-time DA signal requires simultaneous knowledge of the synchronized activity of DA cell subpopulations and assessment of the down-stream functional effect of DA release. Because this is not yet possible solely by experimentation in vivo, we combine computational modeling and fast-scan cyclic voltammetry data to reconstruct the functionally relevant DA signal in DA neuron subpopulations projecting to the NAc core and shell in freely moving rats. The approach provides a novel perspective on real-time DA neuron firing and concurrent activation of presynaptic autoreceptors and postsynaptic targets. We first show that individual differences in DA release arise from differences in autoreceptor feedback. The model predicts that extracellular DA concentrations in NAc core result from constant baseline DA firing, whereas DA concentrations in NAc shell reflect highly dynamic firing patters, including synchronized burst firing and pauses. Our models also predict that this anatomical difference in DA signaling is exaggerated by intravenous infusion of cocaine.

SIGNIFICANCE STATEMENT

Orchestrated signaling from mesolimbic dopamine (DA) neurons is important for initiating appropriate behavior in response to salient stimuli. Thus, subpopulations of mesolimbic DA neurons show different in vitro properties and synaptic inputs depending on their specific projections to the core and shell subterritories of the nucleus accumbens (NAc). However, the functional consequence of these differences is unknown. Here we analyze and model DA dynamics in different areas of the NAc to establish the real-time DA signal. In freely behaving animals, we find that the DA signal from mesencephalic neurons projecting to the NAc shell is dominated by synchronized bursts and pauses, whereas signaling is uniform for core-projecting neurons; this difference is amplified by cocaine.

The pharmacology of effort-related choice behavior: Dopamine, depression, and individual differences

This review paper is focused upon the involvement of mesolimbic dopamine (DA) and related brain systems in effort-based processes. Interference with DA transmission affects instrumental behavior in a manner that interacts with the response requirements of the task, such that rats with impaired DA transmission show a heightened sensitivity to ratio requirements. Impaired DA transmission also affects effort-related choice behavior, which is assessed by tasks that offer a choice between a preferred reinforcer that has a high work requirement vs. less preferred reinforcer that can be obtained with minimal effort. Rats and mice with impaired DA transmission reallocate instrumental behavior away from food-reinforced tasks with high response costs, and show increased selection of low reinforcement/low cost options. Tests of effort-related choice have been developed into models of pathological symptoms of motivation that are seen in disorders such as depression and schizophrenia. These models are being employed to explore the effects of conditions associated with various psychopathologies, and to assess drugs for their potential utility as treatments for effort-related symptoms. Studies of the pharmacology of effort-based choice may contribute to the development of treatments for symptoms such as psychomotor slowing, fatigue or anergia, which are seen in depression and other disorders.

Frontostriatal systems comprising connections between ventral medial prefrontal cortex and nucleus accumbens subregions differentially regulate motor impulse control in rats

RATIONALE

Deficits in impulse control are prevalent in several neuropsychiatric disorders that are based on impaired frontostriatal communication. The ventral medial prefrontal cortex (vmPFC) and the nucleus accumbens (NAc) are key substrates of impulse control in rats. The NAc core and shell are considered to be differentially involved suggesting a functional distinction between the connections of the vmPFC and particular NAc subregions concerning impulse control.

OBJECTIVES/METHODS

In the present study, simultaneous inactivation of the rats' vmPFC and NAc core or shell by contralateral microinfusion of the GABAA receptor agonist muscimol was used to investigate their relevance for impulse control in the five-choice serial reaction time task (5-CSRTT).

RESULTS

Disconnection of the vmPFC and NAc shell produced specific impairments in inhibitory control, indicated by significantly increased premature responding and an enhanced number of time-out responses, closely resembling the effects of bilateral inactivation of either the vmPFC or NAc shell previously reported using the same task. In contrast, disconnection of the vmPFC and NAc core only slightly increased the rate of omissions and latency of reward collection indicating attentional and motivational deficits.

CONCLUSIONS

Our results extend previous findings indicating the functional specialisation of frontostriatal networks and show a differential contribution of specific vmPFC-NAc connections to behavioural control depending on the NAc subregion. We conclude that the regulation of impulse control in rats requires an intact connection between the vmPFC and the NAc shell, while the vmPFC-NAc core projection seems to be of minor importance.

Nucleus accumbens core and shell inactivation differentially affects impulsive behaviours in rats

Impulsivity is a multifactorial phenomenon, determined by deficits in decision-making (impulsive choice) and impulse control (impulsive action). Recent findings indicate that impulsive behaviour is not only top-down controlled by cortical areas, but also modulated at subcortical level. The nucleus accumbens (NAc) might be a key substrate in cortico-limbic-striatal circuits involved in impulsive behaviour. Dissociable effects of the NAc subregions in various behavioural paradigms point to a potential functional distinction between NAc core and shell concerning different types of impulsivity. The present study used reversible inactivation of the rats' NAc core and shell via bilateral microinfusion of the GABAA receptor agonist muscimol (0.05μg/0.3μl) and fluorophore-conjugated muscimol (FCM, 0.27μg/0.3μl) in order to study their contribution to different aspects of impulse control in a 5-choice serial reaction time task (5-CSRTT) and impulsive choice in a delay-based decision-making T-maze task. Acute inactivation of NAc core as well as shell by muscimol increased impulsive choice, with higher impairments of the rats' waiting capacity in the T-maze following core injections compared to shell. Intra-NAc shell infusion of muscimol also induced specific impulse control deficits in the 5-CSRTT, while deactivation of the core caused severe general impairments in task performance. FCM did not affect animal behaviour. Our findings reveal clear involvement of NAc shell in both forms of impulsivity. Both subareas play a key role in the regulation of impulsive decision-making, but show functional dichotomy regarding impulse control with the core being more implicated in motivational and motor aspects.

Lesions of the nucleus accumbens core modulate development of matching behavior

Background

The development of choice is a crucial determinant in the performance of appetitive responses. Given two options with different reinforcement rates, animals match their relative rate of responding to the relative rates of reinforcement (i.e., matching behavior). A previous study has shown that the nucleus accumbens core (AcbC) is involved in the performance of matching behavior in trained animals. However, the role of the AcbC in the acquisition of matching behavior has not been addressed.

Results

We conducted a series of experimental sessions to examine the role of the AcbC on the development of matching behavior. Instrumental responding was measured in rats with excitotoxic lesions of the AcbC. Rats were given two options that differed in the relative rate of reinforcement under concurrent variable-interval schedules. The locations of the more frequently reinforced option and the alternative option were randomly switched between sessions. Lesions of the AcbC accelerated the development of matching behavior compared to the sham-operated group. The AcbC-lesioned rats exhibited closer conformity to the matching law than shams when the options were in the same positions as in the previous session (the same condition), but not when the option locations had been switched (the different condition). The AcbC rats showed smaller probabilities of switching behavior between alternatives than shams. Post-reinforcement pausing was not affected by the AcbC lesion. Neither numbers of rewards obtained nor number of lever presses were different between the AcbC-lesioned rats and shams over session blocks.

Conclusions

Our results suggest that the AcbC plays a regulatory role in the development of matching behavior through switching probabilities rather than perception of reward magnitude. The differential effect of AcbC lesions on the matching behavior between the same and different conditions suggests influence of the spontaneous recovery, that is, reversion to a previously reinforced choice at the beginning of the next session, on the development of matching behavior in the AcbC-lesioned rats.

Dopamine dependency for acquisition and performance of Pavlovian conditioned response

During Pavlovian conditioning, pairing of a neutral conditioned stimulus (CS) with a reward leads to conditioned reward-approach responses (CRs) that are elicited by presentation of the CS. CR behaviors can be sign tracking, in which animals engage the CS, or goal tracking, in which animals go to the reward location. We investigated CR behaviors in mice with only ∼5% of normal dopamine in the striatum using a Pavlovian conditioning paradigm. These mice had severely impaired acquisition of the CR, which was ameliorated by pharmacological restoration of dopamine synthesis with l-dopa. Surprisingly, after they had learned the CR, its expression decayed only gradually in following sessions that were conducted without l-dopa treatment. To assess specific contributions of dopamine signaling in the dorsal or ventral striatum, we performed virus-mediated restoration of dopamine synthesis in completely dopamine-deficient (DD) mice. Mice with dopamine signaling only in the dorsal striatum did not acquire a CR, whereas mice with dopamine signaling only in in the ventral striatum acquired a CR. The CR in mice with dopamine signaling only in the dorsal striatum was restored by subjecting the mice to instrumental training in which they had to interact with the CS to obtain rewards. We conclude that dopamine is essential for learning and performance of CR behavior that is predominantly goal tracking. Furthermore, although dopamine signaling in the ventral striatum is sufficient to support a CR, dopamine signaling only in the dorsal striatum can also support a CR under certain circumstances.

Effort-related motivational effects of the VMAT-2 inhibitor tetrabenazine: implications for animal models of the motivational symptoms of depression

Motivated behaviors are often characterized by a high degree of behavioral activation, and work output and organisms frequently make effort-related decisions based upon cost/benefit analyses. Moreover, people with major depression and other disorders often show effort-related motivational symptoms such as anergia, psychomotor retardation, and fatigue. It has been suggested that tasks measuring effort-related choice behavior could be used as animal models of the motivational symptoms of depression, and the present studies characterized the effort-related effects of the vesicular monoamine transport (VMAT) inhibitor tetrabenazine. Tetrabenazine produces depressive symptoms in humans and, because of its selective inhibition of VMAT-2, it preferentially depletes dopamine (DA). Rats were assessed using a concurrent fixed-ratio 5/chow feeding choice task that is known to be sensitive to dopaminergic manipulations. Tetrabenazine shifted response choice in rats, producing a dose-related decrease in lever pressing and a concomitant increase in chow intake. However, it did not alter food intake or preference in parallel free-feeding choice studies. The effects of tetrabenazine on effort-related choice were reversed by the adenosine A2A antagonist MSX-3 and the antidepressant bupropion. A behaviorally active dose of tetrabenazine decreased extracellular DA in nucleus accumbens and increased expression of DARPP-32 in accumbens medium spiny neurons in a pattern indicative of reduced transmission at both D1 and D2 DA receptors. These experiments demonstrate that tetrabenazine, which is used in animal models to produce depression-like effects, can alter effort-related choice behavior. These studies have implications for the development of animal models of the motivational symptoms of depression and related disorders.

Mapping brain circuits of reward and motivation: in the footsteps of Ann Kelley

Ann Kelley was a scientific pioneer in reward neuroscience. Her many notable discoveries included demonstrations of accumbens/striatal circuitry roles in eating behavior and in food reward, explorations of limbic interactions with hypothalamic regulatory circuits, and additional interactions of motivation circuits with learning functions. Ann Kelley's accomplishments inspired other researchers to follow in her footsteps, including our own laboratory group. Here we describe results from several lines of our research that sprang in part from earlier findings by Kelley and colleagues. We describe hedonic hotspots for generating intense pleasure 'liking', separate identities of 'wanting' versus 'liking' systems, a novel role for dorsal neostriatum in generating motivation to eat, a limbic keyboard mechanism in nucleus accumbens for generating intense desire versus intense dread, and dynamic limbic transformations of learned memories into motivation. We describe how origins for each of these themes can be traced to fundamental contributions by Ann Kelley.

Heads for learning, tails for memory: reward, reinforcement and a role of dopamine in determining behavioral relevance across multiple timescales

Dopamine has long been tightly associated with aspects of reinforcement learning and motivation in simple situations where there are a limited number of stimuli to guide behavior and constrained range of outcomes. In naturalistic situations, however, there are many potential cues and foraging strategies that could be adopted, and it is critical that animals determine what might be behaviorally relevant in such complex environments. This requires not only detecting discrepancies with what they have recently experienced, but also identifying similarities with past experiences stored in memory. Here, we review what role dopamine might play in determining how and when to learn about the world, and how to develop choice policies appropriate to the situation faced. We discuss evidence that dopamine is shaped by motivation and memory and in turn shapes reward-based memory formation. In particular, we suggest that hippocampal-striatal-dopamine networks may interact to determine how surprising the world is and to either inhibit or promote actions at time of behavioral uncertainty.

The role of melanocortins and Neuropeptide Y in food reward

The Neuropeptide Y and the melanocortin peptides are two well-described hypothalamic feeding peptides regulating energy balance. Predominantly expressed within the arcuate nucleus, these neurons project to different brain areas and modulate various aspects of feeding. Hedonic feeding, where one overindulges in palatable food consumption beyond one's nutritional necessities, is one such aspect regulated by NPY/melanocortin signaling. Research suggests that NPY/melanocortin regulate hedonic aspects of feeding through its projections to the brain reward circuitry (ventral tegmental area, lateral hypothalamus, nucleus accumbens etc.), however, exact target areas have not yet been identified. The current work explores literature to provide a mechanistic explanation for the effects of these peptides on food reward.

Nucleus accumbens neurotransmission and effort-related choice behavior in food motivation: effects of drugs acting on dopamine, adenosine, and muscarinic acetylcholine receptors

Mesolimbic dopamine (DA) is a critical component of the brain circuitry regulating behavioral activation and effort-related processes. Although nucleus accumbens (NAc) DA depletions or antagonism leave aspects of appetite and primary food motivation intact, rats with impaired DA transmission reallocate their instrumental behavior away from food-reinforced tasks with high response requirements, and instead select less effortful food-seeking behaviors. Previous work showed that adenosine A2A antagonists can reverse the effects of DA D2 antagonists on effort-related choice, and that stimulation of adenosine A2A receptors produces behavioral effects that are similar to those induced by DA antagonism. The present review summarizes the literature on the role of NAc DA and adenosine in effort-related processes, and also presents original data on the effects of local stimulation of muscarinic acetylcholine receptors in NAc core. Local injections of the muscarinic agonist pilocarpine directly into NAc core produces shifts in effort-related choice behavior similar to those induced by DA antagonism or A2A receptor stimulation, decreasing lever pressing but increasing chow intake in rats responding on a concurrent fixed ratio/chow feeding choice task. In contrast, injections into a neostriatal control site dorsal to the NAc were ineffective. The actions of pilocarpine on this task were attenuated by co-administration of the muscarinic antagonist scopolamine. Thus, drugs that act on DA, adenosine A2A, and muscarinic receptors regulate effort-related choice behavior, which may have implications for the treatment of psychiatric symptoms such as psychomotor slowing, fatigue or anergia that can be observed in depression and other disorders.

The mysterious motivational functions of mesolimbic dopamine

Nucleus accumbens dopamine is known to play a role in motivational processes, and dysfunctions of mesolimbic dopamine may contribute to motivational symptoms of depression and other disorders, as well as features of substance abuse. Although it has become traditional to label dopamine neurons as "reward" neurons, this is an overgeneralization, and it is important to distinguish between aspects of motivation that are differentially affected by dopaminergic manipulations. For example, accumbens dopamine does not mediate primary food motivation or appetite, but is involved in appetitive and aversive motivational processes including behavioral activation, exertion of effort, approach behavior, sustained task engagement, Pavlovian processes, and instrumental learning. In this review, we discuss the complex roles of dopamine in behavioral functions related to motivation.

The behavioral pharmacology of effort-related choice behavior: dopamine, adenosine and beyond

For many years, it has been suggested that drugs that interfere with dopamine (DA) transmission alter the "rewarding" impact of primary reinforcers such as food. Research and theory related to the functions of mesolimbic DA are undergoing a substantial conceptual restructuring, with the traditional emphasis on hedonia and primary reward yielding to other concepts and lines of inquiry. The present review is focused upon the involvement of nucleus accumbens DA in effort-related choice behavior. Viewed from the framework of behavioral economics, the effects of accumbens DA depletions and antagonism on food-reinforced behavior are highly dependent upon the work requirements of the instrumental task, and DA-depleted rats show a heightened sensitivity to response costs, especially ratio requirements. Moreover, interference with accumbens DA transmission exerts a powerful influence over effort-related choice behavior. Rats with accumbens DA depletions or antagonism reallocate their instrumental behavior away from food-reinforced tasks that have high response requirements, and show increased selection of low reinforcement/low cost options. Nucleus accumbens DA and adenosine interact in the regulation of effort-related functions, and other brain structures (anterior cingulate cortex, amygdala, ventral pallidum) also are involved. Studies of the brain systems regulating effort-based processes may have implications for understanding drug abuse, as well as symptoms such as psychomotor slowing, fatigue or anergia in depression and other neurological disorders.

Cannabinoids and value-based decision making: implications for neurodegenerative disorders

In recent years, disturbances in cognitive function have been increasingly recognized as important symptomatic phenomena in neurodegenerative diseases, including Parkinson's Disease (PD). Value-based decision making in particular is an important executive cognitive function that is not only impaired in patients with PD, but also shares neural substrates with PD in basal ganglia structures and the dopamine system. Interestingly, the endogenous cannabinoid system modulates dopamine function and subsequently value-based decision making. This review will provide an overview of the interdisciplinary research that has influenced our understanding of value-based decision making and the role of dopamine, particularly in the context of reinforcement learning theories, as well as recent animal and human studies that demonstrate the modulatory role of activation of cannabinoid receptors by exogenous agonists or their naturally occurring ligands. The implications of this research for the symptomatology of and potential treatments for PD are also discussed.

The novel adenosine A(2A) antagonist prodrug MSX-4 is effective in animal models related to motivational and motor functions

Adenosine A(2A) and dopamine D2 receptors interact to regulate diverse aspects of ventral and dorsal striatal functions related to motivational and motor processes, and it has been suggested that adenosine A(2A) antagonists could be useful for the treatment of depression, parkinsonism and other disorders. The present experiments were performed to characterize the effects of MSX-4, which is an amino acid ester prodrug of the potent and selective adenosine A(2A) receptor antagonist MSX-2, by assessing its ability to reverse pharmacologically induced motivational and motor impairments. In the first group of studies, MSX-4 reversed the effects of the D2 antagonist eticlopride on a concurrent lever pressing/chow feeding task that is used as a measure of effort-related choice behavior. MSX-4 was less potent after intraperitoneal administration than the comparison compound, MSX-3, though both were equally efficacious. With this task, MSX-4 was orally active in the same dose range as MSX-3. MSX-4 also reversed the locomotor suppression induced by eticlopride in the open field, but did not induce anxiogenic effects as measured by the relative amount of interior activity. Behaviorally active doses of MSX-4 also attenuated the increase in c-Fos and pDARPP-32(Thr34) expression in nucleus accumbens core that was induced by injections of eticlopride. In addition, MSX-4 suppressed the oral tremor induced by the anticholinesterase galantamine, which is consistent with an antiparkinsonian profile. These actions of MSX-4 indicate that this compound could have potential utility as a treatment for parkinsonism, as well as some of the motivational symptoms of depression and other disorders.

Nucleus accumbens dopamine modulates response rate but not response timing in an interval timing task

While previous work has demonstrated that systemic dopamine manipulations can modulate temporal perception by altering the speed of internal clock processes, the neural site of this modulation remains unclear. Based on recent research suggesting that changes in incentive salience can alter the perception of time, as well as work showing that nucleus accumbens (NAc) shell dopamine (DA) levels modulate the incentive salience of discriminative stimuli that predict instrumental outcomes, we assessed whether microinjections of DA agents into the NAc shell would impact temporal perception. Rats were trained on either a 10-s or 30-s temporal production procedure and received intra-NAc shell microinfusions of sulpiride, amphetamine, and saline. Results showed that NAc DA modulations had no effect on response timing, but intra-NAc shell sulpiride microinfusions significantly decreased response rates relative to saline and amphetamine. Our findings therefore suggest that neither NAc shell DA levels, nor the resultant changes in incentive salience signaled by this structure, impact temporal control.

Adenosine A2A receptor antagonism and genetic deletion attenuate the effects of dopamine D2 antagonism on effort-based decision making in mice

Brain dopamine (DA) and adenosine interact in the regulation of behavioral activation and effort-related processes. In the present studies, a T-maze task was developed in mice for the assessment of effort-related decision making. With this task, the two arms of the maze have different reinforcement densities, and a vertical barrier is positioned in the arm with the higher density (HD), presenting the animal with an effort-related challenge. Under control conditions mice prefer the HD arm, and climb the barrier to obtain the larger amount of food. The DA D(2) receptor antagonist haloperidol decreased selection of the HD arm and increased selection of the arm with the low density of reinforcement. However, the HD arm was still the preferred choice in haloperidol-treated mice trained with barriers in both arms. Pre-feeding the mice to reduce food motivation dramatically increased omissions, an effect that was distinct from the actions of haloperidol. Co-administration of theophylline, a nonselective adenosine receptor antagonist, partially reversed the effects of haloperidol. This effect seems to be mediated by the A(2A) receptor but not the A(1) receptor, since the A(2A) antagonist MSX-3, but not the A(1) antagonist CPT, dose dependently reversed the effects of haloperidol on effort-related choice and on c-Fos expression in the dorsal striatum and nucleus accumbens. In addition, adenosine A(2A) receptor knockout mice were resistant to the effects of haloperidol on effort-related choice in the maze. These results indicate that DA D(2) and adenosine A(2A) receptors interact to regulate effort-related decision making and effort expenditure in mice.

Dopamine in nucleus accumbens: salience modulation in latent inhibition and overshadowing

Latent inhibition (LI) is demonstrated when non-reinforced pre-exposure to a to-be-conditioned stimulus retards later learning. Learning is similarly retarded in overshadowing, in this case using the relative intensity of competing cues to manipulate associability. Electrolytic/excitotoxic lesions to shell accumbens (NAc) and systemic amphetamine both reliably abolish LI. Here a conditioned emotional response procedure was used to demonstrate LI and overshadowing and to examine the role of dopamine (DA) within NAc. Experiment 1 showed that LI but not overshadowing was abolished by systemic amphetamine (1.0 mg/kg i.p.). In Experiment 2, 6-hydroxydopamine (6-OHDA) was used to lesion DA terminals within NAc: both shell- and core- (plus shell-)lesioned rats showed normal LI and overshadowing. Experiment 3 compared the effects of amphetamine microinjected at shell and core coordinates prior to conditioning: LI, but not overshadowing, was abolished by 10.0 but not 5.0 µg/side amphetamine injected in core but not shell NAc. These results suggest that the abolition of LI produced by NAc shell lesions is not readily reproduced by regionally restricted DA depletion within NAc; core rather than shell NAc mediates amphetamine-induced abolition of LI; overshadowing is modulated by different neural substrates.