Prefrontostriatal circuitry regulates effort-related decision making

A Common Stay-on-Goal Mechanism in the Anterior Cingulate Cortex for Information and Effort Choices

Humans and nonhumans alike often make choices to gain information, even when the information cannot be used to change the outcome. Prior research has shown that the anterior cingulate cortex (ACC) is important for evaluating options involving reward-predictive information. Here we studied the role of ACC in information choices using optical inhibition to evaluate the contribution of this region during specific epochs of decision-making. Rats could choose between an uninformative option followed by a cue that predicted reward 50% of the time versus a fully informative option that signaled outcomes with certainty but was rewarded only 20% of the time. Reward seeking during the informative S+ cue decreased following ACC inhibition, indicating a causal contribution of this region in supporting reward expectation to a cue signaling reward with certainty. Separately in a positive control experiment and in support of a known role for this region in sustaining high-effort behavior for preferred rewards, we observed reduced lever presses and lower breakpoints in effort choices following ACC inhibition. The lack of changes in reward latencies in both types of decisions indicate the motivational value of rewards remained intact, revealing instead a common role for ACC in maintaining persistence toward certain and valuable rewards.

Multi-level encoding of reward, effort, and choice across the frontal cortex and basal ganglia during cost-benefit decision-making

Adaptive value-guided decision-making requires weighing up the costs and benefits of pursuing an available opportunity. Though neurons across frontal cortical-basal ganglia circuits have been repeatedly shown to represent decision-related parameters, it is unclear whether and how this information is coordinated. To address this question, we performed large-scale single-unit recordings simultaneously across 5 medial/orbital frontal and basal ganglia regions as rats decided whether to pursue varying reward payoffs available at different effort costs. Single neurons encoding combinations of decision variables (reward, effort, and choice) were represented within all recorded regions. Coactive cell assemblies, ensembles of neurons that repeatedly coactivated within short time windows (<25 ms), represented the same decision variables despite the members often having diverse individual coding properties. Together, these findings demonstrate a multi-level encoding structure for cost-benefit computations where individual neurons are coordinated into larger assemblies that can represent task variables independently of their constituent components.

A common stay-on-goal mechanism in anterior cingulate cortex for information and effort choices

Humans and non-humans alike often make choices to gain information, even when the information cannot be used to change the outcome. Prior research has shown the anterior cingulate cortex (ACC) is important for evaluating options involving reward-predictive information. Here we studied the role of ACC in information choices using optical inhibition to evaluate the contribution of this region during specific epochs of decision making. Rats could choose between an uninformative option followed by a cue that predicted reward 50% of the time vs. a fully informative option that signaled outcomes with certainty, but was rewarded only 20% of the time. Reward seeking during the informative S+ cue decreased following ACC inhibition, indicating a causal contribution of this region in supporting reward expectation to a cue signaling reward with certainty. Separately in a positive control experiment and in support of a known role for this region in sustaining high-effort behavior for preferred rewards, we observed reduced lever presses and lower breakpoints in effort choices following ACC inhibition. The lack of changes in reward latencies in both types of decisions indicate the motivational value of rewards remained intact, revealing instead a common role for ACC in maintaining persistence toward certain and valuable rewards.

Graphical Summary

Significance Statement

We often make choices to gain information, even when the information cannot be used to change the outcome. Here we investigated the precise timing of the role of the anterior cingulate cortex (ACC) in decisions that involve seeking certain versus uncertain rewards. By optically inhibiting ACC neurons, we demonstrate that this region is crucial for maintaining persistence toward rewards signaled with certainty, without altering the motivational value of the reward itself. In a positive control experiment, we also confirm that ACC is important in effort-based choice. The findings reveal a common role for ACC in maintaining persistence toward certain and valuable rewards, necessary for making optimal decisions. These results have implications for understanding psychiatric disorders involving maladaptive reward-seeking behavior.

Cell- and Pathway-Specific Disruptions in the Accumbens of Fragile X Mouse

Fragile X syndrome (FXS) is a genetic cause of intellectual disability and autism spectrum disorder. The mesocorticolimbic system, which includes the prefrontal cortex (PFC), basolateral amygdala (BLA), and nucleus accumbens core (NAcC), is essential for regulating socioemotional behaviors. We employed optogenetics to compare the functional properties of the BLA→NAcC, PFC→NAcC, and reciprocal PFC↔BLA pathways in Fmr1-/y::Drd1a-tdTomato male mice. In FXS mice, the PFC↔BLA reciprocal pathway was unaffected, while significant synaptic modifications occurred in the BLA/PFC→NAcC pathways. We observed distinct changes in D1 striatal projection neurons (SPNs) and separate modifications in D2 SPNs. In FXS mice, the BLA/PFC→NAcC-D2 SPN pathways demonstrated heightened synaptic strength. Focusing on the BLA→NAcC pathway, linked to autistic symptoms, we found increased AMPAR and NMDAR currents and elevated spine density in D2 SPNs. Conversely, the amplified firing probability of BLA→NAcC-D1 SPNs was not accompanied by increased synaptic strength, AMPAR and NMDAR currents, or spine density. These pathway-specific alterations resulted in an overall enhancement of excitatory-to-spike coupling, a physiologically relevant index of how efficiently excitatory inputs drive neuronal firing, in both BLA→NAcC-D1 and BLA→NAcC-D2 pathways. Finally, the absence of fragile X messenger ribonucleoprotein 1 (FMRP) led to impaired long-term depression specifically in BLA→D1 SPNs. These distinct alterations in synaptic transmission and plasticity within circuits targeting the NAcC highlight the potential role of postsynaptic mechanisms in selected SPNs in the observed circuit-level changes. This research underscores the heightened vulnerability of the NAcC in the context of FMRP deficiency, emphasizing its pivotal role in the pathophysiology of FXS.

Neural activity ramps in frontal cortex signal extended motivation during learning

Learning requires the ability to link actions to outcomes. How motivation facilitates learning is not well understood. We designed a behavioral task in which mice self-initiate trials to learn cue-reward contingencies and found that the anterior cingulate region of the prefrontal cortex (ACC) contains motivation-related signals to maximize rewards. In particular, we found that ACC neural activity was consistently tied to trial initiations where mice seek to leave unrewarded cues to reach reward-associated cues. Notably, this neural signal persisted over consecutive unrewarded cues until reward-associated cues were reached, and was required for learning. To determine how ACC inherits this motivational signal we performed projection-specific photometry recordings from several inputs to ACC during learning. In doing so, we identified a ramp in bulk neural activity in orbitofrontal cortex (OFC)-to-ACC projections as mice received unrewarded cues, which continued ramping across consecutive unrewarded cues, and finally peaked upon reaching a reward-associated cue, thus maintaining an extended motivational state. Cellular resolution imaging of OFC confirmed these neural correlates of motivation, and further delineated separate ensembles of neurons that sequentially tiled the ramp. Together, these results identify a mechanism by which OFC maps out task structure to convey an extended motivational state to ACC to facilitate goal-directed learning.

Neural activity ramps in frontal cortex signal extended motivation during learning

Learning requires the ability to link actions to outcomes. How motivation facilitates learning is not well understood. We designed a behavioral task in which mice self-initiate trials to learn cue-reward contingencies and found that the anterior cingulate region of the prefrontal cortex (ACC) contains motivation-related signals to maximize rewards. In particular, we found that ACC neural activity was consistently tied to trial initiations where mice seek to leave unrewarded cues to reach reward-associated cues. Notably, this neural signal persisted over consecutive unrewarded cues until reward associated cues were reached, and was required for learning. To determine how ACC inherits this motivational signal we performed projection specific photometry recordings from several inputs to ACC during learning. In doing so, we identified a ramp in bulk neural activity in orbitofrontal cortex (OFC)-to-ACC projections as mice received unrewarded cues, which continued ramping across consecutive unrewarded cues, and finally peaked upon reaching a reward associated cue, thus maintaining an extended motivational state. Cellular resolution imaging of OFC confirmed these neural correlates of motivation, and further delineated separate ensembles of neurons that sequentially tiled the ramp. Together, these results identify a mechanism by which OFC maps out task structure to convey an extended motivational state to ACC to facilitate goal-directed learning.

A stress-sensitive frontostriatal circuit supporting effortful reward-seeking behavior

Effort valuation-a process for selecting actions based on the anticipated value of rewarding outcomes and expectations about the work required to obtain them-plays a fundamental role in decision-making. Effort valuation is disrupted in chronic stress states and is supported by the anterior cingulate cortex (ACC), but the circuit-level mechanisms by which the ACC regulates effort-based decision-making are unclear. Here, we show that ACC neurons projecting to the nucleus accumbens (ACC-NAc) play a critical role in effort valuation behavior in mice. Activity in ACC-NAc cells integrates both reward- and effort-related information, encoding a reward-related signal that scales with effort requirements and is necessary for supporting future effortful decisions. Chronic corticosterone exposure reduces motivation, suppresses effortful reward-seeking, and disrupts ACC-NAc signals. Together, our results delineate a stress-sensitive ACC-NAc circuit that supports effortful reward-seeking behavior by integrating reward and effort signals and reinforcing effort allocation in the service of maximizing reward.

Inactivation of phosphodiesterase-4B gene in rat nucleus accumbens shell by CRISPR/Cas9 or positive allosteric modulation of the protein affects the motivation to chronically self-administer nicotine

Large scale human genome wide association studies (GWAS) have identified a growing pool of genes associated with cigarette smoking. One of the most prominent, phosphodiesterase-4B (PDE4B), has been associated with multiple smoking phenotypes. Although PDE4B modulates the half-life of neuronal cAMP, its precise role in smoking behaviors is unknown. To address this knowledge gap, we used a reverse translational approach. We inactivated PDE4B in bilateral medial nucleus accumbens shell (NAcs) neurons by injecting AAV containing a specific gRNA in female transgenic Cas9+ Long Evans rats. These rats then were given 23-h chronic access to nicotine intravenous self-administration (IVSA) under a schedule of increasing fixed ratios (FR). With the increased effort required at FR7, nicotine SA (i.e. active presses and drug infusions) declined significantly in controls, whereas it was maintained in the mutagenized group. A progressive ratio (PR) study also showed significantly greater cumulative nicotine infusions in the PDE4B-edited group. Hence, we hypothesized that enhanced PDE4B protein activity would reduce nicotine IVSA. A positive allosteric modulator, 2-(3-(4-chloro-3-fluorophenyl)-5-ethyl-1H-1,2,4-triazol-1-yl)-N-(3,5-dichlorobenzyl)acetamide (MR-L2), was microinfused into NAcs bilaterally at FR3 or FR5; in both cohorts, MR-L2 acutely reduced nicotine IVSA. In summary, these studies show that the activity of PDE4B regulates the capacity of NAcs to maintain nicotine IVSA in face of the cost of increasing work. This finding and the results of the PR study indicate that PDE4B affects the motivation to obtain nicotine. These reverse translational studies in rats provide insight into the motivational effects of NAcs PDE4B that advance our understanding of the smoking behaviors mapped in human GWAS.

Apathy and Motivation: Biological Basis and Drug Treatment

Apathy is a disabling syndrome associated with poor functional outcomes that is common across a broad range of neurological and psychiatric conditions. Currently, there are no established therapies specifically for the condition, and safe and effective treatments are urgently needed. Advances in the understanding of motivation and goal-directed behavior in humans and animals have shed light on the cognitive and neurobiological mechanisms contributing to apathy, providing an important foundation for the development of new treatments. Here, we review the cognitive components, neural circuitry, and pharmacology of apathy and motivation, highlighting converging evidence of shared transdiagnostic mechanisms. Though no pharmacological treatments have yet been licensed, we summarize trials of existing and novel compounds to date, identifying several promising candidates for clinical use and avenues of future drug development.

Not all discounts are created equal: Regional activity and brain networks in temporal and effort discounting

Reward outcomes associated with costs like time delay and effort investment are generally discounted in decision-making. Standard economic models predict rewards associated with different types of costs are devalued in a similar manner. However, our review of rodent lesion studies indicated partial dissociations between brain regions supporting temporal- and effort-based decision-making. Another debate is whether options involving low and high costs are processed in different brain substrates (dual-system) or in the same regions (single-system). This research addressed these issues using coordinate-based, connectivity-based, and activation network-based meta-analyses to identify overlapping and separable neural systems supporting temporal (39 studies) and effort (20 studies) discounting. Coordinate-based activation likelihood estimation and resting-state connectivity analyses showed immediate-small reward and delayed-large reward choices engaged distinct regions with unique connectivity profiles, but their activation network mapping was found to engage the default mode network. For effort discounting, salience and sensorimotor networks supported low-effort choices, while the frontoparietal network supported high-effort choices. There was little overlap between the temporal and effort networks. Our findings underscore the importance of differentiating different types of costs in decision-making and understanding discounting at both regional and network levels.

Nucleus Accumbens Shell Neurons Encode the Kinematics of Reward Approach Locomotion

The nucleus accumbens (NAc) is considered an interface between motivation and action, with NAc neurons playing an important role in promoting reward approach. However, the encoding by NAc neurons that contributes to this role remains unknown. We recorded 62 NAc neurons in male Wistar rats (n = 5) running towards rewarded locations in an 8-arm radial maze. Variables related to locomotor approach kinematics were the best predictors of the firing rate for most NAc neurons. Nearly 18% of the recorded neurons were inhibited during the entire approach run (locomotion-off cells), suggesting that reduction in firing of these neurons promotes initiation of locomotor approach. 27% of the neurons presented a peak of activity during acceleration followed by a valley during deceleration (acceleration-on cells). Together, these neurons accounted for most of the speed and acceleration encoding identified in our analysis. In contrast, a further 16% of neurons presented a valley during acceleration followed by a peak just prior to or after reaching reward (deceleration-on cells). These findings suggest that these three classes of NAc neurons influence the time course of speed changes during locomotor approach to reward.

Inactivation of phosphodiesterase-4B gene in rat nucleus accumbens shell by CRISPR/Cas9 modulates the motivation to chronically self-administer nicotine

Large scale human genome wide association studies (GWAS) have identified a growing pool of genes associated with cigarette smoking. One of the most prominent, phosphodiesterase-4B (PDE4B), has been associated with multiple smoking phenotypes. Although PDE4B modulates the half-life of neuronal cAMP, its precise role in smoking behaviors is unknown. To address this knowledge gap, we used a reverse translational approach. We inactivated PDE4B in bilateral medial nucleus accumbens shell (NAcs) neurons by injecting AAV containing a specific gRNA in female transgenic Cas9+ Long Evans rats. These rats then were given 23-hour chronic access to nicotine intravenous self-administration (IVSA) under a schedule of increasing fixed ratios (FR). With the increased effort required at FR7, nicotine SA (i.e. active presses and drug infusions) declined significantly in controls, whereas it was maintained in the mutagenized group. A progressive ratio (PR) study also showed significantly greater cumulative nicotine infusions in the mutant group. Hence, we hypothesized that enhanced PDE4B protein activity would reduce nicotine IVSA. A positive allosteric modulator,2-(3-(4-chloro-3-fluorophenyl)-5-ethyl-1H-1,2,4-triazol-1-yl)-N-(3,5-dichlorobenzyl)acetamide (MR-L2), was microinfused into NAcs bilaterally at FR3 or FR5; in both cohorts, MR-L2 acutely reduced nicotine IVSA. In summary, these studies show that the activity of PDE4B regulates the capacity of NAcs to maintain nicotine IVSA in face of the cost of increasing work. This finding and the results of the PR study indicate that PDE4B affects the motivation to obtain nicotine. These reverse translational studies in rats provide insight into the motivational effects of NAcs PDE4B that advance our understanding of the smoking behaviors mapped in human GWAS.

Accumbal adenosine A2A receptor inactivation biases for large and costly rewards in the effort- but not delay-based decision making

The cost-benefit decision-making (CBDM) is critical to normal human activity and a diminished willingness to expend effort to obtain rewards is a prevalent/noted characteristic of neuropsychiatric disorders such as schizophrenia, Parkinson's disease. Numerous studies have identified nucleus accumbens (NAc) as an important locus for CBDM control but their neuromodulatory and behavioral mechanisms remain largely under-explored. Adenosine A_2A receptors (A_2ARs), which are highly concentrated in the striatopallidal neurons, can integrate glutamate and dopamine signals for controlling effort-related choice behaviors. While the involvement of A_2ARs in effort-based decision making is well documented, the role of other decision variables (reward discrimination) in effort-based decision making and the role of A_2AR in delay-based decision making are less clear. In this study, we have developed a well-controlled CBDM behavioral paradigm to manipulate effort/cost and reward independently or in combination, allowing a dissection of four behavioral elements: effort-based CBDM (E-CBDM), delay-based CBDM (D-CBDM), reward discrimination (RD), effort discrimination (ED), and determined the effect of genetic knockdown (KD) of NAc A_2AR on the four behavioral elements. We found that A_2AR KD in NAc increased the choice for larger, more costly reward in the E-CBDM, but not D-CBDM. Furthermore, this high-effort/high-reward bias was attributable to the increased willingness to engage in effort but not the effect of discrimination of reward magnitude. Our findings substantiate an important role of the NAc A_2AR in control of E-CBDM and support that pharmacologically targeting NAc A_2ARs would be a useful strategy for treating the aberrant effort-based decision making in neuropsychiatric disorders.

Critical review of RDoC approaches to the study of motivation with animal models: effort valuation/willingness to work

The NIMH research domain criteria (RDoC) approach was instigated to refocus mental health research on the neural circuits that mediate psychological functions, with the idea that this would foster an understanding of the neural basis of specific psychiatric dysfunctions (i.e. 'symptoms and circuits') and ultimately facilitate treatment. As a general idea, this attempt to go beyond traditional diagnostic categories and focus on neural circuit dysfunctions related to specific symptoms spanning multiple disorders has many advantages. For example, motivational dysfunctions are present in multiple disorders, including depression, schizophrenia, Parkinson's disease, and other conditions. A critical aspect of motivation is effort valuation/willingness to work, and several clinical studies have identified alterations in effort-based decision making in various patient groups. In parallel, formal animal models focusing on the exertion of effort and effort-based decision making have been developed. This paper reviews the literature on models of effort-based motivational function in the context of a discussion of the RDoC approach, with an emphasis on the dissociable nature of distinct aspects of motivation. For example, conditions associated with depression and schizophrenia blunt the selection of high-effort activities as measured by several tasks in animal models (e.g. lever pressing, barrier climbing, wheel running). Nevertheless, these manipulations also leave fundamental aspects of hedonic reactivity, food motivation, and reinforcement intact. This pattern of effects demonstrates that the general emphasis of the RDoC on the specificity of the neural circuits mediating behavioral pathologies, and the dissociative nature of these dysfunctions, is a valid concept. Nevertheless, the specific placement of effort-related processes as simply a 'sub-construct' of 'reward processing' is empirically and conceptually problematic. Thus, while the RDoC is an excellent general framework for new ways to approach research and therapeutics, it still needs further refinement.

Effort-based decision making in response to high-dose androgens: role of dopamine receptors

INTRODUCTION

Anabolic-androgenic steroids (AAS) are performance-enhancing drugs used by both world-class and rank-and-file athletes. AAS abuse has been linked with risky decision-making, ranging from drunk driving to abusing multiple drugs. Our lab uses operant behavior in rats to test the effects of AAS (testosterone) on decision making. In our previous study, testosterone caused rats to work harder for food reward during an effort discounting (ED) task. ED is sensitive to dopamine in the nucleus accumbens, and AAS alter accumbens dopamine receptor expression. Accordingly, we determined if testosterone increases response to dopamine receptor antagonists during ED.

METHODS

Rats were treated chronically with high-dose testosterone (7.5 mg/kg; n = 9) or vehicle (n = 9). We measured baseline preference for the large reward in an ED task, where rats choose between a small easy reward (one lever press for one sugar pellet) and a large difficult reward (2, 5, 10, or 15 presses for three pellets). Preference for the large reward was measured after administration of D1-like (SCH23390, 0.01 mg/kg) or D2-like (eticlopride, 0.06 mg/kg) receptor antagonists.

RESULTS

At baseline, testosterone- and vehicle-treated rats showed similar preference for the large reward lever (FR5, testosterone: 68.6 ± 9.7% and vehicle: 85.7 ± 2.5%). SCH23390 reduced large reward preference significantly in both groups (FR5, testosterone: 41.3 ± 9.2%; vehicle: 49.1 ± 8.2%; F(1,16) = 17.7; P < 0.05). Eticlopride decreased large reward preference in both groups, but more strongly in testosterone-treated rats (FR5: testosterone: 37.0 ± 9.7%; vehicle: 56.3 ± 7.8%; F(1,16) = 35.3; P < 0.05).

CONCLUSION

Testosterone increases response to dopamine D2-like receptor blockade, and this contributes to previously observed changes in decision-making behaviors.

The ventral striatum contributes to the activity of the motor cortex and motor outputs in monkeys

The ventral striatum (VSt) is thought to be involved in the vigor of motivated behavior and is suggested to be a limbic-motor interface between limbic areas involved in motivational processes and neural circuits regulating behavioral outputs. However, there is little direct evidence demonstrating the involvement of the VSt in motor control for motivated behaviors. To clarify the functional role of the VSt in motor control, we investigated the effect of reversible pharmacological inactivation of the VSt on the oscillatory activity of the sensorimotor cortices and motor outputs in two macaque monkeys. VSt inactivation reduced movement-related activities of the primary motor cortex and premotor area at 15–120 Hz and increased those at 5–7 Hz. These changes were accompanied by reduced torque outputs but had no effect on the correct performance rate. The present study provides direct evidence that the VSt regulates activities of the motor cortices and motor output.

Prenatal dexamethasone exposure alters effort decision making and triggers nucleus accumbens and anterior cingulate cortex functional changes in male rats

Daily, individuals select actions based on cost-benefit to allocate resources into goal-directed actions. Different brain regions coordinate this complex decision, including the nucleus accumbens (NAc), anterior cingulate cortex (ACC), and ventral tegmental area (VTA). In utero exposure to synthetic glucocorticoids (iuGC), such as dexamethasone, triggers prominent motivation deficits but the impact of this exposure in the ACC-NAc and/or ACC-VTA circuits is unknown. Here, we show that iuGC exposure causes decreased motivation for natural rewards (food) and impaired effort-based decision-making. Importantly, reduced neuronal activation (number of c-fos⁺ neurons) was observed in the NAc core and ACC of iuGC rats in comparison to CTR rats after performing the effort-based decision-making task. In addition, iuGC treatment led to increased NAc and ACC basal neuronal activity. Electrophysiological recordings during optogenetic modulation of ACC terminals in the NAc revealed that the ACC-NAc circuit is dysfunctional in iuGC animals. These data suggest that iuGC animals present motivational and effort-based decision-making deficits that can be associated with the observed ACC-NAc dysfunction.

Disrupted Decision-Making: EcoHIV Inoculation in Cocaine Dependent Rats

Independently, chronic cocaine use and HIV-1 viral protein exposure induce neuroadaptations in the frontal-striatal circuit as evidenced by both clinical and preclinical studies; how the frontal-striatal circuit responds to HIV-1 infection following chronic drug use, however, has remained elusive. After establishing experience with both sucrose and cocaine self-administration, a pretest-posttest experimental design was utilized to evaluate preference judgment, a simple form of decision-making dependent upon the integrity of frontal-striatal circuit function. During the pretest assessment, male rats exhibited a clear preference for cocaine, whereas female animals preferred sucrose. Two posttest evaluations (3 days and 6 weeks post inoculation) revealed that, independent of biological sex, inoculation with chimeric HIV (EcoHIV), but not saline, disrupted decision-making. Prominent structural alterations in the frontal-striatal circuit were evidenced by synaptodendritic alterations in pyramidal neurons in the medial prefrontal cortex. Thus, the EcoHIV rat affords a valid animal model to critically investigate how the frontal-striatal circuit responds to HIV-1 infection following chronic drug use.

Vigor, Effort-Related Aspects of Motivation and Anhedonia

In this chapter we provide an overview of the pharmacological and circuit mechanisms that determine the willingness to expend effort in pursuit of rewards. A particular focus will be on the role of the mesolimbic dopamine system, as well the contributing roles of limbic and cortical brains areas involved in the evaluation, selection, and invigoration of goal-directed actions. We begin with a review of preclinical studies, which have provided key insights into the brain systems that are necessary and sufficient for effort-based decision-making and have characterized novel compounds that enhance selection of high-effort activities. Next, we summarize translational studies identifying and expanding this circuitry in humans. Finally, we discuss the relevance of this work for understanding common motivational impairments as part of the broader anhedonia symptom domain associated with mental illness, and the identification of new treatment targets within this circuitry to improve motivation and effort-expenditure.

Effects of the dopamine depleting agent tetrabenazine on detailed temporal parameters of effort-related choice responding

The dopamine-depleting agent tetrabenazine alters effort-based choice, suppressing food-reinforced behaviors with high response requirements, while increasing selection of low-cost options. In the present experiments, rats were tested on a concurrent fixed ratio 5/chow feeding choice task, in which high-carbohydrate Bio-serv pellets reinforced lever pressing and lab chow was concurrently available. Detailed timing of lever pressing was monitored with an event recording system, and the temporal characteristics of operant behavior seen after 1.0 mg/kg tetrabenazine or vehicle injections were analyzed. Tetrabenazine shifted choice, decreasing lever pressing but increasing chow intake. There was a small effect on the interresponse-time distribution within ratios, but marked increases in the total duration of pauses in responding. The postreinforcement-pause (PRP) distribution was bimodal, but tetrabenazine did not increase the duration of PRPs. Tetrabenazine increased time feeding and duration and number of feeding bouts, but did not affect feeding rate or total time spent lever pressing for pellets and consuming chow. Thus, TBZ appears to predominantly affect the relative allocation of lever pressing versus chow, with little alteration in consummatory motor acts involved in chow intake. Tetrabenazine is used to model motivational symptoms in psychopathology, and these effects in rats could have implications for psychiatric research.

Interactions Between Experience, Genotype and Sex in the Development of Individual Coping Strategies

Coping strategies, the first line of defense against adversities, develop through experience. There is consistent evidence that both genotype and sex contribute to the development of dysfunctional coping, leading to maladaptive outcomes of adverse experiences or to adaptive coping that fosters rapid recovery even from severe stress. However, how these factors interact to influence the development of individual coping strategies is just starting to be investigated. In the following review, we will consider evidence that experience, sex, and genotype influence the brain circuits and neurobiological processes involved in coping with adversities and discuss recent results pointing to the specific effects of the interaction between early experiences, genotype, and stress in the development of functional and dysfunctional coping styles.

Event-based control of autonomic and emotional states by the anterior cingulate cortex

Despite being an intensive area of research, the function of the anterior cingulate cortex (ACC) remains somewhat of a mystery. Human imaging studies implicate the ACC in various cognitive functions, yet surgical ACC lesions used to treat emotional disorders have minimal lasting effects on cognition. An alternative view is that ACC regulates autonomic states, consistent with its interconnectivity with autonomic control regions and that stimulation evokes changes in autonomic/emotional states. At the cellular level, ACC neurons are highly multi-modal and promiscuous, and can represent a staggering array of task events. These neurons nevertheless combine to produce highly event-specific ensemble patterns that likely alter activity in downstream regions controlling emotional and autonomic tone. Since neuromodulators regulate the strength of the ensemble activity patterns, they would regulate the impact these patterns have on downstream targets. Through these mechanisms, the ACC may determine how strongly to react to the very events its ensembles represent. Pathologies arise when specific event-related representations gain excessive control over autonomic/emotional states.

Dorsomedial prefrontal neural ensembles reflect changes in task utility that culminate in task quitting

When performing a physically demanding behavior, sometimes the optimal choice is to quit the behavior rather than persist to minimize energy expenditure for the benefits gained. The dorsomedial prefrontal cortex (dmPFC), consisting of the anterior cingulate cortex and secondary motor area, likely contributes toward such utility assessments. Here, we examined how male rat dmPFC single unit and ensemble-level activity corresponded to changes in task utility and quitting in an effortful weight lifting task. Rats carried out two task paradigms: one that became progressively more physically demanding over time and a second fixed effort version. Rats could quit the task at any time. Dorsomedial PFC neurons were highly responsive to each behavioral stage of the task, consisting of rope pulling, reward retrieval, and reward area leaving. Activity was highest early in sessions, commensurate with the highest relative task utility, then decreased until the point of quitting. Neural ensembles consistently represented the sequential behavioral phases of the task. However, these representations were modified over time and became more distinct over the course of the session. These results suggest that dmPFC neurons represent behavioral states that are dynamically modified as behaviors lose their utility, culminating in task quitting.NEW & NOTEWORTHY When carrying out a physically demanding task, animals must continually assess whether to persist or quit. In this study, we recorded neurons in the dorsomedial prefrontal cortex (dmPFC) of rats as they carried out a challenging weightlifting task, up to the point of quitting. We demonstrate that dmPFC neurons form a representation of the task that is modified, via a decrease in firing rate, by the decreasing the utility of the task that may signal quitting.

Disruptions in effort-based decision-making following acute optogenetic stimulation of ventral tegmental area dopamine cells

The dopamine system has been implicated in decision-making particularly when associated with effortful behavior. We examined acute optogenetic stimulation of dopamine cells in the ventral tegmental area (VTA) as mice engaged in an effort-based decision-making task. Tyrosine hydroxylase-Cre mice were injected with Cre-dependent ChR2 or eYFP control virus in the VTA. While eYFP control mice showed effortful discounting, stimulation of dopamine cells in ChR2 mice disrupted effort-based decision-making by reducing choice toward the lever associated with a preferred outcome and greater effort. Surprisingly, disruptions in effortful discounting were observed in subsequent test sessions conducted in the absence of optogenetic stimulation, however during these sessions ChR2 mice displayed enhanced high choice responding across trial blocks. These findings suggest increases in VTA dopamine cell activity can disrupt effort-based decision-making in distinct ways dependent on the timing of optogenetic stimulation.

Go with the flow: A neuroscientific view on being fully engaged

Flow is a state of full task absorption, accompanied with a strong drive and low levels of self-referential thinking. Flow is likely when there is a match between a person's skills and the task challenge. Despite its relevance for human performance and the vast body of research on flow, there is currently still relatively little insight in its underlying neurocognitive mechanisms. In this paper, we discuss a set of large brain networks that may be involved in establishing the core dimensions of flow. We propose that dopaminergic and noradrenergic systems mediate the intrinsic motivation and activate mood states that are typical for flow. The interaction between three large-scale attentional networks, namely the Default Mode Network, Central Executive Network and the Salience Network is proposed to play a role in the strong task engagement, low self-referential thinking, feedback and feelings of control in flow. The proposed relationships between flow and the brain networks may support the generation of new hypotheses and can guide future research in this field.

Mapping Disease Course Across the Mood Disorder Spectrum Through a Research Domain Criteria Framework

BACKGROUND

The National Institute of Mental Health Research Domain Criteria (RDoC) initiative aims to establish a neurobiologically valid framework for classifying mental illness. Here, we examined whether the RDoC construct of reward learning and three aspects of its underlying neurocircuitry predicted symptom trajectories in individuals with mood pathology.

METHODS

Aligning with the RDoC approach, we recruited individuals (n = 80 with mood disorders [58 unipolar and 22 bipolar] and n = 32 control subjects; 63.4% female) based on their performance on a laboratory-based reward learning task rather than clinical diagnosis. We then assessed 1) anterior cingulate cortex prediction errors using electroencephalography, 2) striatal reward prediction errors using functional magnetic resonance imaging, and 3) medial prefrontal cortex glutamatergic function (mPFC Gln/Glu) using ¹H magnetic resonance spectroscopy. Severity of anhedonia, (hypo)mania, and impulsivity were measured at baseline, 3 months, and 6 months.

RESULTS

Greater homogeneity in aspects of brain function (mPFC Gln/Glu) was observed when individuals were classified according to reward learning ability rather than diagnosis. Furthermore, mPFC Gln/Glu levels predicted more severe (hypo)manic symptoms cross-sectionally, predicted worsening (hypo)manic symptoms longitudinally, and explained greater variance in future (hypo)manic symptoms than diagnostic information. However, rather than being transdiagnostic, this effect was specific to individuals with bipolar disorder. Prediction error indices were unrelated to symptom severity.

CONCLUSIONS

Although findings are preliminary and require replication, they suggest that heightened mPFC Gln/Glu warrants further consideration as a predictor of future (hypo)mania. Importantly, this work highlights the value of an RDoC approach that works in tandem with, rather than independent of, traditional diagnostic frameworks.

Dorsomedial striatal contributions to different forms of risk/reward decision making

Optimal decision making involving reward uncertainty is integral to adaptive goal-directed behavior. In some instances, these decisions are guided by internal representations of reward history, whereas in other situations, external cues inform a decision maker about how likely certain actions are to yield reward. Different regions of the frontal lobe form distributed networks with striatal and amygdalar regions that facilitate different types of risk/reward decision making. The dorsal medial striatum (DMS) is one key output region of the prefrontal cortex, yet there have been few preclinical studies investigating the involvement of the DMS in different forms of risk/reward decision making. The present study addressed this issue, wherein separate groups of male rats were trained on one of two tasks where they chose between a small/certain or a large/risky reward. In a probabilistic discounting task, reward probabilities changed systematically over blocks of trials (100-6.25% or 6.25-100%), requiring rats to use internal representations of reward history to guide choice. Cue-guided decision-making was assessed with a "Blackjack" task, where different auditory cues indicated the odds associated with the large/risky option (50 or 12.5%). Inactivation of the DMS with GABA agonists impaired adjustments in choice biases during probabilistic discounting, resulting in either increases or decreases in risky choice as the probabilities associated with the large/risky reward decreased or increased over a session. In comparison, DMS inactivation increased risky choices on poor-odds trials on the Blackjack task, which was associated with a reduced impact that non-rewarded choices had on subsequent choices. DMS inactivation also impaired performance of an auditory conditional discrimination. These findings highlight a previously uncharacterized role for the DMS in facilitating flexible action selection during multiple forms of risk/reward decision making.

Dissociating the effects of dopamine D2 receptors on effort-based versus value-based decision making using a novel behavioral approach

Cost-benefit decision making is essential for organisms to adapt to their ever-changing environment. Most studies of cost-benefit decision making involve choice conditions in which effort and value are varied simultaneously. This prevents identification of the aspects of cost-benefit decision making that are affected by experimental manipulations. We developed operant assays to isolate the individual impacts of effort and value manipulations on cost-benefit decision making. In the concurrent effort choice (CEC) task, mice choose between exerting two distinct types of effort: the number of responses and the duration of a response, to earn the same reward. By parametrically varying response cost, psychometric functions are obtained that reflect how the two types of effort scale against one another. Direct manipulations of effort shift the functions. Because reward value is held constant in this task, differences in scaling of the two response types must be related to the effort manipulations. In the concurrent value choice (CVC) task, mice make the same type of response to earn rewards of different value (e.g., pellets vs. sucrose solutions). Here the effort required to earn one reward type is parametrically varied to obtain the psychometric function that scales the value of the two rewards into the number of responses subjects will pay to earn one reward over the other. Direct value manipulations shift these functions. We tested the effect of the dopamine D2 receptor antagonist, haloperidol, on performance in the CEC and CVC assays and found that D2R signaling is important for effort-based, but not value-based decision making. (PsycINFO Database Record (c) 2020 APA, all rights reserved).

The dopamine depleting agent tetrabenazine alters effort-related decision making as assessed by mouse touchscreen procedures

RATIONALE

Effort-based decision-making tasks allow animals to choose between preferred reinforcers that require high effort to obtain vs. low-effort/low reward options. Mesolimbic dopamine (DA) and related neural systems regulate effort-based choice. Tetrabenazine (TBZ) is a vesicular monoamine transport type-2 inhibitor that blocks DA storage and depletes DA. In humans, TBZ induces motivational dysfunction and depression. TBZ has been shown reliably to induce a low-effort bias in rats, but there are fewer mouse studies.

OBJECTIVES

The present studies used touchscreen operant procedures (Bussey-Saksida chambers) to assess the effects of TBZ on effort-based choice in mice.

METHODS

C57BL6 mice were trained to press an elevated lit panel on the touchscreen on a fixed ratio 1 schedule reinforced by strawberry milkshake, vs. approaching and consuming a concurrently available but less preferred food pellets (Bio-serv).

RESULTS

TBZ (2.0-8.0 mg/kg IP) shifted choice, producing a dose-related decrease in panel pressing but an increase in pellet intake. In contrast, reinforcer devaluation by pre-feeding substantially decreased both panel pressing and pellet intake. In free-feeding choice tests, mice strongly preferred the milkshake vs. the pellets, and TBZ had no effect on milkshake intake or preference, indicating that the TBZ-induced low-effort bias was not due to changes in primary food motivation or preference. TBZ significantly decreased tissue levels of nucleus accumbens DA.

CONCLUSION

The DA depleting agent TBZ induced an effort-related motivational dysfunction in mice, which may have clinical relevance for assessing novel drug targets for their potential use as therapeutic agents in patients with motivation impairments.

A low dosage of the dopamine D2-receptor antagonist sulpiride affects effort allocation for reward regardless of trait extraversion

Dopamine (DA) is known to be involved in various aspects of reward processing and goal-directed behavior. The present preregistered study aims at directly accessing the causal influence of DA activity on reward motivation in humans, while also accounting for trait extraversion. Therefore, we examined the effect of a single dose of the DA D2 receptor antagonist sulpiride (200 mg) on effort allocation in a modified version of the Effort-Expenditure for Reward Task (EEfRT). Based on its presumably DA increasing action, we expected the low dose of sulpiride to increase participants' willingness to allocate effort during the modified EEfRT relative to placebo, especially in trials with low probability of reward attainment. Further, we expected a moderating effect of trait extraversion on the effects of sulpiride. Two hundred and three healthy male participants were tested in a randomized, double-blind between-subjects design. Contrary to our expectations, sulpiride reduced the average number of clicks within the modified EEfRT and did not interact with reward attributes, suggesting a more global and not reward-specific effect of sulpiride. Furthermore, trait extraversion did not moderate the effect of sulpiride. Our results provide initial support for the validity of the modified version of the EEfRT, suggesting a possible inhibiting effect of a low dose of sulpiride on approach motivation regardless of trait extraversion. However, given the mixed pattern of findings and the possible confounding role of motoric abilities, further studies examining these effects are clearly warranted.

Chemogenetic Modulation and Single-Photon Calcium Imaging in Anterior Cingulate Cortex Reveal a Mechanism for Effort-Based Decisions

The ACC is implicated in effort exertion and choices based on effort cost, but it is still unclear how it mediates this cost-benefit evaluation. Here, male rats were trained to exert effort for a high-value reward (sucrose pellets) in a progressive ratio lever-pressing task. Trained rats were then tested in two conditions: a no-choice condition where lever-pressing for sucrose was the only available food option, and a choice condition where a low-value reward (lab chow) was freely available as an alternative to pressing for sucrose. Disruption of ACC, via either chemogenetic inhibition or excitation, reduced lever-pressing in the choice, but not in the no-choice, condition. We next looked for value coding cells in ACC during effortful behavior and reward consumption phases during choice and no-choice conditions. For this, we used in vivo miniaturized fluorescence microscopy to reliably track responses of the same cells and compare how ACC neurons respond during the same effortful behavior where there was a choice versus when there was no-choice. We found that lever-press and sucrose-evoked responses were significantly weaker during choice compared with no-choice sessions, which may have rendered them more susceptible to chemogenetic disruption. Together, findings from our interference experiments and neural recordings suggest that a mechanism by which ACC mediates effortful decisions is in the discrimination of the utility of available options. ACC regulates these choices by providing a stable population code for the relative value of different options.

SIGNIFICANCE STATEMENT The ACC is implicated in effort-based decision-making. Here, we used chemogenetics and in vivo calcium imaging to explore its mechanism. Rats were trained to lever press for a high-value reward and tested in two conditions: a no-choice condition where lever-pressing for the high-value reward was the only option, and a choice condition where a low-value reward was also available. Inhibition or excitation of ACC reduced effort toward the high-value option, but only in the choice condition. Neural responses in ACC were weaker in the choice compared with the no-choice condition. A mechanism by which ACC regulates effortful decisions is in providing a stable population code for the discrimination of the utility of available options.

Impairment of cost-benefit decision making in morphine-dependent rats is partly mediated via the alteration of BDNF and p-CREB levels in the nucleus accumbens

The ability to choose goals based on decision usefulness or the time required to reach the goals chosen are important aspects of decision making. There is considerable evidence in the literature indicating the fact that drug abuse affects different aspects of cognition. In the current study, we assessed the effects of morphine dependence and its withdrawal on cost-benefit decision making and furthermore the involvement of BDNF and p-CREB in the nucleus accumbens, a key brain area involved in decision making was measured. Different groups of male Wistar rats were trained in an effort-based and/or delay-based form of cost-benefit T-maze decision-making task. Thereafter, the animals were morphine dependent and the percentage of the high reward preference was evaluated. After behavioral tests, the BDNF level, and p-CREB/CREB ratio were measured by Western blot analysis. The results showed that during effort-based but not delay-based decision making, BDNF and p-CREB levels increased. During effort-based decision making in morphine dependent rats, BDNF decreased but there was no significant change in p-CREB. Besides, during delay-based decision making in the morphine dependent group, both BDNF and p-CREB did not show any significant change. These findings revealed that BDNF and p-CREB/CREB ratio in the NAc are essential factors for effort-based but not delay-based decision making. In addition, impairment of effort-based decision making in morphine dependent rats is related to the decrease of BDNF level but not p-CREB/CREB ratio in the NAc. However, delay-based decision making defects in morphine dependent rats did not associate with the change in BDNF and p-CREB levels in the NAc.

Cingulum and abnormal psychological stress response in schizophrenia

Stress is implicated in many aspects of schizophrenia, including heightened distress intolerance. We examined how affect and microstructure of major brain tracts involved in regulating affect may contribute to distress intolerance in schizophrenia. Patients with schizophrenia spectrum disorders (n = 78) and community controls (n = 95) completed diffusion weighted imaging and performed psychological stress tasks. Subjective affect was collected pre and post stressors. Individuals who did not persist during one or both stress tasks were considered distress intolerant (DI), and otherwise distress tolerant (DT). Fractional anisotropy (FA) of the dorsal cingulum showed a significant diagnosis x DT/DI phenotype interaction (p = 0.003). Post-hoc tests showed dorsal cingulum FA was significantly lower in DI patients compared with DI controls (p < 0.001), but not different between DT groups (p = 0.27). Regarding affect responses to stress, irritability showed the largest stress-related change (p < 0.001), but irritability changes were significantly reduced in DI patients compared to DI controls (p = 0.006). The relationship between irritability change and performance errors also differed among patients (ρ = -0.29, p = 0.011) and controls (ρ = 0.21, p = 0.042). Further modeling highlighted the explanatory power of dorsal cingulum for predicting DI even after performance and irritability were taken into account. Distress intolerance during psychological stress exposure is related to microstructural properties of the dorsal cingulum, a key structure for cognitive control and emotion regulation. In schizophrenia, the affective response to psychological stressors is abnormal, and distress intolerant patients had significantly reduced dorsal cingulum FA compared to distress intolerant controls. The findings provide new insight regarding distress intolerance in schizophrenia.

Dopamine D1 receptors in the medial orbitofrontal cortex support effort-related responding in rats

Rodent studies on effort-related responding provide a tool to analyze basal aspects of motivation and to model psychiatric motivational dysfunctions reflecting low exertion of effort or reduced behavioral activation. It turned out that dopamine (DA) signaling in brain areas such as nucleus accumbens are essential in regulating effort-related motivational function and could play a major role in motivational dysfunction in psychiatric disorders. Recent rodent studies revealed that the medial orbitofrontal cortex (mOFC) is another key component of the neural circuitry regulating effort-related motivational function. The mOFC receives prominent DA input, however, the behavioral role of mOFC DA signaling is unknown. Here, we investigated whether DA signaling in the mOFC supports effort-related responding in rats. Results demonstrate that an intra-mOFC D1 receptor blockade markedly reduced effort-related responding in a progressive ratio task. Notably, the magnitude of this effect was comparable to the one caused by a systemic DA depletion induced by the VMAT-2 inhibitor tetrabenazine or by a satiety-induced motivational downshift. Collectively, our data show for the first time that D1 receptor activity in the mOFC plays a critical role in high effort responding. These results support findings in humans pointing to a role of the mOFC in effort-related responding. It is well known that the mOFC becomes dysfunctional in depression and schizophrenia. Our data point to the possibility that reduced mOFC DA activity could contribute to effort-related motivational symptoms in these disorders and support the notion that the DA system may be a drug target to treat effort-related motivational symptoms.

Reward Modulates Affective Priming Effect in Cognitive Conflict Processing: Electrophysiological Evidence

Previous research demonstrated that cognitive conflict could induce an affective priming effect, and the stage (detection/resolution) of conflict processing led to different directions (positive/negative) of the affective priming effect. We suggested that rewards play a critical role in the affective priming effect on conflict resolution. The present study used event-related potentials (ERPs), using the arrow flanker task as primes and choosing specific affective words as targets to investigate the affective priming effect induced by cognitive conflict during the resolution stage. Our question was whether rewards created a modulating effect. Participants were asked to judge the congruency of the prime stimuli and then evaluate the valence of the target words. For behavioral results, the conflict effect was significant, and the reward promoted the behavioral performance of participants. For ERP results, enhanced N2 amplitudes for incongruent primes indicated a significant conflict effect. More importantly, as expected, in the rewarded condition, the enhanced N400 amplitudes for positive targets following incongruent primes were found, indicating a positive priming effect. However, in the unrewarded condition, the reduced N400 amplitudes for positive targets following incongruent primes were found, indicating conflict resolution hindered the processing of positive stimuli. These findings suggested that cognitive conflict-induced the positive priming effect during the resolution stage and that rewards had a moderating effect on the positive priming effect.

Naloxone-precipitated withdrawal ameliorates impairment of cost-benefit decision making in morphine-treated rats: Involvement of BDNF, p-GSK3-β, and p-CREB in the amygdala

Several studies indicated that morphine administration impairs cognitive brain functions. Therefore, in the current study, we investigated the effect of subchronic exposure to morphine and its withdrawal on effort- and/or delay-based forms of cost-benefit decision making and alterations in p-CREB/CREB ratio, p-GSK3β/GSK3β ratio, and BDNF level during decision making in the amygdala. Our data displayed an impairment of both forms of cost-benefit decision making following subchronic exposure to morphine. However, preference of high reward/high effort and/or high delay reward increased after naloxone injection. In molecular section, levels of BDNF and p-CREB/CREB ratio increased during cost-benefit decision making while p-GSK3β/GSK3β ratio decreased in both forms of decision making. In morphine-treated rats, level of BDNF and p-CREB/CREB ratio reduced during both forms of decision making while p-GSK3β/GSK3β ratio increased during delay-based and did not have a significant difference with the control group during effort-based decision making. On the withdrawal day, BDNF level raised while p-GSK3β/GSK3β ratio attenuated compared to morphine-treated group in both form of decision making. In addition, p-CREB/CREB ratio increased only during delay-based decision making on the withdrawal day. In conclusion, our data revealed that subchronic exposure to morphine interferes with the cost-benefit decision making may be via changes in level of BDNF, p-CREB/CREB and p-GSK3β/GSK3β ratio in the amygdala.

Optogenetic Stimulation of the Anterior Cingulate Cortex Ameliorates Autistic-Like Behaviors in Rats Induced by Neonatal Isolation, Caudate Putamen as a Site for Alteration

Epigenetic agents, such as neonatal isolation during neurodevelopmental period of life, can change various regions of the brain. It may further induce psychological disorders such as autistic-like phenomena. This study indicated the role of chronic increased anterior cingulate cortex (ACC) output on alteration of caudate putamen (CPu) as a main behavior regulator region of the brain in adult maternal deprived (MD) rats. For making an animal model, neonates were isolated from their mothers in postnatal days (PND 1-10, 3 h/day). Subsequently, they bilaterally received pLenti-CaMKIIa-hChR2 (H134R)-mCherry-WPRE virus in ACC area via stereotaxic surgery in PND50. After 22 days, these regions were exposed to blue laser (473 nm) for six consecutive days (15 min/day). Then, behavioral deficits were tested and were compared with control group in the following day. Animals were immediately killed and their brains were prepared for tissue processing. Results showed that neonatal isolation induces autistic-like behaviors and leads to overexpression of NMDAR1 and Nox2-gp91phox proteins and elevation of catalase activity in the CPu regions of the adult offspring compared with control group. Chronic optogenetic stimulation of ACC neurons containing (ChR2+) led to significant reduction in the appearance of stereotypical behavior and alien-phobia in MD rats. The amount of NMDAR1 and Nox2-gp91phox expression and the catalase activity in CPu were reduced after this treatment. Therefore, autistic-like behavior seems to be related with elevation of NMDAR1 and Nox2-gp91phox protein levels that enhance the effect of glutamatergic projection on CPu regions. Optogenetic treatment also could ameliorate behavioral deficits by modulating these protein densities.

Quantity versus quality: Convergent findings in effort-based choice tasks

Organisms must frequently make cost-benefit decisions based on time, risk, and effort in choosing rewards to pursue. Various tasks have been developed to assess effort-based choice in rats, and experimenters have found largely similar results across tasks and brain regions. In this review, we focus primarily on the convergence of different effort-based choice tasks where quality or quantity of reward are manipulated. In the former, the rat is typically presented with the option to work for a preferred reward or select a less preferred, but freely-available reward. In such paradigms, the rewards are of different identities but are confirmed to differ qualitatively in value by a food preference task when both are freely-available. In the latter task type, rats are required to select between higher magnitude versus lower magnitudes of the same reward, but each with a similar effort requirement. We discuss the strengths/limitations of these paradigms, and describe brain regions that have been probed that result in converging or equivocal findings. Results are also reviewed with reference to a need for future work, and the broader impacts and implications of studies probing the mechanisms of effort.

Brain mechanisms underlying apathy

The past few decades have seen growing interest in the neuropsychiatric syndrome of apathy, conceptualised as a loss of motivation manifesting as a reduction of goal-directed behaviour. Apathy occurs frequently, and with substantial impact on quality of life, in a broad range of neurological and psychiatric conditions. Apathy is also consistently associated with neuroimaging changes in specific medial frontal cortex and subcortical structures, suggesting that disruption of a common systems-level mechanism may underlie its development, irrespective of the condition that causes it. In parallel with this growing recognition of the clinical importance of apathy, significant advances have been made in understanding normal motivated behaviour in humans and animals. These developments have occurred at several different conceptual levels, from work linking neural structures and neuromodulatory systems to specific aspects of motivated behaviour, to higher order computational models that aim to unite these findings within frameworks for normal goal-directed behaviour. In this review we develop a conceptual framework for understanding pathological apathy based on this current understanding of normal motivated behaviour. We first introduce prominent theories of motivated behaviour-which often involves sequences of actions towards a goal that needs to be maintained across time. Next, we outline the behavioural effects of disrupting these processes in animal models, highlighting the specific effects of these manipulations on different components of motivated behaviour. Finally, we relate these findings to clinical apathy, demonstrating the homologies between this basic neuroscience work and emerging behavioural and physiological evidence from patient studies of this syndrome.

Anterior cingulate cortex encoding of effortful behavior

An animal’s ability to assess the value of their behaviors to minimize energy use while maximizing goal achievement is critical to its survival. The anterior cingulate cortex (ACC) has been previously shown to play a critical role in this behavioral optimization process, especially when animals are faced with effortful behaviors. In the present study, we designed a novel task to investigate the role of the ACC in evaluating behaviors that varied in effort but all resulted in the same outcome. We recorded single unit activity from the ACC as rats ran back and forth in a shuttle box that could be tilted to different tilt angles (0, 15, and 25°) to manipulate effort. Overall, a majority of ACC neurons showed selective firing to specific effort conditions. During effort expenditure, ACC units showed a consistent firing rate bias toward the downhill route compared with the more difficult uphill route, regardless of the tilt angle of the apparatus. Once rats completed a run and received their fixed reward, ACC units also showed a clear firing rate preference for the single condition with the highest relative value (25° downhill). To assess effort preferences, we used a choice version of our task and confirmed that rats prefer downhill routes to uphill routes when given the choice. Overall, these results help to elucidate the functional role of the ACC in monitoring and evaluating effortful behaviors that may then bias decision-making toward behaviors with the highest utility.

NEW & NOTEWORTHY We developed a novel effort paradigm to investigate how the anterior cingulate cortex (ACC) responds to behaviors with varied degrees of physical effort and how changes in effort influence the ACC’s evaluation of behavioral outcomes. Our results provide evidence for a wider role of the ACC in its ability to motivate effortful behaviors and evaluate the outcome of multiple behaviors within an environment.

Adenosine A2A Receptors in the Rat Prelimbic Medial Prefrontal Cortex Control Delay-Based Cost-Benefit Decision Making

Adenosine A_2A receptors (A_2ARs) were recently described to control synaptic plasticity and network activity in the prefrontal cortex (PFC). We now probed the role of these PFC A_2AR by evaluating the behavioral performance (locomotor activity, anxiety-related behavior, cost-benefit decision making and working memory) of rats upon downregulation of A_2AR selectively in the prelimbic medial PFC (PLmPFC) via viral small hairpin RNA targeting the A_2AR (shA_2AR). The most evident alteration observed in shA_2AR-treated rats, when compared to sh-control (shCTRL)-treated rats, was a decrease in the choice of the large reward upon an imposed delay of 15 s assessed in a T-maze-based cost-benefit decision-making paradigm, suggestive of impulsive decision making. Spontaneous locomotion in the open field was not altered, suggesting no changes in exploratory behavior. Furthermore, rats treated with shA_2AR in the PLmPFC also displayed a tendency for higher anxiety levels in the elevated plus maze (less entries in the open arms), but not in the open field test (time spent in the center was not affected). Finally, working memory performance was not significantly altered, as revealed by the spontaneous alternation in the Y-maze test and the latency to reach the platform in the repeated trial Morris water maze. These findings constitute the first direct demonstration of a role of PFC A_2AR in the control of behavior in physiological conditions, showing their major contribution for the control of delay-based cost-benefit decisions.

Thalamic afferents to prefrontal cortices from ventral motor nuclei in decision-making

The focus of this literature review is on the three interacting brain areas that participate in decision‐making: basal ganglia, ventral motor thalamic nuclei, and medial prefrontal cortex, with an emphasis on the participation of the ventromedial and ventral anterior motor thalamic nuclei in prefrontal cortical function. Apart from a defining input from the mediodorsal thalamus, the prefrontal cortex receives inputs from ventral motor thalamic nuclei that combine to mediate typical prefrontal functions such as associative learning, action selection, and decision‐making. Motor, somatosensory and medial prefrontal cortices are mainly contacted in layer 1 by the ventral motor thalamic nuclei and in layer 3 by thalamocortical input from mediodorsal thalamus. We will review anatomical, electrophysiological, and behavioral evidence for the proposed participation of ventral motor thalamic nuclei and medial prefrontal cortex in rat and mouse motor decision‐making.

Dysfunctional effort-based decision-making underlies apathy in genetic cerebral small vessel disease

Apathy is a syndrome of reduced motivation that commonly occurs in patients with cerebral small vessel disease, including those with the early onset form, CADASIL (cerebral autosomal dominant arteriopathy with subcortical infarcts and leukoencephalopathy). The cognitive mechanisms underlying apathy are poorly understood and treatment options are limited. We hypothesized that disrupted effort-based decision-making, the cognitive process by which potential rewards and the effort cost required to obtain them is integrated to drive behaviour, might underlie the apathetic syndrome. Nineteen patients with a genetic diagnosis of CADASIL, as a model of 'pure' vascular cognitive impairment, and 19 matched controls were assessed using two different behavioural paradigms and MRI. On a decision-making task, participants decided whether to accept or reject sequential offers of monetary reward in return for exerting physical effort via handheld dynamometers. Six levels of reward and six levels of effort were manipulated independently so offers spanned the full range of possible combinations. Choice, decision time and force metrics were recorded. Each participant's effort and reward sensitivity was estimated using a computational model of choice. On a separate eye movement paradigm, physiological reward sensitivity was indexed by measuring pupillary dilatation to increasing monetary incentives. This metric was related to apathy status and compared to the behavioural metric of reward sensitivity on the decision-making task. Finally, high quality diffusion imaging and tract-based spatial statistics were used to determine whether tracts linking brain regions implicated in effort-based decision-making were disrupted in apathetic patients. Overall, apathetic patients with CADASIL rejected significantly more offers on the decision-making task, due to reduced reward sensitivity rather than effort hypersensitivity. Apathy was also associated with blunted pupillary responses to incentives. Furthermore, these independent behavioural and physiological markers of reward sensitivity were significantly correlated. Non-apathetic patients with CADASIL did not differ from controls on either task, whilst actual motor performance of apathetic patients in both tasks was also normal. Apathy was specifically associated with reduced fractional anisotropy within tracts connecting regions previously associated with effort-based decision-making. These findings demonstrate behavioural, physiological and anatomical evidence that dysfunctional effort-based decision-making underlies apathy in patients with CADASIL, a model disorder for sporadic small vessel disease. Reduced incentivization by rewards rather than hypersensitivity to effort costs drives this altered pattern of behaviour. The study provides empirical evidence of a cognitive mechanism for apathy in cerebral small vessel disease, and identifies a promising therapeutic target for interventions to improve this debilitating condition.

Cell-Type-Specific Afferent Innervation of the Nucleus Accumbens Core and Shell

The nucleus accumbens (NAc) is clearly implicated in reward processing and drug addiction, as well as in numerous neurological and psychiatric disorders; nevertheless, the circuit mechanisms underlying the diverse functions of the NAc remain poorly understood. Here, we characterized the whole-brain and monosynaptic inputs to two main projection cell types – D1 dopamine receptor expressing medium spiny neurons (D1R-MSNs) and D2 dopamine receptor expressing medium spiny neurons (D2R-MSNs) – within the NAc core and NAc shell by rabies-mediated trans-synaptic tracing. We discovered that D1R-MSNs and D2R-MSNs in both NAc subregions receive similar inputs from diverse sources. Inputs to the NAc core are broadly scattered, whereas inputs to the NAc shell are relatively concentrated. Furthermore, we identified numerous brain areas providing important contrasting inputs to different NAc subregions. The anterior cortex preferentially innervates the NAc core for both D1R-MSNs and D2R-MSNs, whereas the lateral hypothalamic area (LH) preferentially targets D1R-MSNs in the NAc shell. Characterizing the cell-type-specific connectivity of different NAc subregions lays a foundation for studying how diverse functions of the NAc are mediated by specific pathways.

Striatal dopamine D2 receptors regulate effort but not value-based decision making and alter the dopaminergic encoding of cost

Deficits in goal-directed motivation represent a debilitating symptom for many patients with schizophrenia. Impairments in motivation can arise from deficits in processing information about effort and or value, disrupting effective cost-benefit decision making. We have previously shown that upregulated dopamine D2 receptor expression within the striatum (D2R-OE mice) decreases goal-directed motivation. Here, we determine the behavioral and neurochemical mechanisms behind this deficit. Female D2R-OE mice were tested in several behavioral paradigms including recently developed tasks that independently assess the impact of Value or Effort manipulations on cost-benefit decision making. In vivo microdialysis was used to measure extracellular dopamine in the striatum during behavior. In a value-based choice task, D2R-OE mice show normal sensitivity to changes in reward value and used reward value to guide their actions. In an effort-based choice task, D2R-OE mice evaluate the cost of increasing the number of responses greater relative to the effort cost of longer duration responses compared to controls. This shift away from choosing to repeatedly execute a response is accompanied by a dampening of extracellular dopamine in the striatum during goal-directed behavior. In the ventral striatum, extracellular dopamine level negatively correlates with response cost in controls, but this relationship is lost in D2R-OE mice. These results show that D2R signaling in the striatum, as observed in some patients with schizophrenia, alters the relationship between effort expenditure and extracellular dopamine. This dysregulation produces motivation deficits that are specific to effort but not value-based decision making, paralleling the effort-based motivational deficits observed in schizophrenia.

Current Status of Cognitive Remediation for Psychiatric Disorders: A Review

Cognition is an important factor that affects daily functioning and quality of life. Impairment in cognitive function is a common symptom present in various psychological disorders, which hinders patients from functioning normally. Given that cognitive impairment has devastating effects, enhancing this in patients should lead to improvements in compromised quality of life and functioning, including vocational functioning. Over the past 50 years, several attempts have been made to improve impaired cognition, and empirical evidence for cognitive remediation (CR) has accumulated that supports its efficacy for treating schizophrenia. More recently, CR has been successfully applied in the treatment of depressive disorders, bipolar disorders, attention deficit/hyperactivity disorder, and anorexia nervosa. This study critically reviews recent CR studies and suggests their future direction. This study aimed to provide a modern definition of CR, and examine the current status of empirical evidence and representative CR programs that are widely used around the world.

The Psychopharmacology of Effort-Related Decision Making: Dopamine, Adenosine, and Insights into the Neurochemistry of Motivation

Effort-based decision making is studied using tasks that offer choices between high-effort options leading to more highly valued reinforcers versus low-effort/low-reward options. These tasks have been used to study the involvement of neural systems, including mesolimbic dopamine and related circuits, in effort-related aspects of motivation. Moreover, such tasks are useful as animal models of some of the motivational symptoms that are seen in people with depression, schizophrenia, Parkinson's disease, and other disorders. The present review will discuss the pharmacology of effort-related decision making and will focus on the use of these tasks for the development of drug treatments for motivational dysfunction. Research has identified pharmacological conditions that can alter effort-based choice and serve as models for depression-related symptoms (e.g., the vesicular monoamine transport-2 inhibitor tetrabenazine and proinflammatory cytokines). Furthermore, tests of effort-based choice have identified compounds that are particularly useful for stimulating high-effort work output and reversing the deficits induced by tetrabenazine and cytokines. These studies indicate that drugs that act by facilitating dopamine transmission, as well as adenosine A2A antagonists, are relatively effective at reversing effort-related impairments. Studies of effort-based choice may lead to the identification of drug targets that could be useful for treating motivational treatments that are resistant to commonly used antidepressants such as serotonin transport inhibitors.

The anatomy of apathy: A neurocognitive framework for amotivated behaviour

Apathy is a debilitating syndrome associated with many neurological disorders, including several common neurodegenerative diseases such as Parkinson's disease and Alzheimer's disease, and focal lesion syndromes such as stroke. Here, we review neuroimaging studies to identify anatomical correlates of apathy, across brain disorders. Our analysis reveals that apathy is strongly associated with disruption particularly of dorsal anterior cingulate cortex (dACC), ventral striatum (VS) and connected brain regions. Remarkably, these changes are consistent across clinical disorders and imaging modalities. Review of the neuroimaging findings allows us to develop a neurocognitive framework to consider potential mechanisms underlying apathy. According to this perspective, an interconnected group of brain regions - with dACC and VS at its core - plays a crucial role in normal motivated behaviour. Specifically we argue that motivated behaviour requires a willingness to work, to keep working, and to learn what is worth working for. We propose that deficits in any one or more of these processes can lead to the clinical syndrome of apathy, and outline specific approaches to test this hypothesis. A richer neurobiological understanding of the mechanisms underlying apathy should ultimately facilitate development of effective therapies for this disabling condition.