Steep effort discounting of a preferred reward over a freely-available option in prolonged methamphetamine withdrawal in male rats

Cholinergic and dopaminergic-mediated motivated behavior in healthy states and in substance use and mood disorders

Acetylcholine is an important neuromodulator of the mesolimbic dopamine (DA) system, which itself is a mediator of motivated behavior. Motivated behavior can be described by two primary components, termed directional and activational motivation, both of which can be examined and dissociated using effort-choice tasks. The directional component refers to motivated behavior directed towards reinforcing stimuli and away from aversive stimuli. Behaviors characterized by increased vigor, persistence, and work output are considered to reflect activational components of motivation. Disruption of DA signaling has been shown to decrease activational components of motivation, while leaving directional features intact. Facilitation of DA release promotes the activational aspects of motivated behavior. In this review, we discuss cholinergic and DA regulation of motivated behaviors. We place emphasis on effort-choice processes and the ability of effort-choice tasks to examine and dissociate changes of motivated behavior in the context of substance use and mood disorders. Furthermore, we consider how altered cholinergic transmission impacts motivated behavior across disease states, and the possible role of cholinergic dysregulation in the etiology of these illnesses. Finally, we suggest that treatments targeting cholinergic activity may be useful in ameliorating motivational disruptions associated with substance use and comorbid substance use and mood disorders.

Translational opportunities in animal and human models to study alcohol use disorder

Animal and human laboratory paradigms offer invaluable approaches to study the complex etiologies and mechanisms of alcohol use disorder (AUD). We contend that human laboratory models provide a "bridge" between preclinical and clinical studies of AUD by allowing for well-controlled experimental manipulations in humans with AUD. As such, examining the consilience between experimental models in animals and humans in the laboratory provides unique opportunities to refine the translational utility of such models. The overall goal of the present review is to provide a systematic description and contrast of commonly used animal paradigms for the study of AUD, as well as their human laboratory analogs if applicable. While there is a wide breadth of animal species in AUD research, the paradigms discussed in this review rely predominately on rodent research. The overarching goal of this effort is to provide critical analysis of these animal models and to link them to human laboratory models of AUD. By systematically contrasting preclinical and controlled human laboratory models, we seek to identify opportunities to enhance their translational value through forward and reverse translation. We provide future directions to reconcile differences between animal and human work and to improve translational research for AUD.

Effort-reward balance and work motivation in rats: Effects of context and order of experience

Being motivated means exerting effort toward a goal. The 'law of least work' emphasizes a preference for exerting relatively less effort. The law crosses boundaries among species and between physical and mental work. Organisms should be highly sensitive to shifts in effort-reward balance (ERB) in order to make optimal choices. We used a free operant-foraging task to investigate changes in ERB on choice between options requiring more or less effort. Results showed a consistent preference for the option with less effort and insensitivity to shifts in ERB. A second aim explored the influence of order of experience on effort choice. Choice for the more effortful option significantly increased after experiencing an equal effort-reward relationship during the initial free operant-foraging session. This relative increase in choice for the effortful option persisted even after effort-reward imbalance. The findings highlight the importance of contextual factors such as order of experience when examining the impact of shifting effort-reward associations. Instead of ignoring or reducing order effects, the sequence of experience (e.g. for shifts in ERB) could be manipulated to enhance or reduce value of outcomes or effort itself.

Comparison of acute treatment with delayed-onset versus rapid-acting antidepressants on effort-related choice behaviour

RATIONALE

Reward-related impairments are common in major depressive disorder (MDD) and may contribute to the loss of interest in pleasurable activities. A novel approach to studying reward-related decision-making are effort-based tasks; however, direct comparisons between delayed-onset and rapid-acting antidepressants (ADs) have not yet been carried out.

OBJECTIVES

To investigate the effects of conventional delayed-onset ADs versus rapid-acting ADs, ketamine and scopolamine, on effort-related choice behaviour.

METHODS

Female Lister hooded rats were trained in an operant effort for reward task (EfRT) where animals choose between working for a high value-high effort reward and consuming low value-low effort chow. Using a within-subject study design, animals were then tested following acute treatment with different monoaminergic ADs, and the rapid-acting ADs ketamine or scopolamine.

RESULTS

Consistent with previous findings, we found choice behaviour was sensitive to dopaminergic manipulations. We observed that pre-feeding altered choice behaviour and that the use of high or low value reward differentially affected behaviour. Monoamine re-uptake inhibitors and rapid-acting ADs resulted in similar, general patterns of reduced motivation without any evidence for specific effects, and we did not observe any clear differences between these classes of antidepressant.

CONCLUSIONS

Motivational changes induced by dopaminergic manipulations and pre-feeding differentially affect effort choice behaviour. However, both conventional delayed-onset ADs and ketamine and scopolamine appear to have detrimental effects on motivation in this task at the higher doses tested without any evidence of specificity for effort-related choice behaviour, in contrast to their specificity in tasks which look at more cognitive aspects of reward processing.

Chemogenetic Modulation and Single-Photon Calcium Imaging in Anterior Cingulate Cortex Reveal a Mechanism for Effort-Based Decisions

The ACC is implicated in effort exertion and choices based on effort cost, but it is still unclear how it mediates this cost-benefit evaluation. Here, male rats were trained to exert effort for a high-value reward (sucrose pellets) in a progressive ratio lever-pressing task. Trained rats were then tested in two conditions: a no-choice condition where lever-pressing for sucrose was the only available food option, and a choice condition where a low-value reward (lab chow) was freely available as an alternative to pressing for sucrose. Disruption of ACC, via either chemogenetic inhibition or excitation, reduced lever-pressing in the choice, but not in the no-choice, condition. We next looked for value coding cells in ACC during effortful behavior and reward consumption phases during choice and no-choice conditions. For this, we used in vivo miniaturized fluorescence microscopy to reliably track responses of the same cells and compare how ACC neurons respond during the same effortful behavior where there was a choice versus when there was no-choice. We found that lever-press and sucrose-evoked responses were significantly weaker during choice compared with no-choice sessions, which may have rendered them more susceptible to chemogenetic disruption. Together, findings from our interference experiments and neural recordings suggest that a mechanism by which ACC mediates effortful decisions is in the discrimination of the utility of available options. ACC regulates these choices by providing a stable population code for the relative value of different options.

SIGNIFICANCE STATEMENT The ACC is implicated in effort-based decision-making. Here, we used chemogenetics and in vivo calcium imaging to explore its mechanism. Rats were trained to lever press for a high-value reward and tested in two conditions: a no-choice condition where lever-pressing for the high-value reward was the only option, and a choice condition where a low-value reward was also available. Inhibition or excitation of ACC reduced effort toward the high-value option, but only in the choice condition. Neural responses in ACC were weaker in the choice compared with the no-choice condition. A mechanism by which ACC regulates effortful decisions is in providing a stable population code for the discrimination of the utility of available options.

Quantity versus quality: Convergent findings in effort-based choice tasks

Organisms must frequently make cost-benefit decisions based on time, risk, and effort in choosing rewards to pursue. Various tasks have been developed to assess effort-based choice in rats, and experimenters have found largely similar results across tasks and brain regions. In this review, we focus primarily on the convergence of different effort-based choice tasks where quality or quantity of reward are manipulated. In the former, the rat is typically presented with the option to work for a preferred reward or select a less preferred, but freely-available reward. In such paradigms, the rewards are of different identities but are confirmed to differ qualitatively in value by a food preference task when both are freely-available. In the latter task type, rats are required to select between higher magnitude versus lower magnitudes of the same reward, but each with a similar effort requirement. We discuss the strengths/limitations of these paradigms, and describe brain regions that have been probed that result in converging or equivocal findings. Results are also reviewed with reference to a need for future work, and the broader impacts and implications of studies probing the mechanisms of effort.

LiCl Pretreatment Ameliorates Adolescent Methamphetamine Exposure-Induced Long-Term Alterations in Behavior and Hippocampal Ultrastructure in Adulthood in Mice

BACKGROUND

Adolescent methamphetamine exposure causes a broad range of neurobiological deficits in adulthood. Glycogen synthase kinase-3β is involved in various cognitive and behavioral processes associated with methamphetamine exposure. This study aims to investigate the protective effects of the glycogen synthase kinase-3β inhibitor lithium chloride on adolescent methamphetamine exposure-induced long-term alterations in emotion, cognition, behavior, and molecule and hippocampal ultrastructure in adulthood.

METHODS

A behavioral test battery was used to investigate the protective effects of lithium chloride on adolescent methamphetamine exposure-induced long-term emotional, cognitive, and behavioral impairments in mice. Western blotting and immunohistochemistry were used to detect glycogen synthase kinase-3β activity levels in the medial prefrontal cortex and dorsal hippocampus. Electron microscopy was used to analyze changes in synaptic ultrastructure in the dorsal hippocampus. Locomotor sensitization with a methamphetamine (1 mg/kg) challenge was examined 80 days after adolescent methamphetamine exposure.

RESULTS

Adolescent methamphetamine exposure induced long-term alterations in locomotor activity, novel spatial exploration, and social recognition memory; increases in glycogen synthase kinase-3β activity in dorsal hippocampus; and decreases in excitatory synapse density and postsynaptic density thickness in CA1. These changes were ameliorated by lithium chloride pretreatment. Adolescent methamphetamine exposure-induced working memory deficits in Y-maze spontaneous alternation test and anxiety-like behavior in elevated-plus maze test spontaneously recovered after long-term methamphetamine abstinence. No significant locomotor sensitization was observed after long-term methamphetamine abstinence.

CONCLUSIONS

Hyperactive glycogen synthase kinase-3β contributes to adolescent chronic methamphetamine exposure-induced behavioral and hippocampal impairments in adulthood. Our results suggest glycogen synthase kinase-3β may be a potential target for the treatment of deficits in adulthood associated with adolescent methamphetamine abuse.

Rodent Models of Adaptive Value Learning and Decision-Making

Real-world decisions are rarely as straightforward as choosing between clearly "good" vs. "bad" options. More often, options must be evaluated carefully because they differ in relative value. For example, we typically learn about (and make decisions between) options in comparison, where one outcome may be more costly or risky than the other. Several neuropsychiatric conditions are characterized by atypical evaluation of effort and risk costs, including major depression, schizophrenia, autism, obsessive-compulsive disorder, and substance use disorders. Aberrant value learning and decision-making have long been considered a cognitive-behavioral endophenotype of these disorders and can be modeled in rodents. This chapter presents two general methodological domains that the experimenter can manipulate in animal decision-making tasks: risk and effort. Here, we present detailed methods of rodent tasks frequently employed within these domains: probabilistic reversal learning (PRL) and effort choice. These tasks recruit regions within rodent frontal cortex, the amygdala, and the striatum, and performance is heavily modulated by dopamine, making these assays highly valid measures in the study of behavioral and substance addictions, in particular.

Sustained attention but not effort-based decision-making predicts treatment motivation change in people with methamphetamine dependence

BACKGROUND

Early treatment motivation is a meaningful predictor of clinical outcomes in the context of methamphetamine dependence (MD). Cognitive deficits associated with MD can have a significant impact on motivational fluctuations during early treatment. We specifically examined if sustained attention and effort-based decision-making predict early treatment motivation change in individuals with MD. We hypothesised that both variables would be significant predictors of individual differences in treatment motivation change.

METHODS

We conducted a longitudinal, observational, cohort study on individuals with MD (N = 72, Age, M = 31.1, SD = 7.3, 29% female). Participants were assessed with cognitive tests of sustained attention (continuous performance test) and effort-based decision-making (effort expenditure for rewards task) within three weeks of entering treatment and rated their treatment motivation at baseline and at follow up six weeks later (n = 50). Multiple regression was used to examine the predictive value of cognitive variables after controlling for nuisance variables.

RESULTS

Cognitive measures significantly predicted change in treatment motivation after accounting for nuisance variables, F(5,43) = 2.89, p = .025. Analysis of individual predictors showed that sustained attention, but not decision-making, was a significant negative predictor of improvement in treatment motivation (β = -0.34, p = .015).

CONCLUSIONS

Poorer attentional function was associated with limited improvement in motivation during early treatment. These findings help to characterise cognitive predictors of treatment motivation and suggest directions for tailored treatment programs. Individuals entering treatment with attentional deficits may benefit from adjustments to therapy and/or cognitive remediation.

Persistent effect of withdrawal from intravenous methamphetamine self-administration on brain activation and behavioral economic indices involving an effort cost

Exposure to drugs of abuse produces maladaptive changes in cost-benefit decision-making, including the evaluation of time and risk. Studies probing the effects of drug exposure on such evaluations have primarily used experimenter-administered drug regimens. Similarly, while much is known about the neural bases of effort, there have been relatively fewer investigations of the effects of drug experience on effort-based choices. We recently reported that experimenter-administered methamphetamine (meth) resulted in steeper discounting of effort for food rewards in rats, when assessed in protracted withdrawal. Here, we studied rats that underwent withdrawal from weeks of meth intravenous self-administration that later could freely select between a high effort, preferred option (progressive ratio lever pressing for sucrose pellets) versus a low effort, less preferred option (freely-available lab chow). We found decreased effort for the preferred reward and changes in a behavioral economic index demonstrating an increased sensitivity to effort in meth-experienced rats. Critically, the decreased effort for the preferred option was only present in the context of a competing option, not when it was the only option. We also confirmed rats preferred sucrose pellets over chow when both were freely available. These long-lasting changes were accompanied by decreased c-Fos activation in ventral striatum and basolateral amygdala, regions known to be important in effort-based choices. Taken together with our previous observations, these results suggest a robust and enduring effect of meth on value-based decision-making, and point to the underlying neural mechanisms that support the evaluation of an effort cost.

Dopamine, Effort-Based Choice, and Behavioral Economics: Basic and Translational Research

Operant behavior is not only regulated by factors related to the quality or quantity of reinforcement, but also by the work requirements inherent in performing instrumental actions. Moreover, organisms often make effort-related decisions involving economic choices such as cost/benefit analyses. Effort-based decision making is studied using behavioral procedures that offer choices between high-effort options leading to relatively preferred reinforcers vs. low effort/low reward choices. Several neural systems, including the mesolimbic dopamine (DA) system and other brain circuits, are involved in regulating effort-related aspects of motivation. Considerable evidence indicates that mesolimbic DA transmission exerts a bi-directional control over exertion of effort on instrumental behavior tasks. Interference with DA transmission produces a low-effort bias in animals tested on effort-based choice tasks, while increasing DA transmission with drugs such as DA transport blockers tends to enhance selection of high-effort options. The results from these pharmacology studies are corroborated by the findings from recent articles using optogenetic, chemogenetic and physiological techniques. In addition to providing important information about the neural regulation of motivated behavior, effort-based choice tasks are useful for developing animal models of some of the motivational symptoms that are seen in people with various psychiatric and neurological disorders (e.g., depression, schizophrenia, Parkinson’s disease). Studies of effort-based decision making may ultimately contribute to the development of novel drug treatments for motivational dysfunction.