Prior cocaine self-administration does not impair the ability to delay gratification in rats during diminishing returns

Previous exposure to drugs of abuse produces impairments in studies of reversal learning, delay discounting and response inhibition tasks. While these studies contribute to the understanding of normal decision-making and how it is impaired by drugs of abuse, they do not fully capture how decision-making impacts the ability to delay gratification for greater long-term benefit. To address this issue, we used a diminishing returns task to study decision-making in rats that had previously self-administered cocaine. This task was designed to test the ability of the rat to choose to delay gratification in the short-term to obtain more reward over the course of the entire behavioral session. Rats were presented with two choices. One choice had a fixed amount of time delay needed to obtain reward [i.e. fixed delay (FD)], while the other choice had a progressive delay (PD) that started at 0 s and progressively increased by 1 s each time the PD option was selected. During the 'reset' variation of the task, rats could choose the FD option to reset the time delay associated with the PD option. Consistent with previous results, we found that prior cocaine exposure reduced rats' overall preference for the PD option in post-task reversal testing during 'no-reset' sessions, suggesting that cocaine exposure made rats more sensitive to the increasing delay of the PD option. Surprisingly, however, we found that rats that had self-administered cocaine 1-month prior, adapted behavior during 'reset' sessions by delaying gratification to obtain more reward in the long run similar to control rats.

From Prediction to Action: Dissociable Roles of Ventral Tegmental Area and Substantia Nigra Dopamine Neurons in Instrumental Reinforcement

Reward seeking requires the coordination of motor programs to achieve goals. Midbrain dopamine neurons are critical for reinforcement, and their activation is sufficient for learning about cues, actions, and outcomes. Here we examine in detail the mechanisms underlying the ability of ventral tegmental area (VTA) and substantia nigra (SNc) dopamine neurons to support instrumental learning. By exploiting numerous behavioral tasks in combination with time-limited optogenetic manipulations in male and female rats, we reveal that VTA and SNc dopamine neurons generate reinforcement through separable psychological processes. VTA dopamine neurons imbue actions and their associated cues with motivational value that allows flexible and persistent pursuit, whereas SNc dopamine neurons support time-limited, precise, action-specific learning that is nonscalable and inflexible. This architecture is reminiscent of actor-critic reinforcement learning models with VTA and SNc instructing the critic and actor, respectively. Our findings indicate that heterogeneous dopamine systems support unique forms of instrumental learning that ultimately result in disparate reward-seeking strategies.SIGNIFICANCE STATEMENT Dopamine neurons in the midbrain are essential for learning, motivation, and movement. Here we describe in detail the ability of VTA and SNc dopamine neurons to generate instrumental reinforcement, a process where an agent learns about actions they can emit to earn reward. While rats will avidly work and learn to respond for activation of VTA and SNc dopamine neurons, we find that only VTA dopamine neurons imbue actions and their associated cues with motivational value that spur continued pursuit of reward. Our data support a hypothesis that VTA and SNc dopamine neurons engage distinct psychological processes that have consequences for our understanding of these neurons in health and disease.

Rats delay gratification during a time-based diminishing returns task

The rat is a common animal model used to uncover the neural underpinnings of decision making and their disruption in psychiatric illness. Here, we ask if rats can perform a decision-making task that assesses self-control by delayed gratification in the context of diminishing returns. In this task, rats could choose to press one of two levers. One lever was associated with a fixed delay (FD) schedule that delivered reward after a fixed time delay (10 s). The other lever was associated with a progressive delay (PD) schedule; the delay increased by a fixed amount of time (1 s) after each PD lever press. Rats were tested under two conditions: a reset condition where rats could reset the PD schedule back to its initial 0-s delay by pressing the FD lever and a no-reset condition in which resetting the PD schedule was unavailable. We found that rats adapted behavior within reset sessions by delaying gratification to obtain more reward in the long run. That is, they selected the FD lever with the longer delay to reset the PD delay back to zero prior to the equality point, thus achieving more reward over the course of the session. These results are consistent with other species, demonstrating that rats can also maximize the net rate of reward by selecting an option that is not immediately beneficial. Moreover, use of this task in rodents might provide insights into how the brain governs normal and abnormal behavior, as well as treatments that can improve self-control. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Prior Cocaine Exposure Increases Firing to Immediate Reward While Attenuating Cue and Context Signals Related to Reward Value in the Insula

The insula contributes to behavioral control and is disrupted by substance abuse, yet we know little about the neural signals underlying these functions or how they are disrupted after chronic drug self-administration. Here, male and female rats self-administered either cocaine (experimental group) or sucrose (control) for 12 consecutive days. After a 1 month withdrawal period, we recorded from insula while rats performed a previously learned reward-guided decision-making task. Cocaine-exposed rats were more sensitive to value manipulations and were faster to respond. These behavioral changes were accompanied by elevated counts of neurons in the insula that increased firing to reward. These neurons also fired more strongly at the start of long-delay trials, when a more immediate reward would be expected, and fired less strongly in anticipation of the actual delivery of delayed rewards. Although reward-related firing to immediate reward was enhanced after cocaine self-administration, reward-predicting cue and context signals were attenuated. In addition to revealing novel firing patterns unique to insula, our data suggest changes in such neural activity likely contribute to impaired decision making observed after drug use.SIGNIFICANCE STATEMENT The insula plays a clear role in drug addiction and drug-induced impairments of decision making, yet there is little understanding of its underlying neural signals. We found that chronic cocaine self-administration reduces cue and context encoding in insula while enhancing signals related to immediate reward. These changes in neural activity likely contribute to impaired decision making and impulsivity observed after drug use.

Ventral Tegmental Dopamine Neurons Participate in Reward Identity Predictions

Dopamine (DA) neurons in the ventral tegmental area (VTA) and substantia nigra (SNc) encode reward prediction errors (RPEs) and are proposed to mediate error-driven learning. However, the learning strategy engaged by DA-RPEs remains controversial. RPEs might imbue predictive cues with pure value, independently of representations of their associated outcome. Alternatively, RPEs might promote learning about the sensory features (the identity) of the rewarding outcome. Here, we show that, although both VTA and SNc DA neuron activation reinforces instrumental responding, only VTA DA neuron activation during consumption of expected sucrose reward restores error-driven learning and promotes formation of a new cue→sucrose association. Critically, expression of VTA DA-dependent Pavlovian associations is abolished following sucrose devaluation, a signature of identity-based learning. These findings reveal that activation of VTA- or SNc-DA neurons engages largely dissociable learning processes with VTA-DA neurons capable of participating in outcome-specific predictive learning, and the role of SNc-DA neurons appears limited to reinforcement of instrumental responses.

The impact of drugs of abuse on executive function: characterizing long-term changes in neural correlates following chronic drug exposure and withdrawal in rats

Addiction has long been characterized by diminished executive function, control, and impulsivity management. In particular, these deficits often manifest themselves as impairments in reversal learning, delay discounting, and response inhibition. Understanding the neurobiological substrates of these behavioral deficits is of paramount importance to our understanding of addiction. Within the cycle of addiction, periods during and after withdrawal represent a particularly difficult point of intervention in that the negative physical symptoms associated with drug removal and drug craving increase the likelihood that the patient will relapse and return to drug use in order to abate these symptoms. Moreover, it is often during this time that drug induced deficits in executive function hinder the ability of the patient to refrain from drug use. Thus, it is necessary to understand the physiological and behavioral changes associated with withdrawal and drug craving-largely manifesting as deficits in executive control-to develop more effective treatment strategies. In this review, we address the long-term impact that drugs of abuse have on the behavioral and neural correlates that give rise to executive control as measured by reversal learning, delay discounting, and stop-signal tasks, focusing particularly on our work using rats as a model system.

Lever Insertion as a Salient Stimulus Promoting Insensitivity to Outcome Devaluation

Flexible and efficient decision-making in complex environments can be achieved through constant interactions between the goal-directed and habitual systems. While goal-directed behavior is considered dependent upon Response-Outcome (R-O) associations, habits instead rely on Stimulus-Response (S-R) associations. However, the stimuli that support the S-R association underlying habitual responding in typical instrumental procedures are poorly defined. To resolve this issue, we designed a discrete-trials procedure, in which rats must wait for lever insertion and complete a sequence of five lever presses to obtain a reward (20% sucrose or grain-based pellets). Lever insertion thus constituted an audio-visual stimulus signaling the opportunity for reward. Using sensory-specific satiety-induced devaluation, we found that rats trained with grain-based pellets remained sensitive to outcome devaluation over the course of training with this procedure whereas rats trained with a solution of 20% sucrose rapidly developed habit, and that insensitivity to outcome devaluation in rats trained with sucrose did not result from a bias in general satiety. Importantly, although rats trained with pellets were sensitive to satiety-induced devaluation, their performance was not affected by degradation of instrumental contingency and devaluation by conditioned taste aversion (CTA), suggesting that these rats may also have developed habitual responding. To test whether the discrete-trials procedure biases subjects towards habitual responding, we compared discrete-trials to free-running instrumental responding, and found that rats trained with sucrose in a fixed-ratio 5 (FR5) procedure with continuous presentation of the lever were goal-directed. Together, these results demonstrate that discrete presentations of a stimulus predictive of reward availability promoted the formation of S-R habit in rats trained with liquid sucrose. Further research is necessary to explain inconsistencies in sensitivity to outcome devaluation when rats are trained with grain-based pellets.

The Prodrug 4-Chlorokynurenine Causes Ketamine-Like Antidepressant Effects, but Not Side Effects, by NMDA/GlycineB-Site Inhibition

Currently approved antidepressant drug treatment typically takes several weeks to be effective. The noncompetitive N-methyl-d-aspartate (NMDA) receptor antagonist ketamine has shown efficacy as a rapid-acting treatment of depression, but its use is associated with significant side effects. We assessed effects following blockade of the glycineB co-agonist site of the NMDA receptor, located on the GluN1 subunit, by the selective full antagonist 7-chloro-kynurenic acid (7-Cl-KYNA), delivered by systemic administration of its brain-penetrant prodrug 4-chlorokynurenine (4-Cl-KYN) in mice. Following administration of 4-Cl-KYN, 7-Cl-KYNA was promptly recovered extracellularly in hippocampal microdialysate of freely moving animals. The behavioral responses of the animals were assessed using measures of ketamine-sensitive antidepressant efficacy (including the 24-hour forced swim test, learned helplessness test, and novelty-suppressed feeding test). In these tests, distinct from fluoxetine, and similar to ketamine, 4-Cl-KYN administration resulted in rapid, dose-dependent and persistent antidepressant-like effects following a single treatment. The antidepressant effects of 4-Cl-KYN were prevented by pretreatment with glycine or the α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptor antagonist 2,3-dihydroxy-6-nitro-7-sulfamoyl-benzo[f]quinoxaline-2,3-dione (NBQX). 4-Cl-KYN administration was not associated with the rewarding and psychotomimetic effects of ketamine, and did not induce locomotor sensitization or stereotypic behaviors. Our results provide further support for antagonism of the glycineB site for the rapid treatment of treatment-resistant depression without the negative side effects seen with ketamine or other channel-blocking NMDA receptor antagonists.