Withdrawal from escalated cocaine self-administration impairs reversal learning by disrupting the effects of negative feedback on reward exploitation: a behavioral and computational analysis

Intrinsic Functional Connectivity between the Anterior Insular and Retrosplenial Cortex as a Moderator and Consequence of Cocaine Self-Administration in Rats

While functional brain imaging studies in humans suggest that chronic cocaine use alters functional connectivity (FC) within and between key large-scale brain networks, including the default mode network (DMN), the salience network (SN), and the central executive network (CEN), cross-sectional studies in humans are challenging to obtain brain FC prior to cocaine use. Such information is critical to reveal the relationship between individual's brain FC and the subsequent development of cocaine dependence and brain changes during abstinence. Here, we performed a longitudinal study examining functional magnetic resonance imaging (fMRI) data in male rats (n = 7), acquired before cocaine self-administration (baseline), on 1 d of abstinence following 10 d of cocaine self-administration, and again after 30 d of experimenter-imposed abstinence. Using repeated-measures analysis of variance (ANOVA) with network-based statistics (NBS), significant connectivity changes were found between anterior insular cortex (AI) of the SN, retrosplenial cortex (RSC) of the DMN, somatosensory cortex, and caudate-putamen (CPu), with AI-RSC FC showing the most robust changes between baseline and 1 d of abstinence. Additionally, the level of escalated cocaine intake is associated with AI-RSC and AI-CPu FC changes between 1 d and 30 d of abstinence; further, the subjects' AI-RSC FC prior to cocaine intake is a significant moderator for the AI-RSC changes during abstinence. These results provide novel insights into the roles of AI-RSC FC before and after cocaine intake and suggest this circuit to be a potential target to modulate large-scale network and associated behavioral changes in cocaine use disorders.

Neurobehavioral Precursors of Compulsive Cocaine Seeking in Dual Frontostriatal Circuits

Background

Only some individuals who use drugs recreationally eventually develop a substance use disorder, characterized in part by the rigid engagement in drug foraging behavior (drug seeking), which is often maintained in the face of adverse consequences (i.e., is compulsive). The neurobehavioral determinants of this individual vulnerability have not been fully elucidated.

Methods

Using a prospective longitudinal study involving 39 male rats, we combined multidimensional characterization of behavioral traits of vulnerability to stimulant use disorder (impulsivity and stickiness) and resilience (sign tracking and sensation seeking/locomotor reactivity to novelty) with magnetic resonance imaging to identify the structural and functional brain correlates of the later emergence of compulsive drug seeking in drug-naïve subjects. We developed a novel behavioral procedure to investigate the individual tendency to persist in drug-seeking behavior in the face of punishment in a drug-free state in subjects with a prolonged history of cocaine seeking under the control of the conditioned reinforcing properties of a drug-paired Pavlovian conditioned stimulus.

Results

In drug-naïve rats, the tendency to develop compulsive cocaine seeking was characterized by behavioral stickiness-related functional hypoconnectivity between the prefrontal cortex and posterior dorsomedial striatum in combination with impulsivity-related structural alterations in the infralimbic cortex, anterior insula, and nucleus accumbens.

Conclusions

These findings show that the vulnerability to developing compulsive cocaine-seeking behavior stems from preexisting structural or functional changes in two distinct corticostriatal systems that underlie deficits in impulse control and goal-directed behavior.

Computational modelling of reinforcement learning and functional neuroimaging of probabilistic reversal for dissociating compulsive behaviours in gambling and cocaine use disorders

BACKGROUND

Individuals with cocaine use disorder or gambling disorder demonstrate impairments in cognitive flexibility: the ability to adapt to changes in the environment. Flexibility is commonly assessed in a laboratory setting using probabilistic reversal learning, which involves reinforcement learning, the process by which feedback from the environment is used to adjust behavior.

AIMS

It is poorly understood whether impairments in flexibility differ between individuals with cocaine use and gambling disorders, and how this is instantiated by the brain. We applied computational modelling methods to gain a deeper mechanistic explanation of the latent processes underlying cognitive flexibility across two disorders of compulsivity.

METHOD

We present a re-analysis of probabilistic reversal data from individuals with either gambling disorder (n = 18) or cocaine use disorder (n = 20) and control participants (n = 18), using a hierarchical Bayesian approach. Furthermore, we relate behavioural findings to their underlying neural substrates through an analysis of task-based functional magnetic resonanceimaging (fMRI) data.

RESULTS

We observed lower 'stimulus stickiness' in gambling disorder, and report differences in tracking expected values in individuals with gambling disorder compared to controls, with greater activity during reward expected value tracking in the cingulate gyrus and amygdala. In cocaine use disorder, we observed lower responses to positive punishment prediction errors and greater activity following negative punishment prediction errors in the superior frontal gyrus compared to controls.

CONCLUSIONS

Using a computational approach, we show that individuals with gambling disorder and cocaine use disorder differed in their perseverative tendencies and in how they tracked value neurally, which has implications for psychiatric classification.

From Computation to Clinic

Theory-driven and data-driven computational approaches to psychiatry have enormous potential for elucidating mechanism of disease and providing translational linkages between basic science findings and the clinic. These approaches have already demonstrated utility in providing clinically relevant understanding, primarily via back translation from clinic to computation, revealing how specific disorders or symptoms map onto specific computational processes. Nonetheless, forward translation, from computation to clinic, remains rare. In addition, consensus regarding specific barriers to forward translation-and on the best strategies to overcome these barriers-is limited. This perspective review brings together expert basic and computationally trained researchers and clinicians to 1) identify challenges specific to preclinical model systems and clinical translation of computational models of cognition and affect, and 2) discuss practical approaches to overcoming these challenges. In doing so, we highlight recent evidence for the ability of computational approaches to predict treatment responses in psychiatric disorders and discuss considerations for maximizing the clinical relevance of such models (e.g., via longitudinal testing) and the likelihood of stakeholder adoption (e.g., via cost-effectiveness analyses).

Causal role of the dorsolateral prefrontal cortex in modulating the balance between Pavlovian and instrumental systems in the punishment domain

Previous literature suggests that a balance between Pavlovian and instrumental decision-making systems is critical for optimal decision-making. Pavlovian bias (i.e., approach toward reward-predictive stimuli and avoid punishment-predictive stimuli) often contrasts with the instrumental response. Although recent neuroimaging studies have identified brain regions that may be related to Pavlovian bias, including the dorsolateral prefrontal cortex (dlPFC), it is unclear whether a causal relationship exists. Therefore, we investigated whether upregulation of the dlPFC using transcranial current direct stimulation (tDCS) would reduce Pavlovian bias. In this double-blind study, participants were assigned to the anodal or the sham group; they received stimulation over the right dlPFC for 3 successive days. On the last day, participants performed a reinforcement learning task known as the orthogonalized go/no-go task; this was used to assess each participant's degree of Pavlovian bias in reward and punishment domains. We used computational modeling and hierarchical Bayesian analysis to estimate model parameters reflecting latent cognitive processes, including Pavlovian bias, go bias, and choice randomness. Several computational models were compared; the model with separate Pavlovian bias parameters for reward and punishment domains demonstrated the best model fit. When using a behavioral index of Pavlovian bias, the anodal group showed significantly lower Pavlovian bias in the punishment domain, but not in the reward domain, compared with the sham group. In addition, computational modeling showed that Pavlovian bias parameter in the punishment domain was lower in the anodal group than in the sham group, which is consistent with the behavioral findings. The anodal group also showed a lower go bias and choice randomness, compared with the sham group. These findings suggest that anodal tDCS may lead to behavioral suppression or change in Pavlovian bias in the punishment domain, which will help to improve comprehension of the causal neural mechanism.

Sex-dependent effects of early life stress on reinforcement learning and limbic cortico-striatal functional connectivity

Major depressive disorder (MDD) is a stress-related condition hypothesized to involve aberrant reinforcement learning (RL) with positive and negative stimuli. The present study investigated whether repeated early maternal separation (REMS) stress, a procedure widely recognized to cause depression-like behaviour, affects how subjects learn from positive and negative feedback. The REMS procedure was implemented by separating male and female rats from their dam for 6 h each day from post-natal day 5-19. Control rat offspring were left undisturbed during this period. Rats were tested as adults for behavioral flexibility and feedback sensitivity on a probabilistic reversal learning task. A computational approach based on RL theory was used to derive latent behavioral variables related to reward learning and flexibility. To assess underlying brain substrates, a seed-based functional MRI connectivity analysis was applied both before and after an additional adulthood stressor in control and REMS rats. Female but not male rats exposed to REMS stress showed increased response 'stickiness' (repeated responses regardless of reward outcome). Following repeated adulthood stress, reduced functional connectivity from the basolateral amygdala (BLA) to the dorsolateral striatum (DLS), cingulate cortex (Cg), and anterior insula (AI) cortex was observed in females. By contrast, control male rats exposed to the second stressor showed impaired learning from negative feedback (i.e., non-reward) and reduced functional connectivity from the BLA to the DLS and AI compared to maternally separated males. RL in male rats exposed to REMS was unaffected. The fMRI data further revealed that connectivity between the mOFC and other prefrontal cortical and subcortical structures was positively correlated with response 'stickiness'. These findings reveal differences in how females and males respond to early life adversity and subsequent stress. These effects may be mediated by functional divergence in resting-state connectivity between the basolateral amygdala and fronto-striatal brain regions.

Adolescent reinforcement-learning trajectories predict cocaine-taking behaviors in adult male and female rats

The anatomical, structural, and functional adaptations that occur in the brain during adolescence are thought to facilitate improvements in decision-making functions that are known to occur during this stage of development. The mechanisms that underlie these neural adaptations are not known, but deviations in developmental trajectories have been proposed to contribute to the emergence of mental illness, including addiction. Direct evidence supporting this hypothesis, however, has been limited. Here, we used a recently developed reversal-learning protocol to investigate the predictive relationship between adolescent decision-making trajectories and cocaine-taking behaviors in adulthood. Decision-making functions in the reversal-learning task were assessed throughout adolescence and into adulthood in male and female Long-Evans rats. Trial-by-trial choice data was fitted with a reinforcement-learning model to quantify the degree to which choice behavior of individual rats was influenced by rewarded (e.g., ∆₊ parameter) and unrewarded (e.g., ∆₀ parameter) outcomes. We report that reversal-learning performance improved during adolescence and that this was due to an increase in value updating for rewarded outcomes (e.g., ∆₊ parameter). Furthermore, the rate of change in the ∆₊ parameter predicted individual differences in the ∆₊ parameter and, notably, cocaine-taking behaviors in adulthood: Rats that had a shallower adolescent trajectory were found to have a lower ∆₊ parameter and greater cocaine self-administration in adulthood. These data indicate that adolescent development plays a critical role in drug use susceptibility. Future studies aimed at understanding the neurobiological mechanisms that underlie these age-related changes in decision-making could provide new insights into the biobehavioral mechanisms mediating addiction susceptibility.

Brain systems in cocaine abstinence-induced anxiety-like behavior in rodents: A review

Cocaine use disorder (CUD) is a significant public health issue that generates substantial personal, familial, and economic burdens. Still, there are no FDA-approved pharmacotherapies for CUD. Cocaine-dependent individuals report anxiety during withdrawal, and alleviation of anxiety and other negative affective states may be critical for maintaining drug abstinence. However, the neurobiological mechanisms underlying abstinence-related anxiety in humans or anxiety-like behavior in rodents are not fully understood. This review summarizes investigations regarding anxiety-like behavior in mice and rats undergoing cocaine abstinence, as assessed using four of the most common anxiety-related assays: the elevated plus (or its derivative, the elevated zero) maze, open field test, light-dark transition test, and defensive burying task. We first summarize available evidence that cocaine abstinence generates anxiety-like behavior that persists throughout protracted abstinence. Then, we examine investigations concerning neuropeptide, neurotransmitter, and neuromodulator systems in cocaine abstinence-induced anxiety-like behavior. Throughout, we discuss how differences in sex, rodent strain, cocaine dose and dosing strategy and abstinence duration interact to generate anxiety-like behavior.

Reinforcement learning detuned in addiction: integrative and translational approaches

Suboptimal decision-making strategies have been proposed to contribute to the pathophysiology of addiction. Decision-making, however, arises from a collection of computational components that can independently influence behavior. Disruptions in these different components can lead to decision-making deficits that appear similar behaviorally, but differ at the computational, and likely the neurobiological, level. Here, we discuss recent studies that have used computational approaches to investigate the decision-making processes underlying addiction. Studies in animal models have found that value updating following positive, but not negative, outcomes is predictive of drug use, whereas value updating following negative, but not positive, outcomes is disrupted following drug self-administration. We contextualize these findings with studies on the circuit and biological mechanisms of decision-making to develop a framework for revealing the biobehavioral mechanisms of addiction.

Cannabis Use and Neuroadaptation: A Call for Δ 9 -Tetrahydrocannabinol Challenge Studies

Currently, the assessment of the neurobehavioral consequences of repeated cannabis use is restricted to studies in which brain function of chronic cannabis users is compared to that of non-cannabis using controls. The assumption of such studies is that changes in brain function of chronic users are caused by repeated and prolonged exposure to acute cannabis intoxication. However, differences in brain function between chronic cannabis users and non-users might also arise from confounding factors such as polydrug use, alcohol use, withdrawal, economic status, or lifestyle conditions. We propose a methodology that highlights the relevance of acute Δ⁹-tetrahydrocannabinol (THC) dosing studies for a direct assessment of neuroadaptations in chronic cannabis users. The approach includes quantification of neurochemical, receptor, and functional brain network changes in response to an acute cannabis challenge, as well as stratification of cannabis using groups ranging from occasional to cannabis-dependent individuals. The methodology allows for an evaluation of THC induced neuroadaptive and neurocognitive changes across cannabis use history, that can inform neurobiological models on reward driven, compulsive cannabis use.

Unlocking the reinforcement-learning circuits of the orbitofrontal cortex

Neuroimaging studies have consistently identified the orbitofrontal cortex (OFC) as being affected in individuals with neuropsychiatric disorders. OFC dysfunction has been proposed to be a key mechanism by which decision-making impairments emerge in diverse clinical populations, and recent studies employing computational approaches have revealed that distinct reinforcement-learning mechanisms of decision-making differ among diagnoses. In this perspective, we propose that these computational differences may be linked to select OFC circuits and present our recent work that has used a neurocomputational approach to understand the biobehavioral mechanisms of addiction pathology in rodent models. We describe how combining translationally analogous behavioral paradigms with reinforcement-learning algorithms and sophisticated neuroscience techniques in animals can provide critical insights into OFC pathology in biobehavioral disorders. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Prior Cocaine Use Alters the Normal Evolution of Information Coding in Striatal Ensembles during Value-Guided Decision-Making

Substance use disorders (SUDs) are characterized by maladaptive behavior. The ability to properly adjust behavior according to changes in environmental contingencies necessitates the interlacing of existing memories with updated information. This can be achieved by assigning learning in different contexts to compartmentalized "states." Though not often framed this way, the maladaptive behavior observed in individuals with SUDs may result from a failure to properly encode states because of drug-induced neural alterations. Previous studies found that the dorsomedial striatum (DMS) is important for behavioral flexibility and state encoding, suggesting the DMS may be an important substrate for these effects. Here, we recorded DMS neural activity in cocaine-experienced male rats during a decision-making task where blocks of trials represented distinct states to probe whether the encoding of state and state-related information is affected by prior drug exposure. We found that DMS medium spiny neurons (MSNs) and fast-spiking interneurons (FSIs) encoded such information and that prior cocaine experience disrupted the evolution of representations both within trials and across recording sessions. Specifically, DMS MSNs and FSIs from cocaine-experienced rats demonstrated higher classification accuracy of trial-specific rules, defined by response direction and value, compared with those drawn from sucrose-experienced rats, and these overly strengthened trial-type representations were related to slower switching behavior and reaction times. These data show that prior cocaine experience paradoxically increases the encoding of state-specific information and rules in the DMS and suggest a model in which abnormally specific and persistent representation of rules throughout trials in DMS slows value-based decision-making in well trained subjects.SIGNIFICANCE STATEMENT Substance use disorders (SUDs) may result from a failure to properly encode rules guiding situationally appropriate behavior. The dorsomedial striatum (DMS) is thought to be important for such behavioral flexibility and encoding that defines the situation or "state." This suggests that the DMS may be an important substrate for the maladaptive behavior observed in SUDs. In the current study, we show that prior cocaine experience results in over-encoding of state-specific information and rules in the DMS, which may impair normal adaptive decision-making in the task, akin to what is observed in SUDs.

Dysregulation of Decision Making Related to Metabotropic Glutamate 5, but Not Midbrain D3, Receptor Availability Following Cocaine Self-administration in Rats

BACKGROUND

Compulsive patterns of drug use are thought to be the consequence of drug-induced adaptations in the neural mechanisms that enable behavior to be flexible. Neuroimaging studies have found evidence of robust alterations in glutamate and dopamine receptors within brain regions that are known to be critical for decision-making processes in cocaine-dependent individuals, and these changes have been argued to be the consequence of persistent drug use. The causal relationships among drug-induced alterations, cocaine taking, and maladaptive decision-making processes, however, are difficult to establish in humans.

METHODS

We assessed decision making in adult male rats using a probabilistic reversal learning task and used positron emission tomography with the [¹¹C]-(+)-PHNO and [¹⁸F]FPEB radioligands to quantify regional dopamine D_2/3 and metabotropic glutamate 5 (mGlu5) receptor availability, respectively, before and after 21 days of cocaine or saline self-administration. Tests of motivation and relapse-like behaviors were also conducted.

RESULTS

We found that self-administration of cocaine, but not of saline, disrupted behavior in the probabilistic reversal learning task measured by selective impairments in negative-outcome updating and also increased cortical mGlu5 receptor availability following 2 weeks of forced abstinence. D_2/3 and, importantly, midbrain D₃ receptor availability was not altered following 2 weeks of abstinence from cocaine. Notably, the degree of the cocaine-induced increase in cortical mGlu5 receptor availability was related to the degree of disruption in negative-outcome updating.

CONCLUSIONS

These findings suggest that cocaine-induced changes in mGlu5 signaling may be a mechanism by which disruptions in negative-outcome updating emerge in cocaine-dependent individuals.

Midbrain D3 Receptor Availability Predicts Escalation in Cocaine Self-administration

BACKGROUND

Results from neuroimaging studies suggest that disruptions in flexible decision-making functions in substance-dependent individuals are a consequence of drug-induced neural adaptations. In addicted populations, however, the causal relationship between biobehavioral phenotypes of susceptibility and addiction consequence is difficult to dissociate. Indeed, evidence from animals suggests that poor decision making due to preexisting biological factors can independently enhance the risk for developing addiction-like behaviors. Neuroimaging studies in animals provide a unique translational approach for the identification of the neurobiological mechanisms that mediate susceptibility to addiction.

METHODS

We used positron emission tomography in rats to quantify regional dopamine D_2/3 receptors and metabotropic glutamate receptor 5 (mGluR5) and assessed decision making using a probabilistic reversal learning task. Susceptibility to self-administer cocaine was then quantified for 21 days followed by tests of motivation and relapse-like behaviors.

RESULTS

We found that deficits specifically in reward-guided choice behavior on the probabilistic reversal learning task predicted greater escalation of cocaine self-administration behavior and greater motivation for cocaine and, critically, were associated with higher midbrain D₃ receptor availability. Additionally, individual differences in midbrain D₃ receptor availability independently predicted the rate of escalation in cocaine-taking behaviors. No differences in mGluR5 availability, responses during tests of extinction, or cue-induced reinstatement were observed between the groups.

CONCLUSIONS

These findings indicate that our identified D₃-mediated decision-making phenotype can be used as a behavioral biomarker for assessment of cocaine use susceptibility in human populations.

Computational theory-driven studies of reinforcement learning and decision-making in addiction: What have we learned?

Computational psychiatry provides a powerful new approach for linking the behavioral manifestations of addiction to their precise cognitive and neurobiological substrates. However, this emerging area of research is still limited in important ways. While research has identified features of reinforcement learning and decision-making in substance users that differ from health, less emphasis has been placed on capturing addiction cycles/states dynamically, within-person. In addition, the focus on few behavioral variables at a time has precluded more detailed consideration of related processes and heterogeneous clinical profiles. We propose that a longitudinal and multidimensional examination of value-based processes, a type of dynamic "computational fingerprint", will provide a more complete understanding of addiction as well as aid in developing better tailored and timed interventions.

A reinforcement-learning model of active avoidance behavior: Differences between Sprague Dawley and Wistar-Kyoto rats

Avoidance behavior is a typically adaptive response performed by an organism to avert harmful situations. Individuals differ remarkably in their tendency to acquire and perform new avoidance behaviors, as seen in anxiety disorders where avoidance becomes pervasive and inappropriate. In rodent models of avoidance, the inbred Wistar-Kyoto (WKY) rat demonstrates increased learning and expression of avoidance compared to the outbred Sprague Dawley (SD) rat. However, underlying mechanisms that contribute to these differences are unclear. Computational modeling techniques can help identify factors that may not be easily decipherable from behavioral data alone. Here, we utilize a reinforcement learning (RL) model approach to better understand strain differences in avoidance behavior. An actor-critic model, with separate learning rates for action selection (in the actor) and state evaluation (in the critic), was applied to individual data of avoidance acquisition from a large cohort of WKY and SD rats. Latent parameters were extracted, such as learning rate and subjective reinforcement value of foot shock, that were then compared across groups. The RL model was able to accurately represent WKY and SD avoidance behavior, demonstrating that the model could simulate individual performance. The model determined that the perceived negative value of foot shock was significantly higher in WKY than SD rats, whereas learning rate in the actor was lower in WKY than SD rats. These findings demonstrate the utility of computational modeling in identifying underlying processes that could promote strain differences in behavioral performance.

Adolescent-onset vs. adult-onset cocaine use: Impact on cognitive functioning in animal models and opportunities for translation

Animal models are poised to make key contributions to the study of cognitive deficits associated with chronic cocaine use in people. Advantages of animal models include use of a longitudinal experimental design that can control for drug use history and onset-age, sex, drug consumption, and abstinence duration. Twenty-two studies were reviewed (13 in adult male rats, 5 in adolescent vs. adult male rats, 3 in adult male monkeys, and 1 in adult female monkeys), and it was demonstrated repeatedly that male animals with adult-onset cocaine self-administration exposure had impairments in sustained attention, decision making, stimulus-reward learning, working memory, and cognitive flexibility, but not habit learning and spatial learning and memory. These findings have translational relevance because adult cocaine users exhibit a similar range of cognitive deficits. In the limited number of studies available, male rats self-administering cocaine during adolescence were less susceptible than adults to impairment in cognitive flexibility, stimulus-reward learning, and decision making, but were more susceptible than adults to impairment in working memory, a finding also reported in the few studies performed in early-onset cocaine users. These findings suggest that animal models can help fill an unmet need for investigating important but yet-to-be-fully-addressed research questions in people. Research priorities include further investigation of differences between adolescents and adults as well as between males and females following chronic cocaine self-administration. A comprehensive understanding of the broad range of cognitive consequences of chronic cocaine use and the role of developmental plasticity can be of value for improving neuropsychological recovery efforts.

The Neurobiology of Impulsive Decision-Making and Reinforcement Learning in Nonhuman Animals

Impulsive decisions are those that favor immediate over delayed rewards, involve the acceptance of undue risk or uncertainty, or fail to adapt to environmental changes. Pathological levels of impulsive decision-making have been observed in individuals with mental illness, but there may be substantial heterogeneity in the processes that drive impulsive choices. Understanding this behavioral heterogeneity may be critical for understanding associated diverseness in the neural mechanisms that give rise to impulsivity. The application of reinforcement learning algorithms in the deconstruction of impulsive decision-making phenotypes can help bridge the gap between biology and behavior and provide insights into the biobehavioral heterogeneity of impulsive choice. This chapter will review the literature on the neurobiological mechanisms of impulsive decision-making in nonhuman animals; specifically, the role of the amine neuromodulatory systems (dopamine, serotonin, norepinephrine, and acetylcholine) in impulsive decision-making and reinforcement learning processes is discussed. Ultimately, the integration of reinforcement learning algorithms with sophisticated behavioral and neuroscience techniques may be critical for advancing the understanding of the neurochemical basis of impulsive decision-making.

Tonic exploration governs both flexibility and lapses

In many cognitive tasks, lapses (spontaneous errors) are tacitly dismissed as the result of nuisance processes like sensorimotor noise, fatigue, or disengagement. However, some lapses could also be caused by exploratory noise: randomness in behavior that facilitates learning in changing environments. If so, then strategic processes would need only up-regulate (rather than generate) exploration to adapt to a changing environment. This view predicts that more frequent lapses should be associated with greater flexibility because these behaviors share a common cause. Here, we report that when rhesus macaques performed a set-shifting task, lapse rates were negatively correlated with perseverative error frequency across sessions, consistent with a common basis in exploration. The results could not be explained by local failures to learn. Furthermore, chronic exposure to cocaine, which is known to impair cognitive flexibility, did increase perseverative errors, but, surprisingly, also improved overall set-shifting task performance by reducing lapse rates. We reconcile these results with a state-switching model in which cocaine decreases exploration by deepening attractor basins corresponding to rule states. These results support the idea that exploratory noise contributes to lapses, affecting rule-based decision-making even when it has no strategic value, and suggest that one key mechanism for regulating exploration may be the depth of rule states.