Relative insensitivity to time-out punishments induced by win-paired cues in a rat gambling task

Noradrenergic regulation of cue-guided decision making and impulsivity is doubly dissociable across frontal brain regions

RATIONALE

Win-paired stimuli can promote risk taking in experimental gambling paradigms in both rats and humans. We previously demonstrated that atomoxetine, a noradrenaline reuptake inhibitor, and guanfacine, a selective α2A adrenergic receptor agonist, reduced risk taking on the cued rat gambling task (crGT), a rodent assay of risky choice in which wins are accompanied by salient cues. Both compounds also decreased impulsive premature responding.

OBJECTIVE

The key neural loci mediating these effects were unknown. The lateral orbitofrontal cortex (lOFC) and the medial prefrontal cortex (mPFC), which are highly implicated in risk assessment, action selection, and impulse control, receive dense noradrenergic innervation. We therefore infused atomoxetine and guanfacine directly into either the lOFC or prelimbic (PrL) mPFC prior to task performance.

RESULTS

When infused into the lOFC, atomoxetine improved decision making score and adaptive lose-shift behaviour in males, but not in females, without altering motor impulsivity. Conversely, intra-PrL atomoxetine improved impulse control in risk preferring animals of both sexes, but did not alter decision making. Guanfacine administered into the PrL, but not lOFC, also altered motor impulsivity in all subjects, though in the opposite direction to atomoxetine.

CONCLUSIONS

These data highlight a double dissociation between the behavioural effects of noradrenergic signaling across frontal regions with respect to risky choice and impulsive action. Given that the influence of noradrenergic manipulations on motor impulsivity could depend on baseline risk preference, these data also suggest that the noradrenaline system may function differently in subjects that are susceptible to the risk-promoting lure of win-associated cues.

Risk-promoting effects of reward-paired cues in human sign- and goal-trackers

Animal research suggests trait-like individual variation in the degree of incentive salience attribution to reward-predictive cues, defined phenotypically as sign-tracking (high) and goal-tracking (low incentive salience attribution). While these phenotypes have been linked to addiction features in rodents, their translational validity is less clear. Here, we examined whether sign- and goal-tracking in healthy human volunteers modulates the effects of reward-paired cues on decision making. Sign-tracking was measured in a Pavlovian conditioning paradigm as the amount of eye gaze fixation on the reward-predictive cue versus the location of impending reward delivery. In Study 1 (Cherkasova et al., 2018), participants were randomly assigned to perform a binary choice task in which rewards were either accompanied (cued, n = 63) or unaccompanied (uncued, n = 68) by money images and casino jingles. In Study 2, participants (n = 58) performed cued and uncued versions of the task in a within-subjects design. Across both studies, cues promoted riskier choice. Sign-tracking was not associated with risky choice in either study. Goal-tracking rather than sign-tracking was significantly associated with greater risk-promoting effects of cues in Study 1 but not in Study 2, although the direction of findings was consistent across both studies. These findings are at odds with the notion of sign-trackers being preferentially susceptible to the influence of reward cues on behavior and point to the role of mechanisms besides incentive salience in mediating such influences.

Statistical power and false positive rates for interdependent outcomes are strongly influenced by test type: Implications for behavioral neuroscience

Statistical errors in preclinical science are a barrier to reproducibility and translation. For instance, linear models (e.g., ANOVA, linear regression) may be misapplied to data that violate assumptions. In behavioral neuroscience and psychopharmacology, linear models are frequently applied to interdependent or compositional data, which includes behavioral assessments where animals concurrently choose between chambers, objects, outcomes, or types of behavior (e.g., forced swim, novel object, place/social preference). The current study simulated behavioral data for a task with four interdependent choices (i.e., increased choice of a given outcome decreases others) using Monte Carlo methods. 16,000 datasets were simulated (1000 each of 4 effect sizes by 4 sample sizes) and statistical approaches evaluated for accuracy. Linear regression and linear mixed effects regression (LMER) with a single random intercept resulted in high false positives (>60%). Elevated false positives were attenuated in an LMER with random effects for all choice-levels and a binomial logistic mixed effects regression. However, these models were underpowered to reliably detect effects at common preclinical sample sizes. A Bayesian method using prior knowledge for control subjects increased power by up to 30%. These results were confirmed in a second simulation (8000 datasets). These data suggest that statistical analyses may often be misapplied in preclinical paradigms, with common linear methods increasing false positives, but potential alternatives lacking power. Ultimately, using informed priors may balance statistical requirements with ethical imperatives to minimize the number of animals used. These findings highlight the importance of considering statistical assumptions and limitations when designing research studies.

Computational approaches to modeling gambling behaviour: Opportunities for understanding disordered gambling

Computational modeling has become an important tool in neuroscience and psychiatry research to provide insight into the cognitive processes underlying normal and pathological behavior. There are two modeling frameworks, reinforcement learning (RL) and drift diffusion modeling (DDM), that are well-developed in cognitive science, and have begun to be applied to Gambling Disorder. RL models focus on explaining how an agent uses reward to learn about the environment and make decisions based on outcomes. The DDM is a binary choice framework that breaks down decision making into psychologically meaningful components based on choice reaction time analyses. Both approaches have begun to yield insight into aspects of cognition that are important for, but not unique to, gambling, and thus relevant to the development of Gambling Disorder. However, these approaches also oversimplify or neglect various aspects of decision making seen in real-world gambling behavior. Gambling Disorder presents an opportunity for 'bespoke' modeling approaches to consider these neglected components. In this review, we discuss studies that have used RL and DDM frameworks to investigate some of the key cognitive components in gambling and Gambling Disorder. We also include an overview of Bayesian models, a methodology that could be useful for more tailored modeling approaches. We highlight areas in which computational modeling could enable progression in the investigation of the cognitive mechanisms relevant to gambling.

D2/3 Agonist during Learning Potentiates Cued Risky Choice

Impulse control and/or gambling disorders can be triggered by dopamine agonist therapies used to treat Parkinson's disease, but the cognitive and neurobiological mechanisms underlying these adverse effects are unknown. Recent data show that adding win-paired sound and light cues to the rat gambling task (rGT) potentiates risky decision-making and impulsivity via the dopamine system, and that changing dopaminergic tone has a greater influence on behavior while subjects are learning task contingencies. Dopamine agonist therapy may therefore be potentiating risk-taking by amplifying the behavioral impact of gambling-related cues on novel behavior. Here, we show that ropinirole treatment in male rats transiently increased motor impulsivity but robustly and progressively increased choice of the high-risk/high-reward options when administered during acquisition of the cued but not uncued rGT. Early in training, ropinirole increased win-stay behavior after large unlikely wins on the cued rGT, indicative of enhanced model-free learning, which mediated the drug's effect on later risk preference. Ex vivo cFos imaging showed that both chronic ropinirole and the addition of win-paired cues suppressed the activity of dopaminergic midbrain neurons. The ratio of midbrain:prefrontal cFos+ neurons was lower in animals with suboptimal choice patterns and tended to predict risk preference across all rats. Network analyses further suggested that ropinirole induced decoupling of the dopaminergic cells of the VTA and nucleus accumbens but only when win-paired cues were present. Frontostriatal activity uninformed by the endogenous dopaminergic teaching signal therefore appeared to perpetuate risky choice, and ropinirole exaggerated this disconnect in synergy with reward-paired cues.SIGNIFICANCE STATEMENT D2/3 receptor agonists, used to treat Parkinson's disease, can cause gambling disorder through an unknown mechanism. Ropinirole increased risky decision-making in rats, but only when wins were paired with casino-inspired sounds and lights. This was mediated by increased win-stay behavior after large unlikely wins early in learning, indicating enhanced model-free learning. cFos imaging showed that ropinirole suppressed activity of midbrain dopamine neurons, an effect that was mimicked by the addition of win-paired cues. The degree of risky choice rats exhibited was uniquely predicted by the ratio of midbrain dopamine:PFC activity. Depriving the PFC of the endogenous dopaminergic teaching signal may therefore drive risky decision-making on-task, and ropinirole acts synergistically with win-paired cues to amplify this.

Long-Term Outcomes of Adolescent THC Exposure on Translational Cognitive Measures in Adulthood in an Animal Model and Computational Assessment of Human Data

Importance

Although perceived as relatively harmless and nonaddictive, adolescent cannabis use significantly increases the likelihood of developing cannabis use disorder in adulthood, especially for high-potency cannabis. Risky decision-making is associated with chronic cannabis use, but given confounds of human studies, it remains unclear whether adolescent cannabis exposure and Δ9-tetrahydrocannabinol (THC) potency specifically predicts risky decision-making or influences cognitive response to the drug later in life.

Objective

To leverage a human data set of cannabis users and a rat model to evaluate the long-term outcomes of adolescent THC exposure on adult decision-making and impulse control.

Design, Setting, and Participants

This translational rat study tested the link between adolescent THC exposure and adulthood decision-making. A reanalysis of a previously published dataset of human chronic cannabis users was conducted to evaluate decision-making phenotypes. Computational modeling assessed the human and animal results in a single framework. Data were collected from 2017 to 2020 and analyzed from 2020 to 2022.

Main Outcomes and Measures

Decision-making was measured by the Iowa Gambling Task (IGT) and Rat Gambling Task (rGT). Impulse control was assessed in the rat model. Computational modeling was used to determine reward and punishment learning rates and learning strategy used by cannabis users and THC-exposed rats. Cell-specific molecular measures were conducted in the prefrontal cortex and amygdala.

Results

Of 37 participants, 24 (65%) were male, and the mean (SD) age was 33.0 (8.3) years. Chronic cannabis users (n = 22; mean [SE] IGT score, -5.182 [1.262]) showed disadvantageous decision-making compared with controls (n = 15; mean [SE] IGT score, 7.133 [2.687]; Cohen d = 1.436). Risky choice was associated with increased reward learning (mean [SE] IGT score: cannabis user, 0.170 [0.018]; control, 0.046 [0.008]; Cohen d = 1.895) and a strategy favoring exploration vs long-term gains (mean [SE] IGT score: cannabis user, 0.088 [0.012]; control, 0.020 [0.002]; Cohen d = 2.218). Rats exposed to high-dose THC but not low-dose THC during adolescence also showed increased risky decision-making (mean [SE] rGT score: vehicle, 46.17 [7.02]; low-dose THC, 69.45 [6.01]; high-dose THC, 21.97 [11.98]; Cohen d = 0.433) and elevated reward learning rates (mean [SE] rGT score: vehicle, 0.17 [0.01]; low-dose THC, 0.10 [0.01]; high-dose THC, 0.24 [0.06]; Cohen d = 1.541) during task acquisition. These animals were also uniquely susceptible to increased cognitive impairments after reexposure to THC in adulthood, which was correlated with even greater reward learning (r = -0.525; P < .001) and a shift in strategy (r = 0.502; P < .001), similar to results seen in human cannabis users. Molecular studies revealed that adolescent THC dose differentially affected cannabinoid-1 receptor messenger RNA expression in the prelimbic cortex and basolateral amygdala in a layer- and cell-specific manner. Further, astrocyte glial fibrillary acidic protein messenger RNA expression associated with cognitive deficits apparent with adult THC reexposure.

Conclusions and Relevance

In this translational study, high-dose adolescent THC exposure was associated with cognitive vulnerability in adulthood, especially with THC re-exposure. These data also suggest a link between astrocytes and cognition that altogether provides important insights regarding the neurobiological genesis of risky cannabis use that may help promote prevention and treatment efforts.

Large-N Rat Data Enables Phenotyping of Risky Decision-Making: A Retrospective Analysis of Brain Injury on the Rodent Gambling Task

Decision-making is substantially altered after brain injuries. Patients and rats with brain injury are more likely to make suboptimal, and sometimes risky choices. Such changes in decision-making may arise from alterations in how sensitive individuals are to outcomes. To assess this, we compiled and harmonized a large dataset from four studies of TBI, each of which evaluated behavior on the Rodent Gambling Task (RGT). We then determined whether the following were altered: (1) sensitivity to overall contingencies, (2) sensitivity to immediate outcomes, or (3) general choice phenotypes. Overall sensitivity was evaluated using the matching law, immediate sensitivity by looking at the probability of switching choices given a win or loss, and choice phenotypes by k-means clustering. We found significant reductions in sensitivity to the overall outcomes and a bias toward riskier alternatives in TBI rats. However, the substantial individual variability led to poor overall fits in matching analyses. We also found that TBI caused a significant reduction in the tendency to repeatedly choose a given option, but no difference in win- or loss-specific sensitivity. Finally, clustering revealed 5 distinct decision-making phenotypes and TBI reduced membership in the “optimal” type. The current findings support a hypothesis that TBI reduces sensitivity to contingencies. However, in the case of tasks such as the RGT, this is not a simple shift to indiscriminate or less discriminate responding. Rather, TBI rats are more likely to develop suboptimal preferences and frequently switch choices. Treatments will have to consider how this behavior might be corrected.

Serotonin 2C Antagonism in the Lateral Orbitofrontal Cortex Ameliorates Cue-Enhanced Risk Preference and Restores Sensitivity to Reinforcer Devaluation in Male Rats

Previous research has indicated that reward-paired cues can enhance disadvantageous risky choice in both humans and rodents. Systemic administration of a serotonin 2C receptor antagonist can attenuate this cue-induced risk preference in rats. However, the neurocognitive mechanisms mediating this effect are currently unknown. We therefore assessed whether the serotonin 2C receptor antagonist RS 102221 is able to attenuate cue-enhanced risk preference via its actions in the lateral orbitofrontal cortex (lOFC) or prelimbic (PrL) area of the medial prefrontal cortex (mPFC). A total of 32 male Long–Evans rats were trained on the cued version of the rat gambling task (rGT), a rodent analog of the human Iowa gambling task, and bilateral guide cannulae were implanted into the lOFC or PrL. Intra-lOFC infusions of the 5-HT_2C antagonist RS 102221 reduced risky choice in animals that showed a preference for the risky options of the rGT at baseline. This effect was not observed in optimal decision-makers, nor those that received infusions targeting the PrL. Given prior data showing that 5-HT_2C antagonists also improve reversal learning through the same neural locus, we hypothesized that reward-concurrent cues may amplify risky decision-making through cognitive inflexibility. We therefore devalued the sugar pellet rewards used in the cued rGT (crGT) through satiation and observed that decision-making patterns did not shift unless animals also received intra-lOFC RS 102221. Collectively, these data suggest that the lOFC is one critical site through which reward-concurrent cues promote risky choice patterns that are insensitive to reinforcer devaluation, and that 5-HT_2C antagonism may optimize choice by facilitating exploration.

Dopamine neurons gate the intersection of cocaine use, decision making, and impulsivity

Gambling and substance use disorders are highly comorbid. Both clinical populations are impulsive and exhibit risky decision-making. Drug-associated cues have long been known to facilitate habitual drug-seeking, and the salient audiovisual cues embedded within modern gambling products may likewise encourage problem gambling. The dopamine neurons of the ventral tegmental area (VTA) are exquisitely sensitive to drugs of abuse, uncertain rewards, and reward-paired cues and may therefore be the common neural substrate mediating synergistic features of both disorders. To test this hypothesis, we first gained specific inhibitory control over VTA dopamine neurons by transducing a floxed inhibitory DREADD (AAV5-hSyn-DIO-hM4D(Gi)-mCherry) in rats expressing Cre recombinase in tyrosine hydroxylase neurons. We then trained rats in our cued rat gambling task (crGT), inhibiting dopamine neurons throughout task acquisition and performance, before allowing them to self-administer cocaine in the same diurnal period as crGT sessions. The trajectories of addiction differ in women and men, and the dopamine system may differ functionally across the sexes; therefore, we used male and female rats here. We found that inhibition of VTA dopamine neurons decreased cue-induced risky choice and reduced motor impulsivity in males, but surprisingly, enhanced risky decision making in females. Inhibiting VTA dopamine neurons also prevented cocaine-induced changes in decision making in both sexes, but nevertheless drove all animals to consume more cocaine. These findings show that chronic dampening of dopamine signalling can have both protective and deleterious effects on addiction-relevant behaviours, depending on biological sex and dependent variable of interest.

Sign tracking predicts suboptimal behavior in a rodent gambling task

RATIONALE

Reward-associated cues can promote maladaptive behavior, including risky decision-making in a gambling setting. A propensity for sign tracking over goal tracking-i.e., interaction with a reward-predictive cue rather than the site of reward-demonstrates an individual's tendency to transfer motivational value to a cue. However, the relationship of sign tracking to risky decision-making remains unclear.

OBJECTIVES

To determine whether sign tracking predicts risky choice, we used a Pavlovian conditioned approach task to evaluate the tendency of male rats to sign track to a lever cue and then trained rats on a rodent gambling task (rGT) with win-associated cues. We also tested the effects of D-amphetamine, quinpirole (a D₂/D₃ receptor agonist), and PD128907 (a D₃ receptor agonist) on gambling behavior in sign tracker and goal tracker individuals.

RESULTS

Increased sign tracking relative to goal tracking was associated with suboptimal performance on the rGT, including decreased selection of the optimal choice, increased selection of a high-risk/high-reward option, and increased impulsive premature choices. Amphetamine increased choices of a low-risk/low-reward option at the expense of optimal and high-risk choices, whereas quinpirole and PD128907 had little effect on choice allocation, but reduced impulsivity. Drug effects were similar across sign tracker and goal tracker individuals.

CONCLUSIONS

Cue reactivity, as measured by sign tracking, is predictive and may be an important driver of risky and impulsive choices in a gambling setting laden with salient audiovisual cues. Evaluating an individual's sign tracking behavior may be an avenue to predict vulnerability to pathological gambling and the efficacy of treatments.

Pharmacological evidence of a cholinergic contribution to elevated impulsivity and risky decision-making caused by adding win-paired cues to a rat gambling task

BACKGROUND

Pairing rewards with sensory stimulation, in the form of auditory and visual cues, increases risky decision-making in both rats and humans. Understanding the neurobiological basis of this effect could help explain why electronic gambling machines are so addictive, and inform treatment development for compulsive gambling and gaming. Numerous studies implicate the dopamine system in mediating the motivational influence of reward-paired cues; recent data suggest the cholinergic system also plays a critical role. Previous work also indicates that cholinergic drugs alter decision-making under uncertainty.

AIMS

We investigated whether the addition of reward-concurrent cues to the rat gambling task (crGT) altered the effects of peripherally administered cholinergic compounds.

METHODS

Muscarinic and nicotinic agonists and antagonists were administered to 16 male, Long-Evans rats trained on the crGT. Measures of optimal/risky decision-making and motor impulsivity were the main dependent variables of interest.

RESULTS

The muscarinic receptor antagonist scopolamine improved decision-making overall, decreasing selection of one of the risky options while increasing choice of the more advantageous options. The muscarinic agonist oxotremorine increased choice latency but did not significantly affect option preference. Neither the nicotinic antagonist mecamylamine nor the agonist nicotine affected choice patterns, but mecamylamine decreased premature responding, an index of motor impulsivity.

CONCLUSIONS

These results contrast sharply from those obtained previously using the uncued rGT, and suggest that the deleterious effects of win-paired cues on decision-making and impulse control may result from elevated cholinergic tone.

A reinforcement-learning model of active avoidance behavior: Differences between Sprague Dawley and Wistar-Kyoto rats

Avoidance behavior is a typically adaptive response performed by an organism to avert harmful situations. Individuals differ remarkably in their tendency to acquire and perform new avoidance behaviors, as seen in anxiety disorders where avoidance becomes pervasive and inappropriate. In rodent models of avoidance, the inbred Wistar-Kyoto (WKY) rat demonstrates increased learning and expression of avoidance compared to the outbred Sprague Dawley (SD) rat. However, underlying mechanisms that contribute to these differences are unclear. Computational modeling techniques can help identify factors that may not be easily decipherable from behavioral data alone. Here, we utilize a reinforcement learning (RL) model approach to better understand strain differences in avoidance behavior. An actor-critic model, with separate learning rates for action selection (in the actor) and state evaluation (in the critic), was applied to individual data of avoidance acquisition from a large cohort of WKY and SD rats. Latent parameters were extracted, such as learning rate and subjective reinforcement value of foot shock, that were then compared across groups. The RL model was able to accurately represent WKY and SD avoidance behavior, demonstrating that the model could simulate individual performance. The model determined that the perceived negative value of foot shock was significantly higher in WKY than SD rats, whereas learning rate in the actor was lower in WKY than SD rats. These findings demonstrate the utility of computational modeling in identifying underlying processes that could promote strain differences in behavioral performance.

Probing relationships between reinforcement learning and simple behavioral strategies to understand probabilistic reward learning

BACKGROUND

Reinforcement learning (RL) and win stay/lose shift model accounts of decision making are both widely used to describe how individuals learn about and interact with rewarding environments. Though mutually informative, these accounts are often conceptualized as independent processes and so the potential relationships between win stay/lose shift tendencies and RL parameters have not been explored.

NEW METHOD

We introduce a methodology to directly relate RL parameters to behavioral strategy. Specifically, by calculating a truncated multivariate normal distribution of RL parameters given win stay/lose shift tendencies from simulating these tendencies across the parameter space, we maximize the normal distribution for a given set of win stay/lose shift tendencies to approximate reinforcement learning parameters.

RESULTS

We demonstrate novel relationships between win stay/lose shift tendencies and RL parameters that challenge conventional interpretations of lose shift as a metric of loss sensitivity. Further, we demonstrate in both simulated and empirical data that this method of parameter approximation yields reliable parameter recovery.

COMPARISON WITH EXISTING METHOD

We compare this method against the conventionally used maximum likelihood estimation method for parameter approximation in simulated noisy and empirical data. For simulated noisy data, we show that this method performs similarly to maximum likelihood estimation. For empirical data, however, this method provides a more reliable approximation of reinforcement learning parameters than maximum likelihood estimation.

CONCLUSIONS

We demonstrate the existence of relationships between win stay/lose shift tendencies and RL parameters and introduce a method that leverages these relationships to enable recovery of RL parameters exclusively from win stay/lose shift tendencies.

Translating concepts of risk and loss in rodent models of gambling and the limitations for clinical applications

Gambling involves placing something of value at risk in exchange for the opportunity to potentially gain something of greater value in return. A variety of gambling paradigms have been designed to study the maladaptive decision-making that underlies problematic gambling. Central to these gambling models are the definitions of "risk" and "loss", especially when translating the results from rodent studies to clinical applications. Risk and loss are not mutually exclusive but rather share some overlap. With careful interpretation and consideration of the limitations of these behavioral paradigms, results from rodent models may provide insights into the neurobiology of risky decision-making that leads to problematic gambling in humans.

Understanding Addiction Using Animal Models

Drug addiction is a neuropsychiatric disorder with grave personal consequences that has an extraordinary global economic impact. Despite decades of research, the options available to treat addiction are often ineffective because our rudimentary understanding of drug-induced pathology in brain circuits and synaptic physiology inhibits the rational design of successful therapies. This understanding will arise first from animal models of addiction where experimentation at the level of circuits and molecular biology is possible. We will review the most common preclinical models of addictive behavior and discuss the advantages and disadvantages of each. This includes non-contingent models in which animals are passively exposed to rewarding substances, as well as widely used contingent models such as drug self-administration and relapse. For the latter, we elaborate on the different ways of mimicking craving and relapse, which include using acute stress, drug administration or exposure to cues and contexts previously paired with drug self-administration. We further describe paradigms where drug-taking is challenged by alternative rewards, such as appetitive foods or social interaction. In an attempt to better model the individual vulnerability to drug abuse that characterizes human addiction, the field has also established preclinical paradigms in which drug-induced behaviors are ranked by various criteria of drug use in the presence of negative consequences. Separation of more vulnerable animals according to these criteria, along with other innate predispositions including goal- or sign-tracking, sensation-seeking behavior or impulsivity, has established individual genetic susceptibilities to developing drug addiction and relapse vulnerability. We further examine current models of behavioral addictions such as gambling, a disorder included in the DSM-5, and exercise, mentioned in the DSM-5 but not included yet due to insufficient peer-reviewed evidence. Finally, after reviewing the face validity of the aforementioned models, we consider the most common standardized tests used by pharmaceutical companies to assess the addictive potential of a drug during clinical trials.