Increased Goal Tracking in Adolescent Rats Is Goal-Directed and Not Habit-Like

Fractionating impulsivity and reward-related phenotypes in adolescent mice

Adolescence is a developmental period characterized by changes in the brain and behavior, including heightened reward seeking, increased impulsivity, and elevated risk-taking behavior. It is also a sensitive period for the development of a number of behavioral and psychiatric disorders associated with pathological phenotypes of reward processing and impulsivity. Landmark human studies are charting the development of impulsivity and other reward-related phenotypes to identify the facets and timecourse of the adolescent phenotype. Collecting similar data from mice is important to enable molecular, cellular, and circuit-level interrogation of adolescent maturation of reward, motivation, and impulsive behavior. These complex phenotypes have traditionally been difficult to assay in adolescent mice. Here, using a combination of approaches including homecage testing, we tested a number of facets of reward seeking, impulsivity, motivation, and incentive salience attribution during adolescent development. We found that adolescent mice show increased reward seeking, impulsive action, and motivation. Interestingly, we found no effect of adolescence on impulsive choice, sign-tracking, reward-learning, or conditioned reinforcement. Overall, our studies set the stage for approaches to study multi-faceted phenotypes related to impulsivity and other reward-related behaviors in adolescent mice to examine the developmental trajectories of brain and behavior.

Maturation of striatal dopamine supports the development of habitual behavior through adolescence

Developmental trajectories during the transition from adolescence to adulthood contribute to the establishment of stable, adult forms of operation. Understanding the neural mechanisms underlying this transition is crucial for identifying variability in normal development and the onset of psychiatric disorders, which typically emerge during this time. Habitual behaviors can serve as a model for understanding brain mechanisms underlying the stabilization of adult behavior, while also conferring risk for psychopathologies. Dopaminergic (DA) processes in the basal ganglia are thought to facilitate the formation of habits; however, developmental trajectories of habits and the brain systems supporting them have not been characterized in vivo in developing humans. The current study examined trajectories of habitual behavior from adolescence to adulthood and sought to understand how the maturing striatal DA system may act as a potential mechanism in the process of habit formation. We used data from two longitudinal studies (combined n = 217, 10 - 32 years of age, 1-3 visits each, 320 total sessions) to characterize normative developmental trajectories of basal ganglia tissue iron concentration (a proxy for DA-related neurophysiology) and goal-direct and habitual control behaviors in a two-stage decision-making task. Tissue iron concentrations across the basal ganglia and habitual responding during the two-stage sequential decision-making task both increased with age (all p < 0.001). Importantly, habitual responding was associated with tissue iron concentrations in the putamen (F = 4.34, p = 0.014), such that increases in habitual responding were supported by increases in putamen tissue iron concentration during childhood through late adolescence. Exploratory analyses of further subdivisions of anatomical regions found that this association was specific to the posterior putamen. These results provide novel evidence in humans that habitual behavior continues to mature into adulthood and may be supported by increased specialization of reward systems.

Propensity for risky choices despite lower cue reactivity in adolescent rats

Adolescence is a time of heightened risk-taking across species. Salient audiovisual cues associated with rewards are a common feature of gambling environments and have been connected to increased risky decision-making. We have previously shown that, in adult male rats, sign tracking - a behavioral measure of cue reactivity - predicts an individual's propensity for suboptimal risky choices in a rodent gambling task (rGT) with win-paired cues. However, adolescents perform less sign tracking than adult animals, suggesting that they are less cue-reactive than adults in some circumstances. Therefore, we investigated the performance of adolescent male rats on the rGT with win cues and examined its relationship with their sign-tracking behavior. We found that adolescents make more risky choices and fewer optimal choices on the rGT compared with adults, evidence of the validity of the rGT as a model of adolescent gambling behavior. We also confirmed that adolescents perform less sign tracking than adults, and we found that, unlike in adults, adolescents' sign tracking was unrelated to their risk-taking in the rGT. This implies that adolescent risk-taking is less likely than that of adults to be driven by reward-related cues. Finally, we found that adults trained on the rGT as adolescents retained an adolescent-like propensity toward risky choices, suggesting that early exposure to a gambling environment may have a long-lasting impact on risk-taking behavior.

Reward-Mediated, Model-Free Reinforcement-Learning Mechanisms in Pavlovian and Instrumental Tasks Are Related

Model-free and model-based computations are argued to distinctly update action values that guide decision-making processes. It is not known, however, if these model-free and model-based reinforcement learning mechanisms recruited in operationally based instrumental tasks parallel those engaged by pavlovian-based behavioral procedures. Recently, computational work has suggested that individual differences in the attribution of incentive salience to reward predictive cues, that is, sign- and goal-tracking behaviors, are also governed by variations in model-free and model-based value representations that guide behavior. Moreover, it is not appreciated if these systems that are characterized computationally using model-free and model-based algorithms are conserved across tasks for individual animals. In the current study, we used a within-subject design to assess sign-tracking and goal-tracking behaviors using a pavlovian conditioned approach task and then characterized behavior using an instrumental multistage decision-making (MSDM) task in male rats. We hypothesized that both pavlovian and instrumental learning processes may be driven by common reinforcement-learning mechanisms. Our data confirm that sign-tracking behavior was associated with greater reward-mediated, model-free reinforcement learning and that it was also linked to model-free reinforcement learning in the MSDM task. Computational analyses revealed that pavlovian model-free updating was correlated with model-free reinforcement learning in the MSDM task. These data provide key insights into the computational mechanisms mediating associative learning that could have important implications for normal and abnormal states.SIGNIFICANCE STATEMENT Model-free and model-based computations that guide instrumental decision-making processes may also be recruited in pavlovian-based behavioral procedures. Here, we used a within-subject design to test the hypothesis that both pavlovian and instrumental learning processes were driven by common reinforcement-learning mechanisms. Sign-tracking and goal-tracking behaviors were assessed in rats using a pavlovian conditioned approach task, and then instrumental behavior was characterized using an MSDM task. We report that sign-tracking behavior was associated with greater model-free, but not model-based, learning in the MSDM task. These data suggest that pavlovian and instrumental behaviors may be driven by conserved reinforcement-learning mechanisms.

Quantifying the instrumental and noninstrumental underpinnings of Pavlovian responding with the Price equation

The Price equation is a mathematical expression of selectionist and non-selectionist pressures on biological, cultural, and behavioral change. We use it here to specify instrumental and noninstrumental behaviors as they arise within the context of the Pavlovian autoshaping procedure, for rats trained under reward certainty and reward uncertainty. The point of departure for this endeavor is that some portion of autoshaped behavior referred to as goal-tracking appears instrumental-a function of resource attainment (the individual approaches the location where the unconditioned stimulus is to be delivered). By contrast, some other portion of autoshaped behavior referred to as sign-tracking is noninstrumental-irrelevant to making contact with the to-be-delivered unconditioned stimulus. A Price equation model is proposed that unifies our understanding of Pavlovian autoshaping behavior by isolating operant and respondent influences on goal-tracking (instrumental) and sign-tracking (noninstrumental) behavior.

Sign tracking predicts suboptimal behavior in a rodent gambling task

RATIONALE

Reward-associated cues can promote maladaptive behavior, including risky decision-making in a gambling setting. A propensity for sign tracking over goal tracking-i.e., interaction with a reward-predictive cue rather than the site of reward-demonstrates an individual's tendency to transfer motivational value to a cue. However, the relationship of sign tracking to risky decision-making remains unclear.

OBJECTIVES

To determine whether sign tracking predicts risky choice, we used a Pavlovian conditioned approach task to evaluate the tendency of male rats to sign track to a lever cue and then trained rats on a rodent gambling task (rGT) with win-associated cues. We also tested the effects of D-amphetamine, quinpirole (a D₂/D₃ receptor agonist), and PD128907 (a D₃ receptor agonist) on gambling behavior in sign tracker and goal tracker individuals.

RESULTS

Increased sign tracking relative to goal tracking was associated with suboptimal performance on the rGT, including decreased selection of the optimal choice, increased selection of a high-risk/high-reward option, and increased impulsive premature choices. Amphetamine increased choices of a low-risk/low-reward option at the expense of optimal and high-risk choices, whereas quinpirole and PD128907 had little effect on choice allocation, but reduced impulsivity. Drug effects were similar across sign tracker and goal tracker individuals.

CONCLUSIONS

Cue reactivity, as measured by sign tracking, is predictive and may be an important driver of risky and impulsive choices in a gambling setting laden with salient audiovisual cues. Evaluating an individual's sign tracking behavior may be an avenue to predict vulnerability to pathological gambling and the efficacy of treatments.

Incubation of Oxycodone Craving Following Adult-Onset and Adolescent-Onset Oxycodone Self-Administration in Male Rats

Relapse is a major obstacle to curb the ongoing epidemic of prescription opioid abuse. We and others previously demonstrated that oxycodone seeking in adult rats progressively increases after abstinence from oxycodone self-administration (incubation of oxycodone craving). In humans, the onset of oxycodone use in adolescents may increase individuals' vulnerability to later opioid addiction. However, little is known about incubation of oxycodone craving after adolescent-onset oxycodone self-administration in rats. In the first study, we trained single-housed adolescent (postnatal day 35 at start) and adult (postnatal day 77 at start) male Sprague-Dawley rats to self-administer oxycodone (0.1 mg/kg/infusion, 6 h/day for 10 days) and then tested oxycodone relapse on both abstinence day 1 and day 15. Given that social experience is critical for neurobehavioral development in adolescents, we performed the second study using group-housed adolescent and adult rats. In both studies, we observed no age differences in oxycodone self-administration and incubated oxycodone seeking on abstinence day 15. However, on abstinence day 1, we observed decreased oxycodone seeking in adolescents compared with adults. This pattern of data led to elevated incubation slopes in adolescent rats compared with adult rats. Finally, group-housed rats exhibited attenuated oxycodone seeking compared with single-housed rats on abstinence day 15, but not on day 1. Taken together, these data suggest that adolescents may be resistant to oxycodone relapse during early abstinence, but this resistance dissipates quickly during the transition between adolescent and young adulthood. In addition, group-housing plays a protective role against incubated oxycodone craving.

Sign Tracking in an Enriched Environment: A Potential Ecologically Relevant Animal Model of Adaptive Behavior Change

When an object conditioned stimulus (CS) is paired with a food unconditioned stimulus (US), anticipatory goal-directed action directed at the US location (goal tracking) is accompanied by behavior directed at the object CS (sign tracking). Sign-tracking behavior appears to be compulsive and habit-like and predicts increased vulnerability to the addictive potential of drugs in animal models. A large body of the literature also suggests that environmental enrichment protects against the development of addiction-prone phenotypes. Thus, we investigated whether compulsive-like sign tracking develops in environmentally enriched rats trained directly in their enriched home environment. We demonstrate that adolescent enriched-housed male Sprague-Dawley rats readily sign track a 5% ethanol bottle CS in their home environment and at a rate higher than adolescent standard-housed rats. We also show that enriched adolescent rats sign track less than enriched adult-trained rats and that acute isolation stress affects sign- and goal-tracking performance of adolescents and adults differently. Sign tracking increased more in the adult than the adolescent rats. Whereas the younger rats showed a decrease in goal tacking after the final stressor manipulation, the adults showed increased goal tracking. Our results are consistent with recent studies, which suggest that although sign tracking performance is compulsive-like, it is not as inflexible and habit-like as previously assumed. Testing in an enriched home environment with object CSs having greater affordance than "neutral" lever CSs may provide greater ecological relevance for investigating the development and expression of adaptive and compulsive-like behaviors in translational research.

Does learning history shape the associability of outcomes? Further tests of the outcome predictability effect

In recent years, several studies of human predictive learning demonstrated better learning about outcomes that have previously been experienced as consistently predictable compared to outcomes previously experienced as less predictable, namely the outcome predictability effect. As this effect may have wide-reaching implications for current theories of associative learning, the present study aimed to examine the generality of the effect with a human goal-tracking paradigm, employing three different designs to manipulate the predictability of outcomes in an initial training phase. In contrast to the previous studies, learning in a subsequent phase, when every outcome was equally predictable by novel cues, was not reliably affected by the outcomes' predictability in the first phase. This lack of an outcome predictability effect provides insights into the parameters of the effect and its underlying mechanisms.

Basolateral Amygdala to Nucleus Accumbens Communication Differentially Mediates Devaluation Sensitivity of Sign- and Goal-Tracking Rats

Rats rely on communication between the basolateral amygdala (BLA) and nucleus accumbens (NAc) to express lever directed approach in a Pavlovian lever autoshaping (PLA) task that distinguishes sign- and goal-tracking rats. During PLA, sign-tracking rats preferentially approach an insertable lever cue, while goal-tracking rats approach a foodcup where rewards are delivered. While sign-tracking rats inflexibly respond to cues even after the associated reward is devalued, goal-tracking rats flexibly reduce responding to cues during outcome devaluation. Here, we sought to determine whether BLA-NAc communication, which is necessary for sign, but not goal-tracking, drives a rigid appetitive approach of sign-tracking rats that are insensitive to manipulations of outcome value. Using a contralateral chemogenetic inactivation design, we injected contralateral BLA and NAc core with inhibitory DREADD (hm4Di-mCherry) or control (mCherry) constructs. To determine sign- and goal-tracking groups, we trained rats in five PLA sessions in which brief lever insertion predicts food pellet delivery. We sated rats on training pellets (devalued condition) or chow (valued condition) before systemic clozapine injections (0.1 mg/kg) to inactivate BLA and contralateral NAc during two outcome devaluation probe tests, in which we measured lever and foodcup approach. Contralateral BLA-NAc chemogenetic inactivation promoted a flexible lever approach in sign-tracking rats but disrupted the flexible foodcup approach in goal-tracking rats. Consistent with a prior BLA-NAc disconnection lesion study, we find contralateral chemogenetic inactivation of BLA and NAc core reduces lever, but not the foodcup approach in PLA. Together these findings suggest rigid appetitive associative encoding in BLA-NAc of sign-tracking rats hinders the expression of flexible behavior when outcome value changes.

Reward-predictive cues elicit excessive reward seeking in adolescent rats

Impulsive behavior during adolescence may stem from developmental imbalances between motivational and cognitive-control systems, producing greater urges to pursue reward and weakened capacities to inhibit such actions. Here, we developed a Pavlovian-instrumental transfer (PIT) protocol to assay rats' ability to suppress cue-motivated reward seeking based on changes in reward expectancy. Traditionally, PIT studies focus on how reward-predictive cues motivate instrumental reward-seeking behavior (lever pressing). However, cues signaling imminent reward delivery also elicit countervailing focal-search responses (food-port entry). We first examined how reward expectancy (cue-reward probability) influences expression of these competing behaviors. Adult male rats increased rates of lever pressing when presented with cues signaling lower probabilities of reward but focused their activity at the food cup on trials with cues that signaled higher probabilities of reward. We then compared adolescent and adult male rats in their responsivity to cues signaling different reward probabilities. In contrast to adults, adolescent rats did not flexibly adjust patterns of responding based on the expected likelihood of reward delivery but increased their rate of lever pressing for both weak and strong cues. These findings indicate that control over cue-motivated behavior is fundamentally dysregulated during adolescence, providing a model for studying neurobiological mechanisms of adolescent impulsivity.