Effects of orbitofrontal cortex lesions on autoshaped lever pressing and reversal learning

Nucleus accumbens core acetylcholine receptors modulate the balance of flexible and inflexible cue-directed motivation

Sign-tracking is a conditioned response where animals interact with reward-predictive cues due to the cues having motivational value, or incentive salience. The nucleus accumbens core (NAc) has been implicated in mediating the sign-tracking response. Additionally, acetylcholine (ACh) transmission throughout the striatum has been attributed to both incentive motivation and behavioral flexibility. Here, we demonstrate a role for NAc ACh receptors in the flexibility of sign-tracking. Sign-tracking animals were exposed to an omission contingency, in which vigorous sign-tracking was punished by reward omission. Animals rapidly adjusted their behavior, but they maintained sign-tracking in a less vigorous manner that did not cancel reward. Within this context of sign-tracking being persistent yet flexible in structure, blockade of NAc nicotinic receptors (nAChRs) led to a persistence in the initial sign-tracking response during omission followed by a period of change in the makeup of sign-tracking, whereas blockade of muscarinic receptors (mAChRs) oppositely enhanced the omission-related development of the new sign-tracking behaviors. Later, once omission learning had occurred, nAChR blockade uniquely led to reduced sign-tracking and elevated reward-directed behaviors instead. These results indicate that NAc ACh receptors have opposing roles in maintaining learned patterns of sign-tracking, with nAChRs having a special involvement in regulating the structure of the sign-tracking response.

Basolateral amygdala and orbitofrontal cortex, but not dorsal hippocampus, are necessary for the control of reward-seeking by occasion setters

Reward-seeking in the world is driven by cues that can have ambiguous predictive and motivational value. To produce adaptive, flexible reward-seeking, it is necessary to exploit occasion setters, other distinct features in the environment, to resolve the ambiguity of Pavlovian reward-paired cues. Despite this, very little research has investigated the neurobiological underpinnings of occasion setting, and as a result little is known about which brain regions are critical for occasion setting. To address this, we exploited a recently developed task that was amenable to neurobiological inquiry where a conditioned stimulus is only predictive of reward delivery if preceded in time by the non-overlapping presentation of a separate cue-an occasion setter. This task required male rats to maintain and link cue-triggered expectations across time to produce adaptive reward-seeking. We interrogated the contributions of the basolateral amygdala and orbitofrontal cortex to occasion setting as these regions are thought to be critical for the computation and exploitation of state value, respectively. Reversible inactivation of either structure prior to the occasion-setting task resulted in a profound inability of rats to use the occasion setter to guide reward-seeking. In contrast, inactivation of the dorsal hippocampus, a region fundamental for context-specific responding was without effect nor did inactivation of the basolateral amygdala or orbitofrontal cortex in a standard Pavlovian conditioning preparation affect conditioned responding. We conclude that neural activity within the orbitofrontal cortex and basolateral amygdala circuit is necessary to update and resolve ambiguity in the environment to promote cue-driven reward-seeking.

Reversal learning in young and middle-age neurotypicals: Individual difference reaction time considerations

Reversal learning is frequently used to assess components of executive function that contribute to understanding age-related cognitive differences. Reaction time (RT) is less characterized in the reversal learning literature, perhaps due to the daunting task of analyzing the entire RT distribution, but has been deemed a generally sensitive measure of cognitive aging. The current study extends our prior work to further characterize distributional properties of the reversal RT distribution and to distinguish groups of individuals with fractionated profiles of performance, which may be of clinical importance within the context of cognitive aging. Participant sample included young (n = 43) and community-dwelling, healthy, middle-aged (n = 139) adults. To explore individual differences, recursive partitioning analysis achieved a high classification rate by specifying decision tree rules that split participants into young and middle-aged groups. Mu (μ, efficient RT) was the most successful parameter in distinguishing age groups while sigma ( σ ) and tau ( τ , ex-Gaussian indices of intra-individual variability) revealed more subtle individual differences. Accuracy measures did not contribute to separating the groups, suggesting that fractionated components of RT, as opposed to accuracy, can distinguish differences between young and middle-aged participants.

A distributional and theoretical analysis of reaction time in the reversal task across adulthood

Reversal learning assesses components of executive function important for understanding cognitive changes with age. Extant reversal learning literature has largely assessed measures of accuracy, but reaction time (RT) has not yet been well characterized, perhaps due to the daunting task of analyzing non-normal RT distributions. The current study contributes to the literature by examining distributional and theoretical aspects of the entire RT distribution in addition to accuracy. Participant sample included young (N = 43) and community-dwelling, healthy, middle-aged (N = 139) adults. Results showed a Normal-3 Mixture distribution best fits the sample as a whole, with the ex-Gaussian distribution passing visual inspection. Age related significantly to various measures of RT (p's < 0.5); older age was associated with higher both efficient and overall RT, perhaps due to a more conservative criterion of decision-making. In a generalized adaptive elastic net regression, RT explained age-related differences in performance while accuracy did not contribute. Specifically, middle-aged adults were slower in efficient RT and had increased intra-individual variability which has been previously linked to poorer frontal lobe processes and age-related cognitive decline. Overall, these findings highlight the importance of examining the entire RT distribution and measuring RT as a fractionated construct to further explain age-related differences in reversal learning, even in middle-aged individuals.

Voluntary alcohol access during adolescence/early adulthood, but not during adulthood, causes faster omission contingency learning

In omission contingency training, rodents learn to suppress their natural tendency to approach or touch a reward-predictive cue (termed "autoshaping" or "sign-tracking" responses) if the approach/touching responses lead to the omission of the reward. Previous research has shown that high levels of alcohol exposure (through alcohol vapor exposure) or adolescent alcohol consumption (with some versions of the omission contingency task) can lead to faster omission contingency learning. However, the alcohol exposure procedures and/or omission contingency task parameters differed between these different demonstrations. It was unclear whether the same voluntary alcohol consumption procedures during adolescence/early adulthood and/or adulthood would lead to faster omission contingency learning in one or both age groups. Here, rats received 6 weeks of chronic intermittent access to 20% alcohol or water from PND 26-66 (adolescence/early adulthood in Exp. 1) or PND 68-108 (adulthood in Exp. 2) and began behavioral training (autoshaping training followed by omission contingency training) several days later. We found no evidence that alcohol access at either age altered the number of trials with a sign-tracking response on the levers during autoshaping training. However, alcohol access during adolescence/early adulthood, but not during adulthood, led to faster learning to withhold responding on the lever under omission contingencies during the subsequent phase. We also found no evidence that the level of alcohol consumption was correlated with sign-tracking behavior in the autoshaping phase or with the suppression of lever-pressing during the omission contingency phase. Our results suggest that adolescent/early adult rats have increased vulnerability, compared with adults, to some long-term behavioral effects of voluntary alcohol consumption.

Behavioral flexibility in a mouse model for obsessive-compulsive disorder: Impaired Pavlovian reversal learning in SAPAP3 mutants

Obsessive-compulsive disorder (OCD) is characterized by obsessive thinking, compulsive behavior and anxiety, and is often accompanied by cognitive deficits. The neuropathology of OCD involves dysregulation of cortical-striatal circuits. Similar to OCD patients, SAPAP3 knockout mice 3 (SAPAP3-/- ) exhibit compulsive behavior (grooming), anxiety and dysregulated cortical-striatal function. However, it is unknown whether SAPAP3-/- display cognitive deficits and how these different behavioral traits relate to one another. SAPAP3-/- and wild-type (WT) littermates were trained in a Pavlovian conditioning task pairing visual cues with the delivery of sucrose solution. After mice learned to discriminate between a reward-predicting conditioned stimulus (CS+) and a non-reward stimulus (CS-), contingencies were reversed (CS+ became CS- and vice versa). Additionally, we assessed grooming, anxiety and general activity. SAPAP3-/- acquired Pavlovian approach behavior similarly to WT, albeit less vigorously and with a different strategy. However, unlike WT, SAPAP3-/- were unable to adapt their behavior after contingency reversal, exemplified by a lack of re-establishing CS+ approach behavior (sign tracking). Surprisingly, such behavioral inflexibility, decreased vigor, compulsive grooming and anxiety were unrelated. This study shows that SAPAP3-/- are capable of Pavlovian learning, but lack flexibility to adapt associated conditioned approach behavior. Thus, SAPAP3-/- not only display compulsive-like behavior and anxiety, but also cognitive deficits, confirming and extending the validity of SAPAP3-/- as a suitable model for the study of OCD. The observation that compulsive-like behavior, anxiety and behavioral inflexibility were unrelated suggests a non-causal relationship between these traits and may be of clinical relevance for the treatment of OCD.

Functional heterogeneity within the rodent lateral orbitofrontal cortex dissociates outcome devaluation and reversal learning deficits

The orbitofrontal cortex (OFC) is critical for updating reward-directed behaviours flexibly when outcomes are devalued or when task contingencies are reversed. Failure to update behaviour in outcome devaluation and reversal learning procedures are considered canonical deficits following OFC lesions in non-human primates and rodents. We examined the generality of these findings in rodents using lesions of the rodent lateral OFC (LO) in instrumental action-outcome and Pavlovian cue-outcome devaluation procedures. LO lesions disrupted outcome devaluation in Pavlovian but not instrumental procedures. Furthermore, although both anterior and posterior LO lesions disrupted Pavlovian outcome devaluation, only posterior LO lesions were found to disrupt reversal learning. Posterior but not anterior LO lesions were also found to disrupt the attribution of motivational value to Pavlovian cues in sign-tracking. These novel dissociable task- and subregion-specific effects suggest a way to reconcile contradictory findings between rodent and non-human primate OFC research.

The role of glutamate signaling in incentive salience: second-by-second glutamate recordings in awake Sprague-Dawley rats

The attribution of incentive salience to reward-predictive stimuli has been shown to be associated with substance abuse-like behavior such as increased drug taking. Evidence suggests that glutamate neurotransmission and sequential N-methyl-D-aspartate (NMDA) activation are involved in the attribution of incentive salience. Here, we further explore the role of second-by-second glutamate neurotransmission in the attribution of incentive salience to reward-predictive stimuli by measuring sign-tracking behavior during a Pavlovian conditioned approach procedure using ceramic-based microelectrode arrays configured for sensitive measures of extracellular glutamate in awake behaving Sprague-Dawley rats. Specifically, we show that there is an increase in extracellular glutamate levels in the prelimbic cortex (PrL) and the nucleus accumbens core (NAcC) during sign-tracking behavior to a food-predictive conditioned stimulus (CS+) compared to the presentation of a non-predictive conditioned stimulus (CS-). Furthermore, the results indicate greater increases in extracellular glutamate levels in the PrL compared to NAcC in response to the CS+, including differences in glutamate release and signal decay. Taken together, the present research suggests that there is differential glutamate signaling in the NAcC and PrL during sign-tracking behavior to a food-predictive CS+.

Disconnection of basolateral amygdala and insular cortex disrupts conditioned approach in Pavlovian lever autoshaping

Previously established individual differences in appetitive approach and devaluation sensitivity observed in goal- and sign-trackers may be attributed to differences in the acquisition, modification, or use of associative information in basolateral amygdala (BLA) pathways. Here, we sought to determine the extent to which communication of associative information between BLA and anterior portions of insular cortex (IC) supports ongoing Pavlovian conditioned approach behaviors in sign- and goal-tracking rats, in the absence of manipulations to outcome value. We hypothesized that the BLA mediates goal-, but not sign- tracking approach through interactions with the IC, a brain region involved in supporting flexible behavior. We first trained rats in Pavlovian lever autoshaping to determine their sign- or goal-tracking tendency. During alternating test sessions, we gave unilateral intracranial injections of vehicle or a cocktail of gamma-aminobutyric acid (GABA) receptor agonists, baclofen and muscimol, unilaterally into the BLA and contralaterally or ipsilaterally into the IC prior to reinforced lever autoshaping sessions. Consistent with our hypothesis we found that contralateral inactivation of BLA and IC increased the latency to approach the food cup and decreased the number of food cup contacts in goal-trackers. While contralateral inactivation of BLA and IC did not affect the total number of lever contacts in sign-trackers, this manipulation increased the latency to approach the lever. Ipsilateral inactivation of BLA and IC did not impact approach behaviors in Pavlovian lever autoshaping. These findings, contrary to our hypothesis, suggest that communication between BLA and IC maintains a representation of initially learned appetitive associations that commonly support the initiation of Pavlovian conditioned approach behavior regardless of whether it is directed at the cue or the location of reward delivery.

NMDA receptor blockade specifically impedes the acquisition of incentive salience attribution

Glutamatergic signaling plays an important role in learning and memory. Using Pavlovian conditioned approach procedures, the mechanisms that drive stimulus-reward learning and memory have been investigated. However, there are instances where reward-predictive stimuli can function beyond being solely predictive and can be attributed with "motivational value" or incentive salience. Using a Pavlovian conditioned approach procedure consisting of two different but equally predictive stimuli (lever vs. tone) we investigated the role NMDA receptor function has in the attribution of incentive salience. The results revealed that the administration of MK-801, an NMDA receptor antagonist, during acquisition of Pavlovian conditioned approach promoted goal-tracking to a lever stimulus, while control animals learned to sign-track. Moreover, within the same animals, the use of a tone stimulus elicited goal-tracking responses that were unaffected by MK-801 pretreatments. Furthermore, a lever CS that elicited sign-tracking served as a more robust conditioned reinforcer than a tone CS that elicited goal-tracking or a lever CS that elicited goal-tracking via MK-801 pretreatments. Collectively, these results demonstrate that NMDA receptor antagonism can alter the stimulus-reward relationship learned and prevent the attribution of incentive salience, rather than impede general learning.

Suboptimal choice in rats: Incentive salience attribution promotes maladaptive decision-making

Stimuli that are more predictive of subsequent reward also function as better conditioned reinforcers. Moreover, stimuli attributed with incentive salience function as more robust conditioned reinforcers. Some theories have suggested that conditioned reinforcement plays an important role in promoting suboptimal choice behavior, like gambling. The present experiments examined how different stimuli, those attributed with incentive salience versus those without, can function in tandem with stimulus-reward predictive utility to promote maladaptive decision-making in rats. One group of rats had lights associated with goal-tracking as the reward-predictive stimuli and another had levers associated with sign-tracking as the reward-predictive stimuli. All rats were first trained on a choice procedure in which the expected value across both alternatives was equivalent but differed in their stimulus-reward predictive utility. Next, the expected value across both alternatives was systematically changed so that the alternative with greater stimulus-reward predictive utility was suboptimal in regard to primary reinforcement. The results demonstrate that in order to obtain suboptimal choice behavior, incentive salience alongside strong stimulus-reward predictive utility may be necessary; thus, maladaptive decision-making can be driven more by the value attributed to stimuli imbued with incentive salience that reliably predict a reward rather than the reward itself.

Isolating the incentive salience of reward-associated stimuli: value, choice, and persistence

Sign- and goal-tracking are differentially associated with drug abuse-related behavior. Recently, it has been hypothesized that sign- and goal-tracking behavior are mediated by different neurobehavioral valuation systems, including differential incentive salience attribution. Herein, we used different conditioned stimuli to preferentially elicit different response types to study the different incentive valuation characteristics of stimuli associated with sign- and goal-tracking within individuals. The results demonstrate that all stimuli used were equally effective conditioned stimuli; however, only a lever stimulus associated with sign-tracking behavior served as a robust conditioned reinforcer and was preferred over a tone associated with goal-tracking. Moreover, the incentive value attributed to the lever stimulus was capable of promoting suboptimal choice, leading to a significant reduction in reinforcers (food) earned. Furthermore, sign-tracking to a lever was more persistent than goal-tracking to a tone under omission and extinction contingencies. Finally, a conditional discrimination procedure demonstrated that sign-tracking to a lever and goal-tracking to a tone were dependent on learned stimulus–reinforcer relations. Collectively, these results suggest that the different neurobehavioral valuation processes proposed to govern sign- and goal-tracking behavior are independent but parallel processes within individuals. Examining these systems within individuals will provide a better understanding of how one system comes to dominate stimulus–reward learning, thus leading to the differential role these systems play in abuse-related behavior.