Effects of selective excitotoxic lesions of the nucleus accumbens core, anterior cingulate cortex, and central nucleus of the amygdala on autoshaping performance in rats

Striatal dopamine represents valence on dynamic regional scales

Adaptive decision making relies on dynamic updating of learned associations where environmental cues come to predict valenced stimuli, such as food or threat. Cue-guided behavior depends on a network of brain systems, including dopaminergic projections to the striatum. Critically, it remains unclear how dopamine signaling across the striatum encodes multi-valent, dynamic learning contexts, where positive and negative associations must be rapidly disambiguated. To understand this, we employed a Pavlovian discrimination paradigm, where cues predicting food or threat were intermingled during conditioning sessions, and their meaning was serially reversed across training. We found that male and female rats readily distinguished these cues and updated their behavior rapidly upon valence reversal. Using fiber photometry, we recorded dopamine signaling in three major striatal subregions - the dorsolateral striatum (DLS), the nucleus accumbens (NAc) core, and the nucleus accumbens medial shell - finding that valence was represented uniquely across all three regions, indicative of local signals biased for value and salience. Further, ambiguity introduced by cue reversals reshaped striatal dopamine on different timelines: nucleus accumbens signals updated more readily than those in the DLS. Together, these results indicate that striatal dopamine flexibly encodes stimulus valence according to region-specific rules, and these signals are dynamically modulated by changing contingencies in the resolution of ambiguity about the meaning of environmental cues.Significance Statement Adaptive decision making relies on updating learned associations to disambiguate predictions of reward or threat. This cue-guided behavior depends on striatal dopamine, but it remains unclear how dopamine signaling encodes multi-valent, dynamic learning contexts. Here, we employed a paradigm where cues predicting positive and negative outcomes were intermingled, and their meaning was serially reversed across time. We recorded dopamine signaling, finding heterogeneous patterns of valence encoding across striatal subregions, and cue reversal reshaped subregional signals on different timelines. Our results suggest that dopamine flexibly encodes dynamic learning contexts to resolve ambiguity about the meaning of environmental cues.

Distinct Action Signals by Subregions in the Nucleus Accumbens during STOP-Change Performance

The nucleus accumbens (NAc) is thought to contribute to motivated behavior by signaling the value of reward-predicting cues and the delivery of anticipated reward. The NAc is subdivided into core and shell, with each region containing different populations of neurons that increase or decrease firing to rewarding events. While there are numerous theories of functions pertaining to these subregions and cell types, most are in the context of reward processing, with fewer considering that the NAc might serve functions related to action selection more generally. We recorded from single neurons in the NAc as rats of both sexes performed a STOP-change task that is commonly used to study motor control and impulsivity. In this task, rats respond quickly to a spatial cue on 80% of trials (GO) and must stop and redirect planned movement on 20% of trials (STOP). We found that the activity of reward-excited neurons signaled accurate response direction on GO, but not STOP, trials and that these neurons exhibited higher precue firing after correct trials. In contrast, reward-inhibited neurons significantly represented response direction on STOP trials at the time of the instrumental response. Finally, the proportion of reward-excited to reward-inhibited neurons and the strength of precue firing decreased as the electrode traversed the NAc. We conclude that reward-excited cells (more common in core) promote proactive action selection, while reward-inhibited cells (more common in shell) contribute to accurate responding on STOP trials that require reactive suppression and redirection of behavior.

Valence ambiguity dynamically shapes striatal dopamine heterogeneity

Adaptive decision making relies on dynamic updating of learned associations where environmental cues come to predict positive and negatively valenced stimuli, such as food or threat. Flexible cue-guided behaviors depend on a network of brain systems, including dopamine signaling in the striatum, which is critical for learning and maintenance of conditioned behaviors. Critically, it remains unclear how dopamine signaling encodes multi-valent, dynamic learning contexts, where positive and negative associations must be rapidly disambiguated. To understand this, we employed a Pavlovian discrimination paradigm, where cues predicting positive and negative outcomes were intermingled during conditioning sessions, and their meaning was serially reversed across training. We found that rats readily distinguished these cues, and updated their behavior rapidly upon valence reversal. Using fiber photometry, we recorded dopamine signaling in three major striatal subregions -,the dorsolateral striatum (DLS), the nucleus accumbens core, and the nucleus accumbens medial shell - and found heterogeneous responses to positive and negative conditioned cues and their predicted outcomes. Valence ambiguity introduced by cue reversal reshaped striatal dopamine on different timelines: nucleus accumbens core and shell signals updated more readily than those in the DLS. Together, these results suggest that striatal dopamine flexibly encodes multi-valent learning contexts, and these signals are dynamically modulated by changing contingencies to resolve ambiguity about the meaning of environmental cues.

Role of nucleus accumbens D1-type medium spiny neurons in the expression and extinction of sign-tracking

While sign-tracking, also known as autoshaping, has been studied for many decades, only recently has the tendency to show sign-tracking behavior been linked to the development and persistence of addiction. Sign-tracking is dependent upon dopamine activity in the nucleus accumbens (NAc). The NAc is comprised predominantly of medium spiny projection neurons (MSN) that can be differentiated by their D1-like or D2-like dopamine receptor expression. Here we determined how reducing activity of D1-type MSNs in the NAc affects the expression and extinction of sign-tracking. To address this, we transfected the NAc of transgenic male and female rats that selectively express Cre recombinase in D1-type MSNs with a DIO viral vector expressing hM4Di. Cre- rats were given the same viral infusion but did not express the hM4Di receptor and therefore served as controls. Rats were then conditioned to associate lever presentations with pellet delivery. After sign-tracking was established, all rats were administered clozapine-n-oxide (CNO) prior to three additional conditioning sessions to assess the effects of NAc D1-MSNs inhibition on sign-tracking in the presence of reward. CNO treatment did not alter the expression of sign-tracking in Cre+ or Cre- rats. Next rats underwent extinction training where lever presentations occurred without pellet delivery and all rats received a CNO injection prior to each extinction session. In these extinction conditions, Cre+ rats exhibited robust extinction of sign-tracking across sessions, whereas Cre- rats did not. To determine if D1-MSN inhibition merely produced a temporary cessation of sign-tracking or instead had facilitated a persistent loss of sign-tracking, we evaluated the reemergence of sign-tracking in a test for reconditioning. During testing, reintroduction of the CS-US pairing did not promote the reemergence of sign-tracking in Cre+ rats, but restored sign-tracking in Cre- rats. Thus, chemogenetic inhibition of NAc D1-MSNs promoted extinction of sign-tracking. Collectively, these data suggest that D1-MSNs play an important role in resistance to extinction that typifies sign-tracking behavior.

Intra-Accumbal D1- But not D2-Like Dopamine Receptor Antagonism Reverses the Inhibitory Effects of Cannabidiol on Extinction and Reinstatement of Methamphetamine Seeking Behavior in Rats

Introduction: Methamphetamine (METH) is an addictive psychostimulant that facilitates dopamine transmission to the nucleus accumbens (NAc), resulting in alterations in the mesocorticolimbic brain regions. Cannabidiol (CBD) is considered the second most abundant component of cannabis and is believed to decrease the METH effects. Reversing psychostimulant-induced abnormalities in the mesolimbic dopamine system is the main mechanism for this effect. Various other mechanisms have been proposed: increasing endocannabinoid system activity and modulating gamma-aminobutyric acid (GABA) and glutamate neurons in NAc. However, the exact CBD action mechanisms in reducing drug addiction and relapse vulnerability remain unclear. Methods and Results: The present study aimed to investigate the effects of intracerebroventricular (ICV) administrating 5, 10, and 50 μg/5 μL CBD solutions on the extinction period and reinstatement phase of a METH-induced conditioned place preference. This research also aimed to examine the NAc D1-like dopamine receptor (D1R) and D2-like dopamine receptor (D2R) roles in the effects of CBD on these phases, as mentioned earlier, using SCH23390 and sulpiride microinjections as an antagonist of D1R and D2R. The obtained results showed that microinjection of CBD (10 and 50 μg/5 μL, ICV) suppressed the METH-induced reinstatement and significantly decreased mean extinction latency in treated groups compared to both vehicles and/or untreated control groups. In addition, the results demonstrated that administrating intra-accumbal SCH23390 (1 and 4 μg/0.5 μL saline) reversed the inhibitory effects of CBD on extinction and reinstatement phases while different doses of sulpiride (0.25, 1, and 4 μg/0.5 μL; dimethyl sulfoxide 12%) could not alter the CBD effects. Conclusions: In summary, this study showed that CBD made shorter extinction latencies and suppressed the METH reinstatement, in part, by interacting with D1R but not D2R in the NAc.

Identification of neural circuits controlling male sexual behavior and sexual motivation by manganese-enhanced magnetic resonance imaging

Introduction

Different techniques have been used to identify the brain regions that control sexual motivation and sexual behavior. However, the influence of sexual experience on the activation of these brain regions in the same subject is unknown. Using manganese-enhanced magnetic resonance imaging (MEMRI), we analyzed the activation of brain regions in the sexual incentive motivation (SIM) and the partner preference PP (tests) on weeks 1, 5, and 10 in male rats tested for 10 weeks. AIM. In experiment 1, we analyzed the possible toxic effects of 16 mg/kg of MnCl2 on male sexual behavior, running wheel, and motor execution. In experiment 2, subjects were tested for SIM and PP using MEMRI.

Methods

In both experiments, a dose of 16 mg/kg (s.c) of chloride manganese (MnCl2) was administered 24 h before subjects were tested and placed immediately thereafter in a 7-Tesla Bruker scanner.

Results

In experiment 1, the dose of 16 mg/kg of MnCl2 did not induce behavioral alterations that could interfere with interpreting the imaging data. In experiment 2, we found a clear preference for the female in both the SIM and PP tests. We found a higher signal intensity in the olfactory bulb (OB) in week 1 of the SIM test compared to the control group. We also found increased signal intensity in the socio-sexual behavior and mesolimbic reward circuits in the SIM test in week 1. In the PP test, we found a higher signal intensity in the ventral tegmental area (VTA) in week 10 compared to the control group. In the same test, we found increased signal intensity in the socio-sexual and mesolimbic reward circuits in week 5 compared to the control group. Cohen's d analysis of the whole brain revealed that as the subjects gained sexual experience we observed a higher brain activation in the OB in the SIM group. The PP group showed higher brain activation in the cortex and subcortical structures as they acquired sexual experience.

Discussion

As the subjects gain sexual experience, more structures of the reward and socio-sexual circuits are recruited, resulting in different, and large brain activations.

Selective chemogenetic inactivation of corticoaccumbal projections disrupts trait choice impulsivity

Impulsive choice has enduring trait-like characteristics and is defined by preference for small immediate rewards over larger delayed ones. Importantly, it is a determining factor in the development and persistence of substance use disorder (SUD). Emerging evidence from human and animal studies suggests frontal cortical regions exert influence over striatal reward processing areas during decision-making in impulsive choice or delay discounting (DD) tasks. The goal of this study was to examine how these circuits are involved in decision-making in animals with defined trait impulsivity. To this end, we trained adolescent male rats to stable behavior on a DD procedure and then re-trained them in adulthood to assess trait-like, conserved impulsive choice across development. We then used chemogenetic tools to selectively and reversibly target corticostriatal projections during performance of the DD task. The prelimbic region of the medial prefrontal cortex (mPFC) was injected with a viral vector expressing inhibitory designer receptors exclusively activated by designer drugs (Gi-DREADD), and then mPFC projections to the nucleus accumbens core (NAc) were selectively suppressed by intra-NAc administration of the Gi-DREADD actuator clozapine-n-oxide (CNO). Inactivation of the mPFC-NAc projection elicited a robust increase in impulsive choice in rats with lower vs. higher baseline impulsivity. This demonstrates a fundamental role for mPFC afferents to the NAc during choice impulsivity and suggests that maladaptive hypofrontality may underlie decreased executive control in animals with higher levels of choice impulsivity. Results such as these may have important implications for the pathophysiology and treatment of impulse control, SUDs, and related psychiatric disorders.

Nucleus accumbens core acetylcholine receptors modulate the balance of flexible and inflexible cue-directed motivation

Sign-tracking is a conditioned response where animals interact with reward-predictive cues due to the cues having motivational value, or incentive salience. The nucleus accumbens core (NAc) has been implicated in mediating the sign-tracking response. Additionally, acetylcholine (ACh) transmission throughout the striatum has been attributed to both incentive motivation and behavioral flexibility. Here, we demonstrate a role for NAc ACh receptors in the flexibility of sign-tracking. Sign-tracking animals were exposed to an omission contingency, in which vigorous sign-tracking was punished by reward omission. Animals rapidly adjusted their behavior, but they maintained sign-tracking in a less vigorous manner that did not cancel reward. Within this context of sign-tracking being persistent yet flexible in structure, blockade of NAc nicotinic receptors (nAChRs) led to a persistence in the initial sign-tracking response during omission followed by a period of change in the makeup of sign-tracking, whereas blockade of muscarinic receptors (mAChRs) oppositely enhanced the omission-related development of the new sign-tracking behaviors. Later, once omission learning had occurred, nAChR blockade uniquely led to reduced sign-tracking and elevated reward-directed behaviors instead. These results indicate that NAc ACh receptors have opposing roles in maintaining learned patterns of sign-tracking, with nAChRs having a special involvement in regulating the structure of the sign-tracking response.

Neuronal Death Caused by HMGB1-Evoked via Inflammasomes from Thrombin-Activated Microglia Cells

Microglial cells are a macrophage-like cell type residing within the CNS. These cells evoke pro-inflammatory responses following thrombin-induced brain damage. Inflammasomes, which are large caspase-1-activating protein complexes, play a critical role in mediating the extracellular release of HMGB1 in activated immune cells. The exact role of inflammasomes in microglia activated by thrombin remains unclear, particularly as it relates to the downstream functions of HMGB1. After receiving microinjections of thrombin, Sprague Dawley rats of 200 to 250 gm were studied in terms of behaviors and immunohistochemical staining. Primary culture of microglia cells and BV-2 cells were used for the assessment of signal pathways. In a water maze test and novel object recognition analysis, microinjections of thrombin impaired rats' short-term and long-term memory, and such detrimental effects were alleviated by injecting anti-HMGB-1 antibodies. After thrombin microinjections, the increased oxidative stress of neurons was aggravated by HMGB1 injections but attenuated by anti-HMGB-1 antibodies. Such responses occurred in parallel with the volume of activated microglia cells, as well as their expressions of HMGB-1, IL-1β, IL-18, and caspase-I. In primary microglia cells and BV-2 cell lines, thrombin also induced NO release and mRNA expressions of iNOS, IL-1β, IL-18, and activated caspase-I. HMGB-1 aggravated these responses, which were abolished by anti-HMGB-1 antibodies. In conclusion, thrombin induced microglia activation through triggering inflammasomes to release HMGB1, contributing to neuronal death. Such an action was counteracted by the anti-HMGB-1 antibodies. The refinement of HMGB-1 modulated the neuro-inflammatory response, which was attenuated in thrombin-associated neurodegenerative disorder.

The Neurodegenerative Elderly Syndrome (NES) hypothesis: Alzheimer and Parkinson are two faces of the same disease

Increasing evidence suggests that Alzheimer's disease (AD) and Parkinson's disease (PD) share monoamine and alpha-synuclein (αSyn) dysfunctions, often beginning years before clinical manifestations onset. The triggers for these impairments and the causes leading these early neurodegenerative processes to become AD or PD remain unclear. We address these issues by proposing a radically new perspective to frame AD and PD: they are different manifestations of one only disease we call "Neurodegenerative Elderly Syndrome (NES)". NES goes through three phases. The seeding stage, which starts years before clinical signs, and where the part of the brain-body affected by the initial αSyn and monoamine dysfunctions, influences the future possible progression of NES towards PD or AD. The compensatory stage, where the clinical symptoms are still silent thanks to compensatory mechanisms keeping monoamine concentrations homeostasis. The bifurcation stage, where NES becomes AD or PD. We present recent literature supporting NES and discuss how this hypothesis could radically change the comprehension of AD and PD comorbidities and the design of novel system-level diagnostic and therapeutic actions.

Engaging endogenous opioid circuits in pain affective processes

The pervasive use of opioid compounds for pain relief is rooted in their utility as one of the most effective therapeutic strategies for providing analgesia. While the detrimental side effects of these compounds have significantly contributed to the current opioid epidemic, opioids still provide millions of patients with reprieve from the relentless and agonizing experience of pain. The human experience of pain has long recognized the perceived unpleasantness entangled with a unique sensation that is immediate and identifiable from the first-person subjective vantage point as "painful." From this phenomenological perspective, how is it that opioids interfere with pain perception? Evidence from human lesion, neuroimaging, and preclinical functional neuroanatomy approaches is sculpting the view that opioids predominately alleviate the affective or inferential appraisal of nociceptive neural information. Thus, opioids weaken pain-associated unpleasantness rather than modulate perceived sensory qualities. Here, we discuss the historical theories of pain to demonstrate how modern neuroscience is revisiting these ideas to deconstruct the brain mechanisms driving the emergence of aversive pain perceptions. We further detail how targeting opioidergic signaling within affective or emotional brain circuits remains a strong avenue for developing targeted pharmacological and gene-therapy analgesic treatments that might reduce the dependence on current clinical opioid options.

Using touchscreen-delivered cognitive assessments to address the principles of the 3Rs in behavioral sciences

Despite considerable advances in both in silico and in vitro approaches, in vivo studies that involve animal model systems remain necessary in many research disciplines. Neuroscience is one such area, with studies often requiring access to a complete nervous system capable of dynamically selecting between and then executing a full range of cognitive and behavioral outputs in response to a given stimulus or other manipulation. The involvement of animals in research studies is an issue of active public debate and concern and is therefore carefully regulated. Such regulations are based on the principles of the 3Rs of Replacement, Reduction and Refinement. In the sub-specialty of behavioral neuroscience, Full/Absolute Replacement remains a major challenge, as the complete ex vivo recapitulation of a system as complex and dynamic as the nervous system has yet to be achieved. However, a number of very positive developments have occurred in this area with respect to Relative Replacement and to both Refinement and Reduction. In this review, we discuss the Refinement- and Reduction-related benefits yielded by the introduction of touchscreen-based behavioral assessment apparatus. We also discuss how data generated by a specific panel of behavioral tasks developed for this platform might substantially enhance monitoring of laboratory animal welfare and provide robust, quantitative comparisons of husbandry techniques to define and ensure maintenance of best practice.

Incentive motivation: 'wanting' roles of central amygdala circuitry

The central nucleus of amygdala (CeA) mediates positively-valenced reward motivation as well as negatively-valenced fear. Optogenetic or neurochemical stimulation of CeA circuitry can generate intense incentive motivation to pursue and consume a paired natural food, sex, or addictive drug reward, and even create maladaptive 'wanting what hurts' such as attraction to a shock rod. Evidence indicates CeA stimulations selectively amplify incentive motivation ('wanting') but not hedonic impact ('liking') of the same reward. Further, valence flips can occur for CeA contributions to motivational salience. That is, CeA stimulation can promote either incentive motivation or fearful motivation, even in the same individual, depending on situation. These findings may carry implications for understanding CeA roles in neuropsychiatric disorders involving aberrant motivational salience, ranging from addiction to paranoia and anxiety disorders.

Neurotransmitters, Cell Types, and Circuit Mechanisms of Motor Skill Learning and Clinical Applications

Animals acquire motor skills to better survive and adapt to a changing environment. The ability to learn novel motor actions without disturbing learned ones is essential to maintaining a broad motor repertoire. During motor learning, the brain makes a series of adjustments to build novel sensory–motor relationships that are stored within specific circuits for long-term retention. The neural mechanism of learning novel motor actions and transforming them into long-term memory still remains unclear. Here we review the latest findings with regard to the contributions of various brain subregions, cell types, and neurotransmitters to motor learning. Aiming to seek therapeutic strategies to restore the motor memory in relative neurodegenerative disorders, we also briefly describe the common experimental tests and manipulations for motor memory in rodents.

Extinction of cue-evoked food-seeking recruits a GABAergic interneuron ensemble in the dorsal medial prefrontal cortex of mice

Animals must quickly adapt food-seeking strategies to locate nutrient sources in dynamically changing environments. Learned associations between food and environmental cues that predict its availability promote food-seeking behaviors. However, when such cues cease to predict food availability, animals undergo "extinction" learning, resulting in the inhibition of food-seeking responses. Repeatedly activated sets of neurons, or "neuronal ensembles," in the dorsal medial prefrontal cortex (dmPFC) are recruited following appetitive conditioning and undergo physiological adaptations thought to encode cue-reward associations. However, little is known about how the recruitment and intrinsic excitability of such dmPFC ensembles are modulated by extinction learning. Here, we used in vivo 2-Photon imaging in male Fos-GFP mice that express green fluorescent protein (GFP) in recently behaviorally activated neurons to determine the recruitment of activated pyramidal and GABAergic interneuron dmPFC ensembles during extinction. During extinction, we revealed a persistent activation of a subset of interneurons which emerged from a wider population of interneurons activated during the initial extinction session. This activation pattern was not observed in pyramidal cells, and extinction learning did not modulate the excitability properties of activated pyramidal cells. Moreover, extinction learning reduced the likelihood of reactivation of pyramidal cells activated during the initial extinction session. Our findings illuminate novel neuronal activation patterns in the dmPFC underlying extinction of food-seeking, and in particular, highlight an important role for interneuron ensembles in this inhibitory form of learning.

Role of nucleus accumbens core but not shell in incubation of methamphetamine craving after voluntary abstinence

We recently introduced an animal model to study incubation of drug craving after prolonged voluntary abstinence, mimicking the human condition of relapse after successful contingency management treatment. Here we studied the role of the nucleus accumbens (NAc) in this model. We trained rats to self-administer a palatable solution (sucrose 1% + maltodextrin 1%, 6 h/day, 6 days) and methamphetamine (6 h/day, 12 days). We then evaluated relapse to methamphetamine seeking after 1 and 15 days of voluntary abstinence, achieved via a discrete choice procedure between the palatable solution and methamphetamine (14 days). We used RNAscope in-situ hybridization to quantify the colabeling of the neuronal activity marker Fos, and dopamine Drd1- and Drd2-expressing medium spiny neurons (MSNs) in NAc core and shell during the incubation tests. Next, we determined the effect of pharmacological inactivation of NAc core and shell by either GABA_A and GABA_B agonists (muscimol + baclofen, 50 + 50 ng/side), Drd1-Drd2 antagonist (flupenthixol, 10 µg/side), or the selective Drd1 or Drd2 antagonists (SCH39166, 1.0 µg/side or raclopride, 1.0 µg/side) during the relapse tests. Incubated methamphetamine seeking after voluntary abstinence was associated with a selective increase of Fos expression in the NAc core, but not shell, and Fos was colabeled with both Drd1- and Drd2-MSNs. NAc core, but not shell, injections of muscimol + baclofen, flupenthixol, SCH39166, and raclopride reduced methamphetamine seeking after 15 days of abstinence. Together, our results suggest that dopamine transmission through Drd1 and Drd2 in NAc core is critical to the incubation of methamphetamine craving after voluntary abstinence.

Circuit directionality for motivation: Lateral accumbens-pallidum, but not pallidum-accumbens, connections regulate motivational attraction to reward cues

Sign-tracking behavior, in which animals interact with a cue that predicts reward, provides an example of how incentive salience can be attributed to cues and elicit motivation. The nucleus accumbens (NAc) and ventral pallidum (VP) are two regions involved in cue-driven motivation. The VP, and NAc subregions including the medial shell and core, are critical for sign-tracking. Further, connections between the medial shell and VP are known to participate in sign-tracking and other motivated behaviors. The NAc lateral shell (NAcLSh) is a distinct and understudied subdivision of the NAc, and its contribution to the process by which reward cues acquire value remains unclear. The NAcLSh has been implicated in reward-directed behavior, and has reciprocal connections with the VP, suggesting that NAcLSh and VP interactions could be important mechanisms for incentive salience. Here, we use DREADDs (Designer Receptors Exclusively Activated by Designer Drugs) and an intersectional viral delivery strategy to produce a biased inhibition of NAcLSh neurons projecting to the VP, and vice versa. We find that disruption of connections from NAcLSh to VP reduces sign-tracking behavior while not affecting consumption of food rewards. In contrast, VP to NAcLSh disruption affected neither sign-tracking nor reward consumption, but did produce a greater shift in animals' behavior more towards the reward source when it was available. These findings indicate that the NAcLSh → VP pathway plays an important role in guiding animals towards reward cues, while VP → NAcLSh back-projections may not and may instead bias motivated behavior towards rewards.

Voluntary alcohol access during adolescence/early adulthood, but not during adulthood, causes faster omission contingency learning

In omission contingency training, rodents learn to suppress their natural tendency to approach or touch a reward-predictive cue (termed "autoshaping" or "sign-tracking" responses) if the approach/touching responses lead to the omission of the reward. Previous research has shown that high levels of alcohol exposure (through alcohol vapor exposure) or adolescent alcohol consumption (with some versions of the omission contingency task) can lead to faster omission contingency learning. However, the alcohol exposure procedures and/or omission contingency task parameters differed between these different demonstrations. It was unclear whether the same voluntary alcohol consumption procedures during adolescence/early adulthood and/or adulthood would lead to faster omission contingency learning in one or both age groups. Here, rats received 6 weeks of chronic intermittent access to 20% alcohol or water from PND 26-66 (adolescence/early adulthood in Exp. 1) or PND 68-108 (adulthood in Exp. 2) and began behavioral training (autoshaping training followed by omission contingency training) several days later. We found no evidence that alcohol access at either age altered the number of trials with a sign-tracking response on the levers during autoshaping training. However, alcohol access during adolescence/early adulthood, but not during adulthood, led to faster learning to withhold responding on the lever under omission contingencies during the subsequent phase. We also found no evidence that the level of alcohol consumption was correlated with sign-tracking behavior in the autoshaping phase or with the suppression of lever-pressing during the omission contingency phase. Our results suggest that adolescent/early adult rats have increased vulnerability, compared with adults, to some long-term behavioral effects of voluntary alcohol consumption.

Behavioral flexibility in a mouse model for obsessive-compulsive disorder: Impaired Pavlovian reversal learning in SAPAP3 mutants

Obsessive-compulsive disorder (OCD) is characterized by obsessive thinking, compulsive behavior and anxiety, and is often accompanied by cognitive deficits. The neuropathology of OCD involves dysregulation of cortical-striatal circuits. Similar to OCD patients, SAPAP3 knockout mice 3 (SAPAP3-/- ) exhibit compulsive behavior (grooming), anxiety and dysregulated cortical-striatal function. However, it is unknown whether SAPAP3-/- display cognitive deficits and how these different behavioral traits relate to one another. SAPAP3-/- and wild-type (WT) littermates were trained in a Pavlovian conditioning task pairing visual cues with the delivery of sucrose solution. After mice learned to discriminate between a reward-predicting conditioned stimulus (CS+) and a non-reward stimulus (CS-), contingencies were reversed (CS+ became CS- and vice versa). Additionally, we assessed grooming, anxiety and general activity. SAPAP3-/- acquired Pavlovian approach behavior similarly to WT, albeit less vigorously and with a different strategy. However, unlike WT, SAPAP3-/- were unable to adapt their behavior after contingency reversal, exemplified by a lack of re-establishing CS+ approach behavior (sign tracking). Surprisingly, such behavioral inflexibility, decreased vigor, compulsive grooming and anxiety were unrelated. This study shows that SAPAP3-/- are capable of Pavlovian learning, but lack flexibility to adapt associated conditioned approach behavior. Thus, SAPAP3-/- not only display compulsive-like behavior and anxiety, but also cognitive deficits, confirming and extending the validity of SAPAP3-/- as a suitable model for the study of OCD. The observation that compulsive-like behavior, anxiety and behavioral inflexibility were unrelated suggests a non-causal relationship between these traits and may be of clinical relevance for the treatment of OCD.

Evidence for model-based encoding of Pavlovian contingencies in the human brain

Prominent accounts of Pavlovian conditioning successfully approximate the frequency and intensity of conditioned responses under the assumption that learning is exclusively model-free; that animals do not develop a cognitive map of events. However, these model-free approximations fall short of comprehensively capturing learning and behavior in Pavlovian conditioning. We therefore performed multivoxel pattern analysis of high-resolution functional MRI data in human participants to test for the encoding of stimulus-stimulus associations that could support model-based computations during Pavlovian conditioning. We found that dissociable sub-regions of the striatum encode predictions of stimulus-stimulus associations and predictive value, in a manner that is directly related to learning performance. Activity patterns in the orbitofrontal cortex were also found to be related to stimulus-stimulus as well as value encoding. These results suggest that the brain encodes model-based representations during Pavlovian conditioning, and that these representations are utilized in the service of behavior.

How Deep Brain Stimulation of the Nucleus Accumbens Affects the Cingulate Gyrus and Vice Versa

The nucleus accumbens (NA) and the cingulate gyrus (CG) are two vital limbic brain structures. They have attracted attention as deep brain stimulation (DBS) targets in the treatment of common refractory psychiatric illness. The primary purpose of this article was to review the current knowledge regarding the way that NA DBS affects the CG and vice versa. Methodologically, a thorough literature review was performed. According to the current literature, NA DBS modulates the function of several brain areas including the CG cortex. It specifically causes activation in the ipsilateral CG cortex and voltage-dependent reduction of its blood oxygenation. It also reverses anterior mid-CG cortex dysfunction and decreases metabolism in the subgenual CG. Moreover, NA DBS that induces mirth inhibits the function of the anterior CG cortex and enhances effective connectivity from anterior CG to the ventral striatum. On the other hand, although it is highly probable that CG DBS affects the NA, the exact nature of its effects remains unclear. Despite the increasing interest in psychiatric DBS, the available data on how NA DBS affects the CG and vice versa are restricted. This conclusion probably reflects the high complexity of the limbic circuits and necessitates further research.

Differential involvement of dopamine receptor subtypes in the acquisition of Pavlovian sign-tracking and goal-tracking responses

RATIONALE

Previous work has identified that different forms of Pavlovian conditioned approach, sign-tracking and goal-tracking, are governed by distinct neurochemical mechanisms when compared in animals predisposed to learning one form vs. the other.

OBJECTIVES

The present study aimed to investigate whether these are also neurochemically distinct processes in a population of animals capable of developing either response when this is manipulated via the use of distinct conditioned stimuli (CS).

METHODS

Rats were trained on one of two Pavlovian conditioning procedures in which the CS was either a lever, which elicits sign-tracking, or an auditory click, which elicits goal-tracking. The differential involvement of dopamine D1- and D2-receptors (D1R; D2R) in the acquisition of approach types was investigated via systemic administration of antagonists selective to one or both receptor subtypes during Pavlovian training.

RESULTS

Results indicate that dopaminergic signalling is important for the acquisition of both sign-tracking and goal-tracking responses. However, whilst development of sign-tracking to a lever depends on activity at both D1R and D2R, development of goal-tracking in response to a click was shown to depend only on activity at D1R.

CONCLUSIONS

We suggest that the importance of D1R activity in both sign- and goal-tracking acquisition reflects a general role in learning Pavlovian associations, which aligns with data implicating dopamine in prediction error processes. In contrast, the selective involvement of D2R activity in sign-tracking acquisition may reflect its importance in motivational processes such as incentive salience attribution.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

Amphetamine Self-Administration and Its Extinction Alter the 5-HT1B Receptor Protein Levels in Designated Structures of the Rat Brain

Manipulation of the serotonin (5-HT)_1B receptors can modify the behavioral effects of amphetamine including its reinforcing properties. Focus of this study was to examine changes in 5-HT_1B receptor protein expression in several brain structures linked to substance drug disorder in different stages of amphetamine addiction—single session of amphetamine self-administration, 20 consecutive days of amphetamine self-administration, and 3 and 14 days of extinction from chronic drug intake. “Yoked” procedure was employed to set apart pharmacological and motivational effects of amphetamine intoxication. Immunohistofluorescence was performed on brain slices containing the following regions: nucleus accumbens (NAc) shell and core, globus pallidum (GP) lateral and ventral, hippocampus (HIP), substantia nigra (SN), and ventral tegmental area (VTA). Single amphetamine session decreased the amount of 5-HT_1B receptors in SN, VTA, and HIP in active and yoked rats. On the contrary, 20 days of chronic amphetamine exposure triggered elevation of 5-HT_1B receptors exclusively in animals that voluntarily administered the drug in NAc core, GP ventral, and HIP. Furthermore, 14-day (but not 3-day) extinction from amphetamine increased the 5-HT_1B receptor expression in ventral and lateral GP, HIP, and SN. This study is the first to demonstrate that exposure to amphetamine and its extinction alter the expression of 5-HT_1B receptors in various rat brain regions, and those changes seem to be transient and region specific. Importantly, since increased expression of 5-HT_1B receptor after chronic amphetamine self-administration was limited only to active group of animals, we suggest that 5-HT_1B receptor is linked to motivational aspect of addiction.

Limbic-motor integration by neural excitations and inhibitions in the nucleus accumbens

The nucleus accumbens (NAc) has often been described as a "limbic-motor interface," implying that the NAc integrates the value of expected rewards with the motor planning required to obtain them. However, there is little direct evidence that the signaling of individual NAc neurons combines information about predicted reward and behavioral response. We report that cue-evoked neural responses in the NAc form a likely physiological substrate for its limbic-motor integration function. Across task contexts, individual NAc neurons in behaving rats robustly encode the reward-predictive qualities of a cue, as well as the probability of behavioral response to the cue, as coexisting components of the neural signal. In addition, cue-evoked activity encodes spatial and locomotor aspects of the behavioral response, including proximity to a reward-associated target and the latency and speed of approach to the target. Notably, there are important limits to the ability of NAc neurons to integrate motivational information into behavior: in particular, updating of predicted reward value appears to occur on a relatively long timescale, since NAc neurons fail to discriminate between cues with reward associations that change frequently. Overall, these findings suggest that NAc cue-evoked signals, including inhibition of firing (as noted here for the first time), provide a mechanism for linking reward prediction and other motivationally relevant factors, such as spatial proximity, to the probability and vigor of a reward-seeking behavioral response.NEW & NOTEWORTHY The nucleus accumbens (NAc) is thought to link expected rewards and action planning, but evidence for this idea remains sparse. We show that, across contexts, both excitatory and inhibitory cue-evoked activity in the NAc jointly encode reward prediction and probability of behavioral responding to the cue, as well as spatial and locomotor properties of the response. Interestingly, although spatial information in the NAc is updated quickly, fine-grained updating of reward value occurs over a longer timescale.

Motivational neural circuits underlying reinforcement learning

Reinforcement learning (RL) is the behavioral process of learning the values of actions and objects. Most models of RL assume that the dopaminergic prediction error signal drives plasticity in frontal-striatal circuits. The striatum then encodes value representations that drive decision processes. However, the amygdala has also been shown to play an important role in forming Pavlovian stimulus-outcome associations. These Pavlovian associations can drive motivated behavior via the amygdala projections to the ventral striatum or the ventral tegmental area. The amygdala may, therefore, play a central role in RL. Here we compare the contributions of the amygdala and the striatum to RL and show that both the amygdala and striatum learn and represent expected values in RL tasks. Furthermore, value representations in the striatum may be inherited, to some extent, from the amygdala. The striatum may, therefore, play less of a primary role in learning stimulus-outcome associations in RL than previously suggested.

Role of Anterior Cingulate Cortex in Instrumental Learning: Blockade of Dopamine D1 Receptors Suppresses Overt but Not Covert Learning

HIGHLIGHTS

Blockade of dopamine D1 receptors in ACC suppressed instrumental learning when overt responding was required.

Covert learning through observation was not impaired.

After treatment with a dopamine antagonist, instrumental learning recovered but not the rat's pretreatment level of effort tolerance.

ACC dopamine is not necessary for acquisition of task-relevant cues during learning, but regulates energy expenditure and effort based decision.

Dopamine activity in anterior cingulate cortex (ACC) is essential for various aspects of instrumental behavior, including learning and effort based decision making. To dissociate learning from physical effort, we studied both observational (covert) learning, and trial-and-error (overt) learning. If ACC dopamine activity is required for task acquisition, its blockade should impair both overt and covert learning. If dopamine is not required for task acquisition, but solely for regulating the willingness to expend effort for reward, i.e., effort tolerance, blockade should impair overt learning but spare covert learning. Rats learned to push a lever for food rewards either with or without prior observation of an expert conspecific performing the same task. Before daily testing sessions, the rats received bilateral ACC microinfusions of SCH23390, a dopamine D1 receptor antagonist, or saline-control infusions. We found that dopamine blockade suppressed overt responding selectively, leaving covert task acquisition through observational learning intact. In subsequent testing sessions without dopamine blockade, rats recovered their overt-learning capacity but not their pre-treatment level of effort tolerance. These results suggest that ACC dopamine is not required for the acquisition of conditioned behaviors and that apparent learning impairments could instead reflect a reduced level of willingness to expend effort due to cortical dopamine blockade.

Poor Sleep Is Associated With Greater Marital Aggression: The Role of Self Control

BACKGROUND/OBJECTIVE

Although sleep problems are linked to relationship difficulties, the mechanisms involved have not been empirically demonstrated. The present study considers self-control as such a mechanism.

PARTICIPANTS

Data were collected from 342 predominantly white, middle-class, married adults.

METHOD

Participants completed online questionnaires about sleep, marital aggression, and self-control, and a virtual voodoo doll task.

RESULTS AND CONCLUSIONS

Sleep problems were associated with higher levels of aggression on all measures, and lower self-control mediated these associations. Associations did not depend on participant gender, presence of children in the home, income, or length of marriage.

A food-predictive cue attributed with incentive salience engages subcortical afferents and efferents of the paraventricular nucleus of the thalamus

The paraventricular nucleus of the thalamus (PVT) has been implicated in behavioral responses to reward-associated cues. However, the precise role of the PVT in these behaviors has been difficult to ascertain since Pavlovian-conditioned cues can act as both predictive and incentive stimuli. The "sign-tracker/goal-tracker" rat model has allowed us to further elucidate the role of the PVT in cue-motivated behaviors, identifying this structure as a critical component of the neural circuitry underlying individual variation in the propensity to attribute incentive salience to reward cues. The current study assessed differences in the engagement of specific PVT afferents and efferents in response to presentation of a food-cue that had been attributed with only predictive value or with both predictive and incentive value. The retrograde tracer fluorogold (FG) was injected into the PVT or the nucleus accumbens (NAc) of rats, and cue-induced c-Fos in FG-labeled cells was quantified. Presentation of a predictive stimulus that had been attributed with incentive value elicited c-Fos in PVT afferents from the lateral hypothalamus, medial amygdala (MeA), and the prelimbic cortex (PrL), as well as posterior PVT efferents to the NAc. PVT afferents from the PrL also showed elevated c-Fos levels following presentation of a predictive stimulus alone. Thus, presentation of an incentive stimulus results in engagement of subcortical brain regions; supporting a role for the hypothalamic-thalamic-striatal axis, as well as the MeA, in mediating responses to incentive stimuli; whereas activity in the PrL to PVT pathway appears to play a role in processing the predictive qualities of reward-paired stimuli.

Associative and sensorimotor cortico-basal ganglia circuit roles in effects of abused drugs

The mammalian forebrain is characterized by the presence of several parallel cortico-basal ganglia circuits that shape the learning and control of actions. Among these are the associative, limbic and sensorimotor circuits. The function of all of these circuits has now been implicated in responses to drugs of abuse, as well as drug seeking and drug taking. While the limbic circuit has been most widely examined, key roles for the other two circuits in control of goal-directed and habitual instrumental actions related to drugs of abuse have been shown. In this review we describe the three circuits and effects of acute and chronic drug exposure on circuit physiology. Our main emphasis is on drug actions in dorsal striatal components of the associative and sensorimotor circuits. We then review key findings that have implicated these circuits in drug seeking and taking behaviors, as well as drug use disorders. Finally, we consider different models describing how the three cortico-basal ganglia circuits become involved in drug-related behaviors. This topic has implications for drug use disorders and addiction, as treatments that target the balance between the different circuits may be useful for reducing excessive substance use.

The Nucleus Accumbens: Mechanisms of Addiction across Drug Classes Reflect the Importance of Glutamate Homeostasis

The nucleus accumbens is a major input structure of the basal ganglia and integrates information from cortical and limbic structures to mediate goal-directed behaviors. Chronic exposure to several classes of drugs of abuse disrupts plasticity in this region, allowing drug-associated cues to engender a pathologic motivation for drug seeking. A number of alterations in glutamatergic transmission occur within the nucleus accumbens after withdrawal from chronic drug exposure. These drug-induced neuroadaptations serve as the molecular basis for relapse vulnerability. In this review, we focus on the role that glutamate signal transduction in the nucleus accumbens plays in addiction-related behaviors. First, we explore the nucleus accumbens, including the cell types and neuronal populations present as well as afferent and efferent connections. Next we discuss rodent models of addiction and assess the viability of these models for testing candidate pharmacotherapies for the prevention of relapse. Then we provide a review of the literature describing how synaptic plasticity in the accumbens is altered after exposure to drugs of abuse and withdrawal and also how pharmacological manipulation of glutamate systems in the accumbens can inhibit drug seeking in the laboratory setting. Finally, we examine results from clinical trials in which pharmacotherapies designed to manipulate glutamate systems have been effective in treating relapse in human patients. Further elucidation of how drugs of abuse alter glutamatergic plasticity within the accumbens will be necessary for the development of new therapeutics for the treatment of addiction across all classes of addictive substances.

The ketamine-like compound methoxetamine substitutes for ketamine in the self-administration paradigm and enhances mesolimbic dopaminergic transmission

RATIONALE

Recently, an increasing number of emergency cases due to a novel ketamine-like drug, methoxetamine (MXE), were reported in several countries. However, very little is known about the neuropsychopharmacological and reinforcing profile of this compound.

OBJECTIVES

Our study aims to investigate the effects of MXE on self-administration (SA) behaviour in comparison to ketamine and on dopaminergic transmission.

METHODS

A SA substitution study was performed in male rats trained to intravenously (IV) self-administer ketamine. At responding stability, rats were exposed to sequential phases of MXE substitution at different dosages (starting from 0.5 and then decreasing to 0.25 and 0.125 mg/kg). Standard electrophysiological techniques were used to record changes in firing activities of ventral tegmental area (VTA) dopamine neurons projecting to the nucleus accumbens (NAc) shell after acute injection of cumulative doses of MXE (0.031-0.5 mg/kg IV). Finally, in vivo microdialysis was performed in freely moving rats to evaluate the effect of acute MXE administration (0.125, 0.25 and 0.5 mg/kg IV) on dopamine release in the NAc shell.

RESULTS

MXE 0.125 and 0.25 mg/kg, but not 0.5 mg/kg, substituted for ketamine SA. MXE also induced a dose-dependent stimulation of firing rate (p < 0.0001) and burst firing (p < 0.05) of NAc-projecting VTA dopamine neurons. Consistently, MXE significantly (p < 0.05) increased dopamine extracellular levels in the NAc shell at 0.5 and 0.25 mg/kg with different time onsets, i.e. at 40 and 100 min, respectively.

CONCLUSIONS

This study, while confirming the reinforcing effects of MXE, highlights an electrophysiological and neurochemical profile predictive of its addictive properties.

Opposing effects of acute and chronic d-amphetamine on decision-making in rats

Amphetamine and other drugs of abuse have both short-term and long-lasting effects on brain function, and drug sensitization paradigms often result in chronic impairments in behavioral flexibility. Here we show that acute amphetamine administration temporarily renders rats less sensitive to reward omission, as revealed by a decrease in lose-shift responding during a binary choice task. Intracerebral infusions of amphetamine into the ventral striatum did not affect lose-shift responding but did increase impulsive behavior in which rats chose to check both reward feeders before beginning the next trial. In contrast to acute systemic and intracerebral infusions, sensitization through repeated exposure induced long-lasting increased sensitivity to reward omission. These treatments did not affect choices on trials following reward delivery (i.e. win-stay responding), and sensitization increased spine density in the sensorimotor striatum. The dichotomous effects of amphetamine on short-term and long-term loss sensitivity, and the null effect on win-stay responding, are consistent with a shift of behavioral control to the sensorimotor striatum after drug sensitization. These data provide a new demonstration of such a shift in a novel task unrelated to drug administration, and suggests that the dominance of sensorimotor control persists over many hundreds of trials after sensitization.

An omission procedure reorganizes the microstructure of sign-tracking while preserving incentive salience

Appetitive sign-tracking, in which reward-paired cues elicit approach that can result in cue interaction, demonstrates how cues acquire motivational value. For example, rats will approach and subsequently interact with a lever insertion cue that signals food delivery upon its retraction. However, lever deflections are rapidly reduced once rats are trained on an omission schedule in which lever interactions cancel food delivery. Here we evaluated the change in sign-tracking response topography in rats exposed to such an omission procedure. Lever deflections dropped precipitously when they canceled reward. However, rats that were on an omission schedule continued to approach, sniff, and contact the lever without pressing it, and did so at comparable rates to rats that were not under an omission schedule. Thus, sign-tracking was maintained, albeit in a different manner, following omission. Such findings show that the motivational attraction to reward cues can be expressed with remarkable persistence and flexibility.

Dissociable hippocampal and amygdalar D1-like receptor contribution to discriminated Pavlovian conditioned approach learning

Pavlovian conditioning is an elementary form of reward-related behavioral adaptation. The mesolimbic dopamine system is widely considered to mediate critical aspects of reward-related learning. For example, initial acquisition of positively-reinforced operant behavior requires dopamine (DA) D1 receptor (D1R) activation in the basolateral amygdala (BLA), central nucleus of the amygdala (CeA), and the ventral subiculum (vSUB). However, the role of D1R activation in these areas on appetitive, non-drug-related, Pavlovian learning is not currently known. In separate experiments, microinfusions of the D1-like receptor antagonist SCH-23390 (3.0 nmol/0.5 μL per side) into the amygdala and subiculum preceded discriminated Pavlovian conditioned approach (dPCA) training sessions. D1-like antagonism in all three structures impaired the acquisition of discriminated approach, but had no effect on performance after conditioning was asymptotic. Moreover, dissociable effects of D1-like antagonism in the three structures on components of discriminated responding were obtained. Lastly, the lack of latent inhibition in drug-treated groups may elucidate the role of D1-like in reward-related Pavlovian conditioning. The present data suggest a role for the D1 receptors in the amygdala and hippocampus in learning the significance of conditional stimuli, but not in the expression of conditional responses.

A Framework for Understanding the Emerging Role of Corticolimbic-Ventral Striatal Networks in OCD-Associated Repetitive Behaviors

Significant interest in the mechanistic underpinnings of obsessive-compulsive disorder (OCD) has fueled research on the neural origins of compulsive behaviors. Converging clinical and preclinical evidence suggests that abnormal repetitive behaviors are driven by dysfunction in cortico-striatal-thalamic-cortical (CSTC) circuits. These findings suggest that compulsive behaviors arise, in part, from aberrant communication between lateral orbitofrontal cortex (OFC) and dorsal striatum. An important body of work focused on the role of this network in OCD has been instrumental to progress in the field. Disease models focused primarily on these regions, however, fail to capture an important aspect of the disorder: affective dysregulation. High levels of anxiety are extremely prevalent in OCD, as is comorbidity with major depressive disorder. Furthermore, deficits in processing rewards and abnormalities in processing emotional stimuli are suggestive of aberrant encoding of affective information. Accordingly, OCD can be partially characterized as a disease in which behavioral selection is corrupted by exaggerated or dysregulated emotional states. This suggests that the networks producing OCD symptoms likely expand beyond traditional lateral OFC and dorsal striatum circuit models, and highlights the need to cast a wider net in our investigation of the circuits involved in generating and sustaining OCD symptoms. Here, we address the emerging role of medial OFC, amygdala, and ventral tegmental area projections to the ventral striatum (VS) in OCD pathophysiology. The VS receives strong innervation from these affect and reward processing regions, and is therefore poised to integrate information crucial to the generation of compulsive behaviors. Though it complements functions of dorsal striatum and lateral OFC, this corticolimbic-VS network is less commonly explored as a potential source of the pathology underlying OCD. In this review, we discuss this network's potential role as a locus of OCD pathology and effective treatment.

Integration of reward signalling and appetite regulating peptide systems in the control of food-cue responses

Understanding the neurobiological substrates that encode learning about food-associated cues and how those signals are modulated is of great clinical importance especially in light of the worldwide obesity problem. Inappropriate or maladaptive responses to food-associated cues can promote over-consumption, leading to excessive energy intake and weight gain. Chronic exposure to foods rich in fat and sugar alters the reinforcing value of foods and weakens inhibitory neural control, triggering learned, but maladaptive, associations between environmental cues and food rewards. Thus, responses to food-associated cues can promote cravings and food-seeking by activating mesocorticolimbic dopamine neurocircuitry, and exert physiological effects including salivation. These responses may be analogous to the cravings experienced by abstaining drug addicts that can trigger relapse into drug self-administration. Preventing cue-triggered eating may therefore reduce the over-consumption seen in obesity and binge-eating disorder. In this review we discuss recent research examining how cues associated with palatable foods can promote reward-based feeding behaviours and the potential involvement of appetite-regulating peptides including leptin, ghrelin, orexin and melanin concentrating hormone. These peptide signals interface with mesolimbic dopaminergic regions including the ventral tegmental area to modulate reactivity to cues associated with palatable foods. Thus, a novel target for anti-obesity therapeutics is to reduce non-homeostatic, reward driven eating behaviour, which can be triggered by environmental cues associated with highly palatable, fat and sugar rich foods.

A selective cannabinoid CB2 agonist attenuates damage and improves memory retention following stroke in mice

AIMS

We have recently demonstrated that treatment with a cannabinoid CB2 agonist was protective in a mouse middle cerebral artery occlusion model of cerebral ischemia/reperfusion injury. The present study aimed to determine whether these protective effects of CB2 agonism would extend to a mouse photoinjury model of permanent ischemia and determine associated alterations in cognition and infarct size.

MAIN METHODS

Mice received three injections of the CB2 selective agonist O-1966 or vehicle 1h prior to and 2 and 5days following induction of stroke. Infarct size was assessed at 1, 3, or 7days post-injury and learning and memory effects of injury and O-1966 treatment were assessed on days 6 and 7 using a novel object recognition task and an operant acquisition and retention procedure.

KEY FINDINGS

O-1966 treated mice had significantly smaller infarct volumes compared with vehicle treated mice. Photoinjury was also associated with a significant memory impairment on day 7 post-injury, and this deficit was reversed with O-1966 treatment. Surprisingly, sham-operated mice receiving O-1966 treatment showed a significant learning deficit in both the recognition and operant tasks compared with vehicle treated sham mice.

SIGNIFICANCE

We conclude that CB2 activation is protective against cognitive deficits and tissue damage following permanent ischemia, but may dysregulate glial or neuronal function of learning and memory circuits in the absence of injury and/or inflammation.

Orbitofrontal lesions eliminate signalling of biological significance in cue-responsive ventral striatal neurons

The ventral striatum has long been proposed as an integrator of biologically significant associative information to drive actions. While inputs from the amygdala and hippocampus have been much studied, the role of prominent inputs from orbitofrontal cortex (OFC) are less well understood. Here we recorded single unit activity from ventral striatum core in rats with sham or ipsilateral neurotoxic lesions of lateral OFC, as they performed an odor-guided spatial choice task. Consistent with prior reports, we found that spiking activity recorded in sham rats during cue sampling was related to both reward magnitude and reward identity, with higher firing rates observed for cues that predicted more reward. Lesioned rats also showed differential activity to the cues, but this activity was unbiased towards larger rewards. These data support a role for OFC in shaping activity in the ventral striatum to represent the biological significance of associative information in the environment.

Neurophysiology of Reward-Guided Behavior: Correlates Related to Predictions, Value, Motivation, Errors, Attention, and Action

Many brain areas are activated by the possibility and receipt of reward. Are all of these brain areas reporting the same information about reward? Or are these signals related to other functions that accompany reward-guided learning and decision-making? Through carefully controlled behavioral studies, it has been shown that reward-related activity can represent reward expectations related to future outcomes, errors in those expectations, motivation, and signals related to goal- and habit-driven behaviors. These dissociations have been accomplished by manipulating the predictability of positively and negatively valued events. Here, we review single neuron recordings in behaving animals that have addressed this issue. We describe data showing that several brain areas, including orbitofrontal cortex, anterior cingulate, and basolateral amygdala signal reward prediction. In addition, anterior cingulate, basolateral amygdala, and dopamine neurons also signal errors in reward prediction, but in different ways. For these areas, we will describe how unexpected manipulations of positive and negative value can dissociate signed from unsigned reward prediction errors. All of these signals feed into striatum to modify signals that motivate behavior in ventral striatum and guide responding via associative encoding in dorsolateral striatum.

Nucleus accumbens neurons track behavioral preferences and reward outcomes during risky decision making

BACKGROUND

To make appropriate decisions, organisms must evaluate the risks and benefits of action selection. The nucleus accumbens (NAc) has been shown to be critical for this processing and is necessary for appropriate risk-based decision-making behavior. However, it is not clear how NAc neurons encode this information to promote appropriate behavioral responding.

METHODS

Here, rats (n = 17) were trained to perform a risky decision-making task in which discrete visual cues predicted the availability to respond for a smaller certain (safer) or larger uncertain (riskier) reward. Electrophysiological recordings were made in the NAc core and shell to evaluate neural activity during task performance.

RESULTS

At test, animals exhibited individual differences in risk-taking behavior; some displayed a preference for the risky option, some the safe option, and some did not have a preference. Electrophysiological analysis indicated that NAc neurons differentially encoded information related to risk versus safe outcomes. Further, during free choice trials, neural activity during reward-predictive cues reflected individual behavioral preferences. In addition, neural encoding of reward outcomes was correlated with risk-taking behavior, with safe-preferring and risk-preferring rats showing differential activity in the NAc core and shell during reward omissions.

CONCLUSIONS

Consistent with previously demonstrated alterations in prospective reward value with effort and delay, NAc neurons encode information during reward-predictive cues and outcomes in a risk task that tracked the rats' preferred responses. This processing appears to contribute to subjective encoding of anticipated outcomes and thus may function to bias future risk-based decisions.

A potential role for the paraventricular nucleus of the thalamus in mediating individual variation in Pavlovian conditioned responses

There is ample evidence to suggest that the paraventricular nucleus of the thalamus (PVT) mediates cue-reward learning, especially as it relates to drug-seeking behavior. However, its exact role in these complex processes remains unknown. Here we will present and discuss data from our own laboratory which suggests that the PVT plays a role in multiple forms of stimulus-reward learning, and does so via distinct neurobiological systems. Using an animal model that captures individual variation in response to reward-associated cues, we are able to parse the incentive from the predictive properties of reward cues and to elucidate the neural circuitry underlying these different forms of cue-reward learning. When rats are exposed to a classical Pavlovian conditioning paradigm, wherein a cue predicts food reward, some rats, termed sign-trackers, approach and manipulate the cue upon its presentation. This behavior is indicative of attributing incentive salience to the cue. That is, the cue gains excessive control over behavior for sign-trackers. In contrast, other rats, termed goal-trackers, treat the cue as a mere predictor, and upon its presentation go to the location of reward delivery. Based on our own data utilizing this model, we hypothesize that the PVT represents a common node, but differentially regulates the sign- vs. goal-tracking response. We postulate that the PVT regulates sign-tracking behavior, or the attribution of incentive salience, via subcortical, dopamine-dependent mechanisms. In contrast, we propose that goal-tracking behavior, or the attribution of predictive value, is the product of “top-down” glutamatergic processing between the prelimbic cortex (PrL) and the PVT. Together, data from our laboratory and others support a role for the PVT in cue-motivated behaviors and suggest that it may be an important locus within the neural circuitry that goes awry in addiction and related disorders.

Impact of appetitive and aversive outcomes on brain responses: linking the animal and human literatures

Decision-making is motivated by the possibility of obtaining reward and/or avoiding punishment. Though many have investigated behavior associated with appetitive or aversive outcomes, few have examined behaviors that rely on both. Fewer still have addressed questions related to how anticipated appetitive and aversive outcomes interact to alter neural signals related to expected value, motivation, and salience. Here we review recent rodent, monkey, and human research that address these issues. Further development of this area will be fundamental to understanding the etiology behind human psychiatric diseases and cultivating more effective treatments.

Blocking in autoshaped lever-pressing procedures with rats

Rats will approach and contact a lever whose insertion into the chamber signals response-independent food delivery. This "autoshaping" or "sign-tracking" phenomenon has recently attracted considerable attention as a platform for studying individual differences in impulsivity, drug sensitization, and other traits associated with vulnerability to drug addiction. Here, we examined two basic stimulus selection phenomena-blocking and overshadowing-in the autoshaped lever pressing of rats. Blocking and overshadowing were decidedly asymmetrical. Previously reinforced lever-extension conditioned stimuli (CSs) completely blocked conditioning to auditory cues (Exps. 1 and 2), and previously nonreinforced lever-extension CSs overshadowed conditioning to auditory cues. By contrast, conditioning to lever-extension CSs was not blocked by either auditory (Exp. 3) or lever-insertion (Exp. 4) cues, and was not overshadowed by auditory cues. Conditioning to a lever-insertion cue was somewhat overshadowed by the presence of another lever, especially in terms of food cup behavior displayed after lever withdrawal. We discuss several frameworks in which the apparent immunity of autoshaped lever pressing to blocking might be understood. Given evidence that different brain systems are engaged when different kinds of cues are paired with food delivery, it is worth considering the possibility that interactions among them in learning and performance may follow different rules. In particular, it is intriguing to speculate that the roles of simple cue-reinforcer contiguity, as well as of individual and aggregate reinforcer prediction errors, may differ across stimulus classes.

The amygdala: securing pleasure and avoiding pain

The amygdala has traditionally been associated with fear, mediating the impact of negative emotions on memory. However, this view does not fully encapsulate the function of the amygdala, nor the impact that processing in this structure has on the motivational limbic corticostriatal circuitry of which it is an important structure. Here we discuss the interactions between different amygdala nuclei with cortical and striatal regions involved in motivation; interconnections and parallel circuitries that have become increasingly understood in recent years. We review the evidence that the amygdala stores memories that allow initially motivationally neutral stimuli to become associated through pavlovian conditioning with motivationally relevant outcomes which, importantly, can be either appetitive (e.g. food) or aversive (e.g. electric shock). We also consider how different psychological processes supported by the amygdala such as conditioned reinforcement and punishment, conditioned motivation and suppression, and conditioned approach and avoidance behavior, are not only psychologically but also neurobiologically dissociable, being mediated by distinct yet overlapping neural circuits within the limbic corticostriatal circuitry. Clearly the role of the amygdala goes beyond encoding aversive stimuli to also encode the appetitive, requiring an appreciation of the amygdala's mediation of both appetitive and fearful behavior through diverse psychological processes.

Reward learning requires activity of matrix metalloproteinase-9 in the central amygdala

Learning how to avoid danger and pursue reward depends on negative emotions motivating aversive learning and positive emotions motivating appetitive learning. The amygdala is a key component of the brain emotional system; however, an understanding of how various emotions are differentially processed in the amygdala has yet to be achieved. We report that matrix metalloproteinase-9 (MMP-9, extracellularly operating enzyme) in the central nucleus of the amygdala (CeA) is crucial for appetitive, but not for aversive, learning in mice. The knock-out of MMP-9 impairs appetitively motivated conditioning, but not an aversive one. MMP-9 is present at the excitatory synapses in the CeA with its activity greatly enhanced after the appetitive training. Finally, blocking extracellular MMP-9 activity with its inhibitor TIMP-1 provides evidence that local MMP-9 activity in the CeA is crucial for the appetitive, but not for aversive, learning.

Measuring reinforcement learning and motivation constructs in experimental animals: relevance to the negative symptoms of schizophrenia

The present review article summarizes and expands upon the discussions that were initiated during a meeting of the Cognitive Neuroscience Treatment Research to Improve Cognition in Schizophrenia (CNTRICS; http://cntrics.ucdavis.edu) meeting. A major goal of the CNTRICS meeting was to identify experimental procedures and measures that can be used in laboratory animals to assess psychological constructs that are related to the psychopathology of schizophrenia. The issues discussed in this review reflect the deliberations of the Motivation Working Group of the CNTRICS meeting, which included most of the authors of this article as well as additional participants. After receiving task nominations from the general research community, this working group was asked to identify experimental procedures in laboratory animals that can assess aspects of reinforcement learning and motivation that may be relevant for research on the negative symptoms of schizophrenia, as well as other disorders characterized by deficits in reinforcement learning and motivation. The tasks described here that assess reinforcement learning are the Autoshaping Task, Probabilistic Reward Learning Tasks, and the Response Bias Probabilistic Reward Task. The tasks described here that assess motivation are Outcome Devaluation and Contingency Degradation Tasks and Effort-Based Tasks. In addition to describing such methods and procedures, the present article provides a working vocabulary for research and theory in this field, as well as an industry perspective about how such tasks may be used in drug discovery. It is hoped that this review can aid investigators who are conducting research in this complex area, promote translational studies by highlighting shared research goals and fostering a common vocabulary across basic and clinical fields, and facilitate the development of medications for the treatment of symptoms mediated by reinforcement learning and motivational deficits.

Nucleus accumbens responses differentiate execution and restraint in reward-directed behavior

Our behavior is powerfully driven by environmental cues that signal the availability of rewarding stimuli. We frequently encounter stimuli-a bowl of candy or an alert from our smartphone-that trigger actions to obtain those rewards, even though there may be positive outcomes associated with not acting. The inability to restrain one's action in the presence of reward-associated cues is one type of impulsive behavior and a component of such maladaptive behaviors as overeating, gambling, and substance abuse. The nucleus accumbens (NAc) is ideally situated to integrate multiple cognitive and affective inputs to bias action via outputs through the basal ganglia. NAc neurons have been shown to respond to cues that predict reward availability, goal-directed behaviors aimed at obtaining them, and delivery of the reward itself. As these processes are typically associated, it is difficult to discern whether signals in the NAc are more closely related to processing reward-predictive aspects of goal-directed behavior or selection of behavioral response. To dissociate these possibilities, we recorded the activity of NAc neurons while rats performed a task in which two different cues both informed rats of reward availability but required them to either press a lever (Go) or withhold pressing (NoGo) to obtain the reward. Individual cue-responsive neurons showed either increases or decreases in activity at cue onset. Increases in activity were larger, and decreases smaller, when rats withheld lever pressing, whether correctly for NoGo trials or in error on Go trials. Thus NAc cue responses correlated with action, regardless of cue type or accuracy.