Neural substrates of appetitive and aversive prediction error

Cannabidiol interactions with oxycodone analgesia in an operant orofacial cutaneous thermal pain assay following oral administration in rats

Previous studies have driven the notion that the cannabis constituent cannabidiol could be an effective adjunct to opioid administration for managing pain. Most of these studies have used experimental rodents with routes of administration, such as subcutaneous and intraperitoneal, that do not correspond with the routes used in clinical practice. In response to this, we tested the ability of cannabidiol co-administration to augment opioid analgesia via the more clinically-relevant oral route of administration. To this end, male and female rats were orally gavaged with cannabidiol (25 mg/kg), oxycodone (1.4 mg/kg), or a combination of both, after which they were tested in an operant thermal orofacial pain assay in which they voluntarily exposed their faces to cutaneous thermal pain to receive a palatable reward. All three drug conditions produced analgesic effects of varying degrees, being most profound in the combination group where a statistically significant enhancement over oxycodone-induced analgesia alone was evident. Additionally, oxycodone administration decreased lick frequencies - a measure of motor coordination of rhythmic movements - which too was magnified by co-administration of cannabidiol. Together these studies provide further support of an ability of cannabidiol to augment opioid effects, particularly analgesia, when administered by a route relevant to human pain management. As such, they encourage the notion that cannabidiol could find utility as an opioid-sparing approach to treating pain.

Structural synaptic signatures of contextual memory retrieval-reactivated hippocampal engram cells

Learning enhances hippocampal engram cell synaptic connectivity which is crucial for engram reactivation and recall to natural cues. Memory retrieval engages only a subset of the learning-activated ensemble, indicating potential differences in synaptic connectivity signatures of reactivated and non-reactivated cells. We probed these differences in structural synaptic connectivity patterns after recent memory retrieval, 72 h after either neutral Context Exploration (CE) or aversive Contextual Fear Conditioning (CFC). Using a combination of eGRASP (enhanced green fluorescent protein (GFP) reconstitution across synaptic partners) and viral-TRAP (targeted recombination in activated populations) to label CA3 synapses onto CA1 engram cells, we investigated differences in spine density, clusters, and morphology between the reactivated and non-reactivated population of the learning ensemble. In doing so, we developed a pipeline for reconstruction and analysis of dendrites and spines, taking nested data structure into account. Our data demonstrate an interplay between reactivation status, context valence or both factors on the number, distribution, and morphology of CA1 engram cell synapses. Despite a lack of differences in spine density, reactivated engram cells encoding an aversive context were characterised by a higher probability of forming spine clusters and a more dynamic spine type signature compared to their non-reactivated counterparts or engram cells encoding a neutral context. Together, our data indicate that the learning-activated ensemble undergoes different trajectories in structural synaptic connectivity during engram refinement.

Parabrachial Calca neurons mediate second-order conditioning

Learning to associate cues, both directly and indirectly, with biologically significant events is essential for survival. Second-order conditioning (SOC) involves forming an association between a previously reinforced conditioned stimulus (CS1) and a new conditioned stimulus (CS2) without the presence of an unconditioned stimulus (US). The neural substrates mediating SOC, however, remain unclear. Parabrachial Calca neurons, which react to the noxious US, also respond to a CS after pairing with a US, suggesting that Calca neurons mediate SOC. We established an aversive SOC behavioral paradigm in mice and monitored Calca neuron activity via single-cell calcium imaging during conditioning and subsequent recall phases. These neurons were activated by both CS1 and CS2 after SOC. Chemogenetically inhibiting Calca neurons during CS1-CS2 pairing attenuated SOC. Thus, reactivation of the US pathway by a learned CS plays an important role in forming the association between the old and a new CS, promoting the formation of second-order memories.

Dopamine and Norepinephrine Differentially Mediate the Exploration-Exploitation Tradeoff

Dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision-making processes. Although two neuromodulators share a synthesis pathway and are coactivated under states of arousal, they engage in distinct circuits and modulatory roles. However, the specific role of each neuromodulator in decision-making, in particular the exploration-exploitation tradeoff, remains unclear. Revealing how each neuromodulator contributes to exploration-exploitation tradeoff is important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration, a direct comparison using the same dynamic decision-making task is needed. Here, we ran male and female mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA antagonist (flupenthixol), a nonselective DA agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol) and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine on exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. The modulatory effect of beta-noradrenergic receptor activity on exploration was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via sensitivity to outcome. Together, these findings suggested that the mechanisms that govern the exploration-exploitation transition are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

Expectation generation and its effect on subsequent pain and visual perception

Bayesian accounts of perception, such as predictive processing, suggest that perceptions integrate expectations and sensory experience, and thus assimilate to expected values. Furthermore, more precise expectations should have stronger influences on perception. We tested these hypotheses in a paradigm that manipulates both the mean value and the precision of cues within-person. Forty-five participants observed cues-presented as ratings from 10 previous participants-with varying cue means, variances (precision), and skewness across trials. Participants reported expectations regarding the painfulness of thermal stimuli or the visual contrast of flickering checkerboards. Subsequently, similar cues were each followed by a visual or noxious thermal stimulus. While perceptions assimilated to expected values in both modalities, cues' precision mainly affected visual ratings. Furthermore, behavioral and computational models revealed that expectations were biased towards extreme values in both modalities, and towards low-pain cues specifically. fMRI analysis revealed that the cues affected systems related to higher-level affective and cognitive processes-including assimilation to the cue mean in a neuromarker of endogenous contributions to pain and in the nucleus accumbens, and activity consistent with aversive prediction-error-like encoding in the periaqueductal gray during pain perception-but not systems related to early perceptual processing. Our findings suggest that predictive processing theories should be combined with mechanisms such as selective attention to better fit empirical findings, and that expectation generation and its perceptual effects are mostly modality-specific and operate on higher-level processes rather than early perception.

Conflict Dynamics of Post-Retrieval Extinction: A Comparative Analysis of Unconditional and Conditional Reminders Using Skin Conductance Responses and EEG

The post-retrieval extinction paradigm, rooted in reconsolidation theory, holds promise for enhancing extinction learning and addressing anxiety and trauma-related disorders. This study investigates the impact of two reminder types, mild US-reminder (US-R) and CS-reminder (CS-R), along with a no-reminder extinction, on fear recovery prevention in a categorical fear conditioning paradigm. Scalp EEG recordings during reminder and extinction processes were conducted in a three-day design. Results show that the US-R group exhibits a distinctive extinction learning pattern, characterized by a slowed-down yet successful process and pronounced theta-alpha desynchronization (source-located in the prefrontal cortex) during CS processing, followed by enhanced synchronization (source-located in the anterior cingulate) after shock cancellation in extinction trials. These neural dynamics correlate with the subtle advantage of US-R in the Day 3 recovery test, presenting faster spontaneous recovery fading and generally lower fear reinstatement responses. Conversely, the CS reminder elicits CS-specific effects in later episodic tests. The unique neural features of the US-R group suggest a larger prediction error and subsequent effortful conflict learning processes, warranting further exploration.

Spatiotemporal Organization of Prefrontal Norepinephrine Influences Neuronal Activity

Norepinephrine (NE), a neuromodulator released by locus ceruleus (LC) neurons throughout the cortex, influences arousal and learning through extrasynaptic vesicle exocytosis. While NE within cortical regions has been viewed as a homogenous field, recent studies have demonstrated heterogeneous axonal dynamics and advances in GPCR-based fluorescent sensors permit direct observation of the local dynamics of NE at cellular scale. To investigate how the spatiotemporal dynamics of NE release in the prefrontal cortex (PFC) affect neuronal firing, we employed in vivo two-photon imaging of layer 2/3 of the PFC in order to observe fine-scale neuronal calcium and NE dynamics concurrently. In this proof of principle study, we found that local and global NE fields can decouple from one another, providing a substrate for local NE spatiotemporal activity patterns. Optic flow analysis revealed putative release and reuptake events which can occur at the same location, albeit at different times, indicating the potential to create a heterogeneous NE field. Utilizing generalized linear models, we demonstrated that cellular Ca2+ fluctuations are influenced by both the local and global NE field. However, during periods of local/global NE field decoupling, the local field drives cell firing dynamics rather than the global field. These findings underscore the significance of localized, phasic NE fluctuations for structuring cell firing, which may provide local neuromodulatory control of cortical activity.

Integration of Prior Expectations and Suppression of Prediction Errors During Expectancy-Induced Pain Modulation: The Influence of Anxiety and Pleasantness

Pain perception arises from the integration of prior expectations with sensory information. Although recent work has demonstrated that treatment expectancy effects (e.g., placebo hypoalgesia) can be explained by a Bayesian integration framework incorporating the precision level of expectations and sensory inputs, the key factor modulating this integration in stimulus expectancy-induced pain modulation remains unclear. In a stimulus expectancy paradigm combining emotion regulation in healthy male and female adults, we found that participants' voluntary reduction in anticipatory anxiety and pleasantness monotonically reduced the magnitude of pain modulation by negative and positive expectations, respectively, indicating a role of emotion. For both types of expectations, Bayesian model comparisons confirmed that an integration model using the respective emotion of expectations and sensory inputs explained stimulus expectancy effects on pain better than using their respective precision. For negative expectations, the role of anxiety is further supported by our fMRI findings that (1) functional coupling within anxiety-processing brain regions (amygdala and anterior cingulate) reflected the integration of expectations with sensory inputs and (2) anxiety appeared to impair the updating of expectations via suppressed prediction error signals in the anterior cingulate, thus perpetuating negative expectancy effects. Regarding positive expectations, their integration with sensory inputs relied on the functional coupling within brain structures processing positive emotion and inhibiting threat responding (medial orbitofrontal cortex and hippocampus). In summary, different from treatment expectancy, pain modulation by stimulus expectancy emanates from emotion-modulated integration of beliefs with sensory evidence and inadequate belief updating.

A conceptual framework for a neurophysiological basis of art therapy for PTSD

Post-traumatic stress disorder (PTSD) is a heterogeneous condition that affects many civilians and military service members. Lack of engagement, high dropout rate, and variable response to psychotherapy necessitates more compelling and accessible treatment options that are based on sound neuroscientific evidence-informed decision-making. Art therapy incorporates elements proven to be effective in psychotherapy, such as exposure, making it a potentially valuable treatment option. This conceptual paper aims to inform the neurophysiological rationale for the use of art therapy as a therapeutic approach for individuals with PTSD. A narrative synthesis was conducted using literature review of empirical research on the neurophysiological effects of art therapy, with supporting literature on neuroaesthetics and psychotherapies to identify art therapy factors most pertinent for PTSD. Findings were synthesized through a proposed framework based on the triple network model considering the network-based dysfunctions due to PTSD. Art therapy's active components, such as concretization and metaphor, active art engagement, emotion processing and regulation, perspective taking and reframing, and therapeutic alliance, may improve symptoms of PTSD and prompt adaptive brain functioning. Given the scarcity of rigorous studies on art therapy's effectiveness and mechanisms of alleviating PTSD symptoms, the suggested framework offers a neurophysiological rationale and a future research agenda to investigate the impact of art therapy as a therapeutic approach for individuals with PTSD.

D1 and D2 medium spiny neurons in the nucleus accumbens core have distinct and valence-independent roles in learning

At the core of value-based learning is the nucleus accumbens (NAc). D1- and D2-receptor-containing medium spiny neurons (MSNs) in the NAc core are hypothesized to have opposing valence-based roles in behavior. Using optical imaging and manipulation approaches in mice, we show that neither D1 nor D2 MSNs signal valence. D1 MSN responses were evoked by stimuli regardless of valence or contingency. D2 MSNs were evoked by both cues and outcomes, were dynamically changed with learning, and tracked valence-free prediction error at the population and individual neuron level. Finally, D2 MSN responses to cues were necessary for associative learning. Thus, D1 and D2 MSNs work in tandem, rather than in opposition, by signaling specific properties of stimuli to control learning.

Genetic disruption of dopamine β-hydroxylase dysregulates innate responses to predator odor in mice

In rodents, exposure to predator odors such as cat urine acts as a severe stressor that engages innate defensive behaviors critical for survival in the wild. The neurotransmitters norepinephrine (NE) and dopamine (DA) modulate anxiety and predator odor responses, and we have shown previously that dopamine β-hydroxylase knockout (Dbh -/-), which reduces NE and increases DA in mouse noradrenergic neurons, disrupts innate behaviors in response to mild stressors such as novelty. We examined the consequences of Dbh knockout on responses to predator odor (bobcat urine) and compared them to Dbh-competent littermate controls. Over the first 10 min of predator odor exposure, controls exhibited robust defensive burying behavior, whereas Dbh -/- mice showed high levels of grooming. Defensive burying was potently suppressed in controls by drugs that reduce NE transmission, while excessive grooming in Dbh -/- mice was blocked by DA receptor antagonism. In response to a cotton square scented with a novel "neutral" odor (lavender), most control mice shredded the material, built a nest, and fell asleep within 90 min. Dbh -/- mice failed to shred the lavender-scented nestlet, but still fell asleep. In contrast, controls sustained high levels of arousal throughout the predator odor test and did not build nests, while Dbh -/- mice were asleep by the 90-min time point, often in shredded bobcat urine-soaked nesting material. Compared with controls exposed to predator odor, Dbh -/- mice demonstrated decreased c-fos induction in the anterior cingulate cortex, lateral septum, periaqueductal gray, and bed nucleus of the stria terminalis, but increased c-fos in the locus coeruleus and medial amygdala. These data indicate that relative ratios of central NE and DA signaling coordinate the type and valence of responses to predator odor.

Optogenetic stimulation of the locus coeruleus enhances appetitive extinction in rats

Extinction is a specific example of learning where a previously reinforced stimulus or response is no longer reinforced, and the previously learned behaviour is no longer necessary and must be modified. Current theories suggest extinction is not the erasure of the original learning but involves new learning that acts to suppress the original behaviour. Evidence for this can be found when the original behaviour recovers following the passage of time (spontaneous recovery) or reintroduction of the reinforcement (i.e. reinstatement). Recent studies have shown that pharmacological manipulation of noradrenaline (NA) or its receptors can influence appetitive extinction; however, the role and source of endogenous NA in these effects are unknown. Here, we examined the role of the locus coeruleus (LC) in appetitive extinction. Specifically, we tested whether optogenetic stimulation of LC neurons during extinction of a food-seeking behaviour would enhance extinction evidenced by reduced spontaneous recovery in future tests. LC stimulation during extinction trials did not change the rate of extinction but did serve to reduce subsequent spontaneous recovery, suggesting that stimulation of the LC can augment reward-related extinction. Optogenetic inhibition of the LC during extinction trials reduced responding during the trials where it was applied, but no long-lasting changes in the retention of extinction were observed. Since not all LC cells expressed halorhodopsin, it is possible that more complete LC inhibition or pathway-specific targeting would be more effective at suppressing extinction learning. These results provide further insight into the neural basis of appetitive extinction, and in particular the role of the LC. A deeper understanding of the physiological bases of extinction can aid development of more effective extinction-based therapies.

Genetic disruption of dopamine β-hydroxylase dysregulates innate responses to predator odor in mice

In rodents, exposure to predator odors such as cat urine acts as a severe stressor that engages innate defensive behaviors critical for survival in the wild. The neurotransmitters norepinephrine (NE) and dopamine (DA) modulate anxiety and predator odor responses, and we have shown previously that dopamine β-hydroxylase knockout (Dbh -/-), which reduces NE and increases DA in mouse noradrenergic neurons, disrupts innate behaviors in response to mild stressors such as novelty. We examined the consequences of Dbh knockout (Dbh -/-) on responses to predator odor (bobcat urine) and compared them to Dbh-competent littermate controls. Over the first 10 min of predator odor exposure, controls exhibited robust defensive burying behavior, whereas Dbh -/- mice showed high levels of grooming. Defensive burying was potently suppressed in controls by drugs that reduce NE transmission, while excessive grooming in Dbh -/- mice was blocked by DA receptor antagonism. In response to a cotton square scented with a novel "neutral" odor (lavender), most control mice shredded the material, built a nest, and fell asleep within 90 min. Dbh -/- mice failed to shred the lavender-scented nestlet, but still fell asleep. In contrast, controls sustained high levels of arousal throughout the predator odor test and did not build nests, while Dbh -/- mice were asleep by the 90-min time point, often in shredded bobcat urine-soaked nesting material. Compared with controls exposed to predator odor, Dbh -/- mice demonstrated decreased c-fos induction in the anterior cingulate cortex, lateral septum, periaqueductal gray, and bed nucleus of the stria terminalis, but increased c-fos in the locus coeruleus and medial amygdala. These data indicate that relative ratios of central NE and DA signaling coordinate the type and valence of responses to predator odor.

Dissociable Contributions of Basolateral Amygdala and Ventrolateral Orbitofrontal Cortex to Flexible Learning Under Uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including the highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, the unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory designer receptors exclusively activated by designer drugs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-/post-reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of the ventrolateral orbitofrontal cortex (vlOFC), but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Dissociable contributions of basolateral amygdala and ventrolateral orbitofrontal cortex to flexible learning under uncertainty

Reversal learning measures the ability to form flexible associations between choice outcomes with stimuli and actions that precede them. This type of learning is thought to rely on several cortical and subcortical areas, including highly interconnected orbitofrontal cortex (OFC) and basolateral amygdala (BLA), and is often impaired in various neuropsychiatric and substance use disorders. However, unique contributions of these regions to stimulus- and action-based reversal learning have not been systematically compared using a chemogenetic approach and particularly before and after the first reversal that introduces new uncertainty. Here, we examined the roles of ventrolateral OFC (vlOFC) and BLA during reversal learning. Male and female rats were prepared with inhibitory DREADDs targeting projection neurons in these regions and tested on a series of deterministic and probabilistic reversals during which they learned about stimulus identity or side (left or right) associated with different reward probabilities. Using a counterbalanced within-subject design, we inhibited these regions prior to reversal sessions. We assessed initial and pre-post reversal changes in performance to measure learning and adjustments to reversals, respectively. We found that inhibition of vlOFC, but not BLA, eliminated adjustments to stimulus-based reversals. Inhibition of BLA, but not vlOFC, selectively impaired action-based probabilistic reversal learning, leaving deterministic reversal learning intact. vlOFC exhibited a sex-dependent role in early adjustment to action-based reversals, but not in overall learning. These results reveal dissociable roles for BLA and vlOFC in flexible learning and highlight a more crucial role for BLA in learning meaningful changes in the reward environment.

Multiple routes to enhanced memory for emotionally relevant events

Events associated with aversive or rewarding outcomes are prioritized in memory. This memory boost is commonly attributed to the elicited affective response, closely linked to noradrenergic and dopaminergic modulation of hippocampal plasticity. Herein we review and compare this 'affect' mechanism to an additional, recently discovered, 'prediction' mechanism whereby memories are strengthened by the extent to which outcomes deviate from expectations, that is, by prediction errors (PEs). The mnemonic impact of PEs is separate from the affective outcome itself and has a distinct neural signature. While both routes enhance memory, these mechanisms are linked to different - and sometimes opposing - predictions for memory integration. We discuss new findings that highlight mechanisms by which emotional events strengthen, integrate, and segment memory.

The Rescorla-Wagner model, prediction error, and fear learning

The Rescorla-Wagner model remains one of the most important and influential theoretical accounts of the conditions under which Pavlovian learning occurs. Moreover, the experimental approaches that inspired the model continue to provide powerful behavioral tools to advance mechanistic understanding of how we and other animals learn to fear and learn to reduce fear. Here we consider key features of the Rescorla-Wagner model as applied to study of fear learning. We review evidence for key insights of the model. First, learning to fear and learning to reduce fear are governed by a common, signed prediction error. Second, this error drives variations in effectiveness of the shock US that are causal to whether and how much fear is learned or lost during a conditioning trial. We also consider behavioral and neural findings inconsistent with the model and which will be essential to understand and advance understanding of fear learning.

From safety to frustration: The neural substrates of inhibitory learning in aversive and appetitive conditioning procedures

Inhibitory associative learning counters the effects of excitatory learning, whether appetitively or aversively motivated. Moreover, the affective responses accompanying the inhibitory associations are of opponent valence to the excitatory conditioned responses. Inhibitors for negative aversive outcomes (e.g. shock) signal safety, while inhibitors for appetitive outcomes (e.g. food reward) elicit frustration and/or disappointment. This raises the question as to whether studies using appetitive and aversive conditioning procedures should demonstrate the same neural substrates for inhibitory learning. We review the neural substrates of appetitive and aversive inhibitory learning as measured in different procedural variants and in the context of the underpinning excitatory conditioning on which it depends. The mesocorticolimbic dopamine pathways, retrosplenial cortex and hippocampus are consistently implicated in inhibitory learning. Further neural substrates identified in some procedural variants may be related to the specific motivation of the learning task and modalities of the learning cues. Finally, we consider the translational implications of our understanding of the neural substrates of inhibitory learning, for obesity and addictions as well as for anxiety disorders.

Dopamine and norepinephrine differentially mediate the exploration-exploitation tradeoff

The catecholamines dopamine (DA) and norepinephrine (NE) have been repeatedly implicated in neuropsychiatric vulnerability, in part via their roles in mediating the decision making processes. Although the two neuromodulators share a synthesis pathway and are co-activated under states of arousal, they engage in distinct circuits and roles in modulating neural activity across the brain. However, in the computational neuroscience literature, they have been assigned similar roles in modulating the latent cognitive processes of decision making, in particular the exploration-exploitation tradeoff. Revealing how each neuromodulator contributes to this explore-exploit process will be important in guiding mechanistic hypotheses emerging from computational psychiatric approaches. To understand the differences and overlaps of the roles of these two catecholamine systems in regulating exploration and exploitation, a direct comparison using the same dynamic decision making task is needed. Here, we ran mice in a restless two-armed bandit task, which encourages both exploration and exploitation. We systemically administered a nonselective DA receptor antagonist (flupenthixol), a nonselective DA receptor agonist (apomorphine), a NE beta-receptor antagonist (propranolol), and a NE beta-receptor agonist (isoproterenol), and examined changes in exploration within subjects across sessions. We found a bidirectional modulatory effect of dopamine receptor activity on the level of exploration. Increasing dopamine activity decreased exploration and decreasing dopamine activity increased exploration. Beta-noradrenergic receptor activity also modulated exploration, but the modulatory effect was mediated by sex. Reinforcement learning model parameters suggested that dopamine modulation affected exploration via decision noise and norepinephrine modulation affected exploration via outcome sensitivity. Together, these findings suggested that the mechanisms that govern the transition between exploration and exploitation are sensitive to changes in both catecholamine functions and revealed differential roles for NE and DA in mediating exploration.

Enhancement of memory consolidation by an avoidance conditioned stimulus: Modulation by the D3 receptor

Conditioned stimuli (CS) paired with foot-shock can enhance memory consolidation. Because the dopamine D3R has been implicated in mediating various responses to CSs, the current study explored its potential role in modulation of memory consolidation by an avoidance CS. Male Sprague-Dawley rats trained to avoid foot-shocks in a two-way signalled active avoidance task (8 sessions, 30 trials per session, 0.8 mA foot-shock) were pre-treated with the D3R antagonist NGB-2904 (Vehicle, 0.1 or 5 mg/kg) and exposed to the CS immediately after the sample phase of an object recognition memory task. Discrimination ratios were assessed 72 h later. Immediate, but not delayed (6 h), post-sample exposure to the CS enhanced object recognition memory and this effect was dose-dependently blocked by NGB-2904. Control experiments with the beta-noradrenergic receptor antagonist propranolol (10 or 20 mg/kg) and D2R antagonist pimozide (0.2 or 0.6 mg/kg) indicated that NGB-2904 targeted post-training memory consolidation. Exploring the pharmacological selectivity of the D3R effect, it was found that: 1) 5 mg/kg NGB-2904 blocked conditioned memory modulation produced by post-sample exposure to a "weak" CS (one day of avoidance training) and concurrent stimulation of catecholamine activity by 10 mg/kg bupropion; 2) post-sample exposure to a "weak" CS and concurrent administration of the D3R agonist 7-OH-DPAT (1 mg/kg) enhanced consolidation of object memory. Finally, because 5 mg/kg NGB-2904 had no effect on modulation by avoidance training in the presence of foot-shocks, the findings herein support the hypothesis that the D3R plays an important role in modulation of memory consolidation by CSs.

Modulation of aversive value coding in the vertebrate and invertebrate brain

Avoiding potentially dangerous situations is key for the survival of any organism. Throughout life, animals learn to avoid environments, stimuli or actions that can lead to bodily harm. While the neural bases for appetitive learning, evaluation and value-based decision-making have received much attention, recent studies have revealed more complex computations for aversive signals during learning and decision-making than previously thought. Furthermore, previous experience, internal state and systems level appetitive-aversive interactions seem crucial for learning specific aversive value signals and making appropriate choices. The emergence of novel methodologies (computation analysis coupled with large-scale neuronal recordings, neuronal manipulations at unprecedented resolution offered by genetics, viral strategies and connectomics) has helped to provide novel circuit-based models for aversive (and appetitive) valuation. In this review, we focus on recent vertebrate and invertebrate studies yielding strong evidence that aversive value information can be computed by a multitude of interacting brain regions, and that past experience can modulate future aversive learning and therefore influence value-based decisions.

A partner's smile is not per se a safety signal: Psychophysiological response patterns to instructed threat and safety

Recent studies on fear conditioning and pain perception suggest that pictures of loved ones (e.g., a romantic partner) may serve as a prepared safety cue that is less likely to signal aversive events. Challenging this view, we examined whether pictures of smiling or angry loved ones are better safety or threat cues. To this end, 47 healthy participants were verbally instructed that specific facial expressions (e.g., happy faces) cue threat of electric shocks and others cue safety (e.g., angry faces). When facial images served as threat cues, they elicited distinct psychophysiological defensive responses (e.g., increased threat ratings, startle reflex, and skin conductance responses) compared to viewing safety cues. Interestingly, instructed threat effects occurred regardless of the person who cued shock threat (partner vs. unknown) and their facial expression (happy vs. angry). Taken together, these results demonstrate the flexible nature of facial information (i.e., facial expression and facial identity) to be easily learned as signals for threat or safety, even when showing loved ones.

The Recruitment of a Neuronal Ensemble in the Central Nucleus of the Amygdala During the First Extinction Episode Has Persistent Effects on Extinction Expression

BACKGROUND

Adaptive behavior depends on the delicate and dynamic balance between acquisition and extinction memories. Disruption of this balance, particularly when the extinction of memory loses control over behavior, is the root of treatment failure of maladaptive behaviors such as substance abuse or anxiety disorders. Understanding this balance requires a better understanding of the underlying neurobiology and its contribution to behavioral regulation.

METHODS

We microinjected Daun02 in Fos-lacZ transgenic rats following a single extinction training episode to delete extinction-recruited neuronal ensembles in the basolateral amygdala (BLA) and central nucleus of the amygdala (CN) and examined their contribution to behavior in an appetitive Pavlovian task. In addition, we used immunohistochemistry and neuronal staining methods to identify the molecular markers of activated neurons in the BLA and CN during extinction learning or retrieval.

RESULTS

CN neurons were preferentially engaged following extinction, and deletion of these extinction-activated ensembles in the CN but not the BLA impaired the retrieval of extinction despite additional extinction training and promoted greater levels of behavioral restoration in spontaneous recovery and reinstatement. Disrupting extinction processing in the CN in turn increased activity in the BLA. Our results also show a specific role for CN PKCδ+ neurons in behavioral inhibition but not during initial extinction learning.

CONCLUSIONS

We showed that the initial extinction-recruited CN ensemble is critical to the acquisition-extinction balance and that greater behavioral restoration does not mean weaker extinction contribution. These findings provide a novel avenue for thinking about the neural mechanisms of extinction and for developing treatments for cue-triggered appetitive behaviors.

Positioning the brainstem within the neural network of threat prediction

In a recent study, Strickland and McDannald dissected the role of brainstem networks in threat prediction. Using probabilistic threat discrimination in rats, the authors demonstrated that brainstem neurons estimate threat probability and generate positive aversive prediction errors after unexpected outcomes. Their findings suggest that, beyond organizing defensive behaviors, brainstem neurons are involved in threat prediction computations.

Habit Formation and the Effect of Repeated Stress Exposures on Cognitive Flexibility Learning in Horses

Horse training exposes horses to an array of cognitive and ethological challenges. Horses are routinely required to perform behaviours that are not aligned to aspects of their ethology, which may delay learning. While horses readily form habits during training, not all of these responses are considered desirable, resulting in the horse being subject to retraining. This is a form of cognitive flexibility and is critical to the extinction of habits and the learning of new responses. It is underpinned by complex neural processes which can be impaired by chronic or repeated stress. Domestic horses may be repeatedly exposed to multiples stressors. The potential contribution of stress impairments of cognitive flexibility to apparent training failures is not well understood, however research from neuroscience can be used to understand horses' responses to training. We trained horses to acquire habit-like responses in one of two industry-style aversive instrumental learning scenarios (moving away from the stimulus-instinctual or moving towards the stimulus-non-instinctual) and evaluated the effect of repeated stress exposures on their cognitive flexibility in a reversal task. We measured heart rate as a proxy for noradrenaline release, salivary cortisol and serum Brain Derived Neurotrophic Factor (BDNF) to infer possible neural correlates of the learning outcomes. The instinctual task which aligned with innate equine escape responses to aversive stimuli was acquired significantly faster than the non-instinctual task during both learning phases, however contrary to expectations, the repeated stress exposure did not impair the reversal learning. We report a preliminary finding that serum BDNF and salivary cortisol concentrations in horses are positively correlated. The ethological salience of training tasks and cognitive flexibility learning can significantly affect learning in horses and trainers should adapt their practices where such tasks challenge innate equine behaviour.

Role of the locus coeruleus and basal forebrain in arousal and attention

Experimental evidence has implicated multiple neurotransmitter systems in either the direct or indirect modulation of cortical arousal and attention circuitry. In this review, we selectively focus on three such systems: 1) norepinephrine (NE)-containing neurons of the locus coeruleus (LC), 2) acetylcholine (ACh)-containing neurons of the basal forebrain (BF), and 3) parvalbumin (PV)-containing gamma-aminobutyric acid neurons of the BF. Whereas BF-PV neurons serve as a rapid and transient arousal system, LC-NE and BF-ACh neuromodulation are typically activated on slower but longer-lasting timescales. Recent findings suggest that the BF-PV system serves to rapidly respond to even subtle sensory stimuli with a microarousal. We posit that salient sensory stimuli, such as those that are threatening or predict the need for a response, will quickly activate the BF-PV system and subsequently activate both the BF-ACh and LC-NE systems if the circumstances require longer periods of arousal and vigilance. We suggest that NE and ACh have overlapping psychological functions with the main difference being the precise internal/environmental sensory situations/contexts that recruit each neurotransmitter system - a goal for future research to determine. Implications of dysfunction of each of these three attentional systems for our understanding of neuropsychiatric conditions are considered. Finally, the contemporary availability of research tools to selectively manipulate and measure the activity of these distinctive neuronal populations promises to answer longstanding questions, such as how various arousal systems influence downstream decision-making and motor responding.

The role of hypoleptinemia in the psychological and behavioral adaptation to starvation: implications for anorexia nervosa

This narrative review aims to pinpoint mental and behavioral effects of starvation, which may be triggered by hypoleptinemia and as such may be amenable to treatment with leptin receptor agonists. The reduced leptin secretion results from the continuous loss of fat mass, thus initiating a graded triggering of diverse starvation related adaptive functions. In light of leptin receptors located in several peripheral tissues and many brain regions adaptations may extend beyond those of the hypothalamus-pituitary-end organ-axes. We focus on gastrointestinal tract and reward system as relevant examples of peripheral and central effects of leptin. Despite its association with extreme obesity, congenital leptin deficiency with its many parallels to a state of starvation allows the elucidation of mental symptoms amenable to treatment with exogenous leptin in both ob/ob mice and humans with this autosomal recessive disorder. For starvation induced behavioral changes with an intact leptin signaling we particularly focus on rodent models for which proof of concept has been provided for the causative role of hypoleptinemia. For humans, we highlight the major cognitive, emotional and behavioral findings of the Minnesota Starvation Experiment to contrast them with results obtained upon a lesser degree of caloric restriction. Evidence for hypoleptinemia induced mental changes also stems from findings obtained in lipodystrophies. In light of the recently reported beneficial cognitive, emotional and behavioral effects of metreleptin-administration in anorexia nervosa we discuss potential implications for the treatment of this eating disorder. We postulate that leptin has profound psychopharmacological effects in the state of starvation.

Honey bees can store and retrieve independent memory traces after complex experiences that combine appetitive and aversive associations

Real-world experiences do often mix appetitive and aversive events. Understanding the ability of animals to extract, store and use this information is an important issue in neurobiology. We used honey bees as model organism to study learning and memory after a differential conditioning that combines appetitive and aversive training trials. First of all, we describe an aversive conditioning paradigm that constitutes a clear opposite of the well known appetitive olfactory conditioning of the proboscis extension response. A neutral odour is presented paired with the bitter substance quinine. Aversive memory is evidenced later as an odour-specific impairment in appetitive conditioning. Then we tested the effect of mixing appetitive and aversive conditioning trials distributed along the same training session. Differential conditioning protocols like this were used before to study the ability to discriminate odours, however they were not focused on whether appetitive and aversive memories are formed. We found that after a differential conditioning, honey bees establish independent appetitive and aversive memories that do not interfere with each other during acquisition or storage. Finally, we moved the question forward to retrieval and memory expression to evaluate what happens when appetitive and the aversive learned odours are mixed during test. Interestingly, opposite memories compete in a way that they do not cancel each other out. Honey bees showed the ability to switch from expressing appetitive to aversive memory depending on their satiation level.

Unrelenting Fear Under Stress: Neural Circuits and Mechanisms for the Immediate Extinction Deficit

Therapeutic interventions for disorders of fear and anxiety rely on behavioral approaches that reduce pathological fear memories. For example, learning that threat-predictive stimuli are no longer associated with aversive outcomes is central to the extinction of conditioned fear responses. Unfortunately, fear memories are durable, long-lasting, and resistant to extinction, particularly under high levels of stress. This is illustrated by the "immediate extinction deficit," which is characterized by a poor long-term reduction of conditioned fear when extinction procedures are attempted within hours of fear conditioning. Here, I will review recent work that has provided new insight into the neural mechanisms underlying resistance to fear extinction. Emerging studies reveal that locus coeruleus norepinephrine modulates amygdala-prefrontal cortical circuits that are critical for extinction learning. These data suggest that stress-induced activation of brain neuromodulatory systems biases fear memory at the expense of extinction learning. Behavioral and pharmacological strategies to reduce stress in patients undergoing exposure therapy might improve therapeutic outcomes.

The nervous system as a solution for implementing closed negative feedback control loops

Behavior can be regarded as the output of a system (action), as a function linking stimulus to response (reaction), or as an abstraction of the bidirectional relationship between the environment and the organism (interaction). When considering the latter possibility, a relevant question arises concerning how an organism can materially and continuously implement such a relationship during its lifetime in order to perpetuate itself. The feedback control approach has taken up the task of answering just that question. During the last several decades, said approach has been progressing and has started to be recognized as a paradigm shift, superseding certain canonical notions in mainstream behavior analysis, cognitive psychology, and even neuroscience. In this paper, we describe the main features of feedback control theory and its associated techniques, concentrating on its critiques of behavior analysis, as well as the commonalities they share. While some of feedback control theory's major critiques of behavior analysis arise from the fact that they focus on different levels of organization, we believe that some are legitimate and meaningful. Moreover, feedback control theory seems to blend with neurobiology more smoothly as compared to canonical behavior analysis, which only subsists in a scattered handful of fields. If this paradigm shift truly takes place, behavior analysts-whether they accept or reject this new currency-should be mindful of the basics of the feedback control approach.

The Role of the Lateral Habenula in Inhibitory Learning from Reward Omission

The lateral habenula (LHb) is a phylogenetically primitive brain structure that plays a key role in learning to inhibit distinct responses to specific stimuli. This structure is activated by primary aversive stimuli, cues predicting an imminent aversive event, unexpected reward omissions, and cues associated with the omission of an expected reward. The most widely described physiological effect of LHb activation is acutely suppressing midbrain dopaminergic signaling. However, recent studies have identified multiple means by which the LHb promotes this effect as well as other mechanisms of action. These findings reveal the complex nature of LHb circuitry. The present paper reviews the role of this structure in learning from reward omission. We approach this topic from the perspective of computational models of behavioral change that account for inhibitory learning to frame key findings. Such findings are drawn from recent behavioral neuroscience studies that use novel brain imaging, stimulation, ablation, and reversible inactivation techniques. Further research and conceptual work are needed to clarify the nature of the mechanisms related to updating motivated behavior in which the LHb is involved. As yet, there is little understanding of whether such mechanisms are parallel or complementary to the well-known modulatory function of the more recently evolved prefrontal cortex.