Functional architecture of dopamine neurons driving fear extinction learning

Dopamine release and dopamine-related gene expression in the amygdala are modulated by the gastrin-releasing peptide in opposite directions during stress-enhanced fear learning and extinction

Fear extinction leads to a decrease of originally acquired fear responses after the threat is no longer present. Fear extinction is adaptive and critical for organism's survival, but deficits in extinction may lead to exaggerated fear in animals or post-traumatic stress disorder (PTSD) in humans. Dopamine has recently emerged as essential for fear extinction and PTSD, however the neural circuits serving this dopamine function are only beginning to be investigated, and the dopamine intracellular signaling pathways are unknown. We generated gastrin-releasing peptide gene knockout (Grp-/-) mice and found that they exhibit enhanced fear memory in a stress-enhanced fear learning (SEFL) paradigm, which combines stress exposure and fear extinction, two features critical for developing PTSD. Using in vivo fiber photometry to record dopamine signals, we found that the susceptibility of Grp-/- mice to SEFL is paralleled by an increase in basolateral amygdala (BLA) dopaminergic binding during fear conditioning and early extinction. Combined optogenetics and ex vivo electrophysiology showed an increase in presynaptic ventral tegmental area (VTA)-BLA connectivity in Grp-/- mice, demonstrating a role of dysregulated input from the VTA on BLA function in the absence of the GRP. When examining gene transcription using RNA-seq and qPCR, we discovered concerted down-regulation in dopamine-related genes in the BLA of Grp-/- mice following long-term SEFL memory recall that was not observed in naïve conditions. These experiments demonstrate that the GRP regulates dopamine function in stress-enhanced fear processing and identify the Grp as the first gene known to regulate dopaminergic control of fear extinction.

Region-specific nucleus accumbens dopamine signals encode distinct aspects of avoidance learning

Avoidance learning-learning to avoid bad outcomes-is an essential survival behavior. Dopamine signals are widely observed in response to aversive stimuli, indicating they could play a role in learning about how to avoid these stimuli.1,2,3,4,5 However, it is unclear what computations dopamine signals perform to support avoidance learning. Furthermore, substantial heterogeneity in dopamine responses to aversive stimuli has been observed across nucleus accumbens (NAc) subregions.3,6,7,8 To understand how heterogeneous dopamine responses to aversive stimuli contribute to avoidance learning, we recorded NAc core (Core) and NAc ventromedial shell (vmShell) dopamine during a task in which mice could avoid a footshock punishment by moving to the opposite side of a 2-chamber apparatus during a 5-s warning cue. Both signals evolved substantially-but differently-with learning. We found that Core and vmShell dopamine signals responded oppositely to shocks at the beginning of training and oppositely to warning cues as cue-shock associations developed in mid-training. Core dopamine responses strengthen with learning and are especially evident during expert performance. vmShell dopamine responses to cues and shocks were present during early learning but were not sustained during expert performance. Our data support a model in which Core dopamine encodes prediction errors that guide the consolidation of avoidance learning, while vmShell dopamine guides initial cue-shock associations by signaling aversive salience.

Aversion-induced dopamine reductions predict drug-taking and escape behaviors

Dopamine release in the nucleus accumbens core (NAcC) has long been associated with the promotion of motivated behavior. However, inhibited dopamine signaling can increase behavior in certain settings, such as during drug self-administration. While aversive environmental stimuli can reduce dopamine, it is unclear whether such stimuli reliably engage this mechanism in different contexts. Here we compared the physiological and behavioral responses to the same aversive stimulus in different designs to determine if there is uniformity in the manner that aversive stimuli are encoded and promote behavior. NAcC dopamine was measured using fiber photometry in male and female rats during cocaine self-administration sessions in which an acutely aversive 90 dB white noise was intermittently presented. In a separate group of rats, aversion-induced changes in dopamine were measured during an escape design in which operant responses terminated aversive white noise. Aversive white noise significantly reduced NAcC dopamine and increased cocaine self-administration in both male and female rats. The same relationship was observed in the escape design, in which white noise reduced dopamine and promoted the performance of escape behavior. In both designs, the magnitude of the dopamine reduction predicted behavioral performance. While prior research demonstrated that pharmacologically reduced dopamine signaling can promote intake, this report demonstrates that this physiological mechanism is naturally engaged by aversive environmental stimuli and is generalizable to non-drug contexts. These findings illustrate a common physiological signature in response to aversion that may promote both adaptive and maladaptive behavior.

A mechanism linking dopamine's roles in reinforcement, movement and motivation

Dopamine neurons (DANs) play seemingly distinct roles in reinforcement, 1-3 motivation, 4,5 and movement, 6,7 and DA-modulating therapies relieve symptoms across a puzzling spectrum of neurologic and psychiatric symptoms. 8 Yet, the mechanistic relationship among these roles is unknown. Here, we show DA's tripartite roles are causally linked by a process in which phasic striatal DA rapidly and persistently recalibrates the propensity to move, a measure of vigor. Using a self-timed movement task, we found that single exposures to reward-related DA transients (both endogenous and exogenously-induced) exerted one-shot updates to movement timing-but in a surprising fashion. Rather than reinforce specific movement times, DA transients quantitatively changed movement timing on the next trial, with larger transients leading to earlier movements (and smaller to later), consistent with a stochastic search process that calibrates the frequency of movement. Both abrupt and gradual changes in external and internal contingencies-such as timing criterion, reward content, and satiety state-caused changes to the amplitude of DA transients that causally altered movement timing. The rapidity and bidirectionality of the one-shot effects are difficult to reconcile with gradual synaptic plasticity, and instead point to more flexible cellular mechanisms, such as DA-dependent modulation of neuronal excitability. Our findings shed light on how natural reinforcement, as well as DA-related disorders such as Parkinson's disease, could affect behavioral vigor.

Nucleus accumbens dopamine encodes the trace period during appetitive Pavlovian conditioning

Pavlovian conditioning tasks have been used to identify the neural systems involved with learning cue-outcome relationships. In delay conditioning, the conditioned stimulus (CS) overlaps or co-terminates with the delivery of the unconditioned stimulus (US). Prior studies demonstrate that dopamine in the nucleus accumbens (NAc) regulates behavioral responding during delay conditioning. Furthermore, the dopamine response to the CS reflects the relative value of the upcoming reward in these tasks. In contrast to delay conditioning, trace conditioning involves a 'trace' period separating the end of the CS and the US delivery. While dopamine has been implicated in trace conditioning, no studies have examined how NAc dopamine responds to reward-related stimuli in these tasks. Here, we developed a within-subject trace conditioning task where distinct CSs signaled either a short trace period (5s) or a long trace period (55s) prior to food reward delivery. Male rats exhibited greater conditioned responding and a faster response latency to the Short Trace CS relative to the Long Trace CS. Voltammetry recordings in the NAc found that the CS-evoked dopamine response increased on Short Trace trials but decreased on Long Trace trials. Conversely, US-evoked dopamine responses were greater on Long Trace trials relative to Short Trace trials. The CS dopamine response correlated with the response latency and not with conditioned responding. Furthermore, the relationship between CS dopamine and latency was best explained by an exponential function. Our results collectively illustrate that the trace period is encoded by the bidirectional NAc dopamine response to the CS during Pavlovian conditioning.

Significance statement

Learning to associate a cue with given outcome is a fundamental process underlying reward seeking behavior. Striatal dopamine is important for associating cues with rewards during Pavlovian conditioning. However, it is unclear how the dopamine system responds to cues during trace conditioning when there is temporal gap between the cue and reward. Here, we performed voltammetry recordings of striatal dopamine levels in male rats during trace conditioning. We find that cue-evoked dopamine signals encode the trace period and is related to the response latency. While prior reports find dopamine neurons signal the relative reward value by increases in dopamine levels, we demonstrate that the dopamine response to reward-predictive cues can signal the reward value through bidirectional changes in dopamine transmission.

Dopamine prediction error signaling in a unique nigrostriatal circuit is critical for associative fear learning

Learning by experience that certain cues in the environment predict danger is crucial for survival. How dopamine (DA) circuits drive this form of associative learning is not fully understood. Here, in male mice, we demonstrate that DA neurons projecting to a unique subregion of the dorsal striatum, the posterior tail of the striatum (TS), encode a prediction error (PE) signal during associative fear learning. These DA neurons are necessary specifically during acquisition of fear learning, but not once the fear memory is formed, and are not required for forming cue-reward associations. Notably, temporally-precise inhibition or excitation of DA terminals in TS impairs or enhances fear learning, respectively. Furthermore, neuronal activity in TS is crucial for the acquisition of associative fear learning and learning-induced activity patterns in TS critically depend on DA input. Together, our results reveal that DA PE signaling in a non-canonical nigrostriatal circuit is important for driving associative fear learning.

Opponent control of reinforcement by striatal dopamine and serotonin

The neuromodulators dopamine (DA) and serotonin (5-hydroxytryptamine; 5HT) powerfully regulate associative learning1-8. Similarities in the activity and connectivity of these neuromodulatory systems have inspired competing models of how DA and 5HT interact to drive the formation of new associations9-14. However, these hypotheses have not been tested directly because it has not been possible to interrogate and manipulate multiple neuromodulatory systems in a single subject. Here we establish a mouse model that enables simultaneous genetic access to the brain's DA and 5HT neurons. Anterograde tracing revealed the nucleus accumbens (NAc) to be a putative hotspot for the integration of convergent DA and 5HT signals. Simultaneous recording of DA and 5HT axon activity, together with genetically encoded DA and 5HT sensor recordings, revealed that rewards increase DA signalling and decrease 5HT signalling in the NAc. Optogenetically dampening DA or 5HT reward responses individually produced modest behavioural deficits in an appetitive conditioning task, while blunting both signals together profoundly disrupted learning and reinforcement. Optogenetically reproducing DA and 5HT reward responses together was sufficient to drive the acquisition of new associations and supported reinforcement more potently than either manipulation did alone. Together, these results demonstrate that striatal DA and 5HT signals shape learning by exerting opponent control of reinforcement.

A posterior insula to lateral amygdala pathway transmits US-offset information with a limited role in fear learning

During fear learning, associations between a sensory cue (conditioned stimulus, CS) and an aversive stimulus (unconditioned stimulus, US) are formed in specific brain circuits. The lateral amygdala (LA) is involved in CS-US integration; however, US pathways to the LA remain understudied. Here, we investigated whether the posterior insular cortex (pInsCx), a hub for aversive state signaling, transmits US information to the LA during fear learning. We find that the pInsCx makes a robust, glutamatergic projection specifically targeting the anterior LA. In vivo Ca2+ imaging reveals that neurons in the pInsCx and anterior LA display US-onset and US-offset responses; imaging combined with axon silencing shows that the pInsCx selectively transmits US-offset information to the anterior LA. Optogenetic silencing, however, does not show a role for US-driven activity in the anterior LA or its pInsCx afferents in fear memory formation. Thus, we describe a cortical projection that carries US-offset information to the amygdala with a limited role in fear learning.

Building Resilience: The Stress Response as a Driving Force for Neuroplasticity and Adaptation

People exhibit an extraordinary capacity to adjust to stressful situations. Here, we argue that the acute stress response is a major driving force behind this adaptive process. In addition to immediately freeing energy reserves, facilitating a rapid and robust neurocognitive response, and helping to reinstate homeostasis, the stress response also critically regulates neuroplasticity. Therefore, understanding the healthy acute stress response is crucial for understanding stress resilience-the maintenance or rapid recovery of mental health during and after times of adversity. Contemporary resilience research differentiates between resilience factors and resilience mechanisms. Resilience factors refer to a broad array of social, psychological, or biological variables that are stable but potentially malleable and predict resilient outcomes. In contrast, resilience mechanisms refer to proximate mechanisms activated during acute stress that enable individuals to effectively navigate immediate challenges. In this article, we review literature related to how neurotransmitter and hormonal changes during acute stress regulate the activation of resilience mechanisms. We integrate literature on the timing-dependent and neuromodulator-specific regulation of neurocognition, episodic memory, and behavioral and motivational control, highlighting the distinct and often synergistic roles of catecholamines (dopamine and norepinephrine) and glucocorticoids. We conclude that stress resilience is bolstered by improved future predictions and the success-based reinforcement of effective coping strategies during acute stress. The resulting generalized memories of success, controllability, and safety constitute beneficial plasticity that lastingly improves self-control under stress. Insight into such mechanisms of resilience is critical for the development of novel interventions focused on prevention rather than treatment of stress-related disorders.

How Dopamine Enables Learning from Aversion

Dopamine is heavily studied for its role in reward learning, but it is becoming increasingly appreciated that dopamine can also enable learning from aversion. Dopamine neurons modulate their firing and neurotransmitter release patterns in response to aversive outcomes. However, there is considerable heterogeneity in the timing and directionality of the modulation. Open questions remain as to the factors that determine this heterogeneity and how varying patterns of responses to aversion in different dopamine-receptive brain regions contribute to value learning, decision-making, and avoidance. Here, we review recent progress in this area and highlight important future directions.

Current State of the Neuroscience of Fear Extinction and Its Relevance to Anxiety Disorders

The elucidation of the functional neuroanatomy of human fear, or threat, extinction has started in the 2000s by a series of enthusiastically greeted functional magnetic resonance imaging (fMRI) studies that were able to translate findings from rodent research about an involvement of the ventromedial prefrontal cortex (vmPFC) and the hippocampus in fear extinction into human models. Enthusiasm has been painfully dampened by a meta-analysis of human fMRI studies by Fullana and colleagues in 2018 who showed that activation in these areas is inconsistent, sending shock waves through the extinction research community. The present review guides readers from the field (as well as non-specialist readers desiring safe knowledge about human extinction mechanisms) during a series of exposures with corrective information. New information about extinction-related brain activation not considered by Fullana et al. will also be presented. After completion of this exposure-based fear reduction program, readers will trust that the reward learning system, the cerebellum, the vmPFC, the hippocampus, and a wider brain network are involved in human fear extinction, along with the neurotransmitters dopamine and noradrenaline. Specific elements of our exposure program include exploitation of the temporal dynamics of extinction, of the spatial heterogeneity of extinction-related brain activation, of functional connectivity methods, and of large sample sizes. Implications of insights from studies in healthy humans for the understanding and treatment of anxiety-related disorders are discussed.

Dynamic representation of appetitive and aversive stimuli in nucleus accumbens shell D1- and D2-medium spiny neurons

The nucleus accumbens (NAc) is a key brain region for motivated behaviors, yet how distinct neuronal populations encode appetitive or aversive stimuli remains undetermined. Using microendoscopic calcium imaging in mice, we tracked NAc shell D1- or D2-medium spiny neurons' (MSNs) activity during exposure to stimuli of opposing valence and associative learning. Despite drift in individual neurons' coding, both D1- and D2-population activity was sufficient to discriminate opposing valence unconditioned stimuli, but not predictive cues. Notably, D1- and D2-MSNs were similarly co-recruited during appetitive and aversive conditioning, supporting a concurrent role in associative learning. Conversely, when contingencies changed, there was an asymmetric response in the NAc, with more pronounced changes in the activity of D2-MSNs. Optogenetic manipulation of D2-MSNs provided causal evidence of the necessity of this population in the extinction of aversive associations. Our results reveal how NAc shell neurons encode valence, Pavlovian associations and their extinction, and unveil mechanisms underlying motivated behaviors.

Dopaminergic circuits controlling threat and safety learning

The ability to learn from experience that certain cues and situations are associated with threats or safety is crucial for survival and adaptive behavior. Understanding the neural substrates of threat and safety learning has high clinical significance because deficits in these forms of learning characterize anxiety disorders. Traditionally, dopamine neurons were thought to uniformly support reward learning by signaling reward prediction errors. However, the dopamine system is functionally more diverse than was initially appreciated and is also critical for processing threat and safety. In this review, I highlight recent studies demonstrating that dopamine neurons generate prediction errors for threat and safety, and describe how dopamine projections to the amygdala, medial prefrontal cortex (mPFC), and striatum regulate associative threat and safety learning.

Innervation density governs crosstalk of GPCR-based norepinephrine and dopamine sensors

GPCR-based fluorescent sensors are widely used to correlate neuromodulatory signaling with brain function. While experiments in transfected cells often reveal selectivity for individual neurotransmitters, sensor specificity in the brain frequently remains uncertain. Pursuing experiments in brain slices and in vivo, we find that norepinephrine and dopamine cross-activate the respective sensors. Non-specific activation occurred when innervation of the cross-reacting transmitter was high, and silencing of specific innervation was indispensable for interpreting sensor fluorescence.

Out with the bad, in with the good: A review on augmented extinction learning in humans

Several leading therapies for anxiety-related disorders rely on the principles of extinction learning. However, despite decades of development and research, many of these treatments remain only moderately effective. Developing techniques to improve extinction learning is an important step towards developing improved and mechanistically-informed exposure-based therapies. In this review, we highlight human research on strategies that might augment extinction learning through reward neurocircuitry and dopaminergic pathways, with an emphasis on counterconditioning and other behaviorally-augmented forms of extinction learning (e.g., novelty-facilitated extinction, positive affect training). We also highlight emerging pharmacological and non-pharmacological methods of augmenting extinction, including L-DOPA and aerobic exercise. Finally, we discuss future directions for augmented extinction learning and memory research, including the need for more work examining the influence of individual differences and psychopathology.

Aversion-induced drug taking and escape behavior involve similar nucleus accumbens core dopamine signaling signatures

Dopamine release in the nucleus accumbens core (NAcC) has long been associated with the promotion of motivated behavior. However, inhibited dopamine signaling can increase behavior in certain settings, such as during drug self-administration. While aversive environmental stimuli can reduce dopamine, it is unclear whether such stimuli reliably engage this mechanism in different contexts. Here we compared the physiological and behavioral responses to the same aversive stimulus in different designs to determine if there is uniformity in the manner that aversive stimuli are encoded and promote behavior. NAcC dopamine was measured using fiber photometry in male and female rats during cocaine self-administration sessions in which an acutely aversive 90 dB white noise was intermittently presented. In a separate group of rats, aversion-induced changes in dopamine were measured in an escape design in which operant responses terminated aversive white noise. Aversive white noise significantly reduced NAcC dopamine and increased cocaine self-administration in both male and female rats. The same relationship was observed in the escape design, in which white noise reduced dopamine and promoted escape attempts. In both designs, the magnitude of the dopamine reduction predicted behavioral performance. While prior research demonstrated that pharmacologically reduced dopamine signaling can promote intake, this report demonstrates that this physiological mechanism is naturally engaged by aversive environmental stimuli and generalizable to non-drug contexts. These findings illustrate a common physiological signature in response to aversion that may promote both adaptive and maladaptive behavior.

Neurobiology and systems biology of stress resilience

Stress resilience is the phenomenon that some people maintain their mental health despite exposure to adversity or show only temporary impairments followed by quick recovery. Resilience research attempts to unravel the factors and mechanisms that make resilience possible and to harness its insights for the development of preventative interventions in individuals at risk for acquiring stress-related dysfunctions. Biological resilience research has been lagging behind the psychological and social sciences but has seen a massive surge in recent years. At the same time, progress in this field has been hampered by methodological challenges related to finding suitable operationalizations and study designs, replicating findings, and modeling resilience in animals. We embed a review of behavioral, neuroimaging, neurobiological, and systems biological findings in adults in a critical methods discussion. We find preliminary evidence that hippocampus-based pattern separation and prefrontal-based cognitive control functions protect against the development of pathological fears in the aftermath of singular, event-type stressors [as found in fear-related disorders, including simpler forms of posttraumatic stress disorder (PTSD)] by facilitating the perception of safety. Reward system-based pursuit and savoring of positive reinforcers appear to protect against the development of more generalized dysfunctions of the anxious-depressive spectrum resulting from more severe or longer-lasting stressors (as in depression, generalized or comorbid anxiety, or severe PTSD). Links between preserved functioning of these neural systems under stress and neuroplasticity, immunoregulation, gut microbiome composition, and integrity of the gut barrier and the blood-brain barrier are beginning to emerge. On this basis, avenues for biological interventions are pointed out.

Prefrontal Dopamine in Flexible Adaptation to Environmental Changes: A Game for Two Players

Deficits in cognitive flexibility have been characterized in affective, anxiety, and neurodegenerative disorders. This paper reviews data, mainly from studies on animal models, that support the existence of a cortical-striatal brain circuit modulated by dopamine (DA), playing a major role in cognitive/behavioral flexibility. Moreover, we reviewed clinical findings supporting misfunctioning of this circuit in Parkinson's disease that could be responsible for some important non-motoric symptoms. The reviewed findings point to a role of catecholaminergic transmission in the medial prefrontal cortex (mpFC) in modulating DA's availability in the nucleus accumbens (NAc), as well as a role of NAc DA in modulating the motivational value of natural and conditioned stimuli. The review section is accompanied by a preliminary experiment aimed at testing weather the extinction of a simple Pavlovian association fosters increased DA transmission in the mpFC and inhibition of DA transmission in the NAc.