The neural substrates of higher-order conditioning: A review

Prefrontal encoding of an internal model for emotional inference

A key function of brain systems mediating emotion is to learn to anticipate unpleasant experiences. Although organisms readily associate sensory stimuli with aversive outcomes, higher-order forms of emotional learning and memory require inference to extrapolate the circumstances surrounding directly experienced aversive events to other indirectly related sensory patterns that were not part of the original experience. This type of learning requires internal models of emotion, which flexibly track directly experienced and inferred aversive associations. Although the brain mechanisms of simple forms of aversive learning have been well studied in areas such as the amygdala1-4, whether and how the brain forms and represents internal models of emotionally relevant associations are not known5. Here we report that neurons in the rodent dorsomedial prefrontal cortex (dmPFC) encode a flexible internal model of emotion by linking sensory stimuli in the environment with aversive events, whether they were directly or indirectly associated with that experience. These representations form through a multi-step encoding mechanism involving recruitment and stabilization of dmPFC cells that support inference. Although dmPFC population activity encodes all salient associations, dmPFC neurons projecting to the amygdala specifically represent and are required to express inferred associations. Together, these findings reveal how internal models of emotion are encoded in the dmPFC to regulate subcortical systems for recall of inferred emotional memories.

Functional dissection of a neuronal brain circuit mediating higher-order associative learning

A central feature characterizing the neural architecture of many species' brains is their capacity to form associative chains through learning. In elementary forms of associative learning, stimuli coinciding with reward or punishment become attractive or repulsive. Notably, stimuli previously learned as attractive or repulsive can themselves serve as reinforcers, establishing a cascading effect whereby they become associated with additional stimuli. When this iterative process is perpetuated, it results in higher-order associations. Here, we use odor conditioning in Drosophila and computational modeling to dissect the architecture of neuronal networks underlying higher-order associative learning. We show that the responsible circuit, situated in the mushroom bodies of the brain, is characterized by parallel processing of odor information and by recurrent excitatory and inhibitory feedback loops that empower odors to gain control over the dopaminergic valence-signaling system. Our findings establish a paradigmatic framework of a neuronal circuit diagram enabling the acquisition of associative chains.

Integrating past experiences

New results help address a longstanding debate regarding which learning strategies allow animals to anticipate negative events based on past associations between sensory stimuli.

Integration of sensory and fear memories in the rat medial temporal lobe

Wong et al., 2019 used a sensory preconditioning protocol to examine how sensory and fear memories are integrated in the rat medial temporal lobe. In this protocol, rats integrate a sound-light (sensory) memory that forms in stage 1 with a light-shock (fear) memory that forms in stage 2 to generate fear responses (freezing) across test presentations of the sound in stage 3. Here, we advance this research by showing that (1) how/when rats integrate the sound-light and light-shock memories (online in stage 2 or at test in stage 3) changes with the number of sound-light pairings in stage 1; and (2) regardless of how/when it occurs, the integration requires communication between two regions of the medial temporal lobe: the perirhinal cortex and basolateral amygdala complex. Thus, 'event familiarity' determines how/when sensory and fear memories are integrated but not the circuitry by which the integration occurs: this remains the same.

Parabrachial Calca neurons mediate second-order conditioning

Learning to associate cues, both directly and indirectly, with biologically significant events is essential for survival. Second-order conditioning (SOC) involves forming an association between a previously reinforced conditioned stimulus (CS1) and a new conditioned stimulus (CS2) without the presence of an unconditioned stimulus (US). The neural substrates mediating SOC, however, remain unclear. Parabrachial Calca neurons, which react to the noxious US, also respond to a CS after pairing with a US, suggesting that Calca neurons mediate SOC. We established an aversive SOC behavioral paradigm in mice and monitored Calca neuron activity via single-cell calcium imaging during conditioning and subsequent recall phases. These neurons were activated by both CS1 and CS2 after SOC. Chemogenetically inhibiting Calca neurons during CS1-CS2 pairing attenuated SOC. Thus, reactivation of the US pathway by a learned CS plays an important role in forming the association between the old and a new CS, promoting the formation of second-order memories.

The selective D3Receptor antagonist VK4-116 reverses loss of insight caused by self-administration of cocaine in rats

Chronic psychostimulant use causes long-lasting changes to neural and cognitive function that persist after long periods of abstinence. As cocaine users transition from drug use to abstinence, a parallel transition from hyperactivity to hypoactivity has been found in orbitofrontal-striatal glucose metabolism and striatal D2/D3-receptor activity. Targeting these changes pharmacologically, using highly selective dopamine D3-receptor (D3R) antagonists and partial agonists, has shown promise in reducing drug-taking, and attenuating relapse in animal models of cocaine and opioid use disorder. However, much less attention has been paid to treating the loss of insight, operationalized as the inability to infer likely outcomes, associated with chronic psychostimulant use. Here we tested the selective D3R antagonist VK4-116 as a treatment for this loss in rats with a prior history of cocaine use. Male and female rats were first trained to self-administer cocaine or a sucrose liquid for 2 weeks. After 4 weeks of abstinence, performance was assessed using a sensory preconditioning (SPC) learning paradigm. Rats were given VK4-116 (15 mg/kg, i.p.) or vehicle 30 min prior to each SPC training session, thus creating four drug-treatment groups: sucrose-vehicle, sucrose-VK4-116, cocaine-vehicle, cocaine-VK4-116. The control groups (sucrose-vehicle, sucrose-VK4-116) showed normal sensory preconditioning, whereas cocaine use (cocaine-vehicle) selectively disrupted responding to the preconditioned cue, an effect that was reversed in the cocaine-VK4-116 group, which demonstrating responding to the preconditioned cue at levels comparable to controls. These preclinical findings demonstrate that highly selective dopamine D3R antagonists, particularly VK4-116, can reverse the long-term negative behavioral consequences of cocaine use.

Divergent recruitment of developmentally defined neuronal ensembles supports memory dynamics

Memories are dynamic constructs whose properties change with time and experience. The biological mechanisms underpinning these dynamics remain elusive, particularly concerning how shifts in the composition of memory-encoding neuronal ensembles influence the evolution of a memory over time. By targeting developmentally distinct subpopulations of principal neurons, we discovered that memory encoding resulted in the concurrent establishment of multiple memory traces in the mouse hippocampus. Two of these traces were instantiated in subpopulations of early- and late-born neurons and followed distinct reactivation trajectories after encoding. The divergent recruitment of these subpopulations underpinned gradual reorganization of memory ensembles and modulated memory persistence and plasticity across multiple learning episodes. Thus, our findings reveal profound and intricate relationships between ensemble dynamics and the progression of memories over time.

Semantic structures facilitate threat memory integration throughout the medial temporal lobe and medial prefrontal cortex

Emotional experiences can profoundly impact our conceptual model of the world, modifying how we represent and remember a host of information even indirectly associated with that experienced in the past. Yet, how a new emotional experience infiltrates and spreads across pre-existing semantic knowledge structures (e.g., categories) is unknown. We used a modified aversive sensory preconditioning paradigm in fMRI (n = 35) to investigate whether threat memories integrate with a pre-established category to alter the representation of the entire category. We observed selective but transient changes in the representation of conceptually related items in the amygdala, medial prefrontal cortex, and occipitotemporal cortex following threat conditioning to a simple cue (geometric shape) pre-associated with a different, but related, set of category exemplars. These representational changes persisted beyond 24 h in the hippocampus and perirhinal cortex. Reactivation of the semantic category during threat conditioning, combined with activation of the hippocampus or medial prefrontal cortex, was predictive of subsequent amygdala reactivity toward novel category members at test. This provides evidence for online integration of emotional experiences into semantic categories, which then promotes threat generalization. Behaviorally, threat conditioning by proxy selectively and retroactively enhanced recognition memory and increased the perceived typicality of the semantic category indirectly associated with threat. These findings detail a complex route through which new emotional learning generalizes by modifying semantic structures built up over time and stored in memory as conceptual knowledge.

What is Learned Determines How Pavlovian Conditioned Fear is Consolidated in the Brain

Activity in the basolateral amygdala complex (BLA) is needed to encode fears acquired through contact with both innate sources of danger (i.e., things that are painful) and learned sources of danger (e.g., being threatened with a gun). However, within the BLA, the molecular processes required to consolidate the two types of fear are not the same: protein synthesis is needed to consolidate the first type of fear (so-called first-order fear) but not the latter (so-called second-order fear). The present study examined why first- and second-order fears differ in this respect. Specifically, it used a range of conditioning protocols in male and female rats, and assessed the effects of a BLA infusion of the protein synthesis inhibitor, cycloheximide, on first- and second-order conditioned fear. The results revealed that the differential protein synthesis requirements for consolidation of first- and second-order fears reflect differences in what is learned in each case. Protein synthesis in the BLA is needed to consolidate fears that result from encoding of relations between stimuli in the environment (stimulus-stimulus associations, typical for first-order fear) but is not needed to consolidate fears that form when environmental stimuli associate directly with fear responses emitted by the animal (stimulus-response associations, typical for second-order fear). Thus, the substrates of Pavlovian fear conditioning in the BLA depend on the way that the environment impinges upon the animal. This is discussed with respect to theories of amygdala function in Pavlovian fear conditioning, and ways in which stimulus-response associations might be consolidated in the brain.

The cognitive (lateral) hypothalamus

Despite the physiological complexity of the hypothalamus, its role is typically restricted to initiation or cessation of innate behaviors. For example, theories of lateral hypothalamus argue that it is a switch to turn feeding 'on' and 'off' as dictated by higher-order structures that render when feeding is appropriate. However, recent data demonstrate that the lateral hypothalamus is critical for learning about food-related cues. Furthermore, the lateral hypothalamus opposes learning about information that is neutral or distal to food. This reveals the lateral hypothalamus as a unique arbitrator of learning capable of shifting behavior toward or away from important events. This has relevance for disorders characterized by changes in this balance, including addiction and schizophrenia. Generally, this suggests that hypothalamic function is more complex than increasing or decreasing innate behaviors.

How common is a common error term? The rules that govern associative learning in sensory preconditioning and second-order conditioning

In standard (first-order) Pavlovian conditioning protocols, pairings of an initially neutral conditioned stimulus (CS) and a biologically significant unconditioned stimulus (US) result in the formation of a CS-US association. The strength of this association is theoretically regulated by prediction error: specifically, the difference between the total level of conditioning supported by the US and the degree to which it is predicted by all stimuli present (i.e., a common error term). In higher-order conditioning protocols (e.g., sensory preconditioning and second-order conditioning), a Pavlovian CS is used to condition responses to other stimuli with which it is paired. At present, it is unknown whether error-correction processes regulate associative learning in higher-order conditioning and, if so, whether these processes are the same as those that regulate formation of a CS-US association in first-order conditioning. Here we review studies that have provided findings relevant to this question: specifically, studies that have examined blocking and/or inhibitory learning in sensory preconditioning and second-order conditioning. These studies show that: (1) animals can form inhibitory associations between relatively neutral sensory stimuli; (2) the learning that occurs in sensory preconditioning and second-order conditioning can be blocked; and, finally, (3) a first-order CS can block conditioning to a second-order CS, and vice versa. The findings are taken to imply that a common error term regulates associative learning in higher-order conditioning, just as it regulates associative learning in first-order conditioning. They are discussed with respect to the nature of the error signal that underlies conditioning and future work that is needed to advance our understanding of the rules that govern different types of learning.

Aberrant orbitofrontal cortical activation interferes with encoding of Pavlovian fear conditioning

Obsessive-compulsive disorder (OCD) patients were usually found with the hyper-activation of the orbitofrontal cortex (OFC) and a deficit in fear extinction learning. The OFC can be subdivided into the lateral OFC (lOFC) and the medial OFC (mOFC). Previous studies have suggested that both subregions are involved in the modulation of negative emotions. However, how aberrant activation of the OFC interacts with the encoding of Pavlovian fear remains unknown. In this study, the lOFC or the mOFC was pharmacologically activated or inactivated before the fear conditioning on Day 1, followed by a context test on Day 2 and a tone test on Day 3 in male Long-Evans rats. We found that for the animals that underwent fear conditioning under aberrant activation of either the lOFC or the mOFC, they showed normal within-session fear expression. However, the acquisition/consolidation of contextual fear was impaired under mOFC activation, while the acquisition/consolidation of cued fear was impaired under either the lOFC or the mOFC activation, in that these animals showed lower freezing compared to controls during the retrieval test. On the other hand, for the animals that underwent fear conditioning under inactivation of either the lOFC or the mOFC, they showed normal within-session fear expression, as well as intact encoding of both the contextual and cued fear. Together, our results suggested that the OFC was not actively engaged in the acquisition of Pavlovian fear conditioning, but aberrant activation of the OFC impaired fear learning.