Understanding Associative Learning Through Higher-Order Conditioning

Prefrontal encoding of an internal model for emotional inference

A key function of brain systems mediating emotion is to learn to anticipate unpleasant experiences. Although organisms readily associate sensory stimuli with aversive outcomes, higher-order forms of emotional learning and memory require inference to extrapolate the circumstances surrounding directly experienced aversive events to other indirectly related sensory patterns that were not part of the original experience. This type of learning requires internal models of emotion, which flexibly track directly experienced and inferred aversive associations. Although the brain mechanisms of simple forms of aversive learning have been well studied in areas such as the amygdala1-4, whether and how the brain forms and represents internal models of emotionally relevant associations are not known5. Here we report that neurons in the rodent dorsomedial prefrontal cortex (dmPFC) encode a flexible internal model of emotion by linking sensory stimuli in the environment with aversive events, whether they were directly or indirectly associated with that experience. These representations form through a multi-step encoding mechanism involving recruitment and stabilization of dmPFC cells that support inference. Although dmPFC population activity encodes all salient associations, dmPFC neurons projecting to the amygdala specifically represent and are required to express inferred associations. Together, these findings reveal how internal models of emotion are encoded in the dmPFC to regulate subcortical systems for recall of inferred emotional memories.

Integrating past experiences

New results help address a longstanding debate regarding which learning strategies allow animals to anticipate negative events based on past associations between sensory stimuli.

Parabrachial Calca neurons mediate second-order conditioning

Learning to associate cues, both directly and indirectly, with biologically significant events is essential for survival. Second-order conditioning (SOC) involves forming an association between a previously reinforced conditioned stimulus (CS1) and a new conditioned stimulus (CS2) without the presence of an unconditioned stimulus (US). The neural substrates mediating SOC, however, remain unclear. Parabrachial Calca neurons, which react to the noxious US, also respond to a CS after pairing with a US, suggesting that Calca neurons mediate SOC. We established an aversive SOC behavioral paradigm in mice and monitored Calca neuron activity via single-cell calcium imaging during conditioning and subsequent recall phases. These neurons were activated by both CS1 and CS2 after SOC. Chemogenetically inhibiting Calca neurons during CS1-CS2 pairing attenuated SOC. Thus, reactivation of the US pathway by a learned CS plays an important role in forming the association between the old and a new CS, promoting the formation of second-order memories.

Divergent recruitment of developmentally defined neuronal ensembles supports memory dynamics

Memories are dynamic constructs whose properties change with time and experience. The biological mechanisms underpinning these dynamics remain elusive, particularly concerning how shifts in the composition of memory-encoding neuronal ensembles influence the evolution of a memory over time. By targeting developmentally distinct subpopulations of principal neurons, we discovered that memory encoding resulted in the concurrent establishment of multiple memory traces in the mouse hippocampus. Two of these traces were instantiated in subpopulations of early- and late-born neurons and followed distinct reactivation trajectories after encoding. The divergent recruitment of these subpopulations underpinned gradual reorganization of memory ensembles and modulated memory persistence and plasticity across multiple learning episodes. Thus, our findings reveal profound and intricate relationships between ensemble dynamics and the progression of memories over time.

Semantic structures facilitate threat memory integration throughout the medial temporal lobe and medial prefrontal cortex

Emotional experiences can profoundly impact our conceptual model of the world, modifying how we represent and remember a host of information even indirectly associated with that experienced in the past. Yet, how a new emotional experience infiltrates and spreads across pre-existing semantic knowledge structures (e.g., categories) is unknown. We used a modified aversive sensory preconditioning paradigm in fMRI (n = 35) to investigate whether threat memories integrate with a pre-established category to alter the representation of the entire category. We observed selective but transient changes in the representation of conceptually related items in the amygdala, medial prefrontal cortex, and occipitotemporal cortex following threat conditioning to a simple cue (geometric shape) pre-associated with a different, but related, set of category exemplars. These representational changes persisted beyond 24 h in the hippocampus and perirhinal cortex. Reactivation of the semantic category during threat conditioning, combined with activation of the hippocampus or medial prefrontal cortex, was predictive of subsequent amygdala reactivity toward novel category members at test. This provides evidence for online integration of emotional experiences into semantic categories, which then promotes threat generalization. Behaviorally, threat conditioning by proxy selectively and retroactively enhanced recognition memory and increased the perceived typicality of the semantic category indirectly associated with threat. These findings detail a complex route through which new emotional learning generalizes by modifying semantic structures built up over time and stored in memory as conceptual knowledge.

Integrating and fragmenting memories under stress and alcohol

Stress can powerfully influence the way we form memories, particularly the extent to which they are integrated or situated within an underlying spatiotemporal and broader knowledge architecture. These different representations in turn have significant consequences for the way we use these memories to guide later behavior. Puzzlingly, although stress has historically been argued to promote fragmentation, leading to disjoint memory representations, more recent work suggests that stress can also facilitate memory binding and integration. Understanding the circumstances under which stress fosters integration will be key to resolving this discrepancy and unpacking the mechanisms by which stress can shape later behavior. Here, we examine memory integration at multiple levels: linking together the content of an individual experience, threading associations between related but distinct events, and binding an experience into a pre-existing schema or sense of causal structure. We discuss neural and cognitive mechanisms underlying each form of integration as well as findings regarding how stress, aversive learning, and negative affect can modulate each. In this analysis, we uncover that stress can indeed promote each level of integration. We also show how memory integration may apply to understanding effects of alcohol, highlighting extant clinical and preclinical findings and opportunities for further investigation. Finally, we consider the implications of integration and fragmentation for later memory-guided behavior, and the importance of understanding which type of memory representation is potentiated in order to design appropriate interventions.

Lipopolysaccharide (LPS) attenuates the primary conditioning of lithium chloride (LiCl)-induced context aversion but not the secondary conditioning of context aversion or taste avoidance

A first-order association can be formed between toxin-induced nausea and a context, as well as nausea and a taste cue. However, comparatively little is understood about second-order associations. The present study examined if the bacterial endotoxin, LPS, could impair the first- and second-order conditioning of context aversion (anticipatory nausea paradigm) and subsequent conditioned taste avoidance (two-bottle task). Adult male Long Evans rats were treated with LiCl (127 mg/kg, intraperitoneal [i.p.]) or vehicle control (NaCl) and then exposed to a distinct context for 4 first-order conditioning trials. LPS (200 μg/kg, i.p.) or NaCl were administered 24 h after each trial. Seventy-two h after the final first-order conditioning trial, rats underwent 2 second-order conditioning trials where they were treated with 2% saccharin (i.p.) and then exposed to the same context. Twenty-four h after the final second-order conditioning trial, rats were tested in a two-bottle task (2 trials), where they were given a choice between water and a palatable 0.2% saccharin solution. LiCl-treated rats demonstrated a context aversion by the 3rd conditioning trial in the anticipatory nausea paradigm. Rats previously exposed to LiCl also displayed a conditioned taste avoidance of saccharin within the two-bottle task. LPS attenuated first-order context aversion but did not alter either second-order context aversion or conditioned taste avoidance in the two-bottle task. This study demonstrated that a secondary association formed within an aversive context could result in a conditioned taste avoidance. Further, LPS may be able to attenuate primary conditioning, but not secondary conditioning.

An integrative systems biology view of host-pathogen interactions: The regulation of immunity and homeostasis is concomitant, flexible, and smart

The systemic bio-organization of humans and other mammals is essentially "preprogrammed", and the basic interacting units, the cells, can be crudely mapped into discrete sets of developmental lineages and maturation states. Over several decades, however, and focusing on the immune system, we and others invoked evidence - now overwhelming - suggesting dynamic acquisition of cellular properties and functions, through tuning, re-networking, chromatin remodeling, and adaptive differentiation. The genetically encoded "algorithms" that govern the integration of signals and the computation of new states are not fully understood but are believed to be "smart", designed to enable the cells and the system to discriminate meaningful perturbations from each other and from "noise". Cellular sensory and response properties are shaped in part by recurring temporal patterns, or features, of the signaling environment. We compared this phenomenon to associative brain learning. We proposed that interactive cell learning is subject to selective pressures geared to performance, allowing the response of immune cells to injury or infection to be progressively coordinated with that of other cell types across tissues and organs. This in turn is comparable to supervised brain learning. Guided by feedback from both the tissue itself and the neural system, resident or recruited antigen-specific and innate immune cells can eradicate a pathogen while simultaneously sustaining functional homeostasis. As informative memories of immune responses are imprinted both systemically and within the targeted tissues, it is desirable to enhance tissue preparedness by incorporating attenuated-pathogen vaccines and informed choice of tissue-centered immunomodulators in vaccination schemes. Fortunately, much of the "training" that a living system requires to survive and function in the face of disturbances from outside or within is already incorporated into its design, so it does not need to deep-learn how to face a new challenge each time from scratch. Instead, the system learns from experience how to efficiently select a built-in strategy, or a combination of those, and can then use tuning to refine its organization and responses. Efforts to identify and therapeutically augment such strategies can take advantage of existing integrative modeling approaches. One recently explored strategy is boosting the flux of uninfected cells into and throughout an infected tissue to rinse and replace the infected cells.