What is Learned Determines How Pavlovian Conditioned Fear is Consolidated in the Brain

Activity in the basolateral amygdala complex (BLA) is needed to encode fears acquired through contact with both innate sources of danger (i.e., things that are painful) and learned sources of danger (e.g., being threatened with a gun). However, within the BLA, the molecular processes required to consolidate the two types of fear are not the same: protein synthesis is needed to consolidate the first type of fear (so-called first-order fear) but not the latter (so-called second-order fear). The present study examined why first- and second-order fears differ in this respect. Specifically, it used a range of conditioning protocols in male and female rats, and assessed the effects of a BLA infusion of the protein synthesis inhibitor, cycloheximide, on first- and second-order conditioned fear. The results revealed that the differential protein synthesis requirements for consolidation of first- and second-order fears reflect differences in what is learned in each case. Protein synthesis in the BLA is needed to consolidate fears that result from encoding of relations between stimuli in the environment (stimulus-stimulus associations, typical for first-order fear) but is not needed to consolidate fears that form when environmental stimuli associate directly with fear responses emitted by the animal (stimulus-response associations, typical for second-order fear). Thus, the substrates of Pavlovian fear conditioning in the BLA depend on the way that the environment impinges upon the animal. This is discussed with respect to theories of amygdala function in Pavlovian fear conditioning, and ways in which stimulus-response associations might be consolidated in the brain.

Minimal circuit motifs for second-order conditioning in the insect mushroom body

In well-established first-order conditioning experiments, the concurrence of a sensory cue with reinforcement forms an association, allowing the cue to predict future reinforcement. In the insect mushroom body, a brain region central to learning and memory, such associations are encoded in the synapses between its intrinsic and output neurons. This process is mediated by the activity of dopaminergic neurons that encode reinforcement signals. In second-order conditioning, a new sensory cue is paired with an already established one that presumably activates dopaminergic neurons due to its predictive power of the reinforcement. We explored minimal circuit motifs in the mushroom body for their ability to support second-order conditioning using mechanistic models. We found that dopaminergic neurons can either be activated directly by the mushroom body's intrinsic neurons or via feedback from the output neurons via several pathways. We demonstrated that the circuit motifs differ in their computational efficiency and robustness. Beyond previous research, we suggest an additional motif that relies on feedforward input of the mushroom body intrinsic neurons to dopaminergic neurons as a promising candidate for experimental evaluation. It differentiates well between trained and novel stimuli, demonstrating robust performance across a range of model parameters.

The Basolateral Amygdala: The Core of a Network for Threat Conditioning, Extinction, and Second-Order Threat Conditioning

Threat conditioning, extinction, and second-order threat conditioning studied in animal models provide insight into the brain-based mechanisms of fear- and anxiety-related disorders and their treatment. Much attention has been paid to the role of the basolateral amygdala (BLA) in such processes, an overview of which is presented in this review. More recent evidence suggests that the BLA serves as the core of a greater network of structures in these forms of learning, including associative and sensory cortices. The BLA is importantly regulated by hippocampal and prefrontal inputs, as well as by the catecholaminergic neuromodulators, norepinephrine and dopamine, that may provide important prediction-error or learning signals for these forms of learning. The sensory cortices may be required for the long-term storage of threat memories. As such, future research may further investigate the potential of the sensory cortices for the long-term storage of extinction and second-order conditioning memories.

How common is a common error term? The rules that govern associative learning in sensory preconditioning and second-order conditioning

In standard (first-order) Pavlovian conditioning protocols, pairings of an initially neutral conditioned stimulus (CS) and a biologically significant unconditioned stimulus (US) result in the formation of a CS-US association. The strength of this association is theoretically regulated by prediction error: specifically, the difference between the total level of conditioning supported by the US and the degree to which it is predicted by all stimuli present (i.e., a common error term). In higher-order conditioning protocols (e.g., sensory preconditioning and second-order conditioning), a Pavlovian CS is used to condition responses to other stimuli with which it is paired. At present, it is unknown whether error-correction processes regulate associative learning in higher-order conditioning and, if so, whether these processes are the same as those that regulate formation of a CS-US association in first-order conditioning. Here we review studies that have provided findings relevant to this question: specifically, studies that have examined blocking and/or inhibitory learning in sensory preconditioning and second-order conditioning. These studies show that: (1) animals can form inhibitory associations between relatively neutral sensory stimuli; (2) the learning that occurs in sensory preconditioning and second-order conditioning can be blocked; and, finally, (3) a first-order CS can block conditioning to a second-order CS, and vice versa. The findings are taken to imply that a common error term regulates associative learning in higher-order conditioning, just as it regulates associative learning in first-order conditioning. They are discussed with respect to the nature of the error signal that underlies conditioning and future work that is needed to advance our understanding of the rules that govern different types of learning.

Understanding Associative Learning Through Higher-Order Conditioning

Associative learning is often considered to require the physical presence of stimuli in the environment in order for them to be linked. This, however, is not a necessary condition for learning. Indeed, associative relationships can form between events that are never directly paired. That is, associative learning can occur by integrating information across different phases of training. Higher-order conditioning provides evidence for such learning through two deceptively similar designs – sensory preconditioning and second-order conditioning. In this review, we detail the procedures and factors that influence learning in these designs, describe the associative relationships that can be acquired, and argue for the importance of this knowledge in studying brain function.

Reinstatement of Cortical Outcome Representations during Higher-Order Learning

Naturalistic learning scenarios are characterized by infrequent experience of external feedback to guide behavior. Higher-order learning mechanisms like second-order conditioning (SOC) may allow stimuli that were never experienced together with reinforcement to acquire motivational value. Despite its explanatory potential for real-world learning, surprisingly little is known about the neural mechanism underlying such associative transfer of value in SOC. Here, we used multivariate cross-session, cross-modality searchlight classification on functional magnetic resonance imaging data obtained from humans during SOC. We show that visual first-order conditioned stimuli (CS) reinstate cortical patterns representing previously paired gustatory outcomes in the lateral orbitofrontal cortex (OFC). During SOC, this OFC region showed increased functional covariation with amygdala, where neural pattern similarity between second-order CS and outcomes increased from early to late stages of SOC. Our data suggest a mechanism by which motivational value is conferred to stimuli that were never paired with reinforcement.

Second-Order Conditioning in Humans

In contrast to the large body of work demonstrating second-order conditioning (SOC) in non-human animals, the evidence for SOC in humans is scant. In this review, I examine the existing literature and suggest theoretical and procedural explanations for why SOC has been so elusive in humans. In particular, I discuss potential interactions with conditioned inhibition, whether SOC is rational, and propose critical parameters needed to obtain the effect. I conclude that SOC is a real but difficult phenomenon to obtain in humans, and suggest directions for future research.

Cortical Contributions to Higher-Order Conditioning: A Review of Retrosplenial Cortex Function

In higher-order conditioning paradigms, such as sensory preconditioning or second-order conditioning, discrete (e.g., phasic) or contextual (e.g., static) stimuli can gain the ability to elicit learned responses despite never being directly paired with reinforcement. The purpose of this mini-review is to examine the neuroanatomical basis of high-order conditioning, by selectively reviewing research that has examined the role of the retrosplenial cortex (RSC) in sensory preconditioning and second-order conditioning. For both forms of higher-order conditioning, we first discuss the types of associations that may occur and then review findings from RSC lesion/inactivation experiments. These experiments demonstrate a role for the RSC in sensory preconditioning, suggesting that this cortical region might contribute to higher-order conditioning via the encoding of neutral stimulus-stimulus associations. In addition, we address knowledge gaps, avenues for future research, and consider the contribution of the RSC to higher-order conditioning in relation to related brain structures.

Second-Order Conditioning and Conditioned Inhibition in Different Moments of the Same Training: The Effect of A+ and AX- Trial Number

The feature negative discrimination (A+/AX-) can result in X gaining excitatory properties (second-order conditioning, SOC) or in X gaining inhibitory properties (conditioned inhibition, CI), a challenging finding for most current associative learning theories. Research on the variables that modulate which of these phenomena would occur is scarce but has clearly identified the trial number as an important variable. In the set of experiments presented here, the effect of trial number was assessed in a magazine training task with rats as a function of both the conditioning sessions and the number of A+ and AX- trials per session, holding constant the total number of trials per session. The results indicated that SOC is most likely to be found at the beginning of training when there are many A+ and few AX- trials, and CI (as assessed by a retardation test) is most likely to be found at the end of training when there are few A+ and many AX- trials. Both phenomena were also found at different moments of training when the number of A+ trials was equal to the number of AX- trials. These results cannot be predicted by acquisition-focused associative models but can be predicted by theories that distinguish between learning and performance.

An optogenetic analogue of second-order reinforcement in Drosophila

In insects, odours are coded by the combinatorial activation of ascending pathways, including their third-order representation in mushroom body Kenyon cells. Kenyon cells also receive intersecting input from ascending and mostly dopaminergic reinforcement pathways. Indeed, in Drosophila, presenting an odour together with activation of the dopaminergic mushroom body input neuron PPL1-01 leads to a weakening of the synapse between Kenyon cells and the approach-promoting mushroom body output neuron MBON-11. As a result of such weakened approach tendencies, flies avoid the shock-predicting odour in a subsequent choice test. Thus, increased activity in PPL1-01 stands for punishment, whereas reduced activity in MBON-11 stands for predicted punishment. Given that punishment-predictors can themselves serve as punishments of second order, we tested whether presenting an odour together with the optogenetic silencing of MBON-11 would lead to learned odour avoidance, and found this to be the case. In turn, the optogenetic activation of MBON-11 together with odour presentation led to learned odour approach. Thus, manipulating activity in MBON-11 can be an analogue of predicted, second-order reinforcement.

Individual variability in behavioral flexibility predicts sign-tracking tendency

Sign-tracking rats show heightened sensitivity to food- and drug-associated cues, which serve as strong incentives for driving reward seeking. We hypothesized that this enhanced incentive drive is accompanied by an inflexibility when incentive value changes. To examine this we tested rats in Pavlovian outcome devaluation or second-order conditioning prior to the assessment of sign-tracking tendency. To assess behavioral flexibility we trained rats to associate a light with a food outcome. After the food was devalued by pairing with illness, we measured conditioned responding (CR) to the light during an outcome devaluation probe test. The level of CR during outcome devaluation probe test correlated with the rats' subsequent tracking tendency, with sign-tracking rats failing to suppress CR to the light after outcome devaluation. To assess Pavlovian incentive learning, we trained rats on first-order (CS+, CS-) and second-order (SOCS+, SOCS-) discriminations. After second-order conditioning, we measured CR to the second-order cues during a probe test. Second-order conditioning was observed across all rats regardless of tracking tendency. The behavioral inflexibility of sign-trackers has potential relevance for understanding individual variation in vulnerability to drug addiction.

Cocaine self-administration abolishes associative neural encoding in the nucleus accumbens necessary for higher-order learning

BACKGROUND

Cocaine use is often associated with diminished cognitive function, persisting even after abstinence from the drug. Likely targets for these changes are the core and shell of the nucleus accumbens (NAc), which are critical for mediating the rewarding aspects of drugs of abuse as well as supporting associative learning. To understand this deficit, we recorded neural activity in the NAc of rats with a history of cocaine self-administration or control subjects while they learned Pavlovian first- and second-order associations.

METHODS

Rats were trained for 2 weeks to self-administer intravenous cocaine or water. Later, rats learned a first-order Pavlovian discrimination where a conditioned stimulus (CS)+ predicted food, and a control (CS-) did not. Rats then learned a second-order association where, absent any food reinforcement, a novel cued (SOC+) predicted the CS+ and another (SOC-) predicted the CS-. Electrophysiological recordings were taken during performance of these tasks in the NAc core and shell.

RESULTS

Both control subjects and cocaine-experienced rats learned the first-order association, but only control subjects learned the second-order association. Neural recordings indicated that core and shell neurons encoded task-relevant information that correlated with behavioral performance, whereas this type of encoding was abolished in cocaine-experienced rats.

CONCLUSIONS

The NAc core and shell perform complementary roles in supporting normal associative learning, functions that are impaired after cocaine experience. This impoverished encoding of motivational behavior, even after abstinence from the drug, might provide a key mechanism to understand why addiction remains a chronically relapsing disorder despite repeated attempts at sobriety.

Adolescent but not adult rats exhibit ethanol-mediated appetitive second-order conditioning

BACKGROUND

Adolescent rats are less sensitive to the sedative effects of ethanol than older animals. They also seem to perceive the reinforcing properties of ethanol. However, unlike neonates or infants, ethanol-mediated appetitive behavior is yet to be clearly shown in adolescents. Appetitive ethanol reinforcement was assessed in adolescent (postnatal day 33, P33) and adult rats (P71) through second-order conditioning (SOC).

METHODS

On P32 or P70, animals were intragastrically administered ethanol (0.5 or 2.0 g/kg) paired with intra-oral pulses of sucrose (CS(1), first-order conditioning phase). CS(1) delivery took place either 5-20 (early pairing) or 30-45 minutes (late pairing) following ethanol administration. The time interval between CS(1) exposure and ethanol administration was 240 minutes in unpaired controls. On P33 or P71, animals were presented the CS(1) (second-order conditioning phase) in a distinctive chamber (CS(2), second-order conditioning). Then they were tested for CS(2) preference.

RESULTS

Early and late paired adolescents, but not adults, had greater preference for the CS(2) than controls, a result indicative of ontogenetic variation in ethanol-mediated reinforcement. During the CS(1)-CS(2) associative phase, paired adolescents given 2.0 g/kg ethanol wall-climbed more than controls. Blood and brain ethanol levels associated with the 0.5 and 2.0 g/kg doses at the onset of each conditioning phase did not differ substantially across age, with mean blood ethanol concentration of 38 and 112 mg%.

CONCLUSIONS

These data indicate age-related differences between adolescent and adult rats in terms of sensitivity to ethanol's motivational effects. Adolescents exhibited high sensitivity for ethanol's appetitive effects. These animals also showed ethanol-mediated behavioral activation during the SOC phase. The SOC preparation provides a valuable conditioning model for assessing ethanol's motivational effects across ontogeny.

Neurotoxic lesions of basolateral, but not central, amygdala interfere with Pavlovian second-order conditioning and reinforcer devaluation effects

Considerable evidence suggests that various discrete nuclei within the amygdala complex are critically involved in the assignment of emotional significance or value to events through associative learning. Much of this evidence comes from aversive conditioning procedures. For example, lesions of either basolateral amygdala (ABL) or the central nucleus (CN) interfere with the acquisition or expression of conditioned fear. The present study examined the effects of selective neurotoxic lesions of either ABL or CN on the acquisition of positive incentive value by a conditioned stimulus (CS) with two appetitive Pavlovian conditioning procedures. In second-order conditioning experiments, rats first received light-food pairings intended to endow the light with reinforcing power. The acquired reinforcing power of the light was then measured by examining its ability to serve as a reinforcer for second-order conditioning of a tone when tone-light pairings were given in the absence of food. Acquisition of second-order conditioning was impaired in rats with ABL lesions but not in rats with CN lesions. In reinforcer devaluation procedures, conditioned responding of rats with ABL lesions was insensitive to postconditioning changes in the value of the reinforcer, whereas rats with CN lesions, like normal rats, were able to spontaneously adjust their CRs to the current value of the reinforcer. The results of both test procedures indicate that ABL, but not CN, is part of a system involved in CSs' acquisition of positive incentive value. Together with evidence that identifies a role for CN in certain changes in attentional processing of CSs in conditioning, these results suggest that separate amygdala subsystems contribute to a variety of processes inherent in associative learning.