A Review of Backward Higher-Order Conditioning: Implications for a Pavlovian Conditioning Analysis of Stimulus Equivalence

Stimulus equivalence (SE) is demonstrated when participants exposed to conditional discrimination training pass tests for reflexivity, symmetry, transitivity, and equivalence (symmetry combined with transitivity). Most theorists attribute the origin of SE to operant processes, but some argue that it results from Pavlovian conditioning. Symmetry is problematic for the latter hypothesis because it seems to require excitatory backward conditioning. However, equivalence tests resemble backward sensory preconditioning (BSP) and backward second-order conditioning (BSOC), two well-established processes. A review of associationistic theories of BSP and BSOC showed that the temporal coding hypothesis (TCH) explains outcomes that other associationistic theories cannot explain (i.e., BSOC and BSP effects after first-order conditioning with delay vs. trace conditioning and forward vs. backward conditioning). The TCH assumes that organisms encode the temporal attributes of stimulus events (e.g., order and interval duration) and this temporal information is integrated across separate phases of training. The TCH seems compatible with a behavioral analysis if direct stimulus control replaces the notion of temporal maps. The TCH perspective does not seem applicable to SE because SE tests are not predictive tasks. This suggests that SE is fundamentally different from BSP and BSOC and a Pavlovian conditioning analysis of SE is inadequate. This conclusion is consistent with previous criticism of a Pavlovian account of SE according to which Pavlovian conditioning cannot be interpreted as stimulus substitution.

The neural substrates of higher-order conditioning: A review

Sensory preconditioned and second-order conditioned responding are each well-documented. The former occurs in subjects (typically rats) exposed to pairings of two relatively neutral stimuli, S2 and S1, and then to pairings of S1 and a motivationally significant event [an unconditioned stimulus (US)]; the latter occurs when the order of these experiences is reversed with rats being exposed to S1-US pairings and then to S2-S1 pairings. In both cases, rats respond when tested with S2 in a manner appropriate to the affective nature of the US, e.g., approach when the US is appetitive and withdrawal when it is aversive. This paper reviews the neural substrates of sensory preconditioning and second-order conditioning. It identifies commonalities and differences in the substrates of these so-called higher-order conditioning protocols and discusses these commonalities/differences in relation to what is learned. In so doing, the review highlights ways in which these types of conditioning enhance our understanding of how the brain encodes and retrieves different types of information to generate appropriate behavior.

Excitotoxic lesions of the perirhinal cortex leave intact rats' gustatory sensory preconditioning

We report findings from two sensory preconditioning experiments in which rats consumed two flavoured solutions, each with two gustatory components (AX and BY), composed of sweet, bitter, salt, and acid elements. After this pre-exposure, rats were conditioned to X by pairing with lithium chloride. Standard sensory preconditioning was observed: Consumption of flavour A was less than that of B. We found that sensory preconditioning was maintained when X was added to A and B. Both experiments included one group of rats with lesions of the perirhinal cortex, which did not influence sensory preconditioning. We discuss our findings in the light of other sensory preconditioning procedures that involve the perirhinal cortex and conclude that differences in experimental variables invoke different mechanisms of sensory preconditioning, which vary in their requirement of the perirhinal cortex.

Not "either-or" but "which-when": A review of the evidence for integration in sensory preconditioning

Sensory preconditioning protocols can be used to assess how the brain integrates memories that share common features. In these protocols, animals are first exposed to pairings of two relatively innocuous stimuli, S2 and S1 (stage 1), and then to pairings of one of these stimuli, S1, with an event of motivational significance (stage 2). Following this training, test presentations of S2 elicit responses appropriate to the motivationally significant event, and these responses are taken to indicate formation of distinct S2-S1 and S1-event memories that are integrated in some way to generate that responding. This paper reviews studies of sensory preconditioning in rats, mice, rabbits and people to determine whether S2-S1 and S1-event memories are integrated through a chaining process at the time of their retrieval (i.e., test presentations of S2 trigger retrieval of S1, and thereby, responses appropriate to the event); or "online" at the time of memory formation (i.e., in stage 2, S1 activates a representation of S2 such that both stimuli associate with the motivationally significant event). It finds that the type of integration is determined by the manner in which stimuli are presented in preconditioning as well as their familiarity. When the stimuli in preconditioning are presented repeatedly and/or serially (i.e., one after the other), the S2-S1 and S1-event memories are chained at the time of retrieval/testing. In contrast, when the stimuli in preconditioning are relatively novel and/or presented simultaneously, the S2-S1 and S1-event memories are integrated online. These statements are related to prior claims regarding the circumstances that promote different types of memory integration and, more generally, mechanisms of information processing in the mammalian brain.

Mechanisms of higher-order learning in the amygdala

Adaptive behaviour is under the potent control of environmental cues. Such cues can acquire value by virtue of their associations with outcomes of motivational significance, be they appetitive or aversive. There are at least two ways through which an environmental cue can acquire value, through first-order and higher-order conditioning. In first-order conditioning, a neutral cue is directly paired with an outcome of motivational significance. In higher-order conditioning, a cue is indirectly associated with motivational events via a directly conditioned first-order stimulus. The present article reviews some of the associations that support learning in first- and higher-order conditioning, as well as the role of the BLA and the molecular mechanisms involved in these two types of learning.

Appetitive and aversive sensory preconditioning in rats is impaired by disruption of the postrhinal cortex

Episodic memory involves binding stimuli and/or events together in time and place. Furthermore, memories become more complex when new experiences influence the meaning of stimuli within the original memory. Thus collectively, complex episodic memory formation and maintenance involves processes such as encoding, storage, retrieval, updating and reconsolidation, which can be studied using animal models of higher-order conditioning. In the present study aversive and appetitive sensory preconditioning paradigms were used to test the hypothesis that the postrhinal cortex (POR), which is a component of the hippocampal memory system, is involved in higher-order conditioning. Drawing on the known role of the POR in contextual learning, Experiment 1 employed a four-phase sensory preconditioning task that involved fear learning and context discrimination in rats with or without permanent lesions of the POR. In parallel, to examine POR function during higher-order conditioning in the absence of a particular spatial arrangement, Experiments 2 and 3 used a three-phase sensory preconditioning paradigm involving phasic stimuli. In Experiment 2, bilateral lesions of the POR were made and in Experiment 3, a chemogenetic approach was used to temporarily inactivate POR neurons during each phase of the paradigm. Evidence of successful sensory preconditioning was observed in sham rats which, during the critical context discrimination test, demonstrated higher levels of freezing behavior when re-exposed to the paired versus the unpaired context, whereas POR-lesioned rats did not. Data from the appetitive sensory preconditioning paradigm also confirmed the hypothesis in that during the critical auditory discrimination test, sham rats showed greater food cup responding following presentations of the paired compared to the unpaired auditory stimulus, whereas POR-lesioned rats did not. Lastly, in Experiment 3, when the POR was inactivated only during preconditioning or only during conditioning, discrimination during the critical auditory test was impaired. Thus, regardless of whether stimulus-stimulus associations were formed between static or phasic stimuli or whether revaluation of the paired stimulus occurred through association with an aversive or an appetitive unconditioned stimulus, the effects were the same; POR lesions disrupted the ability to use higher-order conditioned stimuli to guide prospective behavior.

Manipulating memory associations changes decision-making preferences in a preconditioning task

Memories of past experiences can guide our decisions. Thus, if memories are undermined or distorted, decision making should be affected. Nevertheless, little empirical research has been done to examine the role of memory in reinforcement decision-making. We hypothesized that if memories guide choices in a conditioning decision-making task, then manipulating these memories would result in a change of decision preferences to gain reward. We manipulated participants' memories by providing false feedback that their memory associations were wrong before they made decisions that could lead them to win money. Participants' memory ratings decreased significantly after receiving false feedback. More importantly, we found that false feedback led participants' decision bias to disappear after their memory associations were undermined. Our results suggest that reinforcement decision-making can be altered by false feedback on memories. The results are discussed using memory mechanisms such as spreading activation theories.

Associations and hallucinations in mice and men

Powers et al. (2017, Science, 357(6351), 596-600) report that Pavlovian conditioning can result in the perception of a stimulus in its absence, and that this effect is related to hallucinations outside the laboratory. Considered alongside similar studies in animals, this suggests that associatively produced perceptual processing offers a means to study hallucination-like behaviour in the animal laboratory.

Putting fear in context: Elucidating the role of the retrosplenial cortex in context discrimination in rats

The retrosplenial cortex (RSC), which receives visuo-spatial sensory input and interacts with numerous hippocampal memory system structures, has a well-established role in contextual learning and memory. While it has been demonstrated that RSC function is necessary to learn to recognize a single environment that is directly paired with an aversive event, the role of the RSC in discriminating between two different contexts remains largely unknown. To address this, first order (Experiment 1) and higher order (Experiment 2) fear conditioning paradigms were conducted with sham and RSC-lesioned rats. In Experiment 1 rats were exposed to one context in which shock was delivered and to a second, distinct context without shock. Their ability to discriminate between the contexts was assessed during a re-exposure test. In a second experiment, a new cohort of RSC-lesioned rats was exposed to two contexts made distinct through visual, olfactory and auditory stimuli. In a subsequent conditioning phase, the salience of one of the auditory stimuli was paired to an aversive footshock while the other was not. Similar to Experiment 1, freezing behavior was analyzed upon re-exposure to the contexts in the absence of both the auditory cue and the footshock. The results revealed that RSC is not necessary for rats to use contextual information to successfully discriminate between two contexts under the relatively simple demands involved in this first order conditioning paradigm but that context discrimination is impaired when the processing of complex and/or ambiguous contextual stimuli is required to select appropriate behavioral responses.

Learning theory: a driving force in understanding orbitofrontal function

Since it was demonstrated the orbitofrontal cortex (OFC) is critical to reversal learning, there has been considerable interest in specifying its role in flexible, outcome-guided behavior. Behavioral paradigms from the learning theory tradition, such as outcome devaluation, blocking, Pavlovian to instrumental transfer, and overexpectation have been a driving force in this research. The use of these procedures has revealed OFC's unique role in forming and integrating information about specific features of events and outcomes to drive behavior and learning. These studies highlight the power and importance of learning theory principles in guiding neuroscience research.