Stimulus representation in SOP: I. Theoretical rationalization and some implications

More and Less Fear in Serotonin Transporter Knockout Mice

Recent theories suggest that reduced serotonin transporter (5-HTT) function, which increases serotonin (5-HT) levels at the synapse, enhances neural plasticity and affects sensitivity to environmental cues. This may promote learning about emotionally relevant events. However, the boundaries that define such emotional learning remain to be established. This was investigated using 5-HTT knockout (5-HTTKO) mice which provide a model of long-term elevated 5-HT transmission and are associated with increased anxiety. Compared to wild-type controls, 5-HTTKO mice were faster to discriminate between an auditory cue that predicted footshock (CS+) and a cue predicting no footshock (CS-). Notably, this enhanced discrimination performance was driven not by faster learning that the CS+ predicted footshock, but rather by faster learning that the CS- cue signals the absence of footshock and thus provides temporary relief from fear/anxiety. Similarly, 5-HTTKO mice were also faster to reduce their fear of the CS+ cue during subsequent extinction. These findings are consistent with facilitated inhibitory learning that predicts the absence of potential threats in 5-HTTKO mice. However, 5-HTTKO mice also exhibited increased generalisation of fear learning about ambiguous aversive cues in a novel context, different from the training context. Thus, 5-HTTKO mice can exhibit both more and less fear compared to wild-type controls. Taken together, our results support the idea that loss of 5-HTT function, and corresponding increases in synaptic 5-HT availability, may facilitate learning by priming of aversive memories. This both facilitates inhibitory learning for fear memories but also enhances generalisation of fear.

Food reinforcement and habituation to food are processes related to initiation and cessation of eating

An individual bout of eating involves cues to start eating, as well as cues to terminate eating. One process that determines initiation of eating is food reinforcement. Foods with high reinforcing value are also likely to be consumed in greater quantities. Research suggests both cross-sectional and prospective relationships between food reinforcement and obesity, food reinforcement is positively related to energy intake, and energy intake mediates the relationship between food reinforcement and obesity. A process related to cessation of eating is habituation. Habituation is a general behavioral process that describes a reduction in physiological or affective response to a stimulus, or a reduction in the behavioral responding to obtain a stimulus. Repeated exposure to the same food during a meal can result in habituation to that food and a reduction in consumption. Habituation is also cross-sectionally and prospectively related to body weight, as people who habituate slower consume more in a meal and are more overweight. Research from our laboratory has shown that these two processes independently influence eating, as they can account for almost 60% of the variance in ad libitum intake. In addition, habituation phenotypes show reliable relationships with reinforcing value, such that people who habituate faster also find food less reinforcing. Developing a better understanding of cues to start and stop eating is fundamental to understanding how to modify eating behavior. An overview of research on food reinforcement, habituation and food intake for people with a range of weight status and without eating disorders is provided, and ideas about integrating these two processes that are related to initiation and termination of a bout of eating are discussed.

Dissociating Representations of Time and Number in Reinforcement-Rate Learning by Deletion of the GluA1 AMPA Receptor Subunit in Mice

Theories of learning differ in whether they assume that learning reflects the strength of an association between memories or symbolic encoding of the statistical properties of events. We provide novel evidence for symbolic encoding of informational variables by demonstrating that sensitivity to time and number in learning is dissociable. Whereas responding in normal mice was dependent on reinforcement rate, responding in mice that lacked the GluA1 AMPA receptor subunit was insensitive to reinforcement rate and, instead, dependent on the number of times a cue had been paired with reinforcement. This suggests that GluA1 is necessary for weighting numeric information by temporal information in order to calculate reinforcement rate. Sample sizes per genotype varied between seven and 23 across six experiments and consisted of both male and female mice. The results provide evidence for explicit encoding of variables by animals rather than implicit encoding via variations in associative strength.

Cue duration determines response rate but not rate of acquisition of Pavlovian conditioning in mice

The duration of a conditioned stimulus (CS) is a key determinant of Pavlovian conditioning. Rate estimation theory (RET) proposes that reinforcement rate is calculated over cumulative exposure to a cue and the reinforcement rate of a cue, relative to the background reinforcement rate, determines the speed of acquisition of conditioned responding. Consequently, RET predicts that shorter-duration cues require fewer trials to acquisition than longer-duration cues due to the difference in reinforcement rates. We tested this prediction by reanalysing the results of a previously published experiment. Mice received appetitive Pavlovian conditioning of magazine approach behaviour with a 10-s CS and a 40-s CS. Cue duration did not affect the rate at which responding emerged or the rate at which it peaked. The 10-s CS did elicit higher levels of responding than the 40-s CS. These results are not consistent with rate estimation theory. Instead, they are consistent with an associative analysis that assumes that asymptotic levels of responding reflect the balance between increments and decrements in associative strength across cumulative exposure to a cue.

Effects of conditioned stimulus (CS) duration, intertrial interval, and I/T ratio on appetitive Pavlovian conditioning

Pavlovian learning is influenced by at least 2 temporal variables: The time between the onset of the conditioned stimulus (CS) and presentation of the unconditioned stimulus (US), and the time between successive conditioning trials (the intertrial interval [ITI]). Wagner's Sometimes Opponent Process (SOP) model (e.g., 1981) provides a rich account of the effects of varying the absolute durations of CS and ITI. However, other theories have contrastingly emphasized the role of the relative durations of CS (T) and ITI (I). Three experiments with rats used an appetitive conditioning preparation to separate the two approaches. They manipulated absolute values of I and T over a factor of 6 and compared the effect of varying T and I/T by the same factor. Conditioning was indexed by the rate of foodcup entry during training, during common tests conducted later with different combinations of I and T, and with a reinforcers-to-criterion measure. Experiment 1 found that learning with a 10-s CS was superior to that with a 60-s CS when the I/T ratio was the same. There was little evidence of learning with the 60-s CS. Experiment 2 replicated that result and extended it to show that a 60-s CS still supported little conditioned responding when I/T was increased from 12 to 72. Experiment 3 then examined intermediate CS durations between 10-s and 60-s while I/T was controlled. The results support a role for absolute CS duration rather than the I/T ratio. Explanations based on SOP and behavior systems theory are explored. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Remembering the order of serially presented objects: A matter of time?

Remembering the sequence, in which stimuli are encountered or events have occurred, is a key process in episodic memory and can also facilitate recognition memory. Rodents, when presented with a sequence of objects, will explore the object encountered first; yet, whether this behaviour is because the rodents spontaneously encode the order of stimuli presentation or because of relative familiarity or temporal decay is unknown. Here, we tested sequence memory in rats using a series of spontaneous preference tasks. Experiment 1 demonstrated that when rats are presented with a sequence of four objects, with an inter-sample interval of 5 min or 1 h, they preferentially explored the object presented earlier in the list irrespective of the inter-sample interval. We then demonstrated that such memory for order was not affected by increasing or decreasing the inter-sample interval between the middle objects (Experiment 2). Finally, we showed that memory for order is not a function of absolute object familiarity, as animals showed clear discrimination between the objects presented in the sample phases and a novel object, independent of the sample objects' position in the sequence (Experiment 3). These results show that animals are able to encode the order of objects presented in a sequence, and as such temporal order memory is not achieved using the process of relative or absolute familiarity or temporal decay.

The amount of exposure determines generalization in animal perceptual learning using short inter-stimulus intervals

While human and animal perceptual learning (PL) had sometimes yielded similar results, there is evidence of some striking discrepancies. It has been proposed that such differences reflect the existence of multiple species-specific mechanisms, especially regarding to humans. However, it is also possible that those discrepancies are caused by procedural differences. One of the most important differences between PL experiments in humans and laboratory animals is the inter-stimulus interval (ISI) used. In the former, short ISI reliably produces PL, while in the latter reducing the ISI leads to paradoxical results. We report two experiments with rodents to prove that the length of exposure is a key element under such conditions. In the first experiment we replicated the paradoxical results already present in the literature using a short exposure. In a follow up experiment, we increased the exposure trials and obtained normal PL in animals using short ISI. Our results support current associative theories of PL and highlight the impact of procedural differences on this phenomenon.

The effects of stimulus distribution form during trace conditioning

Three experiments examined the effect of distribution form of the trace interval on trace conditioning. In Experiments 1 and 2, two groups of rats were conditioned to a fixed-duration conditioned stimulus (CS) in a trace interval procedure; rats in Group Fix received a fixed-duration trace interval, whereas for rats in Group Var the trace interval was of variable duration. Responding during the CS was higher in Group Var than in Group Fix, whereas during the trace interval this difference in responding reversed—Group Fix showed higher response rates than Group Var. Experiment 3 examined whether the greater response rate observed during the CS in Group Var was due to a performance effect or the acquisition of greater associative strength by the CS. Following trace conditioning, the rats from Experiment 1 underwent a second phase of delay conditioning with the same CS; a 5-s auditory stimulus was presented in compound with the last 5 s of the 15-s CS, and the unconditioned stimulus (US) was delivered at the offset of the CSs. On test with the auditory stimulus alone, subjects in Group Var showed lower response rates during the auditory stimulus than subjects in Group Fix. We interpreted these findings as evidence that the superior responding in Group Var during the CS was a result of it acquiring greater associative strength than in Group Fix.

Exploring the Effect of Stimulus Similarity on the Summation Effect in Causal Learning

Several contemporary models anticipate that the summation effect is modulated by the similarity between the cues forming a compound. Here, we explore this hypothesis in a series of causal learning experiments. Participants were presented with two visual cues that separately predicted a common outcome and later asked for the outcome predicted by the compound of the two cues. Similarity was varied between groups through changes in shape, spatial position, color, configuration, and rotation. In variance with the predictions of these models, we observed similar and strong levels of summation in both groups across all manipulations of similarity. The effect, however, was significantly reduced by manipulations intended to impact assumptions about the causal independence of the cues forming the compound, but this reduction was independent of stimulus similarity. These results are problematic for similarity-based models and can be more readily explained by rational approaches to causal learning.

The development and present status of the SOP model of associative learning

The Sometimes Opponent Processes (SOP) model in its original form was especially calculated to address how expected unconditioned stimulus (US) and conditioned stimulus (CS) are rendered less effective than their novel counterparts in Pavlovian conditioning. Its several elaborations embracing the essential notion have extended the scope of the model to integrate a much greater number of phenomena of Pavlovian conditioning. Here, we trace the development of the model and add further thoughts about its extension and refinement.

A Rescorla-Wagner drift-diffusion model of conditioning and timing

Computational models of classical conditioning have made significant contributions to the theoretic understanding of associative learning, yet they still struggle when the temporal aspects of conditioning are taken into account. Interval timing models have contributed a rich variety of time representations and provided accurate predictions for the timing of responses, but they usually have little to say about associative learning. In this article we present a unified model of conditioning and timing that is based on the influential Rescorla-Wagner conditioning model and the more recently developed Timing Drift-Diffusion model. We test the model by simulating 10 experimental phenomena and show that it can provide an adequate account for 8, and a partial account for the other 2. We argue that the model can account for more phenomena in the chosen set than these other similar in scope models: CSC-TD, MS-TD, Learning to Time and Modular Theory. A comparison and analysis of the mechanisms in these models is provided, with a focus on the types of time representation and associative learning rule used.

Familiarity-based stimulus generalization of conditioned suppression

We report that stimulus novelty/familiarity is able to modulate stimulus generalization and discuss the theoretical implications of novelty/familiarity coding. Rats in Skinner boxes received clicker → shock pairings before generalization testing to a tone. Before clicker training, different groups of rats received preexposure treatments designed to systematically modulate the clicker and the tone's novelty and familiarity. Rats whose preexposure matched novelty/familiarity (i.e., either both or neither clicker and tone were preexposed) showed enhanced suppression to the tone relative to rats whose preexposure mixed novelty/familiarity (i.e., only clicker or tone was preexposed). This was not the result of sensory preconditioning to clicker and tone. (PsycINFO Database Record

A theoretical analysis of transfer of occasion setting: SOP with replaced elements

The available data on occasion setting led Susan Brandon and Allan Wagner (Brandon and Wagner, 1998; Wagner and Brandon, 2001) to formulate what has come to be known as a replaced-elements conception (REM) of context-dependent cues within the SOP model (Wagner, 1981). In the present paper, we review the development of the theory, and show how, with a few congenial assumptions about shared cues, it can address some of the major regularities concerning when the transfer of occasion setting does or does not occur. Among the particular examples are the relatively unique transfers that have been reported to occur between separate serial discriminations and between targets that have been trained with the same versus different reinforcers.

Do long delay conditioned stimuli develop inhibitory properties?

In long-delay conditioning, a long conditioned stimulus (CS) is paired in its final segments with an unconditioned stimulus. With sufficient training, this procedure usually results in conditioned responding being delayed until the final segment of the CS, a pattern of responding known as inhibition of delay. However, there have been no systematic investigations of the associative structure of long delay conditioning, and whether the initial segment of a long delay CS actually becomes inhibitory is debatable. In an appetitive preparation with rat subjects, the initial segment of long delay CS A passed a retardation (Experiment 1a) but not a summation (Experiment 1b) test for conditioned inhibition. Furthermore, retardation was observed only if long delay conditioning and retardation training occurred in the same context (Experiment 2). Thus, the initial segment of a long delay CS appears to share more characteristics with a latent inhibitor than a conditioned inhibitor. Componential theories of conditioning appear best suited to account for these results.

Relative recency influences object-in-context memory

In two experiments rats received training on an object-in-context (OIC) task, in which they received preexposure to object A in context x, followed by exposure to object B in context y. In a subsequent test both A and B are presented in either context x or context y. Usually more exploration is seen of the object that has not previously been paired with the test context, an effect attributed to the ability to remember where an object was encountered. However, in the typical version of this task, object A has also been encountered less recently than object B at test. This is precisely the arrangement in tests of 'relatively recency' (RR), in which more remotely presented objects are explored more than objects experienced more recently. RR could contaminate performance on the OIC task, by enhancing the OIC effect when animals are tested in context y, and masking it when the test is in context x. This possibility was examined in two experiments, and evidence for superior performance in context y was obtained. The implications of this for theoretical interpretations of recognition memory and the procedures used to explore it are discussed.

An associative analysis of object memory

Different aspects of recognition memory in rodents are commonly assessed using variants of the spontaneous object recognition procedure in which animals explore objects that differ in terms of their novelty, recency, or where they have previously been presented. The present article describes three standard variants of this procedure, and outlines a theory of associative learning, SOP which can offer an explanation of performance on all three types of task. The implications of this for theoretical interpretations of recognition memory and the procedures used to explore it are discussed.

Attribution and expression of incentive salience are differentially signaled by ultrasonic vocalizations in rats

During Pavlovian incentive learning, the affective properties of rewards are thought to be transferred to their predicting cues. However, how rewards are represented emotionally in animals is widely unknown. This study sought to determine whether 50-kHz ultrasonic vocalizations (USVs) in rats may signal such a state of incentive motivation to natural, nutritional rewards. To this end, rats learned to anticipate food rewards and, across experiments, the current physiological state (deprived vs. sated), the type of learning mechanism recruited (Pavlovian vs. instrumental), the hedonic properties of UCS (low vs. high palatable food), and the availability of food reward (continued vs. discontinued) were manipulated. Overall, we found that reward-cues elicited 50-kHz calls as they were signaling a putative affective state indicative of incentive motivation in the rat. Attribution and expression of incentive salience, however, seemed not to be an unified process, and could be teased apart in two different ways: 1) under high motivational state (i.e., hunger), the attribution of incentive salience to cues occurred without being expressed at the USVs level, if reward expectations were higher than the outcome; 2) in all experiments when food rewards were devalued by satiation, reward cues were still able to elicit USVs and conditioned anticipatory activity although reward seeking and consumption were drastically weakened. Our results suggest that rats are capable of representing rewards emotionally beyond apparent, immediate physiological demands. These findings may have translational potential in uncovering mechanisms underlying aberrant and persistent motivation as observed in drug addiction, gambling, and eating disorders.

Hippocampal synaptic plasticity, spatial memory and anxiety

Recent studies using transgenic mice lacking NMDA receptors in the hippocampus challenge the long-standing hypothesis that hippocampal long-term potentiation-like mechanisms underlie the encoding and storage of associative long-term spatial memories. However, it may not be the synaptic plasticity-dependent memory hypothesis that is wrong; instead, it may be the role of the hippocampus that needs to be re-examined. We present an account of hippocampal function that explains its role in both memory and anxiety.

Retrieval-induced forgetting in rats

"Retrieval-induced forgetting" in rats was evaluated using a modified spontaneous object recognition test. The test consisted of a sample phase, retrieval or interference phase, and a test phase with 60-min delay period inserted between the phases. Rats were randomly assigned to one of three groups (control, retrieval and interference) and allowed to explore the field in which two different objects (A, B) were placed in the sample phase. In the retrieval phase, two identical objects (B, B), which were the same as one of the objects presented in the sample phase, were placed again. In the interference phase, two identical objects (C, C), which were novel for animals, were placed. In the test phase, two different objects (A, D), one of which was identical to that presented in sample phase (familiar object) and the other was novel, were placed and the time spent exploring each object was analyzed. While the exploration of the novel object was significantly longer than that of the familiar object in rats subjected to the interference phase, rats subjected to the retrieval phase could not discriminate between the familiar and the novel objects at the test phase. These results demonstrate the "retrieval-induced forgetting" phenomenon in a spontaneous object recognition test in rats.

Acquisition of conditioned responding in a multiple schedule depends on the reinforcement's temporal contingency with each stimulus

Forty mice acquired conditioned responses to stimuli presented in a multiple schedule with variable inter-trial intervals (ITIs). In some trials, reinforcement was preceded by a variable conditioned stimulus (CS), while other trials were reinforced following distinctive fixed-duration CS. A third stimulus was presented but never paired with reinforcement. Subjects in five groups experienced ITIs of different durations. Acquisition of responding to each stimulus depended only on the cycle-to-trial ratio (C/T), and thus on the temporal contingency of each stimulus. Acquisition was unaffected by whether CSs were of fixed or variable duration.

Time to rethink the neural mechanisms of learning and memory

Most studies in the neurobiology of learning assume that the underlying learning process is a pairing - dependent change in synaptic strength that requires repeated experience of events presented in close temporal contiguity. However, much learning is rapid and does not depend on temporal contiguity, which has never been precisely defined. These points are well illustrated by studies showing that the temporal relations between events are rapidly learned- even over long delays- and that this knowledge governs the form and timing of behavior. The speed with which anticipatory responses emerge in conditioning paradigms is determined by the information that cues provide about the timing of rewards. The challenge for understanding the neurobiology of learning is to understand the mechanisms in the nervous system that encode information from even a single experience, the nature of the memory mechanisms that can encode quantities such as time, and how the brain can flexibly perform computations based on this information.

Temporal maps in appetitive Pavlovian conditioning

Previous research suggests animals may integrate temporal information into mental representations, or temporal maps. We examined the parameters under which animals integrate temporal information in three appetitive conditioning experiments. In Experiment 1 the temporal relationship between 2 auditory cues was established during sensory preconditioning (SPC). Subsequently, rats were given first order conditioning (FOC) with one of the cues. Results showed integration of the order of cues between the SPC and FOC training phases. In subsequent experiments we tested the hypothesis that quantitative temporal information can be integrated across phases. In Experiment 2, SPC of two short auditory cues superimposed on a longer auditory cue was followed by FOC of either one of the short cues, or of the long cue at different times in the cue. Contrary to our predictions we did not find evidence of integration of temporal information across the phases of the experiment and instead responding to the SPC cues in Experiment 2 appeared to be dominated by generalization from the FOC cues. In Experiment 3 shorter auditory cues were superimposed on a longer duration light cue but with asynchronous onset and offset of the superimposed cues. There is some evidence consistent with the hypothesis that quantitative discrimination of whether reward should be expected during the early or later parts of a cue could be integrated across experiences. However, the pattern of responding within cues was not indicative of integration of quantitative temporal information. Generalization of expected times of reward during FOC seems to be the dominant determinant of within-cue response patterns in these experiments. Consequently, while we clearly demonstrated the integration of temporal order in the modulation of this dominant pattern we did not find strong evidence of integration of precise quantitative temporal information. This article is part of a Special Issue entitled: Associative and Temporal Learning.

The error in total error reduction

Most models of human and animal learning assume that learning is proportional to the discrepancy between a delivered outcome and the outcome predicted by all cues present during that trial (i.e., total error across a stimulus compound). This total error reduction (TER) view has been implemented in connectionist and artificial neural network models to describe the conditions under which weights between units change. Electrophysiological work has revealed that the activity of dopamine neurons is correlated with the total error signal in models of reward learning. Similar neural mechanisms presumably support fear conditioning, human contingency learning, and other types of learning. Using a computational modeling approach, we compared several TER models of associative learning to an alternative model that rejects the TER assumption in favor of local error reduction (LER), which assumes that learning about each cue is proportional to the discrepancy between the delivered outcome and the outcome predicted by that specific cue on that trial. The LER model provided a better fit to the reviewed data than the TER models. Given the superiority of the LER model with the present data sets, acceptance of TER should be tempered.

Exposure to a fearful context during periods of memory plasticity impairs extinction via hyperactivation of frontal-amygdalar circuits

An issue of increasing theoretical and translational importance is to understand the conditions under which learned fear can be suppressed, or even eliminated. Basic research has pointed to extinction, in which an organism is exposed to a fearful stimulus (such as a context) in the absence of an expected aversive outcome (such as a shock). This extinction process results in the suppression of fear responses, but is generally thought to leave the original fearful memory intact. Here, we investigate the effects of extinction during periods of memory lability on behavioral responses and on expression of the immediate-early gene c-Fos within fear conditioning and extinction circuits. Our results show that long-term extinction is impaired when it occurs during time periods during which the memory should be most vulnerable to disruption (soon after conditioning or retrieval). These behavioral effects are correlated with hyperactivation of medial prefrontal cortex and amygdala subregions associated with fear expression rather than fear extinction. These findings demonstrate that behavioral experiences during periods of heightened fear prevent extinction and prolong the conditioned fear response.

Extinction reveals that primary sensory cortex predicts reinforcement outcome

Primary sensory cortices are traditionally regarded as stimulus analysers. However, studies of associative learning-induced plasticity in the primary auditory cortex (A1) indicate involvement in learning, memory and other cognitive processes. For example, the area of representation of a tone becomes larger for stronger auditory memories and the magnitude of area gain is proportional to the degree that a tone becomes behaviorally important. Here, we used extinction to investigate whether 'behavioral importance' specifically reflects a sound's ability to predict reinforcement (reward or punishment) vs. to predict any significant change in the meaning of a sound. If the former, then extinction should reverse area gains as the signal no longer predicts reinforcement. Rats (n = 11) were trained to bar-press to a signal tone (5.0 kHz) for water-rewards, to induce signal-specific area gains in A1. After subsequent withdrawal of reward, A1 was mapped to determine representational areas. Signal-specific area gains, estimated from a previously established brain-behavior quantitative function, were reversed, supporting the 'reinforcement prediction' hypothesis. Area loss was specific to the signal tone vs. test tones, further indicating that withdrawal of reinforcement, rather than unreinforced tone presentation per se, was responsible for area loss. Importantly, the amount of area loss was correlated with the amount of extinction (r = 0.82, P < 0.01). These findings show that primary sensory cortical representation can encode behavioral importance as a signal's value to predict reinforcement, and that the number of cells tuned to a stimulus can dictate its ability to command behavior.

Context-dependent encoding of fear and extinction memories in a large-scale network model of the basal amygdala

The basal nucleus of the amygdala (BA) is involved in the formation of context-dependent conditioned fear and extinction memories. To understand the underlying neural mechanisms we developed a large-scale neuron network model of the BA, composed of excitatory and inhibitory leaky-integrate-and-fire neurons. Excitatory BA neurons received conditioned stimulus (CS)-related input from the adjacent lateral nucleus (LA) and contextual input from the hippocampus or medial prefrontal cortex (mPFC). We implemented a plasticity mechanism according to which CS and contextual synapses were potentiated if CS and contextual inputs temporally coincided on the afferents of the excitatory neurons. Our simulations revealed a differential recruitment of two distinct subpopulations of BA neurons during conditioning and extinction, mimicking the activation of experimentally observed cell populations. We propose that these two subgroups encode contextual specificity of fear and extinction memories, respectively. Mutual competition between them, mediated by feedback inhibition and driven by contextual inputs, regulates the activity in the central amygdala (CEA) thereby controlling amygdala output and fear behavior. The model makes multiple testable predictions that may advance our understanding of fear and extinction memories.

The amygdaloid complex is one of the key brain structures involved in fear-related processes. A typical way to study neural correlates of fear expression (e.g. freezing response) in the amygdala is to perform a fear conditioning paradigm, which yields a conditioned fear response. This response can be reversed by another procedure called fear extinction. Thanks to the experimental approaches to date we have some understanding about the putative roles of specific subnuclei within the amygdala in the formation of these fear and extinction memories. Here, we complement the experimental studies by providing a computational model that addresses the question of how fear and extinction memories are encoded in the amygdala, and specifically, in the basal nucleus (BA). We propose a specific neural mechanism to explain how the BA may integrate information about a salient, conditioned stimulus and the environment, thereby enabling it to switch the state of the animal from low to high fear and vice versa. We also provide possible explanations for various other behavioral findings, such as the recovery of fear after it had been extinguished (renewal). Finally, we make specific, experimentally testable predictions that need to be addressed in future work.

Variations in selective and nonselective prediction error with the negative dimension of schizotypy

Two human associative-learning experiments investigated the relationship between the negative dimension of schizotypy and selective and nonselective prediction-error learning. Experiment 1 demonstrates that individuals low, but not high, on the introvertive anhedonia dimension of schizotypy demonstrate Kamin blocking, which has been taken to reflect the operation of selective learning (Rescorla & Wagner, 1972). In complement, Experiment 2 demonstrates that individuals high, but not low, on the same dimension demonstrate asymmetrical learning about the components of a compound stimulus that differ in their associative history, which has been suggested to reflect the operation of nonselective learning (Rescorla, 2000). The implications of this double dissociation for understanding the nature of the cognitive deficit in schizophrenia and for theories of learning are considered.

Learning related activation of somatosensory cortex by an auditory stimulus recorded with magnetoencephalography

Advances in non-invasive neuroimaging technology now provide a means of directly observing learning within the brain. Classical conditioning serves as an ideal starting point for examining the dynamic expression of learning within the human brain, since this paradigm is well characterized using multiple levels of analysis in a broad range of species. We used MEG to expand the characterization of conditioned responses (CR) recorded from the human brain with a simultaneous examination of their spatial, temporal and spectral properties. We paired an auditory conditioned stimulus (CS+) with a somatosensory unconditioned stimulus (US). We found that when the US was randomly omitted, presentations of CS+ alone, elicited greater desynchronization of beta-band activity in contralateral somatosensory cortex compared to presentations of an auditory stimulus that was never paired with the US (CS-), and compared the CS+ following a non-reinforced extinction session. This differentiation was largest between 150 and 350ms following US omission. We show that cross-modal CRs in the primary sensorimotor system are predominantly characterized by modulation of ongoing cortical oscillations.

The dynamics of conditioning and extinction

Pigeons responded to intermittently reinforced classical conditioning trials with erratic bouts of responding to the conditioned stimulus. Responding depended on whether the prior trial contained a peck, food, or both. A linear persistence-learning model moved pigeons into and out of a response state, and a Weibull distribution for number of within-trial responses governed in-state pecking. Variations of trial and intertrial durations caused correlated changes in rate and probability of responding and in model parameters. A novel prediction--in the protracted absence of food, response rates can plateau above zero--was validated. The model predicted smooth acquisition functions when instantiated with the probability of food but a more accurate jagged learning curve when instantiated with trial-to-trial records of reinforcement. The Skinnerian parameter was dominant only when food could be accelerated or delayed by pecking. These experiments provide a framework for trial-by-trial accounts of conditioning and extinction that increases the information available from the data, permitting such accounts to comment more definitively on complex contemporary models of momentum and conditioning.

Time and Associative Learning

In a basic associative learning paradigm, learning is said to have occurred when the conditioned stimulus evokes an anticipatory response. This learning is widely believed to depend on the contiguous presentation of conditioned and unconditioned stimulus. However, what it means to be contiguous has not been rigorously defined. Here we examine the empirical bases for these beliefs and suggest an alternative view based on the hypothesis that learning about the temporal relationships between events determines the speed of emergence, vigor and form of conditioned behavior. This temporal learning occurs very rapidly and prior to the appearance of the anticipatory response. The temporal relations are learned even when no anticipatory response is evoked. The speed with which an anticipatory response emerges is proportional to the informativeness of the predictive cue (CS) regarding the rate of occurrence of the predicted event (US). This analysis gives an account of what we mean by "temporal pairing" and is in accord with the data on speed of acquisition and basic findings in the cue competition literature. In this account, learning depends on perceiving and encoding temporal regularities rather than stimulus contiguities.

Multiple learning parameters differentially regulate olfactory generalization

Sensory representations depend strongly on the descending regulation of perceptual processing. Generalization among similar stimuli is a fundamental cognitive process that defines the extent of the variance in physical stimulus properties that becomes categorized together and associated with a common contingency, thereby establishing units of meaning. The olfactory system provides an experimentally tractable model system in which to study the interactions of these physical and psychological factors within the framework of their underlying neurophysiological mechanisms. The authors here show that olfactory associative learning systematically regulates gradients of odor generalization. Specifically, increasing odor-reward pairings, odor concentration, or reward quality--each a determinant of associative learning--significantly transformed olfactory generalization gradients, each narrowing the range of variance in odor quality perceived as likely to share the learned contingency of a conditioned odor stimulus. However, differences in the qualitative features of these three transformations suggest that these different determinants of learning are not necessarily theoretically interchangeable. These results demonstrate that odor representations are substantially shaped by experience and descending influences.

Cross-modal transfer of the conditioned eyeblink response during interstimulus interval discrimination training in young rats

Eyeblink classical conditioning (EBC) was observed across a broad developmental period with tasks utilizing two interstimulus intervals (ISIs). In ISI discrimination, two distinct conditioned stimuli (CSs; light and tone) are reinforced with a periocular shock unconditioned stimulus (US) at two different CS-US intervals. Temporal uncertainty is identical in design with the exception that the same CS is presented at both intervals. Developmental changes in conditioning have been reported in each task beyond ages when single-ISI learning is well developed. The present study sought to replicate and extend these previous findings by testing each task at four separate ages. Consistent with previous findings, younger rats (postnatal day--PD23 and 30) trained in ISI discrimination showed evidence of enhanced cross-modal influence of the short CS-US pairing upon long CS conditioning relative to older subjects. ISI discrimination training at PD43-47 yielded outcomes similar to those in adults (PD65-71). Cross-modal transfer effects in this task therefore appear to diminish between PD30 and PD43-47. Comparisons of ISI discrimination with temporal uncertainty indicated that cross-modal transfer in ISI discrimination at the youngest ages did not represent complete generalization across CSs. ISI discrimination undergoes a more protracted developmental emergence than single-cue EBC and may be a more sensitive indicator of developmental disorders involving cerebellar dysfunction.

Distinct neural mechanisms mediate olfactory memory formation at different timescales

Habituation is one of the oldest forms of learning, broadly expressed across sensory systems and taxa. Here, we demonstrate that olfactory habituation induced at different timescales (comprising different odor exposure and intertrial interval durations) is mediated by different neural mechanisms. First, the persistence of habituation memory is greater when mice are habituated on longer timescales. Second, the specificity of the memory (degree of cross-habituation to similar stimuli) also depends on induction timescale. Third, we demonstrate a pharmacological double dissociation between the glutamatergic mechanisms underlying short- and long-timescale odor habituation. LY341495, a class II/III metabotropic glutamate receptor antagonist, blocked habituation only when the induction timescale was short. Conversely, MK-801, an N-methyl-D-aspartate (NMDA) receptor antagonist, prevented habituation only when the timescale was long. Finally, whereas short-timescale odor habituation is mediated within the anterior piriform cortex, infusion of MK-801 into the olfactory bulbs prevented odor habituation only at longer timescales. Thus, we demonstrate two neural mechanisms underlying simple olfactory learning, distinguished by their persistence and specificity, mediated by different olfactory structures and pharmacological effectors, and differentially utilized based solely on the timescale of odor presentation.

Cellular learning theory: theoretical comment on Cole and McNally (2007)

The idea that learning proceeds as a function of the discrepancy (or error) between expected and obtained outcomes is central to many theories of associative learning. However, remarkably little is known about the neurobiological mechanisms that underlie this learning of predictive errors in fear conditioning, a widely used preparation in studies of cellular and molecular mechanisms of memory. In this issue of Behavioral Neuroscience, S. Cole and G. P. McNally demonstrate an important dissociation between the establishment and regulation of predictive error at the cellular level. Their findings have added a level of complexity to currently established views of the function of NMDA and opioid receptors in learning and memory. This commentary discusses some of the implications of these findings for theoretical and neurobiological approaches to memory, as well as current thinking about the cellular circuitry involved in reward learning and drug abuse.

Simulation of habituation to simple and multiple stimuli

Within the psychological literature there are a number of models that reproduce the defining properties of habituation to a single stimulus. However, most of them do not reproduce the phenomenon of dishabituation shown in empirical studies, consisting in the recovery of a stimulus previously habituated upon the appearance of a novel stimulus. The present work offers a model of habituation which, in addition to reproducing the basic properties of habituation to a stimulus, also does so when more than one stimulus is presented, and thus includes the dishabituation phenomenon. This model consists of two functions, one called "activation" and the other "availability", and is tested by means of simulation of the responses in the context of different stimulus patterns. The results of the simulation show a good qualitative fit to the empirical results on the phenomena of habituation, including dishabituation. In addition, the model is suitable for inclusion in associative models that reproduce classical conditioning, which will make it possible in the future to incorporate into these in a simple way the influence that the habituation of each stimulus may have on its association with other stimuli.

Stimulus specificity of concurrent recovery in the rabbit nictitating membrane response

Three experiments demonstrated that, following the extinction of an established conditioned stimulus (CS; e.g., tone), the pairing of an orthogonal stimulus from another modality (e.g., light) with the unconditioned stimulus (US) results in strong recovery of responding to the extinguished CS. This recovery occurred to about an equal degree regardless of whether or not initial training contained unambiguous stimulus-reinforcer relationships--that is, consistent CS-US pairings--or some degree of ambiguity, including intramodal discrimination training, partial reinforcement, or even cross-modal discrimination training (tone vs. light). Experiments 1 and 2 demonstrated that this recovery of responding was largely specific to the extinguished CS, but moderate generalization to other stimuli from the same modality did appear. The results are discussed with reference to alternative mechanisms applicable to learning-dependent generalization between otherwise distinct CSs. These models assume that such generalization is mediated by either a shared response, shared reinforcer, shared context, or shared hidden units within a layered neural network. A specific layered network is proposed to explain the present results as well as other types of savings seen previously in conditioning of the rabbit nictitating membrane response.

Temporally specific extinction of conditioned responses in the rabbit (Oryctolagus cuniculus) nictitating membrane preparation

Extinguishing a conditioned response (CR) has entailed separating the conditioned stimulus (CS) from the unconditioned stimulus (US). This research reveals that elimination of the rabbit (Oryctolagus cuniculus) nictitating membrane response occurred during continuous CS-US pairings. Initial training contained a mixture of 2 CS-US interstimulus intervals (ISIs): 200 ms and 1,200 ms. The CRs showed double peaks, one for each ISI. When 1 ISI was removed, its CR peak showed the hallmarks of extinction: a decline across sessions, spontaneous recovery between sessions, and rapid reacquisition when the absent ISI was reintroduced. These results support real-time models of conditioning that segment the CS into microstimuli while challenging theories that rely on contextual control, US representations, CS processing, and response inhibition.

Temporal difference models describe higher-order learning in humans

The ability to use environmental stimuli to predict impending harm is critical for survival. Such predictions should be available as early as they are reliable. In pavlovian conditioning, chains of successively earlier predictors are studied in terms of higher-order relationships, and have inspired computational theories such as temporal difference learning. However, there is at present no adequate neurobiological account of how this learning occurs. Here, in a functional magnetic resonance imaging (fMRI) study of higher-order aversive conditioning, we describe a key computational strategy that humans use to learn predictions about pain. We show that neural activity in the ventral striatum and the anterior insula displays a marked correspondence to the signals for sequential learning predicted by temporal difference models. This result reveals a flexible aversive learning process ideally suited to the changing and uncertain nature of real-world environments. Taken with existing data on reward learning, our results suggest a critical role for the ventral striatum in integrating complex appetitive and aversive predictions to coordinate behaviour.

Quantitative models of Pavlovian conditioning

Over the last few years, research on learning and memory has become increasingly interdisciplinary. In the past, theories of learning, as a prerogative of psychologists, were generally formulated in purely verbal terms and evaluated exclusively at the behavioral level. At present, scientists are trying to build theories with a quantitative and biological flavor, seeking to embrace more complex behavioral phenomena. Pavlovian conditioning, one of the simplest and ubiquitous forms of learning, is especially suited for this multiple level analysis (i.e., quantitative, neurobiological, and behavioral), in part because of recent discoveries showing a correspondence between behavioral phenomena and associative properties at the cellular and systems levels, and in part because of its well established quantitative theoretical tradition. The present review, examines the mayor quantitative theories of Pavlovian conditioning and the phenomena to which they have been designed to account. In order to provide researchers from different disciplines with a simple guideline about the rationale of the different theoretical choices, all the models are described through a single formalism based on the neural network connectionist perspective.

Stimulus representation in SOP: II. An application to inhibition of delay

The componential extension of SOP accounts for conditioned response (CR) timing in Pavlovian conditioning by assuming that learning accrues with relative independence to stimulus elements that are differentially occasioned during the duration of the conditioned stimulus (CS). SOP, using a competitive learning rule and the assumption that temporal learning emerges via resolution of what is equivalent to an "AX+BX-" discrimination, predicts a progressive increase in the latency of the CR over training, or what Pavlov refer to as "inhibition of delay." Other componential models, which use noncompetitive learning rules, do not predict inhibition of delay. Either type of model makes the prediction indicated, independently of the length of the CS-unconditioned stimulus (US) interval. We report two experiments that demonstrated inhibition of delay when rabbits were trained with relatively long, but not with short, CS-US intervals. To account for this divergence, we assumed that the SOP stimulus trace involves two kinds of elements, some with a temporally distributed pattern of activity over the duration of the CS duration, and some with a randomly distributed pattern. This stimulus representation, not only allows for inhibition of delay with long but not short CS-US intervals, but in combination with SOP's performance rule deduces CR's with "Weber variability."