1
|
Bouton ME, Maren S, McNally GP. BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING. Physiol Rev 2021; 101:611-681. [PMID: 32970967 PMCID: PMC8428921 DOI: 10.1152/physrev.00016.2020] [Citation(s) in RCA: 144] [Impact Index Per Article: 48.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/19/2022] Open
Abstract
This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.
Collapse
Affiliation(s)
- Mark E Bouton
- Department of Psychological Science, University of Vermont, Burlington, Vermont
| | - Stephen Maren
- Department of Psychological and Brain Sciences and Institute for Neuroscience, Texas A&M University, College Station, Texas
| | - Gavan P McNally
- School of Psychology, University of New South Wales, Sydney, Australia
| |
Collapse
|
2
|
Calcagni G, Caballero-Garrido E, Pellón R. Behavior Stability and Individual Differences in Pavlovian Extended Conditioning. Front Psychol 2020; 11:612. [PMID: 32390896 PMCID: PMC7189120 DOI: 10.3389/fpsyg.2020.00612] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/30/2019] [Accepted: 03/16/2020] [Indexed: 12/05/2022] Open
Abstract
How stable and general is behavior once maximum learning is reached? To answer this question and understand post-acquisition behavior and its related individual differences, we propose a psychological principle that naturally extends associative models of Pavlovian conditioning to a dynamical oscillatory model where subjects have a greater memory capacity than usually postulated, but with greater forecast uncertainty. This results in a greater resistance to learning in the first few sessions followed by an over-optimal response peak and a sequence of progressively damped response oscillations. We detected the first peak and trough of the new learning curve in our data, but their dispersion was too large to also check the presence of oscillations with smaller amplitude. We ran an unusually long experiment with 32 rats over 3,960 trials, where we excluded habituation and other well-known phenomena as sources of variability in the subjects' performance. Using the data of this and another Pavlovian experiment by Harris et al. (2015), as an illustration of the principle we tested the theory against the basic associative single-cue Rescorla–Wagner (RW) model. We found evidence that the RW model is the best non-linear regression to data only for a minority of the subjects, while its dynamical extension can explain the almost totality of data with strong to very strong evidence. Finally, an analysis of short-scale fluctuations of individual responses showed that they are described by random white noise, in contrast with the colored-noise findings in human performance.
Collapse
Affiliation(s)
- Gianluca Calcagni
- Instituto de Estructura de la Materia, CSIC, Madrid, Spain
- *Correspondence: Gianluca Calcagni
| | | | - Ricardo Pellón
- Facultad de Psicología, Universidad Nacional de Educación a Distancia (UNED), Madrid, Spain
| |
Collapse
|
3
|
Roles of context in acquisition of human instrumental learning: Implications for the understanding of the mechanisms underlying context-switch effects. Learn Behav 2018; 45:211-227. [PMID: 28039580 DOI: 10.3758/s13420-016-0256-8] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Four experiments in human instrumental learning explored the associations involving the context that develop after three trials of training on simple discriminations. Experiments 1 and 4 found a deleterious effect of switching the learning context that cannot be explained by the context-outcome binary associations commonly used to explain context-switch effects after short training in human predictive learning and in animal Pavlovian conditioning. Evidence for context-outcome (Experiment 2), context-discriminative stimulus (Experiment 3), and context-instrumental response (Experiment 4) binary associations was found within the same training paradigm, suggesting that contexts became associated with all the elements of the situation, regardless of whether those associations played a role in a specific context-switch effect detected on performance.
Collapse
|
4
|
Leer A, Haesen K, Vervliet B. Beyond Extinction: Prolonged Conditioning and Repeated Threat Exposure Abolish Contextual Renewal of Fear-Potentiated Startle Discrimination but Leave Expectancy Ratings Intact. Front Psychiatry 2018; 9:117. [PMID: 29681867 PMCID: PMC5897540 DOI: 10.3389/fpsyt.2018.00117] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/18/2017] [Accepted: 03/20/2018] [Indexed: 12/15/2022] Open
Abstract
Extinction treatments decrease fear via repeated exposures to the conditioned stimulus (CS) and are associated with a return of fear. Alternatively, fear can be reduced via reductions in the perceived intensity of the unconditioned stimulus (US), e.g., through repeated exposures to the US. Promisingly, the few available studies show that repeated US exposures outperform standard extinction. US exposure treatments can decrease fear via two routes: (1) by weakening the CS-US association (extinction-like mechanism), and/or (2) by weakening the subjective US aversiveness (habituation-like mechanism). The current study further investigated the conditions under which US exposure treatment may reduce renewal, by adding a group in which CS-US pairings continued following fear acquisition. During acquisition, participants learned that one of two visual stimuli (CS+/CS-) predicted the occurrence of an aversive electrocutaneous stimulus (US). Next, the background context changed and participants received one of three interventions: repeated CS exposures, (2) repeated US exposures, or (3) continued CS-US pairings. Following repeated CS exposures, test presentations of the CSs in the original conditioning context revealed intact CS+/CS- differentiation in the fear-potentiated startle reflex, while the differentiation was abolished in the other two groups. Differential US expectancy ratings, on the other hand, were intact in all groups. Skin conductance data were inconclusive because standard context renewal following CS exposures did not occur. Unexpectedly, there was no evidence for a habituation-like process having taken place during US exposures or continued CS-US pairings. The results provide further evidence that US exposures outperform the standard extinction treatment and show that effects are similar when US exposures are part of CS-US pairings.
Collapse
Affiliation(s)
- Arne Leer
- Clinical Psychology, Utrecht University, Utrecht, Netherlands
| | - Kim Haesen
- Center for Excellence on Generalization, University of Leuven, Leuven, Belgium
| | - Bram Vervliet
- Center for Excellence on Generalization, University of Leuven, Leuven, Belgium
| |
Collapse
|
5
|
Trask S, Thrailkill EA, Bouton ME. Occasion setting, inhibition, and the contextual control of extinction in Pavlovian and instrumental (operant) learning. Behav Processes 2016; 137:64-72. [PMID: 27720958 DOI: 10.1016/j.beproc.2016.10.003] [Citation(s) in RCA: 78] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/30/2015] [Revised: 09/20/2016] [Accepted: 10/05/2016] [Indexed: 11/29/2022]
Abstract
An occasion setter is a stimulus that modulates the ability of another stimulus to control behavior. A rich history of experimental investigation has identified several important properties that define occasion setters and the conditions that give rise to occasion setting. In this paper, we first consider the basic hallmarks of occasion setting in Pavlovian conditioning. We then review research that has examined the mechanisms underlying the crucial role of context in Pavlovian and instrumental extinction. In Pavlovian extinction, evidence suggests that the extinction context can function as a negative occasion setter whose role is to disambiguate the current meaning of the conditioned stimulus; the conditioning context can also function as a positive occasion setter. In operant extinction, in contrast, the extinction context may directly inhibit the response, and the conditioning context can directly excite it. We outline and discuss the key results supporting these distinctions.
Collapse
|
6
|
Tallot L, Capela D, Brown BL, Doyère V. Individual trial analysis evidences clock and non-clock based conditioned suppression behaviors in rats. Behav Processes 2016; 124:97-107. [PMID: 26772780 DOI: 10.1016/j.beproc.2016.01.003] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2015] [Revised: 01/01/2016] [Accepted: 01/01/2016] [Indexed: 10/22/2022]
Abstract
We analyzed the temporal pattern of conditioned suppression of lever-pressing for food in rats conditioned with tone-shock pairings using either a 10 or 15s conditioned stimulus (CS)-unconditioned stimulus (US) interval with a CS duration that was three times the CS-US interval. The analysis of average suppression and of individual trials was performed during Probe CS-alone trials and when a short gap was inserted during the CS. The pattern of suppression followed the classical temporal rules: (1) scalar property, (2) a shift in peak suppression due to a gap, compatible with a Stop rule, (3) a three-state pattern of lever-pressing in individual trials, with abrupt start and stop of suppression. The peak of the average suppression curve, but not the middle time, was anticipatory to the programmed US time. The pattern of lever-pressing in individual trials unraveled two types of start of suppression behavior: a clock-based biphasic responding, with a burst of lever-pressing before suppression, and a non-clock based monophasic reduction of lever-pressing close to the CS onset. The non-clock based type of behavior may be responsible for the anticipatory peak time, and the biphasic pattern of lever-pressing may reflect the decision stage described in clock models.
Collapse
Affiliation(s)
- Lucille Tallot
- Univ Paris-Saclay, Univ Paris-Sud, CNRS, Institut des Neurosciences Paris-Saclay (Neuro PSI), UMR 9197 Orsay, France.
| | - Daphné Capela
- Univ Paris-Saclay, Univ Paris-Sud, CNRS, Institut des Neurosciences Paris-Saclay (Neuro PSI), UMR 9197 Orsay, France
| | - Bruce L Brown
- Department of Psychology, Queens College and the Graduate Center, The City University of New York, Flushing, NY 11367, USA
| | - Valérie Doyère
- Univ Paris-Saclay, Univ Paris-Sud, CNRS, Institut des Neurosciences Paris-Saclay (Neuro PSI), UMR 9197 Orsay, France.
| |
Collapse
|
7
|
The effect of the amount of blocking cue training on blocking of appetitive conditioning in mice. Behav Processes 2015; 122:36-42. [PMID: 26562656 PMCID: PMC4721184 DOI: 10.1016/j.beproc.2015.11.007] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/26/2015] [Revised: 11/02/2015] [Accepted: 11/04/2015] [Indexed: 11/24/2022]
Abstract
Blocking of appetitive conditioning in mice has rarely been demonstrated. Blocking occurred when there was 200, but not 80 trials with a visual blocking cue. Blocking occurred independent of trial number with an auditory blocking cue. Post-asymptotic training is necessary under certain conditions for blocking.
Conditioning of a target cue is blocked when it occurs in compound with another cue (blocking cue) that has already received conditioning. Although blocking of appetitive conditioning is commonly used in rodents as a test of selective learning, it has been demonstrated rarely in mice. In order to investigate the conditions that result in blocking in mice two studies tested the effect of the extent of prior blocking cue training on blocking of appetitive conditioning. Mice received either 80 or 200 trials of blocking cue training prior to compound conditioning. A control group received only compound training. Experiment 1 assessed the ability of a visual cue to block conditioning to an auditory target cue. Exposure to the context and the unconditioned stimulus, sucrose pellets, was equated across groups. Blocking was evident in mice that received 200, but not 80 training trials with the visual blocking cue. Responding to the blocking cue was similar across groups. Experiment 2 assessed the ability of an auditory cue to block conditioning to a visual target cue. Blocking was evident in mice trained with 80 and 200 auditory blocking cue trials. The results demonstrate that the strength of blocking in mice is dependent on the modality and experience of the blocking cue. Furthermore, prolonged training of the blocking cue after asymptotic levels of conditioned responding have been reached is necessary for blocking to occur under certain conditions suggesting that the strength of conditioned responding is a limited measure of learning.
Collapse
|
8
|
Bouton ME, Todd TP, León SP. Contextual control of discriminated operant behavior. JOURNAL OF EXPERIMENTAL PSYCHOLOGY-ANIMAL LEARNING AND COGNITION 2015; 40:92-105. [PMID: 24000907 DOI: 10.1037/xan0000002] [Citation(s) in RCA: 43] [Impact Index Per Article: 4.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Previous research has suggested that changing the context after instrumental (operant) conditioning can weaken the strength of the operant response. That result contrasts with the results of studies of Pavlovian conditioning, in which a context switch often does not affect the response elicited by a conditioned stimulus. To begin to make the methods more similar, Experiments 1-3 tested the effects of a context switch in rats on a discriminated operant response (R; lever pressing or chain pulling) that had been reinforced only in the presence of a 30-s discriminative stimulus (S; tone or light). As in Pavlovian conditioning, responses and reinforcers became confined to presentations of the S during training. However, in Experiment 1, after training in Context A, a switch to Context B caused a decrement in responding during S. In Experiment 2, a switch to Context B likewise decreased responding in S when Context B was equally familiar, equally associated with reinforcement, or equally associated with the training of a discriminated operant (a different R reinforced in a different S). However, there was no decrement if Context B had been associated with the same response that was trained in Context A (Experiments 2 and 3). The effectiveness of S transferred across contexts, whereas the strength of the response did not. Experiment 4 found that a continuously reinforced response was also disrupted by context change when the same response manipulandum was used in both training and testing. Overall, the results suggest that the context can have a robust general role in the control of operant behavior. Mechanisms of contextual control are discussed.
Collapse
|
9
|
Morrow JD, Saunders BT, Maren S, Robinson TE. Sign-tracking to an appetitive cue predicts incubation of conditioned fear in rats. Behav Brain Res 2015; 276:59-66. [PMID: 24747659 PMCID: PMC4201891 DOI: 10.1016/j.bbr.2014.04.002] [Citation(s) in RCA: 38] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2014] [Revised: 03/29/2014] [Accepted: 04/01/2014] [Indexed: 12/29/2022]
Abstract
Although post-traumatic stress disorder (PTSD) and addiction are very different disorders, both are characterized by hyperreactivity to trauma- or drug-related cues, respectively. We investigated whether an appetitive conditioning task, Pavlovian conditioned approach, which predicts vulnerability to reinstatement of cocaine-seeking, also predicts fear incubation, which may be a marker for vulnerability to PTSD. We classified rats based on whether they learned to approach and interact with a food predictive cue (sign-trackers), or, whether upon cue presentation they went to the location of impending food delivery (goal-trackers). Rats were then exposed to extensive Pavlovian tone-shock pairings, which causes the fear response to increase or "incubate" over time. We found that the fear incubation effect was only present in sign-trackers. The behavior of goal-trackers was more consistent with a normal fear response-it was most robust immediately after training and decayed slowly over time. Sign-trackers also had lower levels of brain-derived neurotrophic factor (BDNF) protein in the prefrontal cortex than goal-trackers. These results indicate that, while many factors likely contribute to the disproportionate co-occurrence of PTSD and substance abuse, one such factor may be a core psychological trait that biases some individuals to attribute excessive motivational significance to predictive cues, regardless of the emotional valence of those cues. High levels of BDNF in the prefrontal cortex may be protective against developing excessive emotional and motivational responses to salient cues.
Collapse
Affiliation(s)
- Jonathan D Morrow
- Department of Psychiatry, University of Michigan, 4250 Plymouth Road SPC 5767, Ann Arbor, MI 48109-2700, United States; Neuroscience Graduate Program, University of Michigan, 4137 Undergraduate Science Building (USB), 204 Washtenaw Avenue, Ann Arbor, MI 48109-2215, United States.
| | - Benjamin T Saunders
- Department of Psychology, Biopsychology Program, University of Michigan, East Hall, 530 Church Street, Ann Arbor, MI 48109-1043, United States.
| | - Stephen Maren
- Neuroscience Graduate Program, University of Michigan, 4137 Undergraduate Science Building (USB), 204 Washtenaw Avenue, Ann Arbor, MI 48109-2215, United States; Department of Psychology, Biopsychology Program, University of Michigan, East Hall, 530 Church Street, Ann Arbor, MI 48109-1043, United States.
| | - Terry E Robinson
- Neuroscience Graduate Program, University of Michigan, 4137 Undergraduate Science Building (USB), 204 Washtenaw Avenue, Ann Arbor, MI 48109-2215, United States; Department of Psychology, Biopsychology Program, University of Michigan, East Hall, 530 Church Street, Ann Arbor, MI 48109-1043, United States.
| |
Collapse
|
10
|
Abstract
Unhealthy behavior is responsible for much human disease, and a common goal of contemporary preventive medicine is therefore to encourage behavior change. However, while behavior change often seems easy in the short run, it can be difficult to sustain. This article provides a selective review of research from the basic learning and behavior laboratory that provides some insight into why. The research suggests that methods used to create behavior change (including extinction, counterconditioning, punishment, reinforcement of alternative behavior, and abstinence reinforcement) tend to inhibit, rather than erase, the original behavior. Importantly, the inhibition, and thus behavior change more generally, is often specific to the "context" in which it is learned. In support of this view, the article discusses a number of lapse and relapse phenomena that occur after behavior has been changed (renewal, spontaneous recovery, reinstatement, rapid reacquisition, and resurgence). The findings suggest that changing a behavior can be an inherently unstable and unsteady process; frequent lapses should be expected. In the long run, behavior-change therapies might benefit from paying attention to the context in which behavior change occurs.
Collapse
|
11
|
Holmes NM, Westbrook RF. Appetitive context conditioning proactively, but transiently, interferes with expression of counterconditioned context fear. Learn Mem 2014; 21:597-605. [PMID: 25320352 PMCID: PMC4201809 DOI: 10.1101/lm.035089.114] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2014] [Accepted: 07/15/2014] [Indexed: 11/25/2022]
Abstract
Four experiments used rats to study appetitive-aversive transfer. Rats trained to eat a palatable food in a distinctive context and shocked in that context ate and did not freeze when tested 1 d later but froze and did not eat when tested 14 d later. These results were associatively mediated (Experiments 1 and 2), observed when rats were or were not food deprived (Experiments 1 and 2), and were not due to latent inhibition (Experiment 3). In contrast, rats trained to eat in the context and shocked there 13 d later froze and did not eat when tested 1 d after the shocked exposure. However, rats that received an additional eating session in the context 1 d before the shocked exposure ate and did not freeze when tested 1 d after the shocked exposure (Experiment 4). The results show that appetitive conditioning transiently interferes with aversive conditioning. They are discussed in terms of a weak context-shock association becoming stronger with the lapse of time (so-called fear incubation) or of the interference by the context-food association becoming weaker with the lapse of time.
Collapse
Affiliation(s)
- Nathan M Holmes
- School of Psychology, University of New South Wales, Sydney NSW 2052, Australia
| | | |
Collapse
|
12
|
Abstract
Most theories of associative learning assert that conditioned responding to a target cue is a monotonically increasing function of unconditioned-stimulus (US) intensity. In a lick suppression preparation with rats, a cue was paired with a 0.4-, 0.6-, 0.8-, 1.0-, 1.2-, or 1.4-mA footshock in Experiment 1a, and with a 0.3-, 0.8-, 1.3-, or 1.8-mA footshock in Experiment 1b. Subsequent suppression in response to the cue was an inverted-U function of the US intensity. In Experiment 2, we demonstrated that massive extinction of the training context and compound conditioning can each attenuate the response decrement caused by training with a high-intensity US. The sometimes-competing-retrieval model (Stout & Miller, Psychological Review 114:759-783, 2007) provides a better fit to these data than do several other models of associative learning.
Collapse
|
13
|
Urcelay GP, Miller RR. The functions of contexts in associative learning. Behav Processes 2014; 104:2-12. [PMID: 24614400 DOI: 10.1016/j.beproc.2014.02.008] [Citation(s) in RCA: 52] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/19/2013] [Revised: 02/05/2014] [Accepted: 02/09/2014] [Indexed: 11/17/2022]
Abstract
Although contexts play many roles during training and also during testing, over the last four decades theories of learning have predominantly focused on one or the other of the two families of functions served by contexts. In this selective review, we summarize recent data concerning these two functions and their interrelationship. The first function is similar to that of discrete cues, and allows contexts to elicit conditioned responses and compete with discrete events for behavioral control. The second function is modulatory, and similar to that of discrete occasion setters in that in this role contexts do not elicit conditioned responses by themselves, but rather modulate instrumental responding or responding to Pavlovian cues. We first present evidence for these two functions, and then suggest that the spacing of trials, amount of training, and contiguity are three determinants of the degree to which the context will play each function. We also conclude that these two functions are not mutually exclusive, and that future research would benefit from identifying the conditions under which their functions dominate behavioral control. We close by discussing some misconceptions concerning contexts. This article is part of a Special Issue entitled: SQAB 2013: Contextual Con.
Collapse
Affiliation(s)
| | - Ralph R Miller
- Department of Psychology, State University of New York at Binghamton, USA.
| |
Collapse
|
14
|
A fundamental role for context in instrumental learning and extinction. Behav Processes 2014; 104:13-9. [PMID: 24576702 DOI: 10.1016/j.beproc.2014.02.012] [Citation(s) in RCA: 99] [Impact Index Per Article: 9.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2013] [Revised: 02/12/2014] [Accepted: 02/14/2014] [Indexed: 11/23/2022]
Abstract
The purpose of this article is to review recent research that has investigated the effects of context change on instrumental (operant) learning. The first part of the article discusses instrumental extinction, in which the strength of a reinforced instrumental behavior declines when reinforcers are withdrawn. The results suggest that extinction of either simple or discriminated operant behavior is relatively specific to the context in which it is learned: As in prior studies of Pavlovian extinction, ABA, ABC, and AAB renewal effects can all be observed. Further analysis supports the idea that the organism learns to refrain from making a specific response in a specific context, or in more formal terms, an inhibitory context-response association. The second part of the article then discusses research suggesting that the context also controls instrumental behavior before it is extinguished. Several experiments demonstrate that a context switch after either simple or discriminated operant training causes a decrement in the strength of the response. Over a range of conditions, the animal appears to learn a direct association between the context and the response. Under some conditions, it can also learn a hierarchical representation of context and the response-reinforcer relation. Extinction is still more context-specific than conditioning, as indicated by ABC and AAB renewal. Overall, the results establish that the context can play a significant role in both the acquisition and extinction of operant behavior.
Collapse
|
15
|
The dual role of the context in postpeak performance decrements resulting from extended training. Learn Behav 2013; 40:476-93. [PMID: 22359182 DOI: 10.3758/s13420-012-0068-4] [Citation(s) in RCA: 12] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
The context's role in Pavlovian conditioning depends on the trial spacing during training, with massed trials revealing a function akin to that of discrete stimuli, and spaced trials revealing a modulatory function (Urcelay & Miller, Journal of Experimental Psychology. Animal Behavior Processes, 36, 268-280, 2010). Here we examined the contextual determinants of a common but largely ignored effect: attenuated conditioned responding with extended reinforced training (i.e., a postpeak performance deficit [PPD]). Contextual sources of PPDs were investigated in four fear-conditioning experiments with rats. In Experiment 1, as the number of reinforced trials increased, conditioned responding decreased, even when testing occurred outside the training context. Experiment 2 revealed opposing influences of context on the PPD based on trial spacing, which interacted with whether testing occurred in the training context. This finding reconciles Experiment 1's results with previous data (Bouton, Frohardt, Sunsay, Waddell, & Morris, Journal of Experimental Psychology. Animal Behavior Processes, 34, 223-236, 2008). Experiment 3 suggested that extended training with these parameters did not lead to habituation to conditioned or unconditioned stimuli. In Experiment 4, few or many massed training trials were followed orthogonally by context extinction or no context extinction. After many pairings, context extinction reduced the PPD (i.e., enhanced responding), suggesting a competitive role of the context. These results, together with prior data suggesting that context modulates expressions of the PPD, are consistent with the view that contexts can play two distinctly different roles.
Collapse
|
16
|
Witnauer JE, Urcelay GP, Miller RR. The error in total error reduction. Neurobiol Learn Mem 2013; 108:119-35. [PMID: 23891930 DOI: 10.1016/j.nlm.2013.07.018] [Citation(s) in RCA: 9] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/11/2013] [Revised: 07/09/2013] [Accepted: 07/18/2013] [Indexed: 11/19/2022]
Abstract
Most models of human and animal learning assume that learning is proportional to the discrepancy between a delivered outcome and the outcome predicted by all cues present during that trial (i.e., total error across a stimulus compound). This total error reduction (TER) view has been implemented in connectionist and artificial neural network models to describe the conditions under which weights between units change. Electrophysiological work has revealed that the activity of dopamine neurons is correlated with the total error signal in models of reward learning. Similar neural mechanisms presumably support fear conditioning, human contingency learning, and other types of learning. Using a computational modeling approach, we compared several TER models of associative learning to an alternative model that rejects the TER assumption in favor of local error reduction (LER), which assumes that learning about each cue is proportional to the discrepancy between the delivered outcome and the outcome predicted by that specific cue on that trial. The LER model provided a better fit to the reviewed data than the TER models. Given the superiority of the LER model with the present data sets, acceptance of TER should be tempered.
Collapse
Affiliation(s)
- James E Witnauer
- Department of Psychology, State University of New York at Brockport, USA
| | | | - Ralph R Miller
- Department of Psychology, State University of New York at Binghamton, USA.
| |
Collapse
|
17
|
Abstract
While fear and anxiety can grow over time in anxiety disorders, most efforts to model this phenomenon with fear conditioning in rodents cause fear that remains stable or decreases across weeks or months. Here, we describe several methods to induce conditioned fear that grows over the course of 1 month and is sustained for at least 2 months using an extended fear conditioning approach. These methods include a very reliable standard method that causes multiple fear measures to increase over months, as well as alternative methods.
Collapse
Affiliation(s)
- Charles L. Pickens
- National Institute on Alcohol Abuse and Alcoholism, Laboratory of Behavioral and Genomic Neuroscience, 5625 Fishers Lane, Room 2N09, Rockville, MD 20851. . Phone: 301-443-4052. Fax: 301-480-1952
| | - Sam A. Golden
- Mount Sinai School of Medicine, Friedman Brain Institute and Department of Neuroscience, 1425 Madison Avenue, Icahn Building 10-02, New York, NY 10029. . Phone: 212-659-8625. Fax: 212-849-2611
| | - Sunila G. Nair
- University of Washington, Department of Psychiatry and Behavioral Sciences, Harborview Medical Center, 325 9th Ave, Seattle, WA 98104. . Phone: 206-897-5801. Fax: 206-897-5804
| |
Collapse
|
18
|
Thanellou A, Green JT. Cerebellar structure and function in male Wistar-Kyoto hyperactive rats. Behav Neurosci 2013; 127:311-24. [PMID: 23398437 DOI: 10.1037/a0031897] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Previous research has suggested that the Wistar-Kyoto Hyperactive (WKHA) rat strain may model some of the behavioral features associated with attention-deficit/hyperactivity disorder (ADHD). We have shown that, in cerebellar-dependent eyeblink conditioning, male WKHAs emit eyeblink CRs with shortened onset latencies. To further characterize the shortened CR onset latencies seen in male WKHA rats, we examined 750-ms delay conditioning with either a tone conditional stimulus (CS) or a light CS, we extended acquisition training, and we included Wistar rats as an additional, outbred control strain. Our results indicated that WKHAs learned more quickly and showed a shortened CR onset latency to a tone CS compared to both Wistar-Kyoto Hypertensive (WKHT) and Wistars. WKHAs and Wistars show a lengthening of CR onset latency over conditioning with a tone CS and an increasing confinement of CRs to the later part of the tone CS (inhibition of delay). WKHAs learned more quickly to a light CS only in comparison to WKHTs, and showed a shortened CR onset latency only in comparison to Wistars. Wistars showed an increasing confinement of CRs to the late part of the light CS over conditioning. We used unbiased stereology to estimate the number of Purkinje and granule cells in the cerebellar cortex of the three strains. Our results indicated that WKHAs have more granule cells than Wistars and WKHTs and more Purkinje cells than Wistars. Results are discussed in terms of CS processing and cerebellar cortical contributions to EBC.
Collapse
Affiliation(s)
- Alexandra Thanellou
- Department of Psychology, University of Vermont, 2 Colchester Avenue, Burlington, VT 05405-0134, USA
| | | |
Collapse
|
19
|
Performance factors in associative learning: assessment of the sometimes competing retrieval model. Learn Behav 2013; 40:347-66. [PMID: 22927006 DOI: 10.3758/s13420-012-0086-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2022]
Abstract
Previous simulations revealed that the sometimes competing retrieval model (SOCR; Stout & Miller, Psychological Review, 114, 759-783, 2007), which assumes local error reduction, can explain many cue interaction phenomena that elude traditional associative theories based on total error reduction. Here, we applied SOCR to a new set of Pavlovian phenomena. Simulations used a single set of fixed parameters to simulate each basic effect (e.g., blocking) and, for specific experiments using different procedures, used fitted parameters discovered through hill climbing. In simulation 1, SOCR was successfully applied to basic acquisition, including the overtraining effect, which is context dependent. In simulation 2, we applied SOCR to basic extinction and renewal. SOCR anticipated these effects with both fixed parameters and best-fitting parameters, although the renewal effects were weaker than those observed in some experiments. In simulation 3a, feature-negative training was simulated, including the often observed transition from second-order conditioning to conditioned inhibition. In simulation 3b, SOCR predicted the observation that conditioned inhibition after feature-negative and differential conditioning depends on intertrial interval. In simulation 3c, SOCR successfully predicted failure of conditioned inhibition to extinguish with presentations of the inhibitor alone under most circumstances. In simulation 4, cue competition, including blocking (4a), recovery from relative validity (4b), and unblocking (4c), was simulated. In simulation 5, SOCR correctly predicted that inhibitors gain more behavioral control than do excitors when they are trained in compound. Simulation 6 demonstrated that SOCR explains the slower acquisition observed following CS-weak shock pairings.
Collapse
|
20
|
Abstract
Alcohol addiction is a chronically relapsing disorder characterized by compulsive alcohol seeking and use. Alcohol craving and long-lasting vulnerability to relapse present a great challenge for the successful treatment of alcohol addiction. Therefore, relapse prevention has emerged as a critically important area of research, with the need for effective and valid animal models of relapse. This chapter provides an overview of the repertoire of animal models of craving and relapse presently available and employed in alcoholism research. These models include conditioned reinstatement, stress-induced reinstatement, ethanol priming-induced reinstatement, conditioned place preference, Pavlovian spontaneous recovery, the alcohol deprivation effect, and seeking-taking chained schedules. Thus, a wide array of animal models is available that permit investigation of behaviors directed at obtaining access to alcohol, as well as neurobehavioral mechanisms and genetic factors that regulate these behaviors. These models also are instrumental for identifying pharmacological treatment targets and as tools for evaluating the efficacy of potential medications for the prevention of alcohol craving and relapse.
Collapse
|
21
|
Dagnino-Subiabre A, Pérez MÁ, Terreros G, Cheng MY, House P, Sapolsky R. Corticosterone treatment impairs auditory fear learning and the dendritic morphology of the rat inferior colliculus. Hear Res 2012; 294:104-13. [PMID: 23088831 DOI: 10.1016/j.heares.2012.09.008] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/23/2012] [Revised: 09/27/2012] [Accepted: 09/30/2012] [Indexed: 12/18/2022]
Abstract
Stress leads to secretion of the adrenal steroid hormone corticosterone (CORT). The aim of this study was to determine the effects of chronic CORT administration on auditory and visual fear conditioning. Male Sprague-Dawley rats received CORT (400 mg/ml) in their drinking water for 10 consecutive days; this treatment induces stress levels of serum CORT. CORT impaired fear conditioning (F((1,28)) = 11.52, p < 0.01) and extinction (F((1,28)) = 4.86, p < 0.05) of auditory fear learning, but did not affect visual fear conditioning. In addition, we analyzed the CORT effects on the neuronal morphology of the inferior colliculus (flat neurons, auditory mesencephalon, a key brain area for auditory processing) and superior colliculus (wide-field neurons, related to visual processing) by Golgi stain. CORT decreased dendritic arborization of inferior colliculus neurons by approximately 50%, but did not affect superior colliculus neurons. Thus, CORT had more deleterious effects on the auditory fear processing than the visual system in the brain.
Collapse
Affiliation(s)
- Alexies Dagnino-Subiabre
- Laboratory of Behavioral Neurobiology, Center for Neurobiology and Brain Plasticity, Department of Physiology, Faculty of Sciences, Universidad de Valparaíso, Valparaíso, Chile.
| | | | | | | | | | | |
Collapse
|
22
|
Jozefowiez J, Witnauer JE, Miller RR. The temporal pattern of responding in conditioned bar-press suppression: the role of the context switch and training mode. Behav Processes 2011; 89:239-43. [PMID: 22178451 DOI: 10.1016/j.beproc.2011.12.002] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/08/2011] [Revised: 10/21/2011] [Accepted: 12/01/2011] [Indexed: 10/14/2022]
Abstract
The present study examined the temporal pattern of responding in a conditioned bar-press suppression task in rats. Rats were exposed to either a 30-s or a 120-s conditioned stimulus (CS) followed by a footshock. Training took place either while the rats were lever-pressing for water (online), or with the lever removed from the box (offline). They were then exposed to the CS while they were lever-pressing for water, either in the training context or in a different context. Bar-press suppression during the CS was constant across the duration of the CS during training, but was restricted to the initial portion of the CS at the time of testing, especially when subjects were tested in a different context. Those results replicate the reactive (as opposed to anticipatory) pattern observed in a lick suppression procedure by Jozefowiez et al. (2011) and indicate that a change in context at the time of testing might be critical for its expression.
Collapse
|
23
|
Abstract
Cognitive-behavioral therapies (CBTs) have been shown to be efficacious for the treatment of anxiety disorders in children and adolescents. Randomized clinical trials indicate that approximately two-thirds of children treated with CBT will be free of their primary diagnosis at posttreatment. Although several CBT treatment packages have been investigated in youth with diverse anxiety disorders, common core components have been identified. A comprehensive assessment, development of a good therapeutic relationship and working alliance, cognitive restructuring, repeated exposure with reduction of avoidance behavior, and skills training comprise the core procedures for the treatment of anxiety disorders in youth.
Collapse
Affiliation(s)
- Laura D. Seligman
- Associate Professor, Department of Psychology University of Toledo, Toledo, Ohio
| | - Thomas H. Ollendick
- University Distinguished Professor, Department of Psychology, Virginia Polytechnic Institute and State University, Blacksburg, Virginia, Director, Child Study Center, Virginia Polytechnic Institute and State University, Blacksburg, Virginia
| |
Collapse
|
24
|
|
25
|
Pickens CL, Navarre BM, Nair SG. Incubation of conditioned fear in the conditioned suppression model in rats: role of food-restriction conditions, length of conditioned stimulus, and generality to conditioned freezing. Neuroscience 2010; 169:1501-10. [PMID: 20600654 DOI: 10.1016/j.neuroscience.2010.06.036] [Citation(s) in RCA: 24] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/30/2010] [Revised: 06/14/2010] [Accepted: 06/16/2010] [Indexed: 11/19/2022]
Abstract
We recently adapted the conditioned suppression of operant responding method to study fear incubation. We found that food-restricted rats show low fear 2 days after extended (10 d; 100 30-s tone-shock pairings) fear training and high fear after 1-2 months. Here, we studied a potential mechanism of fear incubation: extended food-restriction stress. We also studied whether fear incubation is observed after fear training with a prolonged-duration (6-min) tone conditioned stimulus (CS), and whether conditioned freezing incubates after extended training in rats with or without a concurrent operant task. Conditioned fear was assessed 2 days and 1 month after training. In the conditioned suppression method, fear incubation was reliably observed in rats under moderate food-restriction conditions (18-20 g food/day) that allowed for weight gain, and after extended (10 d), but not limited (1 d), fear training with the 6-min CS. Incubation of conditioned freezing was observed after extended fear training in rats lever-pressing for food and, to a lesser degree, in rats not performing an operant task. Results indicate that prolonged hunger-related stress does not account for fear incubation in the conditioned suppression method, and that fear incubation occurs to a longer-duration (6-min) fear CS. Extended training also leads to robust fear incubation of conditioned freezing in rats performing an operant task and weaker fear incubation in rats not performing an operant task.
Collapse
Affiliation(s)
- C L Pickens
- Behavioral Neuroscience Branch, Intramural Research Program-National Institute on Drug Abuse, National Institutes of Health, Department of Health and Human Services, Baltimore, MD 21218, USA.
| | | | | |
Collapse
|
26
|
Pickens CL, Golden SA, Adams-Deutsch T, Nair SG, Shaham Y. Long-lasting incubation of conditioned fear in rats. Biol Psychiatry 2009; 65:881-6. [PMID: 19167702 PMCID: PMC2740722 DOI: 10.1016/j.biopsych.2008.12.010] [Citation(s) in RCA: 92] [Impact Index Per Article: 6.1] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/11/2008] [Revised: 12/12/2008] [Accepted: 12/16/2008] [Indexed: 11/30/2022]
Abstract
BACKGROUND In 1937, Diven reported that human fear responses to cues previously paired with shock progressively increase or incubate over 24 hours. Since then, fear incubation has been demonstrated in both humans and nonhumans. However, the difficulty of demonstrating long-lasting fear incubation in rodents has hampered the study of the underlying mechanisms of this incubation. Here, we describe a rat procedure where fear reliably incubates over time. METHODS We trained food-restricted rats to lever-press for food pellets in daily 90-min sessions. We then gave each rat 100 30-sec tones co-terminating with a .5-sec .5-mA footshock over 10 days (10 pairings/day). Groups of rats (n = 10-15) were then given four presentations of the tone (the fear cue) 2, 15, 31, or 61 days after fear conditioning training and were assessed for conditioned suppression of lever-pressing. RESULTS We found that conditioned fear responses were significantly higher 31 and 61 days after fear training than after 2 or 15 days. In control experiments, we showed that extensive tone-shock pairing is necessary for the emergence of fear incubation and that it is unlikely that non-associative factors contribute to this incubation. CONCLUSIONS We describe a procedure for generating reliable and long-lasting conditioned fear incubation. Our procedure can be used to study mechanisms of fear incubation and might provide a model for studying the mechanisms of delayed-onset posttraumatic stress disorder that occur in a sub-population of people previously exposed to chronic stressors.
Collapse
Affiliation(s)
- Charles L Pickens
- Behavioral Neuroscience Branch, Intramural Research Program, National Institute on Drug Abuse, National Institutes of Health, Department of Health and Human Services, Baltimore, Maryland 21224, USA
| | | | | | | | | |
Collapse
|
27
|
Holt DJ, Lebron-Milad K, Milad MR, Rauch SL, Pitman RK, Orr SP, Cassidy BS, Walsh JP, Goff DC. Extinction memory is impaired in schizophrenia. Biol Psychiatry 2009; 65:455-63. [PMID: 18986648 PMCID: PMC3740529 DOI: 10.1016/j.biopsych.2008.09.017] [Citation(s) in RCA: 135] [Impact Index Per Article: 9.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/10/2008] [Revised: 08/20/2008] [Accepted: 09/18/2008] [Indexed: 11/18/2022]
Abstract
BACKGROUND Schizophrenia is associated with abnormalities in emotional processing and social cognition, which might result from disruption of the underlying neural mechanism(s) governing emotional learning and memory. To investigate this possibility, we measured the acquisition and extinction of conditioned fear responses and delayed recall of extinction in schizophrenia and control subjects. METHODS Twenty-eight schizophrenia and 18 demographically matched control subjects underwent a 2-day fear conditioning, extinction learning, and extinction recall procedure, in which skin conductance response (SCR) magnitude was used as the index of conditioned responses. RESULTS During fear acquisition, 83% of the control subjects and 57% of the patients showed autonomic responsivity ("responders"), and the patients showed larger SCRs to the stimulus that was not paired with the unconditioned stimulus (CS-) than the control subjects. Within the responder group, there was no difference between the patients and control subjects in levels of extinction learning; however, the schizophrenia patients showed significant impairment, relative to the control subjects, in context-dependent recall of the extinction memory. In addition, delusion severity in the patients correlated with baseline skin conductance levels. CONCLUSIONS These data are consistent with prior evidence for a heightened neural response to innocuous stimuli in schizophrenia and elevated arousal levels in psychosis. The finding of deficient extinction recall in schizophrenia patients who showed intact extinction learning suggests that schizophrenia is associated with a disturbance in the neural processes supporting emotional memory.
Collapse
Affiliation(s)
- Daphne J Holt
- Psychiatric Neuroimaging Research Program, Psychiatry Department, Massachusetts General Hospital, Boston, Massachusetts 02129, USA.
| | | | | | | | | | | | | | | | | |
Collapse
|