Extinction and the associative structure of heterogeneous instrumental chains

Acute food deprivation-induced relapse to heroin seeking after short and long punishment-imposed abstinence in male rats

RATIONAL

Stress is a major trigger for drug relapse in humans and animal models, even after prolonged abstinence. However, animal models for stress-induced relapse were criticized for the lack of predictive and face validity.

OBJECTIVES

Here we investigated the effect of acute food deprivation stress in a novel stress-induced relapse model using voluntary, punishment-imposed abstinence from heroin. We also performed a detailed characterization of the development of punishment-imposed abstinence.

METHODS

Male rats were trained to self-administered heroin (0.1 mg/kg/infusion) for 2 weeks, using the seeking-taking chained schedule. Pressing the 'seeking' lever led to the insertion of the 'taking' lever and pressing the take lever resulted in heroin infusion. Following self-administration training, rats were exposed to 8 or 21 days of heroin-seeking punishment. During punishment, 30% of the completed seek links resulted in a mild escalating footshock instead of take lever presentation. Next, rats were tested for heroin seeking under extinction conditions after 24 h of food deprivation and sated conditions.

RESULTS

Probabilistic punishment of seeking lever responses resulted in gradual suppression of heroin seeking and taking. Exposure to food-deprivation stress induced a robust relapse to heroin seeking after short and long punishment-imposed abstinence periods, without significant effects of time, i.e., no incubation of heroin seeking. Individual differences were observed in the development of punishment-induced abstinence and stress-induced relapse.

CONCLUSIONS

These results suggest that stress is a reliable trigger to relapse even after a prolonged period of punishment-induced, voluntary abstinence.

New functions of the rodent prelimbic and infralimbic cortex in instrumental behavior

The prelimbic and infralimbic cortices of the rodent medial prefrontal cortex mediate the effects of context and goals on instrumental behavior. Recent work from our laboratory has expanded this understanding. Results have shown that the prelimbic cortex is important for the modulation of instrumental behavior by the context in which the behavior is learned (but not other contexts), with context potentially being broadly defined (to include at least previous behaviors). We have also shown that the infralimbic cortex is important in the expression of extensively-trained instrumental behavior, regardless of whether that behavior is expressed as a stimulus-response habit or a goal-directed action. Some of the most recent data suggest that infralimbic cortex may control the currently active behavioral state (e.g., habit vs. action or acquisition vs. extinction) when two states have been learned. We have also begun to examine prelimbic and infralimbic cortex function as key nodes of discrete circuits and have shown that prelimbic cortex projections to an anterior region of the dorsomedial striatum are important for expression of minimally-trained instrumental behavior. Overall, the use of an associative learning perspective on instrumental learning has allowed the research to provide new perspectives on how these two "cognitive" brain regions contribute to instrumental behavior.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Hippocampal-Dependent Inhibitory Learning and Memory Processes in the Control of Eating and Drug Taking

As manifestations of excessive and uncontrolled intake, obesity and drug addiction have generated much research aimed at identifying common neuroadaptations that could underlie both disorders. Much work has focused on changes in brain reward and motivational circuitry that can overexcite eating and drug-taking behaviors. We suggest that the regulation of both behaviors depends on balancing excitation produced by stimuli associated with food and drug rewards with the behavioral inhibition produced by physiological "satiety" and other stimuli that signal when those rewards are unavailable. Our main hypothesis is that dysregulated eating and drug use are consequences of diet- and drug-induced degradations in this inhibitory power. We first outline a learning and memory mechanism that could underlie the inhibition of both food and drug-intake, and we describe data that identifies the hippocampus as a brain substrate for this mechanism. We then present evidence that obesitypromoting western diets (WD) impair the operation of this process and generate pathophysiologies that disrupt hippocampal functioning. Next, we present parallel evidence that drugs of abuse also impair this same learning and memory process and generate similar hippocampal pathophysiologies. We also describe recent findings that prior WD intake elevates drug self-administration, and the implications of using drugs (i.e., glucagon-like peptide- 1 agonists) that enhance hippocampal functioning to treat both obesity and addiction are also considered. We conclude with a description of how both WD and drugs of abuse could initiate a "vicious-cycle" of hippocampal pathophysiology and impaired hippocampal-dependent behavioral inhibition.

Prediction and control of operant behavior: What you see is not all there is

Prediction and control of operant behavior are major goals of behavior analysis. We suggest that achieving these goals can benefit from doing more than identifying the three-term contingency between the behavior, its setting stimulus, and its consequences. Basic research now underscores the idea that prediction and control require consideration of the behavior's history. As one example, if an operant is a goal-directed action, it is controlled by the current value of the reinforcer, as illustrated by the so-called reinforcer devaluation effect. In contrast, if the behavior is a habit, it occurs automatically, without regard to the reinforcer's value, as illustrated by its insensitivity to the reinforcer devaluation effect. History variables that distinguish actions and habits include the extent of their prior practice and their schedule of reinforcement. Other operants can appear to have very low or zero strength. However, if the behavior has reached that level through extinction or punishment, it may precipitously increase in strength by changing the context, allowing time to pass, presenting the reinforcer contingently or noncontingently, or extinguishing an alternative behavior. Behaviors that are not suppressed by extinction or punishment are not affected the same way. When predicting the strength of an operant behavior, what you see is not all there is. The behavior's history counts.

Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control

This article reviews recent research on the extinction of instrumental (or operant) conditioning from the perspective that it is an example of a general retroactive interference process. Previous discussions of interference have focused primarily on findings from Pavlovian conditioning. The present review shows that extinction in instrumental learning has much in common with other examples of retroactive interference in instrumental learning (e.g., omission learning, punishment, second-outcome learning, discrimination reversal learning, and differential reinforcement of alternative behavior). In each, the original learning can be largely retained after conflicting information is learned, and behavior is cued or controlled by the current context. The review also suggests that a variety of stimuli can play the role of context, including room and apparatus cues, temporal cues, drug state, deprivation state, stress state, and recent reinforcers, discrete cues, or behaviors. In instrumental learning situations, the context can control behavior through its direct association with the reinforcer or punisher, through its hierarchical relation with response-outcome associations, or its direct association (inhibitory or excitatory) with the response. In simple instrumental extinction and habit learning, the latter mechanism may play an especially important role.

Maintaining performance in searching dogs: Evidence from a rat model that training to detect a second (irrelevant) stimulus can maintain search and detection responding

Scent-detecting dogs perform a sequence, or chain, of behaviors that, at minimum, includes searching followed by a detection behavior that signals the presence of a target stimulus to the handler. However, when working, dogs often engage in prolonged periods of searching without encountering a target. It is therefore important for trainers to use methods that promote persistent search behavior and target detection accuracy. Laboratory models can provide insights to the important variables that influence search persistence and accuracy. The present experiments examined a rat model of detection dog behavior. Two experiments assessed the use of practice with a single target stimulus to maintain search and detection of another previously-trained target. In Experiment 1, after learning a search→detection chain with two auditory targets, rats received either brief or extended training with only one of the targets before being tested for detection of both targets in extinction. The results suggest that single-target training strengthened the ability of the other target to control the detection behavior. Experiment 2 found that even infrequent target encounters were still effective at maintaining detection behavior to the other trained target. Importantly, the treatment was effective when the target stimuli were from different sensory modalities. Overall, the results support the utility of the rat model of search-dog behavior for evaluating novel training methods. We suggest several useful procedures for enhancing search persistence and accuracy in detection dogs that can be implemented in training protocols.

Effects of outcome devaluation on instrumental behaviors in a discriminated heterogeneous chain

Operant behavior often takes place in a sequence, or chain, of linked responses that lead to a reinforcer. We have recently studied rats performing a discriminated heterogeneous behavior chain that involves the presentation of a discriminative stimulus (e.g., a panel light) to set the occasion for a procurement behavior (e.g., a lever press) that leads to a second stimulus (e.g., a second panel light) that indicates that a consumption response (e.g., a chain pull) will be reinforced. The present study assessed the role played by a representation of the reinforcer in controlling the performance of the responses in this chain. After acquisition of the chain, rats received a reinforcer devaluation treatment in the form of repeated paired, or unpaired, presentations of the food-pellet outcome and lithium-chloride illness. Once paired rats came to reject the pellets, half the animals in each group were tested on procurement, and the other half were tested on consumption. Neither response was affected by the outcome devaluation treatment, although entries into the food cup were suppressed. Combined with other results, the findings suggest that the "goal" for goal-directed procurement responding in a discriminated heterogeneous chain may be the consumption response rather than the primary reinforcer. (PsycINFO Database Record

Factors that influence the persistence and relapse of discriminated behavior chains

Behavior chains are composed of sequences of behaviors that minimally include procurement and then consumption. This review surveys recent research from this laboratory that has examined the properties of discriminated heterogeneous behavior chains. In contrast to another review (Thrailkill and Bouton, 2016a), it discusses work examining what makes chained behavior persistent, and what makes it relapse. Results suggest that responses in a discriminated heterogeneous behavior chain may become associated, so that extinction of either one reduces the strength of the other. Evidence also suggests that the goal of the first (procurement) response may be the next (consumption) response (rather than the upcoming discriminative stimulus, a putative conditioned reinforcer, or the primary reinforcer at the end of the chain). Further studies suggest that methods that promote generalization across acquisition and extinction (partial reinforcement and delivery of noncontingent reinforcers during extinction) lead to greater persistence of the procurement response. A third set of studies analyzed the contextual control and relapse of chained behaviors. The context controls both the acquisition and extinction of chained behaviors. In addition, a separately-extinguished consumption response is renewed when returned to the context of the chain. The research expands our general understanding of the learning processes that govern instrumental behavior as well as our understanding of chains.

Contextual control of chained instrumental behaviors

Recent studies suggest a significant role for context in controlling the acquisition and extinction of simple operant responding. The present experiments examined the contextual control of a heterogeneous behavior chain. Rats first learned a chain in which a discriminative stimulus set the occasion for a procurement response (e.g., pulling a chain), which led to a second discriminative stimulus that occasion-set a consumption response (e.g., pressing a lever) that produced a food-pellet reinforcer. Experiment 1 showed that, after separate extinction of procurement and consumption, each response increased when it was returned to the acquisition context (ABA renewal) or was tested in a new context (AAB renewal). In addition, procurement responding, but not consumption responding, was decreased by changing the context after acquisition. Experiment 2 demonstrated ABA and AAB renewal of procurement and consumption following extinction of the whole chain. This time, the context-switch after acquisition weakened both procurement and consumption. Experiment 3 found that return to the context of the behavior chain renewed a consumption response that had been extinguished separately. Finally, in Experiment 4, rats learned 2 different discriminated heterogeneous chains; consumption extinguished outside its chain was only renewed on return to a chain when it was preceded by its associated procurement response. The results suggest a role for context in the extinction of chained behavior. They also support the view that procurement is influenced by the physical context and that consumption is controlled primarily by the response that precedes it in the chain. (PsycINFO Database Record