Unexpected food outcomes can return a habit to goal-directed action

Levonorgestrel maintains goal-directed behavior in habit-trained intact female rats

Hormonal contraceptives are utilized by millions of women worldwide. However, it remains unclear if these powerful endocrine modulators may alter cognitive function. Habit formation involves the progression of instrumental learning as it goes from being a conscious goal-directed process to a cue-driven automatic habitual motor response. Dysregulated goal and/or habit is implicated in numerous psychopathologies, underscoring the relevance of examining the effect of hormonal contraceptives on goal-directed and habitual behavior. This study examined the effect of levonorgestrel (LNG), a widely used progestin-type contraceptive, on the development of habit in intact female rats. Rats were implanted with subcutaneous capsules that slowly released LNG over the course of the experiment or cholesterol-filled capsules. All female rats underwent operant training followed by reward devaluation to test for habit. One group of females was trained at a level that is sub-threshold to habit, while another group of females was trained to a level well over the habit threshold observed in intact females. The results reveal that all sub-threshold trained rats remained goal-directed irrespective of LGN treatment, suggesting LNG is not advancing habit formation in female rats at this level of reinforcement. However, in rats that were overtrained well above the threshold, cholesterol females showed habitual behavior, thus replicating a portion of our original studies. In contrast, LNG-treated habit-trained rats remained goal-directed, indicating that LNG impedes the development and/or expression of habit following this level of supra-threshold to habit training. Thus, LNG may offset habit formation by sustaining attentional or motivational processes during learning in intact female rats. These results may be clinically relevant to women using this type of hormonal contraceptive as well as in other progestin-based hormone therapies.

The behavioral origins of phylogenic responses and ontogenic habits

An examination of innate behavior and its possible origins suggests parallels with the formation of habitual behavior. Inflexible but adaptive responses-innate reflexive behavior, Pavlovian conditioned responses, and operant habits-may have evolved from variable behavior in phylogeny and ontogeny. This form of "plasticity-first" scientific narrative was unpopular post-Darwin but has recently gained credibility in evolutionary biology. The present article seeks to identify originating events and contingencies contributing to such inflexible but adaptive behavior at both phylogenic and ontogenic levels of selection. In ontogeny, the development of inflexible performance (i.e., habit) from variable operant behavior is reminiscent of the genetic accommodation of initially variable phylogenic traits. The effects characteristic of habit (e.g., unresponsiveness to reinforcer devaluation) are explicable as the result of a conflict between behaviors at distinct levels of selection. The present interpretation validates the practice of seeking hard analogies between evolutionary biology and operant behavior. Finding such parallels implies the validity of a claim that organismal behavior, both innate and learned, is a product of selection by consequences. A complete and coherent account of organismal behavior may ultimately focus on functional selective histories in much the same way evolutionary biology does with its subject matter.

The temporal structure of goal-directed and habitual operant behavior

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Habit and persistence

Voluntary behaviors (operants) can come in two varieties: Goal-directed actions, which are emitted based on the remembered value of the reinforcer, and habits, which are evoked by antecedent cues and performed without the reinforcer's value in active memory. The two are perhaps most clearly distinguished with the reinforcer-devaluation test: Goal-directed actions are suppressed when the reinforcer is separately devalued and responding is tested in extinction, and habitual behaviors are not. But what is the function of habit learning? Habits are often thought to be strong and unusually persistent. The present selective review examines this idea by asking whether habits identified by the reinforcer-devaluation test are more resistant to extinction, resistant to the effects of other contingency change, vulnerable to relapse, resistant to the weakening effects of context change, or permanently in place once they are learned. Surprisingly little evidence supports the idea that habits are permanent or more persistent. Habits are more context-specific than goal-directed actions are. Methods that make behavior persistent do not necessarily work by encouraging habit. The function of habit learning may not be to make a behavior strong or more persistent but to make it automatic and efficient in a particular context.

Stimulus control of habits: Evidence for both stimulus specificity and devaluation insensitivity in a dual-response task

Goal-directed and habitual actions are clearly defined by their associative relations. Whereas goal-directed control can be confirmed via tests of outcome devaluation and contingency-degradation sensitivity, a comparable criterion for positively detecting habits has not been established. To confirm habitual responding, a test of control by the stimulus-response association is required while also ruling out goal-directed control. Here we describe an approach to developing such a test in rats using two discriminative stimuli that set the occasion for two different responses that then earn the same outcome. Performance was insensitive to outcome devaluation and showed stimulus-response specificity, indicative of stimulus-controlled behavior. The reliance of stimulus-response associations was further supported by a lack of sensitivity during the single extinction test session used here. These results demonstrate that two concurrently trained responses can come under habitual control when they share a common outcome. By reducing the ability of one stimulus to signal its corresponding response-outcome association, we found evidence for goal-directed control that can be dissociated from habits. Overall, these experiments provide evidence that tests assessing specific stimulus-response associations can be used to investigate habits.

Resurgence of goal-directed actions and habits

This study investigated how goal-directed and habitual behaviors recover after extinction within the context of the resurgence effect, a form of relapse induced by the removal or worsening of alternative reinforcement. Rats were trained to press a target lever with one reinforcer (O1) for either minimal (4) or extended (16) sessions. An extinction test after the completion of O1 devaluation confirmed that minimal and extended training formed goal-directed and habitual behaviors, respectively. Then, pressing an alternative lever was reinforced with a second reinforcer (O2) while the target response was placed on extinction. When O2 was discontinued, the minimally trained target response resurged with goal-directed status as in the extinction test. However, the extinguished habitual behavior in the extensively trained rats did not recover as a habit but instead with goal-directed status, possibly due to the context specificity of habits or the introduction of a new response-reinforcer contingency. The critical finding that reinforcer devaluation consistently led to less resurgence regardless of the amount of acquisition training provides a clinical implication that coupling differential-reinforcement-of-alternative-behavior (DRA) treatments with the devaluation of the associated reinforcer of problematic behavior could effectively diminish its recurrence.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration (2 h/day). We then exposed them to 4 days of punishment testing, in which footshock (0.4 mA, 0.3 s) was delivered randomly on one-third of trials, immediately following completion of seeking and prior to extension of the taking lever. Before and after punishment testing (4 days pre-punishment and ≥4 days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Learning to stop responding

Learning to stop responding is an important process that allows behavior to adapt to a changing and variable environment. This article reviews recent research in this laboratory and others that has studied how animals learn to stop responding in operant extinction, punishment, and feature-negative learning. Extinction and punishment are shown to be similar in two fundamental ways. First, the response-suppressing effects of both are highly context-specific. Second, the response-suppressing effects of both can be remarkably response-specific: Inhibition of one response transfers little to other responses. Learning to inhibit the response so specifically may result from the correction of "response error," the difference between the level of responding and what the current reinforcer supports. In contrast, the inhibition of responding that develops in feature-negative learning, where the response is reinforced during one discriminative stimulus (A) but not in a compound of A and stimulus B, is less response-specific: The inhibition of responding by stimulus B transfers and inhibits a second response, especially if the second response has itself been inhibited before. The results thus indicate both response-specific and response-general forms of behavioral inhibition. One possibility is that response-specific inhibition is learned when the circumstances encourage the organism to pay attention to the response-to what it is actually doing-as behavioral suppression is learned.

Flexible control of Pavlovian-instrumental transfer based on expected reward value

The Pavlovian-instrumental transfer (PIT) paradigm is widely used to assay the motivational influence of reward-predictive cues, reflected by their ability to invigorate instrumental behavior. Leading theories assume that a cue's motivational properties are tied to predicted reward value. We outline an alternative view that recognizes that reward-predictive cues may suppress rather than motivate instrumental behavior under certain conditions, an effect termed positive conditioned suppression. We posit that cues signaling imminent reward delivery tend to inhibit instrumental behavior, which is exploratory by nature, in order to facilitate efficient retrieval of the expected reward. According to this view, the motivation to engage in instrumental behavior during a cue should be inversely related to the value of the predicted reward, since there is more to lose by failing to secure a high-value reward than a low-value reward. We tested this hypothesis in rats using a PIT protocol known to induce positive conditioned suppression. In Experiment 1, cues signaling different reward magnitudes elicited distinct response patterns. Whereas the one-pellet cue increased instrumental behavior, cues signaling three or nine pellets suppressed instrumental behavior and elicited high levels of food-port activity. Experiment 2 found that reward-predictive cues suppressed instrumental behavior and increased food-port activity in a flexible manner that was disrupted by post-training reward devaluation. Further analyses suggest that these findings were not driven by overt competition between the instrumental and food-port responses. We discuss how the PIT task may provide a useful tool for studying cognitive control over cue-motivated behavior in rodents. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Lack of action monitoring as a prerequisite for habitual and chunked behavior: Behavioral and neural correlates

We previously reported the rapid development of habitual behavior in a discrete-trials instrumental task in which lever insertion and retraction act as reward-predictive cues delineating sequence execution. Here we asked whether lever cues or performance variables reflective of skill and automaticity might account for habitual behavior in male rats. Behavior in the discrete-trials habit-promoting task was compared with two task variants lacking the sequence-delineating cues of lever extension and retraction. We find that behavior is under goal-directed control in absence of sequence-delineating cues but not in their presence, and that skilled performance does not predict goal-directed vs. habitual behavior. Neural activity recordings revealed an engagement of dorsolateral striatum and a disengagement of dorsomedial striatum during the sequence execution of the habit-promoting task, specifically. Together, these results indicate that sequence delineation cues promote habit and differential engagement of striatal subregions during instrumental responding, a pattern that may reflect cue-elicited behavioral chunking.

Intermittent access training produces greater motivation for a non-drug reinforcer than long access training

It has recently been proposed that the intermittent access (IntA) drug self-administration procedure better produces behavioral changes relevant to addiction than the long access (LgA) procedure. In this version of the IntA procedure, the drug is made available for a 5-min period during each half hour of a 6-h session. In contrast, on the LgA procedure, the drug is available continuously for 6 h. Previous studies have found that IntA drug self-administration produces greater drug motivation, measured by increased progressive ratio breakpoints, than LgA self-administration. It has been hypothesized that this effect is due to the rapid, "spiking" brain levels of the drug, and consequent neuroadaptations, experienced by rats during IntA sessions. However, no study has compared the effects of IntA versus LgA training on reinforcer motivation when using a non-drug reinforcer. The present study compared motivation for a saccharin reinforcer after IntA or LgA training. In Experiment 1, separate groups of rats lever-pressed for saccharin on the IntA or LgA procedures. In Experiment 2, a within-subjects design was used where rats pressed one lever on the IntA procedure and another lever on the LgA procedure for saccharin. In both experiments, IntA training produced greater breakpoints than LgA training. As no drug was used here, spiking drug levels could not have been responsible for the increased saccharin motivation observed after IntA training. Instead, it is proposed that differences in stimulus-reinforcer associations learned during IntA versus LgA training may be responsible for the effect. Future research is needed to determine the extent to which such learning factors may contribute to the increased motivation observed after IntA training with drug reinforcers.

Habitual behaviour associated with exposure to high-calorie diet is prevented by an orexin-receptor-1 antagonist

Habitual actions, which are associated with addictive behaviours, contribute to the loss of control of food seeking seen following exposure to calorie-dense foods in rats. Antagonism of orexin-receptor-1 (ORX-R1) has been shown to reduce a range of stimulus-driven feeding behaviours, but have yet to be implicated in the regulation of habitual actions. In the current study, male Long-Evans rats were given 'binge-like' access to high-calorie diet (HCD) or standard chow diet, and were subsequently trained to press a lever for food outcome. When lever responses were tested following outcome devaluation, chow-fed rats displayed goal-directed actions, whereas HCD-exposed rats displayed habitual actions. In study 1, it was shown that systemic administration of the ORX-R1 antagonist, SB-334867, prior to test restored goal-directed behaviour in HCD-exposed rats. In study 2, intra-nigral administration of SB-334867 similarly restored goal-directed behaviour, thereby implicating the substantia nigra as an important site for this effect. This study demonstrates that targeting ORX-R1 reduces habitual food seeking in male rats which may be important for understanding and treating compulsive feeding, obesity and binge eating disorder. This study also implicates the lateral hypothalamus, where ORX is produced, in mediating the expression of habits for the first time, and thus extends on the neurocircuits known to regulate habitual actions. Further investigation is required to determine whether the same effects are also seen in female rats, given that there are recognised sexual dimorphisms in feeding behaviour and a higher incidence of disordered eating in female than male populations.

Making habits measurable beyond what they are not: A focus on associative dual-process models

Habits are the subject of intense international research. Under the associative dual-process model the outcome devaluation paradigm has been used extensively to classify behaviours as being either goal-directed (sensitive to shifts in the value of associated outcomes) or habitual (triggered by stimuli without anticipation of consequences). This has proven to be a useful framework for studying the neurobiology of habit and relevance of habits in clinical psychopathology. However, in recent years issues have been raised about this rather narrow definition of habits in comparison to habitual behaviour experienced in the real world. Specifically, defining habits as the absence of goal-directed control, the very specific set-ups required to demonstrate habit experimentally and the lack of direct evidence for habits as stimulus-response behaviours are viewed as problematic. In this review paper we address key critiques that have been raised about habit research within the framework of the associative dual-process model. We then highlight novel research approaches studying different features of habits with methods that expand beyond traditional paradigms.

Action sequences, habits, and attention in copying strategies

Understanding how culture evolves in society is an extremely difficult task. The bifocal stance theory (BST) deploys two copying strategies which can be linked to dual-system theories of behavior. BST would benefit from incorporating results from these theories, such as the evolution of attention to goals or steps of a behavioral sequence, and the role of the environment in prompting different copying strategies.

Maintained goal-directed control with overtraining on ratio schedules

It is thought that goal-directed control of actions weakens or becomes masked by habits over time. We tested the opposing hypothesis that goal-directed control becomes stronger over time, and that this growth is modulated by the overall action-outcome contiguity. Despite group differences in action-outcome contiguity early in training, rats trained under random and fixed ratio schedules showed equivalent goal-directed control of lever pressing that appeared to grow over time. We confirmed that goal-directed control was maintained after extended training under another type of ratio schedule-continuous reinforcement-using specific satiety and taste aversion devaluation methods. These results add to the growing literature showing that extensive training does not reliably weaken goal-directed control and that it may strengthen it, or at least maintain it.

New functions of the rodent prelimbic and infralimbic cortex in instrumental behavior

The prelimbic and infralimbic cortices of the rodent medial prefrontal cortex mediate the effects of context and goals on instrumental behavior. Recent work from our laboratory has expanded this understanding. Results have shown that the prelimbic cortex is important for the modulation of instrumental behavior by the context in which the behavior is learned (but not other contexts), with context potentially being broadly defined (to include at least previous behaviors). We have also shown that the infralimbic cortex is important in the expression of extensively-trained instrumental behavior, regardless of whether that behavior is expressed as a stimulus-response habit or a goal-directed action. Some of the most recent data suggest that infralimbic cortex may control the currently active behavioral state (e.g., habit vs. action or acquisition vs. extinction) when two states have been learned. We have also begun to examine prelimbic and infralimbic cortex function as key nodes of discrete circuits and have shown that prelimbic cortex projections to an anterior region of the dorsomedial striatum are important for expression of minimally-trained instrumental behavior. Overall, the use of an associative learning perspective on instrumental learning has allowed the research to provide new perspectives on how these two "cognitive" brain regions contribute to instrumental behavior.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Effect of context on the instrumental reinforcer devaluation effect produced by taste-aversion learning

Four experiments manipulated the context in which taste-aversion conditioning occurred when the reinforcer was devalued after instrumental learning. In all experiments, rats learned to lever press in an operant conditioning chamber and then had an aversion to the food-pellet reinforcer conditioned by pairing it with lithium chloride (LiCl) in either that context or a different context. Lever pressing was then tested in extinction to assess its status as a goal-directed action. In Experiment 1, aversion conditioning in the operant conditioning chamber suppressed lever-pressing during the test, but aversion conditioning in the home cage did not. Exposure to the averted pellet in the operant conditioning chamber after conditioning in the home cage did not change this effect (Experiment 2). The same pattern was observed when the different context was a second operant-style chamber (counterbalanced), exposure to the contexts was controlled, and pellets were presented in them in the same manner (Experiment 3). The greater effect of aversion conditioning in the instrumental context was not merely due to potentiated contextual conditioning (Experiment 4). Importantly, consumption tests revealed that the aversion conditioned in the different context had transferred to the test context. Thus, when reinforcer devaluation occurred in a different context, the rats lever pressed in extinction for a reinforcer they would otherwise reject. The results suggest that animals encode contextual information about the reinforcer during instrumental learning and suggest caution in making inferences about action versus habit learning when the reinforcer is devalued in a different context. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Uncertainty and predictiveness modulate attention in human predictive learning

[Correction Notice: An Erratum for this article was reported online in Journal of Experimental Psychology: General on Jan 14 2021 (see record 2021-07705-001). In the article, formatting for UK Research Councils funding was omitted. The author note and copyright line now reflect the standard acknowledgment of and formatting for the funding received for this article. All versions of this article have been corrected.] Attention determines which cues receive processing and are learned about. Learning, however, leads to attentional biases. In the study of animal learning, in some circumstances, cues that have been previously predictive of their consequences are subsequently learned about more than are nonpredictive cues, suggesting that they receive more attention. In other circumstances, cues that have previously led to uncertain consequences are learned about more than are predictive cues. In human learning, there is a clear role for predictiveness, but a role for uncertainty has been less clear. Here, in a human learning task, we show that cues that led to uncertain outcomes were subsequently learned about more than were cues that were previously predictive of their outcomes. This effect occurred when there were few uncertain cues. When the number of uncertain cues was increased, attention switched to predictive cues. This pattern of results was found for cues (1) that were uncertain because they led to 2 different outcomes equally often in a nonpredictable manner and (2) that were used in a nonlinear discrimination and were not predictive individually but were predictive in combination with other cues. This suggests that both the opposing predictiveness and uncertainty effects were determined by the relationship between individual cues and outcomes rather than the predictive strength of combined cues. These results demonstrate that learning affects attention; however, the precise nature of the effect on attention depends on the level of task complexity, which reflects a potential switch between exploration and exploitation of cues. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Habit, choice, and addiction

Addiction was suggested to emerge from the progressive dominance of habits over goal-directed behaviors. However, it is generally assumed that habits do not persist in choice settings. Therefore, it is unclear how drug habits may persist in real-world scenarios where this factor predominates. Here, we discuss the poor translational validity of the habit construct, which impedes our ability to determine its role in addiction. New evidence of habitual behavior in a drug choice setting are then described and discussed. Interestingly, habitual preference did not promote drug choice but instead favored abstinence. Here, we propose several clues to reconcile these unexpected results with the habit theory of addiction, and we highlight the need in experimental research to face the complexity of drug addicts' decision-making environments by investigating drug habits in the context of choice and in the presence of cues. On a theoretical level, we need to consider more complex frameworks, taking into account continuous interactions between goal-directed and habitual systems, and alternative decision-making models more representative of real-world conditions.

Renewal of goal direction with a context change after habit learning

An instrumental action can be goal-directed after a moderate amount of practice and then convert to habit after more extensive practice. Recent evidence suggests, however, that habits can return to action status after different environmental manipulations. The present experiments therefore asked whether habit learning interferes with goal direction in a context-dependent manner like other types of retroactive interference (e.g., extinction, punishment, counterconditioning). In Experiment 1, rats were given a moderate amount of instrumental training to form an action in one context (Context A) and then more extended training of the same response to form a habit in another context (Context B). We then performed reinforcer devaluation with taste aversion conditioning in both contexts, and tested the response in both contexts. The response remained habitual in Context B, but was goal-directed in Context A, indicating renewal of goal direction after habit learning. Experiment 2 expanded on Experiment 1 by testing the response in a third context (Context C). It found that the habitual response also renewed as action in this context. Together, the results establish a parallel between habit and extinction learning: Conversion to habit does not destroy action knowledge, but interferes with it in a context-specific way. They are also consistent with other results suggesting that habit is specific to the context in which it is learned, whereas goal-direction can transfer between contexts. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Context and renewal of habits and goal-directed actions after extinction

Instrumental behaviors that are goal-directed actions after moderate amounts of training can become habits after more extended training. Little research has asked how actions and habits are affected by retroactive interference treatments like extinction. The present experiments begin to fill this gap in the literature. In Experiments 1a and 1b, lever pressing in rats was minimally trained (1a) or extensively trained (1b) in one context (Context A), extinguished in a second context (Context B), and then tested in the acquisition context (Context A). Exposure to both contexts was equated and controlled throughout, and the status of the behavior as action or habit was determined by reinforcer devaluation methods (taste aversion conditioning). Results confirmed that action (1a) and habit (1b) renewed with action or habit status, respectively, when they were returned to Context A. Experiments 2a and 2b then similarly tested action and habit after extinction in an ABC renewal paradigm. Here, lever pressing that was trained in Context A and extinguished in Context B renewed as action in Context C regardless of whether it had been an action or habit before extinction. The apparent conversion of habit to action during renewal testing in Context C was consistent with other results suggesting that habits converted to action when the context was changed at the start of extinction. Together, the results suggest that extinction in a second context inhibits instrumental behaviors trained as either actions or habits in a context-specific manner. They also expand on prior findings suggesting that actions transfer across contexts, and that habits do not. A change of context may be sufficient to convert a habit to goal-directed action. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Some factors that restore goal-direction to a habitual behavior

Recent findings from our laboratory suggest that an extensively-practiced instrumental behavior can appear to be a goal-directed action (rather than a habit) when a second behavior is added and reinforced during intermixed final sessions (Shipman et al., 2018). The present experiments were designed to explore and understand this finding. All used the taste aversion method of devaluing the reinforcer to distinguish between goal-directed actions and habits. Experiment 1 confirmed that reinforcing a second response in a separate context (but not mere exposure to that context) can return an extensively-trained habit to the status of goal-directed action. Experiment 2 showed that training of the second response needs to be intermixed with training of the first response to produce this effect; training the second response after the first-response training was complete preserved the first response as a habit. Experiment 3 demonstrated that reinforcing the second response with a different reinforcer breaks the habit status of the first response. Experiment 4 found that free reinforcers (that were not response-contingent) were sufficient to restore goal-directed performance. Together, the results suggest that unexpected reinforcer delivery can render a habitual response goal-directed again.