Contextual control of instrumental actions and habits

A novel free-operant framework enables experimental habit induction in humans

Habits are a prominent feature of both adaptive and maladaptive behavior. Yet, despite substantial research efforts, there are currently no well-established experimental procedures for habit induction in humans. It is likely that laboratory experimental settings, as well as the session-based structure typically used in controlled experiments (also outside the lab), impose serious constraints on studying habits and other effects that are sensitive to context, motivation, and training duration and frequency. To overcome these challenges, we devised a unique real-world free-operant task structure, implemented through a novel smartphone application, whereby participants could freely enter the app (24 hours a day, 7 days a week) to win rewards. This procedure is free of typical laboratory constraints, yet well controlled. Using the canonical sensitivity to outcome devaluation criterion, we successfully demonstrated habit formation as a function of training duration, a long-standing challenge in the field. Additionally, we show a positive relationship between multiple facets of engagement/motivation and goal-directedness. We suggest that our novel paradigm can be used to study the neurobehavioral and psychological mechanism underlying habits in humans. Moreover, the real-world free-operant framework can potentially be used to examine other instrumental behavior-related questions, with greater face validity in naturalistic conditions.

Acute stress facilitates habitual behavior in female rats

Instrumental behavior can reflect the influence of goal-directed and habitual systems. Contemporary research suggests that stress may facilitate control by the habitual system under conditions where the behavior would otherwise reflect control by the goal-directed system. However, it is unclear how stress modulates the influence of these systems on instrumental responding to achieve this effect, particularly in females. Here, we examine whether a mild psychogenic stressor experienced before acquisition training (Experiment 1), or prior to the test of expression (Experiment 2) would influence goal-directed and habitual control of instrumental responding in female rats. In both experiments, rats acquired an instrumental nose-poke response for a sucrose reward. This was followed by a reinforcer devaluation phase in which half the rats in Stressed and Non-Stressed conditions received pairings of the sucrose pellet with illness induced by lithium chloride until they rejected the pellet when offered. The remaining rats received a control treatment consisting of pellets and illness on separate days (Unpaired). Control by goal-directed and habitual systems was evaluated in a subsequent nonreinforced test of nose poking. The results of Experiment 1 indicated that the Non-Stressed Paired group reduced nose-poking compared to the Unpaired controls, identifying the response as goal directed, whereas the Stressed Paired and Unpaired groups made a similar number of nose pokes identifying the response as habitual despite a similar amount of training. Results from Experiment 2 indicated habitual control of nose-poke responding was present when stress was experienced just prior to the test. Collectively, these data suggest that stress may facilitate habitual control by altering the relative influence of goal-directed and habitual processes underpinning instrumental behavior. These results may be clinically relevant for understanding the contributions of stress to dysregulated instrumental behavior in compulsive pathologies.

The behavioral origins of phylogenic responses and ontogenic habits

An examination of innate behavior and its possible origins suggests parallels with the formation of habitual behavior. Inflexible but adaptive responses-innate reflexive behavior, Pavlovian conditioned responses, and operant habits-may have evolved from variable behavior in phylogeny and ontogeny. This form of "plasticity-first" scientific narrative was unpopular post-Darwin but has recently gained credibility in evolutionary biology. The present article seeks to identify originating events and contingencies contributing to such inflexible but adaptive behavior at both phylogenic and ontogenic levels of selection. In ontogeny, the development of inflexible performance (i.e., habit) from variable operant behavior is reminiscent of the genetic accommodation of initially variable phylogenic traits. The effects characteristic of habit (e.g., unresponsiveness to reinforcer devaluation) are explicable as the result of a conflict between behaviors at distinct levels of selection. The present interpretation validates the practice of seeking hard analogies between evolutionary biology and operant behavior. Finding such parallels implies the validity of a claim that organismal behavior, both innate and learned, is a product of selection by consequences. A complete and coherent account of organismal behavior may ultimately focus on functional selective histories in much the same way evolutionary biology does with its subject matter.

The temporal structure of goal-directed and habitual operant behavior

Operant behavior can reflect the influence of goal-directed and habitual processes. These can be distinguished by changes to response rate following devaluation of the reinforcing outcome. Whether a response is goal directed or habitual depends on whether devaluation affects response rate. Response rate can be decomposed into frequencies of bouts and pauses by analyzing the distribution of interresponse times. This study sought to characterize goal-directed and habitual behaviors in terms of bout-initiation rate, within-bout response rate, bout length, and bout duration. Data were taken from three published studies that compared sensitivity to devaluation following brief and extended training with variable-interval schedules. Analyses focused on goal-directed and habitual responding, a comparison of a habitual response to a similarly trained response that had been converted back to goal-directed status after a surprising event, and a demonstration of contextual control of habit and goal direction in the same subjects. Across experiments and despite responses being clearly distinguished as goal directed and habitual by total response rate, analyses of bout-initiation rate, within-bout rate, bout length, and bout duration did not reveal a pattern that distinguished goal-directed from habitual responding.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Habit and persistence

Voluntary behaviors (operants) can come in two varieties: Goal-directed actions, which are emitted based on the remembered value of the reinforcer, and habits, which are evoked by antecedent cues and performed without the reinforcer's value in active memory. The two are perhaps most clearly distinguished with the reinforcer-devaluation test: Goal-directed actions are suppressed when the reinforcer is separately devalued and responding is tested in extinction, and habitual behaviors are not. But what is the function of habit learning? Habits are often thought to be strong and unusually persistent. The present selective review examines this idea by asking whether habits identified by the reinforcer-devaluation test are more resistant to extinction, resistant to the effects of other contingency change, vulnerable to relapse, resistant to the weakening effects of context change, or permanently in place once they are learned. Surprisingly little evidence supports the idea that habits are permanent or more persistent. Habits are more context-specific than goal-directed actions are. Methods that make behavior persistent do not necessarily work by encouraging habit. The function of habit learning may not be to make a behavior strong or more persistent but to make it automatic and efficient in a particular context.

Positive correlation between measures of habitual responding and motivated responding in mice

Habit and motivation are thought to be separate processes, with motivated behavior often considered to be goal directed, whereas habits are defined by the absence of goal-directed control over behavior. However, there has been increasing interrogation of the binary nature of habitual versus goal-directed behavior. Furthermore, although drug and alcohol exposure can promote the formation of habits, drug seeking itself can also be highly flexible, pointing toward the need for complex consideration of the parallel processes that drive behavior. The goal of the current study was to determine whether there was a relation between motivation-as measured by progressive ratio-and habit-as measured by contingency degradation-and whether this relation was affected by ethanol exposure history and sex. The results showed that these measures were positively correlated such that greater contingency insensitivity was associated with achieving higher break points on the progressive-ratio task. However, this relation depended on reinforcement schedule history, ethanol exposure history, and sex. These point to potential relations between measures of habit and motivation and stress the importance of carefully parsing behavioral findings and assays. These findings are also expected to inform future substance use research, as drug history may affect these relations.

Relating different perspectives on how outcomes of behavior influence behavior

Many researchers have tackled the question of how behavior is influenced by its outcomes. Some have adopted a nonmechanistic (functional) perspective that attempts to describe the influence of outcomes on behavior. Others have adopted a mechanistic (cognitive) perspective that attempts to explain the influence of outcomes on behavior. Orthogonal to this distinction, some have focused on the influence of outcomes that a behavior had in the past, whereas others also consider the influence of outcomes that a behavior might have in the future. In this article, we relate these different perspectives with the goal of reducing misunderstandings and fostering collaborations between researchers who adopt different perspectives on the common question of how behavior is influenced by its outcomes.

Resurgence of goal-directed actions and habits

This study investigated how goal-directed and habitual behaviors recover after extinction within the context of the resurgence effect, a form of relapse induced by the removal or worsening of alternative reinforcement. Rats were trained to press a target lever with one reinforcer (O1) for either minimal (4) or extended (16) sessions. An extinction test after the completion of O1 devaluation confirmed that minimal and extended training formed goal-directed and habitual behaviors, respectively. Then, pressing an alternative lever was reinforced with a second reinforcer (O2) while the target response was placed on extinction. When O2 was discontinued, the minimally trained target response resurged with goal-directed status as in the extinction test. However, the extinguished habitual behavior in the extensively trained rats did not recover as a habit but instead with goal-directed status, possibly due to the context specificity of habits or the introduction of a new response-reinforcer contingency. The critical finding that reinforcer devaluation consistently led to less resurgence regardless of the amount of acquisition training provides a clinical implication that coupling differential-reinforcement-of-alternative-behavior (DRA) treatments with the devaluation of the associated reinforcer of problematic behavior could effectively diminish its recurrence.

Contribution of dorsal versus ventral hippocampus to the hierarchical modulation of goal-directed actions in rats

Adaptive behaviour often necessitates that animals learn about events in a manner that is specific to a particular context or environment. These hierarchical organisations allow the animal to decide which action is the most appropriate when faced with ambiguous or conflicting possibilities. This study examined the role of hippocampus in enabling animals to use the context to guide action selection. We used a hierarchical instrumental outcome devaluation task in which male rats learn that the context provides information about the unique action-outcome relations that are in effect. We first confirmed that rats encode and use hierarchical context-(action-outcome) relations. We then show that chemogenetic inhibition of ventral hippocampus impairs both the encoding and retrieval of these associations, while inhibition of dorsal hippocampus impairs only the retrieval. Importantly, neither dorsal nor ventral hippocampus was required for goal-directed behaviour per se as these impairments only emerged when rats were forced to use the context to identify the current action-outcome relationships. These findings are discussed with respect to the role of the hippocampus and its broader circuitry in the contextual modulation of goal-directed behaviour and the importance of hierarchical associations in flexible behaviour.

Food Restriction Level and Reinforcement Schedule Differentially Influence Behavior during Acquisition and Devaluation Procedures in Mice

Behavioral strategies are often classified based on whether reinforcer value controls reinforcement. Value-sensitive behaviors, in which animals update their actions when reinforcer value is changed, are classified as goal-directed; conversely, value-insensitive actions, where behavior remains consistent when the reinforcer is removed or devalued, are considered habitual. Basic reinforcement schedules can help to bias behavior toward either process: random ratio (RR) schedules are thought to promote the formation of goal-directed behaviors while random intervals (RIs) promote habitual control. However, how the schedule-specific features of these tasks interact with other factors that influence learning to control behavior has not been well characterized. Using male and female mice, we asked how distinct food restriction levels, a strategy often used to increase task engagement, interact with RR and RI schedules to control performance during task acquisition and devaluation procedures. We determined that food restriction level has a stronger effect on the behavior of mice following RR schedules compared with RI schedules, and that it promotes a decrease in response rate during devaluation procedures that is best explained by the effects of extinction rather than devaluation. Surprisingly, food restriction accelerated the decrease in response rates observed following devaluation across sequential extinction sessions, but not within a single session. Our results support the idea that the relationships between schedules and behavioral control strategies are not clear-cut and suggest that an animal's engagement in a task must be accounted for, together with the structure of reinforcement schedules, to appropriately interpret the cognitive underpinnings of behavior.

Early-life seizures alter habit behavior formation and fronto-striatal circuit dynamics

Obsessive compulsive disorder (OCD) can occur comorbidly with epilepsy; both are complex, disruptive disorders that lower quality of life. Both OCD and epilepsy are disorders of hyperexcitable circuits, but it is unclear whether common circuit pathology may underlie the co-occurrence of these two neuropsychiatric disorders. Here, we induced early-life seizures (ELS) in rats to examine habit formation as a model for compulsive behaviors. Compulsive, repetitive behaviors in OCD utilize the same circuitry as habit formation. We hypothesized that rats with ELS could be more susceptible to habit formation than littermate controls, and that altered behavior would correspond to altered signaling in fronto-striatal circuits that underlie decision-making and action initiation. Here, we show instead that rats with ELS were significantly less likely to form habit behaviors compared with control rats. This behavioral difference corresponded with significant alterations to temporal coordination within and between brain regions that underpin the action to habit transition: 1) phase coherence between the lateral orbitofrontal cortex and dorsomedial striatum (DMS) and 2) theta-gamma coupling within DMS. Finally, we used cortical electrical stimulation as a model of transcranial magnetic stimulation (TMS) to show that temporal coordination of fronto-striatal circuits in control and ELS rats are differentially susceptible to potentiating and suppressive stimulation, suggesting that altered underlying circuit physiology may lead to altered response to therapeutic interventions such as TMS.

Investigating habits in humans with a symmetrical outcome-revaluation task

The translation of the outcome-devaluation paradigm to study habit in humans has yielded interesting insights but proven to be challenging. We present a novel, outcome-revaluation task with a symmetrical design, in the sense that half of the available outcomes are always valuable and the other half not-valuable. In the present studies, during the instrumental learning phase, participants learned to respond (Go) to certain stimuli to collect valuable outcomes (and points) while refraining to respond (NoGo) to stimuli signaling not-valuable outcomes. Half of the stimuli were short-trained, while the other half were long-trained. Subsequently, in the test phase, the signaled outcomes were either value-congruent with training (still-valuable and still-not-valuable), or value-incongruent (devalued and upvalued). The change in outcome value on value-incongruent trials meant that participants had to flexibly adjust their behavior. At the end of the training phase, participants completed the self-report behavioral automaticity index - providing an automaticity score for each stimulus-response association. We conducted two experiments using this task, that both provided evidence for stimulus-driven habits as reflected in poorer performance on devalued and upvalued trials relative to still-not-valuable trials and still-valuable trials, respectively. While self-reported automaticity increased with longer training, behavioral flexibility was not affected. After extended training (Experiment 2), higher levels of self-reported automaticity when responding to stimuli signaling valuable outcomes were related to more 'slips of action' when the associated outcome was subsequently devalued. We conclude that the symmetrical outcome-revaluation task provides a promising paradigm for the experimental investigation of habits in humans.

Revealing abrupt transitions from goal-directed to habitual behavior

A fundamental tenet of animal behavior is that decision-making involves multiple 'controllers.' Initially, behavior is goal-directed, driven by desired outcomes, shifting later to habitual control, where cues trigger actions independent of motivational state. Clark Hull's question from 1943 still resonates today: "Is this transition abrupt, or is it gradual and progressive?"1 Despite a century-long belief in gradual transitions, this question remains unanswered2,3 as current methods cannot disambiguate goal-directed versus habitual control in real-time. Here, we introduce a novel 'volitional engagement' approach, motivating animals by palatability rather than biological need. Offering less palatable water in the home cage4,5 reduced motivation to 'work' for plain water in an auditory discrimination task when compared to water-restricted animals. Using quantitative behavior and computational modeling6, we found that palatability-driven animals learned to discriminate as quickly as water-restricted animals but exhibited state-like fluctuations when responding to the reward-predicting cue-reflecting goal-directed behavior. These fluctuations spontaneously and abruptly ceased after thousands of trials, with animals now always responding to the reward-predicting cue. In line with habitual control, post-transition behavior displayed motor automaticity, decreased error sensitivity (assessed via pupillary responses), and insensitivity to outcome devaluation. Bilateral lesions of the habit-related dorsolateral striatum7 blocked transitions to habitual behavior. Thus, 'volitional engagement' reveals spontaneous and abrupt transitions from goal-directed to habitual behavior, suggesting the involvement of a higher-level process that arbitrates between the two.

Partial reinforcement extinction and omission effects in the elimination and recovery of discriminated operant behavior

Three experiments explored how training reinforcement schedules and context influence the elimination and recovery of human operant behavior. In Experiment 1, participants learned a discriminated operant response in Context A before the response was eliminated with extinction in Context B. They then received a final test in each context. Groups were trained with a discriminative stimulus that predicted a reinforced response on either every trial (continuous reinforcement [CRF]) or some of the trials (partial reinforcement [PRF]). Extinction was slower following PRF training (a partial reinforcement extinction effect [PREE]) and extinguished responding increased when tested in Context A ("ABA" renewal). Experiment 2 further confirmed the PREE was obtained equally whether extinction occurred in the training context (Context A) or a new context (Context B) which is consistent with trial-based accounts of the PREE. Experiment 3 used the same design as Experiment 1 to evaluate the influence of training reinforcement on response elimination with an omission contingency. Across the omission training phase in Context B, the decrease in responding occurred more slowly in the PRF-trained group in comparison to the CRF-trained group, perhaps the first demonstration of what might be termed a PRF omission effect. Again, ABA renewal was observed in Context A. Training reinforcement schedule therefore had a similar influence on response elimination with extinction and omission. Elimination and recovery of human instrumental behavior, with extinction or omission, are influenced by training reinforcement schedule and context. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration (2 h/day). We then exposed them to 4 days of punishment testing, in which footshock (0.4 mA, 0.3 s) was delivered randomly on one-third of trials, immediately following completion of seeking and prior to extension of the taking lever. Before and after punishment testing (4 days pre-punishment and ≥4 days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Individual Differences in Corticostriatal White-matter Tracts Predict Successful Daily-life Routine Formation

Despite good intentions, people often fail to cross the "intention-behavior gap," especially when goal achievement requires repeated action. To bridge this gap, the formation of automatized routines may be crucial. However, people may differ in the tendency to switch from goal-directed toward habitual control. To shed light on why some people succeed in forming routines while others struggle, the present study related the automatization of a novel, daily routine to individual differences in white-matter connectivity in corticostriatal networks that have been implicated in goal-directed and habitual control. Seventy-seven participants underwent diffusion-weighted imaging and formed the daily routine of taking a (placebo) pill for 3 weeks. Pill intake was measured by electronic pill boxes, and participants filled out a daily online questionnaire on the subjective automaticity of this behavior. Automatization of pill intake was negatively related to striatal (mainly caudate) connectivity with frontal goal-directed and cognitive control regions, namely, ventromedial pFC and anterior cingulate gyrus. Furthermore, daily pill intake was positively related to individual differences in striatal (mainly caudate) connectivity with cognitive control regions, including dorsolateral and anterior pFC. Therefore, strong control networks may be relevant for implementing a new routine but may not benefit its automatization. We also show that habit tendency (assessed with an outcome-devaluation task), conscientiousness, and daily life regularity were positively related to routine automatization. This translational study moves the field of habit research forward by relating self-reported routine automatization to individual differences in performance on an experimental habit measure and to brain connectivity.

Does stress consistently favor habits over goal-directed behaviors? Data from two preregistered exact replication studies

Instrumental learning is controlled by two distinct parallel systems: goal-directed (action-outcome) and habitual (stimulus-response) processes. Seminal research by Schwabe and Wolf (2009, 2010) has demonstrated that stress renders behavior more habitual by decreasing goal-directed control. More recent studies yielded equivocal evidence for a stress-induced shift towards habitual responding, yet these studies used different paradigms to evaluate instrumental learning or used different stressors. Here, we performed exact replications of the original studies by exposing participants to an acute stressor either before (cf. Schwabe and Wolf, 2009) or directly after (cf. Schwabe and Wolf, 2010) an instrumental learning phase in which they had learned that distinct actions led to distinct, rewarding food outcomes (i.e., instrumental learning). Then, following an outcome devaluation phase in which one of the food outcomes was consumed until participants were satiated, action-outcome associations were tested in extinction. Despite successful instrumental learning and outcome devaluation and increased subjective and physiological stress levels following stress exposure, the stress and no-stress groups in both replication studies responded indifferently to valued and devalued outcomes. That is, non-stressed participants failed to demonstrate goal-directed behavioral control, thereby rendering the critical test of a shift from goal-directed to habitual control in the stress group inapt. Several reasons for these replication failures are discussed, including the rather indiscriminate devaluation of outcomes that may have contributed to indifferent responding during extinction, which emphasize the need to further our understanding of the boundary conditions in research aimed at demonstrating a stress-induced shift towards habitual control.

Food cue reactivity: Neurobiological and behavioral underpinnings

The modern obesogenic environment contains an abundance of food cues (e.g., sight, smell of food) as well cues that are associated with food through learning and memory processes. Food cue exposure can lead to food seeking and excessive consumption in otherwise food-sated individuals, and a high level of food cue responsivity is a risk factor for overweight and obesity. Similar food cue responses are observed in experimental rodent models, and these models are therefore useful for mechanistically identifying the neural circuits mediating food cue responsivity. This review draws from both experimental rodent models and human data to characterize the behavioral and biological processes through which food-associated stimuli contribute to overeating and weight gain. Two rodent models are emphasized - cue-potentiated feeding and Pavlovian-instrumental transfer - that provide insight in the neural circuits and peptide systems underlying food cue responsivity. Data from humans are highlighted that reveal physiological, psychological, and neural mechanisms that connect food cue responsivity with overeating and weight gain. The collective literature identifies connections between heightened food cue responsivity and obesity in both rodents and humans, and identifies underlying brain regions (nucleus accumbens, amygdala, orbitofrontal cortex, hippocampus) and endocrine systems (ghrelin) that regulate food cue responsivity in both species. These species similarities are encouraging for the possibility of mechanistic rodent model research and further human research leading to novel treatments for excessive food cue responsivity in humans.

Cyclic estrogen and progesterone during instrumental acquisition contributes to habit formation in female rats

Habit formation is thought to involve two parallel processes that are mediated by distinct neural substates: one that suppresses goal-directed behavior, and one that facilitates stimulus-response (S-R) learning, which underscores habitual behavior. In previous studies we showed that habitual responding emerges early during instrumental training in gonadally-intact female, compared to male, rats. The present study aimed to determine the role of ovarian hormones during instrumental acquisition in the transition from goal-directed to habitual behavior in female rats. Ovariectomized (OVX) female rats were given subcutaneous silastic capsules that released low levels of 17-β estradiol (E2) to maintain estrogen receptor availability. Rats were assigned to one of three hormone treatment conditions: no additional hormone replacement (Control group), replacement with high E2 (High E2 group), or replacement with high E2 followed by progesterone (High E2 + P4 group). Hormone replacement occurred twice during acquisition to mimic natural hormone fluctuations. At test, the Control and High E2 groups demonstrated responding that was sensitive to devaluation by lithium chloride-induced illness, indicating goal-directed behavior. In contrast, the High E2 + P4 group exhibited a pattern of devaluation-insensitive, habitual responding, that suggested the suppression of goal-directed processes. In a follow-up experiment, similar procedures were conducted, however during acquisition, OVX rats were given cyclic high E2 plus medroxy-progesterone (MPA), a form of progesterone that does not metabolize to neuroactive metabolites. In this group, goal-directed behavior was observed. These data indicate that habit formation is not facilitated in low estrogen states, nor in the presence of cyclic high E2. However, cyclic high E2, together with progesterone during acquisition, appears to facilitate the early emergence of habitual responding. Furthermore, these data suggest that a neuroactive progesterone metabolite, like allopregnanolone, in combination with high cyclic E2, supports this phenomenon.

The Unintentional Nonconformist: Habits Promote Resistance to Social influence

This research tests a novel source of resistance to social influence-the automatic repetition of habit. In three experiments, participants with strong habits failed to align their behavior with others. Specifically, participants with strong habits to drink water in a dining hall or snack while working did not mimic others' drinking or eating, whereas those with weak habits conformed. Similarly, participants with strong habits did not shift expectations that they would act in line with descriptive norms, whereas those with weak habits reported more normative behavioral expectations. This habit resistance was not due to a failure to perceive influence: Both strong and weak habit participants' recalled others' behavior accurately, and it was readily accessible. Furthermore, strong habit participants shifted their normative beliefs but not behavior in line with descriptive norms. Thus, habits create behavioral resistance despite people's recognition and acceptance of social influence.

Animal models of action control and cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) has historically been considered a motor disorder induced by a loss of dopaminergic neurons in the substantia nigra pars compacta. More recently, it has been recognized to have significant non-motor symptoms, most prominently cognitive symptoms associated with a dysexecutive syndrome. It is common in the literature to see motor and cognitive symptoms treated separately and, indeed, there has been a general call for specialized treatment of the latter, particularly in the more severe cases of PD with mild cognitive impairment and dementia. Animal studies have similarly been developed to model the motor or non-motor symptoms. Nevertheless, considerable research has established that segregating consideration of cognition from the precursors to motor movement, particularly movement associated with goal-directed action, is difficult if not impossible. Indeed, on some contemporary views cognition is embodied in action control, something that is particularly prevalent in theory and evidence relating to the integration of goal-directed and habitual control processes. The current paper addresses these issues within the literature detailing animal models of cognitive dysfunction in PD and their neural and neurochemical bases. Generally, studies using animal models of PD provide some of the clearest evidence for the integration of these action control processes at multiple levels of analysis and imply that consideration of this integrative process may have significant benefits for developing new approaches to the treatment of PD.

Cognitive strategies shift information from single neurons to populations in prefrontal cortex

Neurons in primate lateral prefrontal cortex (LPFC) play a critical role in working memory (WM) and cognitive strategies. Consistent with adaptive coding models, responses of these neurons are not fixed but flexibly adjust on the basis of cognitive demands. However, little is known about how these adjustments affect population codes. Here, we investigated ensemble coding in LPFC while monkeys implemented different strategies in a WM task. Although single neurons were less tuned when monkeys used more stereotyped strategies, task information could still be accurately decoded from neural populations. This was due to changes in population codes that distributed information among a greater number of neurons, each contributing less to the overall population. Moreover, this shift occurred for task-relevant, but not irrelevant, information. These results demonstrate that cognitive strategies that impose structure on information held in mind rearrange population codes in LPFC, such that information becomes more distributed among neurons in an ensemble.

Evaluating effects of context changes on resurgence in humans

Laboratory models of relapse provide methods for evaluating challenges to behavioral treatments with differential reinforcement of an alternative response (DRA). Resurgence occurs with the worsening of conditions of reinforcement for appropriate behavior and renewal occurs when transitioning out of a treatment context. Across five experiments, participants recruited via online crowdsourcing pressed onscreen buttons to earn points exchangeable for money and contexts sometimes changed through changes in the background image. Returning to the training context (ABA, Experiment 1) and transitioning to a novel context (ABC, Experiment 2) produced greater resurgence when removing alternative reinforcement in comparison with remaining in the treatment context (ABB). In contrast, we observed little difference in resurgence among AAA, ABB, and AAC context manipulations (Experiment 3) and ABA, ABC, and AAC context manipulations (Experiment 4). In Experiment 5, we evaluated relative contributions of the presence versus absence of context changes (ABA vs. ABB) in combination with or without the removal of alternative reinforcement. Both changing context and removing alternative reinforcement increased responding in isolation and the combination produced greater-than-additive effects. Overall, the present findings demonstrate a consistent effect of removing alternative reinforcement on relapse that, under certain conditions, can be enhanced by context change.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Effect of context on the instrumental reinforcer devaluation effect produced by taste-aversion learning

Four experiments manipulated the context in which taste-aversion conditioning occurred when the reinforcer was devalued after instrumental learning. In all experiments, rats learned to lever press in an operant conditioning chamber and then had an aversion to the food-pellet reinforcer conditioned by pairing it with lithium chloride (LiCl) in either that context or a different context. Lever pressing was then tested in extinction to assess its status as a goal-directed action. In Experiment 1, aversion conditioning in the operant conditioning chamber suppressed lever-pressing during the test, but aversion conditioning in the home cage did not. Exposure to the averted pellet in the operant conditioning chamber after conditioning in the home cage did not change this effect (Experiment 2). The same pattern was observed when the different context was a second operant-style chamber (counterbalanced), exposure to the contexts was controlled, and pellets were presented in them in the same manner (Experiment 3). The greater effect of aversion conditioning in the instrumental context was not merely due to potentiated contextual conditioning (Experiment 4). Importantly, consumption tests revealed that the aversion conditioned in the different context had transferred to the test context. Thus, when reinforcer devaluation occurred in a different context, the rats lever pressed in extinction for a reinforcer they would otherwise reject. The results suggest that animals encode contextual information about the reinforcer during instrumental learning and suggest caution in making inferences about action versus habit learning when the reinforcer is devalued in a different context. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

What Is Learned in Pavlovian Conditioning in Crickets? Revisiting the S-S and S-R Learning Theories

In Pavlovian conditioning in mammals, two theories have been proposed for associations underlying conditioned responses (CRs). One theory, called S-S theory, assumes an association between a conditioned stimulus (CS) and internal representation of an unconditioned stimulus (US), allowing the animal to adjust the CR depending on the current value of the US. The other theory, called S-R theory, assumes an association or connection between the CS center and the CR center, allowing the CS to elicit the CR. Whether these theories account for Pavlovian conditioning in invertebrates has remained unclear. In this article, results of our studies in the cricket Gryllus bimaculatus are reviewed. We showed that after a standard amount of Pavlovian training, crickets exhibited no response to odor CS when water US was devalued by providing it until satiation, whereas after extended training, they exhibited a CR after US devaluation. An increase of behavioral automaticity by extended training has not been reported in Pavlovian conditioning in any other animals, but it has been documented in instrumental conditioning in mammals. Our pharmacological analysis suggested that octopamine neurons mediate US (water) value signals and control execution of the CR after standard training. The control, however, diminishes with extension of training and hence the CR becomes insensitive to the US value. We also found that the nature of the habitual response after extended Pavlovian training in crickets is not the same as that after extended instrumental training in mammals concerning the context specificity. Adaptive significance and evolutionary implications for our findings are discussed.

BEHAVIORAL AND NEUROBIOLOGICAL MECHANISMS OF PAVLOVIAN AND INSTRUMENTAL EXTINCTION LEARNING

This article reviews the behavioral neuroscience of extinction, the phenomenon in which a behavior that has been acquired through Pavlovian or instrumental (operant) learning decreases in strength when the outcome that reinforced it is removed. Behavioral research indicates that neither Pavlovian nor operant extinction depends substantially on erasure of the original learning but instead depends on new inhibitory learning that is primarily expressed in the context in which it is learned, as exemplified by the renewal effect. Although the nature of the inhibition may differ in Pavlovian and operant extinction, in either case the decline in responding may depend on both generalization decrement and the correction of prediction error. At the neural level, Pavlovian extinction requires a tripartite neural circuit involving the amygdala, prefrontal cortex, and hippocampus. Synaptic plasticity in the amygdala is essential for extinction learning, and prefrontal cortical inhibition of amygdala neurons encoding fear memories is involved in extinction retrieval. Hippocampal-prefrontal circuits mediate fear relapse phenomena, including renewal. Instrumental extinction involves distinct ensembles in corticostriatal, striatopallidal, and striatohypothalamic circuits as well as their thalamic returns for inhibitory (extinction) and excitatory (renewal and other relapse phenomena) control over operant responding. The field has made significant progress in recent decades, although a fully integrated biobehavioral understanding still awaits.

Renewal of goal direction with a context change after habit learning

An instrumental action can be goal-directed after a moderate amount of practice and then convert to habit after more extensive practice. Recent evidence suggests, however, that habits can return to action status after different environmental manipulations. The present experiments therefore asked whether habit learning interferes with goal direction in a context-dependent manner like other types of retroactive interference (e.g., extinction, punishment, counterconditioning). In Experiment 1, rats were given a moderate amount of instrumental training to form an action in one context (Context A) and then more extended training of the same response to form a habit in another context (Context B). We then performed reinforcer devaluation with taste aversion conditioning in both contexts, and tested the response in both contexts. The response remained habitual in Context B, but was goal-directed in Context A, indicating renewal of goal direction after habit learning. Experiment 2 expanded on Experiment 1 by testing the response in a third context (Context C). It found that the habitual response also renewed as action in this context. Together, the results establish a parallel between habit and extinction learning: Conversion to habit does not destroy action knowledge, but interferes with it in a context-specific way. They are also consistent with other results suggesting that habit is specific to the context in which it is learned, whereas goal-direction can transfer between contexts. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Reduction of contextual control of conditioned responses by extended Pavlovian training in an insect

The effect of repetitive training on learned behavior has been an important subject in neuroscience. In instrumental conditioning in mammals, learned action early in training is often goal-driven and controlled by outcome expectancy, but as training progresses, it becomes more habitual and insensitive to outcome devaluation. Similarly, we recently showed in Pavlovian conditioning in crickets (Gryllus bimaculatus) that a conditioned response (CR) is initially sensitive to devaluation of the unconditioned stimulus but becomes insensitive to it after extended training. It is known that habitual responses after extended instrumental training are characterized by a higher context specificity than are initial goal-directed actions in mammals. In this study, we investigated whether this is applicable to Pavlovian conditioning in crickets. In crickets that received a standard amount of training to associate an odor with water reward under illumination, CR under illumination was stronger than that in the dark. In crickets that received extended training under illumination, on the other hand, the level of CR did not differ in different light conditions. Further experiments confirmed that context specificity decreases with the development of behavioral automaticity by extended training, as opposed to findings in instrumental training in mammals. We conclude that the nature of habitual behaviors after extended training differs in different learning systems of animals.

Appetitive traits as targets for weight loss: The role of food cue responsiveness and satiety responsiveness

Individuals with overweight or obesity (OW/OB) are at increased risk for significant physical and psychological comorbidities. The current treatment for OW/OB is behavioral weight loss, which provides psychoeducation on nutrition and physical activity, as well as behavior therapy skills. However, behavioral weight loss is not effective for the majority of the individuals who participate. Research suggests that overeating, or eating past nutritional needs, is one of the leading causes of weight gain. Accumulating evidence suggests that appetitive traits, such as food cue responsiveness and satiety responsiveness, are associated with overeating and weight in youth and adults. The following review presents the current literature on the relationship between food cue responsiveness, satiety responsiveness, overeating, and OW/OB. Research suggests that higher food cue responsiveness and lower satiety responsiveness are associated with overeating and OW/OB cross-sectionally and longitudinally. Emerging data suggest that food cue responsiveness and satiety responsiveness may exist along the same continuum and can be targeted to manage overeating and reduce weight. We have developed a treatment model targeting food cue responsiveness and satiety responsiveness to reduce overeating and weight and have preliminary feasibility, acceptability, and efficacy data, with testing currently being conducted in larger trials. Through programs targeting appetitive traits we hope to develop an alternative weight loss model to assist individuals with a propensity to overeat.

Involvement of the rodent prelimbic and medial orbitofrontal cortices in goal-directed action: A brief review

Goal-directed action refers to selecting behaviors based on the expectation that they will be reinforced with desirable outcomes. It is typically conceptualized as opposing habit-based behaviors, which are instead supported by stimulus-response associations and insensitive to consequences. The prelimbic prefrontal cortex (PL) is positioned along the medial wall of the rodent prefrontal cortex. It is indispensable for action-outcome-driven (goal-directed) behavior, consolidating action-outcome relationships and linking contextual information with instrumental behavior. In this brief review, we will discuss the growing list of molecular factors involved in PL function. Ventral to the PL is the medial orbitofrontal cortex (mOFC). We will also summarize emerging evidence from rodents (complementing existing literature describing humans) that it too is involved in action-outcome conditioning. We describe experiments using procedures that quantify responding based on reward value, the likelihood of reinforcement, or effort requirements, touching also on experiments assessing food consumption more generally. We synthesize these findings with the argument that the mOFC is essential to goal-directed action when outcome value information is not immediately observable and must be recalled and inferred.

Context and renewal of habits and goal-directed actions after extinction

Instrumental behaviors that are goal-directed actions after moderate amounts of training can become habits after more extended training. Little research has asked how actions and habits are affected by retroactive interference treatments like extinction. The present experiments begin to fill this gap in the literature. In Experiments 1a and 1b, lever pressing in rats was minimally trained (1a) or extensively trained (1b) in one context (Context A), extinguished in a second context (Context B), and then tested in the acquisition context (Context A). Exposure to both contexts was equated and controlled throughout, and the status of the behavior as action or habit was determined by reinforcer devaluation methods (taste aversion conditioning). Results confirmed that action (1a) and habit (1b) renewed with action or habit status, respectively, when they were returned to Context A. Experiments 2a and 2b then similarly tested action and habit after extinction in an ABC renewal paradigm. Here, lever pressing that was trained in Context A and extinguished in Context B renewed as action in Context C regardless of whether it had been an action or habit before extinction. The apparent conversion of habit to action during renewal testing in Context C was consistent with other results suggesting that habits converted to action when the context was changed at the start of extinction. Together, the results suggest that extinction in a second context inhibits instrumental behaviors trained as either actions or habits in a context-specific manner. They also expand on prior findings suggesting that actions transfer across contexts, and that habits do not. A change of context may be sufficient to convert a habit to goal-directed action. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The Meaning of Behavior: Discriminating Reflex and Volition in the Brain

The ability to establish behaviorally what psychological capacity an animal is deploying-to discern accurately what an animal is doing-is key to functional analyses of the brain. Our current understanding of these capacities suggests, however, that this task is complex; there is evidence that multiple capacities are engaged simultaneously and contribute independently to the control of behavior. As such, establishing the contribution of a cell, circuit, or neural system to any one function requires careful dissection of that role from its influence on other functions and, therefore, the careful selection and design of behavioral tasks fit for that purpose. Here I describe recent research that has sought to utilize behavioral tools to investigate the neural bases of instrumental conditioning, particularly the circuits and systems supporting the capacity for goal-directed action, as opposed to conditioned reflexes and habits, and how these sources of action control interact to generate adaptive behavior.

Sign-tracking behavior is sensitive to outcome devaluation in a devaluation context-dependent manner: implications for analyzing habitual behavior

Motivationally attractive cues can draw in behavior in a phenomenon termed incentive salience. Incentive cue attraction is an important model for animal models of drug seeking and relapse. One question of interest is the extent to which the pursuit of motivationally attractive cues is related to the value of the paired outcome or can become unrelated and habitual. We studied this question using a sign-tracking (ST) paradigm in rats, in which a lever stimulus preceding food reward comes to elicit conditioned lever-interaction behavior. We asked whether reinforcer devaluation by means of conditioned taste aversion, a classic test of habitual behavior, can modify ST to incentive cues, and whether this depends upon the manner in which reinforcer devaluation takes place. In contrast to several recent reports, we conclude that ST is indeed sensitive to reinforcer devaluation. However, this effect depends critically upon the congruence between the context in which taste aversion is learned and the context in which it is tested. When the taste aversion successfully transfers to the testing context, outcome value strongly influences ST behavior, both when the outcome is withheld (in extinction) and when animals can learn from outcome feedback (reacquisition). When taste aversion does not transfer to the testing context, ST remains high. In total, the extent to which ST persists after outcome devaluation is closely related to the extent to which that outcome is truly devalued in the task context. We believe this effect of context on devaluation can reconcile contradictory findings about the flexibility/inflexibility of ST. We discuss this literature and relate our findings to the study of habits generally.

The impacts of acquisition and extinction cues on ABC renewal of voluntary behaviors

In two instrumental conditioning experiments with rats, we examined the impacts of acquisition and extinction cues on ABC renewal of instrumental behavior. Animals were reinforced with food for lever pressing in one context, followed by extinction of the response in a second one. Presentations of a brief tone accompanied extinction in Experiment 1 (extinction cue), and acquisition in Experiment 2 (acquisition cue). A final test in a third context revealed that instrumental responding was decreased in the presence of the extinction cue, whereas it was increased in the presence of the acquisition cue. We discuss theoretical and clinical implications of our results.

Unexpected food outcomes can return a habit to goal-directed action

Three experiments examined the return of a habitual instrumental response to the status of goal-directed action. In all experiments, rats received extensive training in which lever pressing was reinforced with food pellets on a random-interval schedule of reinforcement. In Experiment 1, the extensively-trained response was not affected by conditioning a taste aversion to the reinforcer, and was therefore considered a habit. However, if the response had earned a new and unexpected food pellet during the final training session, the response was affected by taste aversion conditioning to the (first) reinforcer, and had thus been converted to a goal-directed action. In Experiment 3, 30 min of prefeeding with an irrelevant food pellet immediately before the test also converted a habit back to action, as judged by the taste-aversion devaluation method. That result was consistent with difficulty in finding evidence of habit with the sensory-specific satiety method after extensive instrumental training (Experiment 2). The results suggest that an instrumental behavior's status as a habit is not permanent, and that a habit can be returned to action status by associating it with a surprising reinforcer (Experiment 1) or by giving the animal an unexpected prefeeding immediately prior to the action/habit test (Experiment 3).

Inactivation of the prelimbic cortex attenuates operant responding in both physical and behavioral contexts

The present experiments aimed to expand our understanding of the role of the prelimbic cortex (PL) in the contextual control of instrumental behavior. Research has previously shown that the PL is involved when the "physical context," or chamber in which an instrumental behavior is trained, facilitates performance of the instrumental response (Trask, Shipman, Green, & Bouton, 2017). Recently, evidence has suggested that when a sequence of two instrumental behaviors is required to earn a reinforcing outcome, the first response (rather than the physical chamber) can be the "behavioral context" for the second response (Thrailkill, Trott, Zerr, and Bouton, 2016). Could the PL also be involved in this kind of contextual control? Here rats first learned a heterogenous behavior chain in which the first response (i.e., pressing a lever or pulling a chain) was cued by a discriminative stimulus and led to a second stimulus which cued a second response (i.e., pulling a chain or pressing a lever); the second response led to a sucrose reward. When the first and second responses were tested in isolation in the training context, pharmacological inactivation of the PL resulted in a reduction of the first response, but not the second response. When the second response was performed in the "context" of the first response (i.e., as part of the behavior chain) however, PL inactivation reduced the second response. Overall, these results support the idea that the PL is important for mediating the effects of a training context on instrumental responding, whether the context is physical or behavioral.

The Law of Recency: An Episodic Stimulus-Response Retrieval Account of Habit Acquisition

A habit is a regularity in automatic responding to a specific situation. Classical learning psychology explains the emergence of habits by an extended learning history during which the response becomes associated to the situation (learning of stimulus-response associations) as a function of practice ("law of exercise") and/or reinforcement ("law of effect"). In this paper, we propose the "law of recency" as another route to habit acquisition that draws on episodic memory models of automatic response regulation. According to this account, habitual responding results from (a) storing stimulus-response episodes in memory, and (b) retrieving these episodes when encountering the stimulus again. This leads to a reactivation of the response that was bound to the stimulus (c) even in the absence of extended practice and reinforcement. As a measure of habit formation, we used a modified color-word contingency learning (CL) paradigm, in which irrelevant stimulus features (i.e., word meaning) were predictive of the to-be-executed color categorization response. The paradigm we developed allowed us to assess effects of global CL and of an instance-based episodic response retrieval simultaneously within the same experiment. Two experiments revealed robust CL as well as episodic response retrieval effects. Importantly, these effects were not independent: Controlling for response retrieval effects eliminated effects of CL, which supports the claim that habit formation can be mediated by episodic retrieval processes, and that short-term binding effects are not fundamentally separate from long-term learning processes. Our findings have theoretical and practical implications regarding (a) models of long-term learning, and (b) the emergence and change of habitual responding.

Some factors that restore goal-direction to a habitual behavior

Recent findings from our laboratory suggest that an extensively-practiced instrumental behavior can appear to be a goal-directed action (rather than a habit) when a second behavior is added and reinforced during intermixed final sessions (Shipman et al., 2018). The present experiments were designed to explore and understand this finding. All used the taste aversion method of devaluing the reinforcer to distinguish between goal-directed actions and habits. Experiment 1 confirmed that reinforcing a second response in a separate context (but not mere exposure to that context) can return an extensively-trained habit to the status of goal-directed action. Experiment 2 showed that training of the second response needs to be intermixed with training of the first response to produce this effect; training the second response after the first-response training was complete preserved the first response as a habit. Experiment 3 demonstrated that reinforcing the second response with a different reinforcer breaks the habit status of the first response. Experiment 4 found that free reinforcers (that were not response-contingent) were sufficient to restore goal-directed performance. Together, the results suggest that unexpected reinforcer delivery can render a habitual response goal-directed again.

On How Definitions of Habits Can Complicate Habit Research

The core message of this paper is that many of the challenges of habit research can be traced back to the presence of causal elements within the definition of habits. For instance, the idea that habits are stimulus-driven implies that habitual behavior is not causally mediated by goal-representations. The presence of these causal elements in the definition of habits leads to difficulties in establishing empirically whether behavior is habitual. Some of these elements can also impoverish theoretical thinking about the mechanisms underlying habitual behavior. I argue that habit research would benefit from eliminating any reference to specific S-R association formation theories from the definition of habits. Which causal elements are retained in the definition of habits depends on the goals of researchers. However, regardless of the definition that is selected, it is good to be aware of the implications of the definition of habits for empirical and theoretical research on habits.

Goal-directed control on interval schedules does not depend on the action-outcome correlation

When an organism's action is based on an anticipation of its consequences, that action is said to be goal-directed. It has long been thought that goal-directed control is made possible by experiencing a strong correlation between response rates and reward rates (Dickinson, 1985). To test this idea, we designed a set of experiments to determine whether the response rate-reward rate correlation is a reliable predictor of goal-directed control on interval schedules. In Experiment 1, rats were trained on random interval (RI) schedules in which the response rate-reward rate correlation was manipulated across groups. In tests of reward devaluation, rats behaved in a goal-directed manner regardless of the experienced correlation. In Experiment 2, rats once again experienced either a strong or weak correlation, but on RI schedules with lower overall reward densities. This time, behavior appeared habitual regardless of the experienced correlation. Experiment 3 confirmed that the density of the RI schedule influences goal-directed control, and also revealed that extensive training on these schedules resulted in goal-directed action. Finally, in Experiment 4 goal-directed responding was greater and emerged sooner on fixed than random interval schedules, but, again, was manifest after extensive training on the RI schedule. Taken together, our data suggest that goal-directed and habitual control are not determined by the correlation between response rates and reward rates. We discuss the importance of temporal uncertainty, action-outcome contiguity, and reinforcement probability in goal-directed control. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Meta-analysis of randomised controlled trials testing behavioural interventions to promote household action on climate change

No consensus exists regarding which are the most effective mechanisms to promote household action on climate change. We present a meta-analysis of randomised controlled trials comprising 3,092,678 observations, which estimates the effects of behavioural interventions holding other factors constant. Here we show that behavioural interventions promote climate change mitigation to a very small degree while the intervention lasts (d = −0.093 95% CI −0.160, −0.055), with no evidence of sustained positive effects once the intervention ends. With the exception of recycling, most household mitigation behaviours show a low behavioural plasticity. The intervention with the highest average effect size is choice architecture (nudges) but this strategy has been tested in a limited number of behaviours. Our results do not imply behavioural interventions are less effective than alternative strategies such as financial incentives or regulations, nor exclude the possibility that behavioural interventions could have stronger effects when used in combination with alternative strategies.

It is not clear which are the most effective mechanisms to achieve sustainable lifestyle behaviour. Here the authors study the impact of behavioural interventions excluding economic incentives by performing a large-scale meta-analysis and find that these interventions promote sustainable behaviours to a small degree in the short-term with no evidence of sustained positive effects once the intervention is completed.

Context-Dependent and Context-Independent Effects of D1 Receptor Antagonism in the Basolateral and Central Amygdala during Cocaine Self-Administration

One way that drugs of abuse perturb the dopamine system is by triggering large amounts of extracellular dopamine to efflux into limbic regions. The basolateral (BLA) and central (CeA) nuclei of the amygdala have been shown to play distinct roles in value representation of primary and conditioned reward. However, the precise role of dopaminergic receptors in the BLA and the CeA during reward-related behaviors remains to be determined. Here we investigate the effects of dopamine D1 receptor blockade in the BLA and the CeA during asymptotic performance of cocaine self-administration and in a novel application of contextual renewal under continued access conditions. After more than three weeks of chained seek-take self-administration of cocaine, male Long Evans rats were given a bilateral intra-BLA or intra-CeA infusion of the D1 antagonist SCH-23390 (2 µg/0.3 µl) for multiple days. Intra-BLA D1 receptor blockade before, but not after the self-administration session, gradually suppressed drug seeking and taking responses and persisted with a change in context with continued D1 blockade. In contrast, intra-CeA D1 receptor blockade caused a rapid reduction in self-administration that showed renewal with a change in context with continued D1 blockade. Further, conditioned place aversion developed with intra-BLA but not intra-CeA infusions. Collectively, these results demonstrate that dopamine D1 receptors in the BLA and CeA both contribute to drug seeking and taking, but may do so through distinct mechanisms.

Prediction and control of operant behavior: What you see is not all there is

Prediction and control of operant behavior are major goals of behavior analysis. We suggest that achieving these goals can benefit from doing more than identifying the three-term contingency between the behavior, its setting stimulus, and its consequences. Basic research now underscores the idea that prediction and control require consideration of the behavior's history. As one example, if an operant is a goal-directed action, it is controlled by the current value of the reinforcer, as illustrated by the so-called reinforcer devaluation effect. In contrast, if the behavior is a habit, it occurs automatically, without regard to the reinforcer's value, as illustrated by its insensitivity to the reinforcer devaluation effect. History variables that distinguish actions and habits include the extent of their prior practice and their schedule of reinforcement. Other operants can appear to have very low or zero strength. However, if the behavior has reached that level through extinction or punishment, it may precipitously increase in strength by changing the context, allowing time to pass, presenting the reinforcer contingently or noncontingently, or extinguishing an alternative behavior. Behaviors that are not suppressed by extinction or punishment are not affected the same way. When predicting the strength of an operant behavior, what you see is not all there is. The behavior's history counts.

Retrieval practice after multiple context changes, but not long retention intervals, reduces the impact of a final context change on instrumental behavior

Recent evidence from this laboratory suggests that a context switch after operant learning consistently results in a decrement in responding. One way to reduce this decrement is to train the response in multiple contexts. One interpretation of this result, rooted in stimulus sampling theory, is that conditioning of a greater number of common stimulus elements arising from more contexts causes better generalization to new contexts. An alternative explanation is that each change of context causes more effortful retrieval, and practice involving effortful retrieval results in learning that is better able to transfer to new situations. The current experiments were designed to differentiate between these two explanations for the first time in an animal learning and memory task. Experiment 1 demonstrated that the detrimental impact of a context change on an instrumental nose-poking response can be reduced by training the response in multiple contexts. Experiment 2 then found that a training procedure which inserted extended retention intervals between successive training sessions did not reduce the detrimental impact of a final context change. This occurred even though the inserted retention intervals had a detrimental impact on responding (and, thus, presumably retrieval) similar to the effect that context switches had in Experiment 1. Together, the results suggest that effortful retrieval practice may not be sufficient to reduce the negative impact of a context change on instrumental behavior. A common elements explanation which supposes that physical and temporal contextual cues do not overlap may account for the findings more readily.

Extinction of instrumental (operant) learning: interference, varieties of context, and mechanisms of contextual control

This article reviews recent research on the extinction of instrumental (or operant) conditioning from the perspective that it is an example of a general retroactive interference process. Previous discussions of interference have focused primarily on findings from Pavlovian conditioning. The present review shows that extinction in instrumental learning has much in common with other examples of retroactive interference in instrumental learning (e.g., omission learning, punishment, second-outcome learning, discrimination reversal learning, and differential reinforcement of alternative behavior). In each, the original learning can be largely retained after conflicting information is learned, and behavior is cued or controlled by the current context. The review also suggests that a variety of stimuli can play the role of context, including room and apparatus cues, temporal cues, drug state, deprivation state, stress state, and recent reinforcers, discrete cues, or behaviors. In instrumental learning situations, the context can control behavior through its direct association with the reinforcer or punisher, through its hierarchical relation with response-outcome associations, or its direct association (inhibitory or excitatory) with the response. In simple instrumental extinction and habit learning, the latter mechanism may play an especially important role.