Shifting the balance between goals and habits: Five failures in experimental habit induction

Action sequence learning, habits, and automaticity in obsessive-compulsive disorder

This study investigates the goal/habit imbalance theory of compulsion in obsessive-compulsive disorder (OCD), which postulates enhanced habit formation, increased automaticity, and impaired goal/habit arbitration. It directly tests these hypotheses using newly developed behavioral tasks. First, OCD patients and healthy participants were trained daily for a month using a smartphone app to perform chunked action sequences. Despite similar procedural learning and attainment of habitual performance (measured by an objective automaticity criterion) by both groups, OCD patients self-reported higher subjective habitual tendencies via a recently developed questionnaire. Subsequently, in a re-evaluation task assessing choices between established automatic and novel goal-directed actions, both groups were sensitive to re-evaluation based on monetary feedback. However, OCD patients, especially those with higher compulsive symptoms and habitual tendencies, showed a clear preference for trained/habitual sequences when choices were based on physical effort, possibly due to their higher attributed intrinsic value. These patients also used the habit-training app more extensively and reported symptom relief post-study. The tendency to attribute higher intrinsic value to familiar actions may be a potential mechanism leading to compulsions and an important addition to the goal/habit imbalance hypothesis in OCD. We also highlight the potential of smartphone app training as a habit reversal therapeutic tool.

Effects of sleep deprivation on food-related Pavlovian-instrumental transfer: a randomized crossover experiment

Recent research suggests that insufficient sleep elevates the risk of obesity. Although the mechanisms underlying the relationship between insufficient sleep and obesity are not fully understood, preliminary evidence suggests that insufficient sleep may intensify habitual control of behavior, leading to greater cue-elicited food-seeking behavior that is insensitive to satiation. The present study tested this hypothesis using a within-individual, randomized, crossover experiment. Ninety-six adults underwent a one-night normal sleep duration (NSD) condition and a one-night total sleep deprivation (TSD) condition. They also completed the Pavlovian-instrumental transfer paradigm in which their instrumental responses for food in the presence and absence of conditioned cues were recorded. The sleep × cue × satiation interaction was significant, indicating that the enhancing effect of conditioned cues on food-seeking responses significantly differed across sleep × satiation conditions. However, this effect was observed in NSD but not TSD, and it disappeared after satiation. This finding contradicted the hypothesis but aligned with previous literature on the effect of sleep disruption on appetitive conditioning in animals-sleep disruption following learning impaired the expression of appetitive behavior. The present finding is the first evidence for the role of sleep in Pavlovian-instrumental transfer effects. Future research is needed to further disentangle how sleep influences motivational mechanisms underlying eating.

A novel free-operant framework enables experimental habit induction in humans

Habits are a prominent feature of both adaptive and maladaptive behavior. Yet, despite substantial research efforts, there are currently no well-established experimental procedures for habit induction in humans. It is likely that laboratory experimental settings, as well as the session-based structure typically used in controlled experiments (also outside the lab), impose serious constraints on studying habits and other effects that are sensitive to context, motivation, and training duration and frequency. To overcome these challenges, we devised a unique real-world free-operant task structure, implemented through a novel smartphone application, whereby participants could freely enter the app (24 hours a day, 7 days a week) to win rewards. This procedure is free of typical laboratory constraints, yet well controlled. Using the canonical sensitivity to outcome devaluation criterion, we successfully demonstrated habit formation as a function of training duration, a long-standing challenge in the field. Additionally, we show a positive relationship between multiple facets of engagement/motivation and goal-directedness. We suggest that our novel paradigm can be used to study the neurobehavioral and psychological mechanism underlying habits in humans. Moreover, the real-world free-operant framework can potentially be used to examine other instrumental behavior-related questions, with greater face validity in naturalistic conditions.

Acute stress facilitates habitual behavior in female rats

Instrumental behavior can reflect the influence of goal-directed and habitual systems. Contemporary research suggests that stress may facilitate control by the habitual system under conditions where the behavior would otherwise reflect control by the goal-directed system. However, it is unclear how stress modulates the influence of these systems on instrumental responding to achieve this effect, particularly in females. Here, we examine whether a mild psychogenic stressor experienced before acquisition training (Experiment 1), or prior to the test of expression (Experiment 2) would influence goal-directed and habitual control of instrumental responding in female rats. In both experiments, rats acquired an instrumental nose-poke response for a sucrose reward. This was followed by a reinforcer devaluation phase in which half the rats in Stressed and Non-Stressed conditions received pairings of the sucrose pellet with illness induced by lithium chloride until they rejected the pellet when offered. The remaining rats received a control treatment consisting of pellets and illness on separate days (Unpaired). Control by goal-directed and habitual systems was evaluated in a subsequent nonreinforced test of nose poking. The results of Experiment 1 indicated that the Non-Stressed Paired group reduced nose-poking compared to the Unpaired controls, identifying the response as goal directed, whereas the Stressed Paired and Unpaired groups made a similar number of nose pokes identifying the response as habitual despite a similar amount of training. Results from Experiment 2 indicated habitual control of nose-poke responding was present when stress was experienced just prior to the test. Collectively, these data suggest that stress may facilitate habitual control by altering the relative influence of goal-directed and habitual processes underpinning instrumental behavior. These results may be clinically relevant for understanding the contributions of stress to dysregulated instrumental behavior in compulsive pathologies.

Characterizing Human Habits in the Lab

Habits pose a fundamental puzzle for those aiming to understand human behavior. They pervade our everyday lives and dominate some forms of psychopathology but are extremely hard to elicit in the lab. In this Registered Report, we developed novel experimental paradigms grounded in computational models, which suggest that habit strength should be proportional to the frequency of behavior and, in contrast to previous research, independent of value. Specifically, we manipulated how often participants performed responses in two tasks varying action repetition without, or separately from, variations in value. Moreover, we asked how this frequency-based habitization related to value-based operationalizations of habit and self-reported propensities for habitual behavior in real life. We find that choice frequency during training increases habit strength at test and that this form of habit shows little relation to value-based operationalizations of habit. Our findings empirically ground a novel perspective on the constituents of habits and suggest that habits may arise in the absence of external reinforcement. We further find no evidence for an overlap between different experimental approaches to measuring habits and no associations with self-reported real-life habits. Thus, our findings call for a rigorous reassessment of our understanding and measurement of human habitual behavior in the lab.

Devaluation of response-produced safety signals reveals circuits for goal-directed versus habitual avoidance in dorsal striatum

Active avoidance responses (ARs) are instrumental behaviors that prevent harm. Adaptive ARs may contribute to active coping, whereas maladaptive avoidance habits are implicated in anxiety and obsessive-compulsive disorders. The AR learning mechanism has remained elusive, as successful avoidance trials produce no obvious reinforcer. We used a novel outcome-devaluation procedure in rats to show that ARs are positively reinforced by response-produced feedback (FB) cues that develop into safety signals during training. Males were sensitive to FB-devaluation after moderate training, but not overtraining, consistent with a transition from goal-directed to habitual avoidance. Using chemogenetics and FB-devaluation, we also show that goal-directed vs. habitual ARs depend on dorsomedial vs. dorsolateral striatum, suggesting a significant overlap between the mechanisms of avoidance and rewarded instrumental behavior. Females were insensitive to FB-devaluation due to a remarkable context-dependence of counterconditioning. However, degrading the AR-FB contingency suggests that both sexes rely on safety signals to perform goal-directed ARs.

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

A neural autopilot theory of habit: Evidence from consumer purchases and social media use

This article applies a two-process "neural autopilot" model to field data. The autopilot model hypothesizes that habitual choice occurs when the reward from a behavior has low numerical "doubt" (i.e., reward prediction errors are small). The model toggles between repeating a previous choice (habit) when doubt is low and making a goal-directed choice when doubt is high. The model has ingredients established in animal learning and cognitive neuroscience and is simple enough to make nonobvious predictions. In two empirical applications, we fit the model to field data on purchases of canned tuna and posting on the Chinese social media site Weibo. This style of modeling is called "structural" because there is a theoretical model of how different variables influence choices by agents (the "structure"), which tightly restricts how hidden variables lead to observed choices. There is empirical support for the model, more strongly for tuna purchases than for Weibo posting, relative to a baseline "reduced-form" model in which current choices are correlated with past choices without a mechanistic (structural) explanation. An interesting set of predictions can also be derived about how consumers react to different kinds of changes in prices and qualities of goods (this is called "counterfactual analysis").

Multiple Memory Subsystems: Reconsidering Memory in the Mind and Brain

The multiple-memory-systems framework-that distinct types of memory are supported by distinct brain systems-has guided learning and memory research for decades. However, recent work challenges the one-to-one mapping between brain structures and memory types central to this taxonomy, with key memory-related structures supporting multiple functions across substructures. Here we integrate cross-species findings in the hippocampus, striatum, and amygdala to propose an updated framework of multiple memory subsystems (MMSS). We provide evidence for two organizational principles of the MMSS theory: First, opposing memory representations are colocated in the same brain structures; second, parallel memory representations are supported by distinct structures. We discuss why this burgeoning framework has the potential to provide a useful revision of classic theories of long-term memory, what evidence is needed to further validate the framework, and how this novel perspective on memory organization may guide future research.

Measuring context-response associations that drive habits

People achieve important life outcomes of health, financial security, and productivity by repeating operant behavior. To identify whether such operants reflect goal pursuit or habit, the present research introduces a new paradigm that yields objective measures of learning and controls for the motivations of goal pursuit. In two experiments, participants practiced a sequential task of making sushi and then completed a test of the strength of cue-response (habit) associations in memory. Finally, they repeated the sushi task without instructions while under cognitive load (designed to impede deliberation about goals). As predicted, greater task practice yielded stronger cue-response associations, which in turn promoted task success. Practice did not improve performance by enhancing goal intentions or other task motivations. We conclude that repetition facilitates performance by creating mental associations that automatically activate practiced, habitual responses upon perception of recurring context cues.

Model-free decision making resists improved instructions and is enhanced by stimulus-response associations

Human behaviour may be thought of as supported by two different computational-learning mechanisms, model-free and model-based respectively. In model-free strategies, stimulus-response associations are strengthened when actions are followed by a reward and weakened otherwise. In model-based learning, previous to selecting an action, the current values of the different possible actions are computed based on a detailed model of the environment. Previous research with the two-stage task suggests that participants' behaviour usually shows a mixture of both strategies. But, interestingly, a recent study by da Silva and Hare (2020) found that participants primarily deploy model-based behaviour when they are given detailed instructions about the structure of the task. In the present study, we reproduce this essential experiment. Our results confirm that improved instructions give rise to a stronger model-based component. Crucially, we also found a significant effect of reward that became stronger under conditions that favoured the development of strong stimulus-response associations. This suggests that the effect of reward, often taken as indicator of a model-free component, is related to stimulus-response learning.

Instant habits versus flexible tenacity: Do implementation intentions accelerate habit formation?

Implementation intentions (strategic "if-then" plans) have been shown to support behaviour change. This may be achieved by mentally forming stimulus-response associations, thereby promoting habit formation. Does this deliberate attempt to instal "strategic automaticity" only offer advantages, or does it also come at the cost of reduced flexibility that characterises learnt habits? To investigate this, we tested healthy, young participants on a computerised instrumental learning task. Critically, we introduced implementation intentions ("if I see stimulus X, then I will respond") versus goal intentions ("for outcome Z, I will respond)" during instrumental acquisition, and subsequently assessed behavioural flexibility in an outcome-revaluation test. In Experiment 1, we conducted a between-subjects manipulation of strategic planning, and in Experiment 2, a within-subject manipulation. We hypothesised that implementation intentions would lead to strong stimulus-response associations and consequently impair performance when the signalled outcome value changed and therefore required a different response, while benefitting performance when the outcome value (and required response) remained the same. We found that implementation intentions supported instrumental learning, but impaired test performance overall (most robustly in Experiment 2), irrespective of whether the signalled outcome value had changed. We argue that this general detrimental effect of implementation intentions on test performance is likely a consequence of their negative effect on stimulus-outcome learning. Our findings warrant caution when applying if-then plans to situations where the agent does not already possess perfect knowledge of behavioural contingencies.While implementation intentions may support efficient and fast behavioural execution, this may come at the expense of behavioural flexibility.

The Willpower Paradox: Possible and Impossible Conceptions of Self-Control

Self-control denotes the ability to override current desires to render behavior consistent with long-term goals. A key assumption is that self-control is required when short-term desires are transiently stronger (more preferred) than long-term goals and people would yield to temptation without exerting self-control. We argue that this widely shared conception of self-control raises a fundamental yet rarely discussed conceptual paradox: How is it possible that a person most strongly desires to perform a behavior (e.g., eat chocolate) and at the same time desires to recruit self-control to prevent themselves from doing it? A detailed analysis reveals that three common assumptions about self-control cannot be true simultaneously. To avoid the paradox, any coherent theory of self-control must abandon either the assumption (a) that recruitment of self-control is an intentional process, or (b) that humans are unitary agents, or (c) that self-control consists in overriding the currently strongest desire. We propose a taxonomy of different kinds of self-control processes that helps organize current theories according to which of these assumptions they abandon. We conclude by outlining unresolved questions and future research perspectives raised by different conceptions of self-control and discuss implications for the question of whether self-control can be considered rational.

Action slips in food choices: A measure of habits and goal-directed control

We report a new, simple instrumental action-slip task, which sets goal-directed action against putative S-R associations. On each training trial, participants were presented with one of two stimuli (blue or green coloured screen). One stimulus (S1) signalled that one joystick response (R1-left or right push) would earn one of two rewards (O1-jellybeans or Pringles points). A second stimulus (S2) signalled a different instrumental relationship (S2:R2-O2). On each test trial, participants were told which outcome could be earnt (O1/O2) on that trial. They were required to withhold responding until the screen changed colour to S1 or S2. On congruent test trials, the stimulus presented (e.g., S1) was associated with the same response (R1) as the outcome available on that trial (O1). On incongruent test trials, in contrast, the outcome (e.g., O1) preceded a stimulus that was associated with a different response (e.g., S2). Hence, in order to obtain the outcome (O1) on incongruent trials, participants were required to suppress any tendency they might have to make the response associated with the stimulus (R2 in response to S2). In two experiments, participants made more errors on incongruent than congruent trials. This result suggests that, on incongruent trials, the stimulus drove responding (e.g., S2 increased R2 responding) in a manner that was inconsistent with goal-directed action (e.g., R1 responding to obtain O1)-an action slip. The results are discussed in terms of popular dual-process theories of instrumental action and a single-process alternative.

Food Restriction Level and Reinforcement Schedule Differentially Influence Behavior during Acquisition and Devaluation Procedures in Mice

Behavioral strategies are often classified based on whether reinforcer value controls reinforcement. Value-sensitive behaviors, in which animals update their actions when reinforcer value is changed, are classified as goal-directed; conversely, value-insensitive actions, where behavior remains consistent when the reinforcer is removed or devalued, are considered habitual. Basic reinforcement schedules can help to bias behavior toward either process: random ratio (RR) schedules are thought to promote the formation of goal-directed behaviors while random intervals (RIs) promote habitual control. However, how the schedule-specific features of these tasks interact with other factors that influence learning to control behavior has not been well characterized. Using male and female mice, we asked how distinct food restriction levels, a strategy often used to increase task engagement, interact with RR and RI schedules to control performance during task acquisition and devaluation procedures. We determined that food restriction level has a stronger effect on the behavior of mice following RR schedules compared with RI schedules, and that it promotes a decrease in response rate during devaluation procedures that is best explained by the effects of extinction rather than devaluation. Surprisingly, food restriction accelerated the decrease in response rates observed following devaluation across sequential extinction sessions, but not within a single session. Our results support the idea that the relationships between schedules and behavioral control strategies are not clear-cut and suggest that an animal's engagement in a task must be accounted for, together with the structure of reinforcement schedules, to appropriately interpret the cognitive underpinnings of behavior.

Investigating habits in humans with a symmetrical outcome-revaluation task

The translation of the outcome-devaluation paradigm to study habit in humans has yielded interesting insights but proven to be challenging. We present a novel, outcome-revaluation task with a symmetrical design, in the sense that half of the available outcomes are always valuable and the other half not-valuable. In the present studies, during the instrumental learning phase, participants learned to respond (Go) to certain stimuli to collect valuable outcomes (and points) while refraining to respond (NoGo) to stimuli signaling not-valuable outcomes. Half of the stimuli were short-trained, while the other half were long-trained. Subsequently, in the test phase, the signaled outcomes were either value-congruent with training (still-valuable and still-not-valuable), or value-incongruent (devalued and upvalued). The change in outcome value on value-incongruent trials meant that participants had to flexibly adjust their behavior. At the end of the training phase, participants completed the self-report behavioral automaticity index - providing an automaticity score for each stimulus-response association. We conducted two experiments using this task, that both provided evidence for stimulus-driven habits as reflected in poorer performance on devalued and upvalued trials relative to still-not-valuable trials and still-valuable trials, respectively. While self-reported automaticity increased with longer training, behavioral flexibility was not affected. After extended training (Experiment 2), higher levels of self-reported automaticity when responding to stimuli signaling valuable outcomes were related to more 'slips of action' when the associated outcome was subsequently devalued. We conclude that the symmetrical outcome-revaluation task provides a promising paradigm for the experimental investigation of habits in humans.

Revealing abrupt transitions from goal-directed to habitual behavior

A fundamental tenet of animal behavior is that decision-making involves multiple 'controllers.' Initially, behavior is goal-directed, driven by desired outcomes, shifting later to habitual control, where cues trigger actions independent of motivational state. Clark Hull's question from 1943 still resonates today: "Is this transition abrupt, or is it gradual and progressive?"1 Despite a century-long belief in gradual transitions, this question remains unanswered2,3 as current methods cannot disambiguate goal-directed versus habitual control in real-time. Here, we introduce a novel 'volitional engagement' approach, motivating animals by palatability rather than biological need. Offering less palatable water in the home cage4,5 reduced motivation to 'work' for plain water in an auditory discrimination task when compared to water-restricted animals. Using quantitative behavior and computational modeling6, we found that palatability-driven animals learned to discriminate as quickly as water-restricted animals but exhibited state-like fluctuations when responding to the reward-predicting cue-reflecting goal-directed behavior. These fluctuations spontaneously and abruptly ceased after thousands of trials, with animals now always responding to the reward-predicting cue. In line with habitual control, post-transition behavior displayed motor automaticity, decreased error sensitivity (assessed via pupillary responses), and insensitivity to outcome devaluation. Bilateral lesions of the habit-related dorsolateral striatum7 blocked transitions to habitual behavior. Thus, 'volitional engagement' reveals spontaneous and abrupt transitions from goal-directed to habitual behavior, suggesting the involvement of a higher-level process that arbitrates between the two.

Failures of Urge Suppression in Obsessive-Compulsive Disorder: Behavioral Modeling Using a Blink Suppression Task

Many individuals with obsessive-compulsive disorder (OCD) report sensory-based urges (e.g. 'not-just-right experiences') in addition to, or instead of, concrete fear-based obsessions. These sensations may be comparable to normative "urges-for-action" (UFA), such as the urge to blink. While research has identified altered functioning of brain regions related to UFA in OCD, little is known about behavioral patterns of urge suppression in the disorder. Using an urge-to-blink task as a model for sensory-based urges, this study compared failures of urge suppression between OCD patients and controls by measuring eyeblinks during 60-second blocks of instructed blink suppression. Cox shared frailty models estimated the hazard of first blinks during each 60-second block and recurrent blinks following each initial erroneous blink. OCD patients demonstrated a higher hazard of first and recurrent blinks compared to controls, suggesting greater difficulty resisting repetitive sensory-based urges. Within OCD, relationships between task outcomes and symptom severity were inconsistent. Findings provide support for a deficit in delaying initial urge-induced actions and terminating subsequent actions in OCD, which is not clearly related to clinical heterogeneity. Elucidating the nature of behavioral resistance to urges is relevant for informing conceptualizations of obsessive-compulsive psychopathology and optimizing treatment outcomes.

Cortical glutamate and GABA are related to compulsive behaviour in individuals with obsessive compulsive disorder and healthy controls

There has been little analysis of neurochemical correlates of compulsive behaviour to illuminate its underlying neural mechanisms. We use 7-Tesla proton magnetic resonance spectroscopy (¹H-MRS) to assess the balance of excitatory and inhibitory neurotransmission by measuring glutamate and GABA levels in anterior cingulate cortex (ACC) and supplementary motor area (SMA) of healthy volunteers and participants with Obsessive-Compulsive Disorder (OCD). Within the SMA, trait and clinical measures of compulsive behaviour are related to glutamate levels, whereas a behavioural index of habitual control correlates with the glutamate:GABA ratio. Participants with OCD also show the latter relationship in the ACC while exhibiting elevated glutamate and lower GABA levels in that region. This study highlights SMA mechanisms of habitual control relevant to compulsive behaviour, common to the healthy sub-clinical and OCD populations. The results also demonstrate additional involvement of anterior cingulate in the balance between goal-directed and habitual responding in OCD.

The Situated Assessment Method (SAM2): Establishing individual differences in habitual behavior

From the perspectives of grounded, situated, and embodied cognition, we have developed a new approach for assessing individual differences. Because this approach is grounded in two dimensions of situatedness-situational experience and the Situated Action Cycle-we refer to it as the Situated Assessment Method (SAM2). Rather than abstracting over situations during assessment of a construct (as in traditional assessment instruments), SAM2 assesses a construct in situations where it occurs, simultaneously measuring factors from the Situated Action Cycle known to influence it. To demonstrate this framework, we developed the SAM2 Habitual Behavior Instrument (SAM2 HBI). Across three studies with a total of 442 participants, the SAM2 HBI produced a robust and replicable pattern of results at both the group and individual levels. Trait-level measures of habitual behavior exhibited large reliable individual differences in the regularity of performing positive versus negative habits. Situational assessments established large effects of situations and large situation by individual interactions. Several sources of evidence demonstrated construct and content validity for SAM2 measures of habitual behavior. At both the group and individual levels, these measures were associated with factors from the Situated Action Cycle known to influence habitual behavior in the literature (consistency, automaticity, immediate reward, long-term reward). Regressions explained approximately 65% of the variance at the group level and a median of approximately 75% at the individual level. SAM2 measures further exhibited well-established interactions with personality measures for self-control and neuroticism. Cognitive-affective processes from the Situated Action Cycle explained nearly all the variance in these interactions. Finally, a composite measure of habitualness established habitual behaviors at both the group and individual levels. Additionally, a composite measure of reward was positively related to the composite measure of habitualness, increasing with self-control and decreasing with neuroticism.

Pathways to the persistence of drug use despite its adverse consequences

The persistence of drug taking despite its adverse consequences plays a central role in the presentation, diagnosis, and impacts of addiction. Eventual recognition and appraisal of these adverse consequences is central to decisions to reduce or cease use. However, the most appropriate ways of conceptualizing persistence in the face of adverse consequences remain unclear. Here we review evidence that there are at least three pathways to persistent use despite the negative consequences of that use. A cognitive pathway for recognition of adverse consequences, a motivational pathway for valuation of these consequences, and a behavioral pathway for responding to these adverse consequences. These pathways are dynamic, not linear, with multiple possible trajectories between them, and each is sufficient to produce persistence. We describe these pathways, their characteristics, brain cellular and circuit substrates, and we highlight their relevance to different pathways to self- and treatment-guided behavior change.

Anxiety as a disorder of uncertainty: implications for understanding maladaptive anxiety, anxious avoidance, and exposure therapy

In cognitive-behavioral conceptualizations of anxiety, exaggerated threat expectancies underlie maladaptive anxiety. This view has led to successful treatments, notably exposure therapy, but is not consistent with the empirical literature on learning and choice alterations in anxiety. Empirically, anxiety is better described as a disorder of uncertainty learning. How disruptions in uncertainty lead to impairing avoidance and are treated with exposure-based methods, however, is unclear. Here, we integrate concepts from neurocomputational learning models with clinical literature on exposure therapy to propose a new framework for understanding maladaptive uncertainty functioning in anxiety. Specifically, we propose that anxiety disorders are fundamentally disorders of uncertainty learning and that successful treatments, particularly exposure therapy, work by remediating maladaptive avoidance from dysfunctional explore/exploit decisions in uncertain, potentially aversive situations. This framework reconciles several inconsistencies in the literature and provides a path forward to better understand and treat anxiety.

What can machine learning teach us about habit formation? Evidence from exercise and hygiene

We apply a machine learning technique to characterize habit formation in two large panel data sets with objective measures of 1) gym attendance (over 12 million observations) and 2) hospital handwashing (over 40 million observations). Our Predicting Context Sensitivity (PCS) approach identifies context variables that best predict behavior for each individual. This approach also creates a time series of overall predictability for each individual. These time series predictability values are used to trace a habit formation curve for each individual, operationalizing the time of habit formation as the asymptotic limit of when behavior becomes highly predictable. Contrary to the popular belief in a "magic number" of days to develop a habit, we find that it typically takes months to form the habit of going to the gym but weeks to develop the habit of handwashing in the hospital. Furthermore, we find that gymgoers who are more predictable are less responsive to an intervention designed to promote more gym attendance, consistent with past experiments showing that habit formation generates insensitivity to reward devaluation.

Individual Differences in Corticostriatal White-matter Tracts Predict Successful Daily-life Routine Formation

Despite good intentions, people often fail to cross the "intention-behavior gap," especially when goal achievement requires repeated action. To bridge this gap, the formation of automatized routines may be crucial. However, people may differ in the tendency to switch from goal-directed toward habitual control. To shed light on why some people succeed in forming routines while others struggle, the present study related the automatization of a novel, daily routine to individual differences in white-matter connectivity in corticostriatal networks that have been implicated in goal-directed and habitual control. Seventy-seven participants underwent diffusion-weighted imaging and formed the daily routine of taking a (placebo) pill for 3 weeks. Pill intake was measured by electronic pill boxes, and participants filled out a daily online questionnaire on the subjective automaticity of this behavior. Automatization of pill intake was negatively related to striatal (mainly caudate) connectivity with frontal goal-directed and cognitive control regions, namely, ventromedial pFC and anterior cingulate gyrus. Furthermore, daily pill intake was positively related to individual differences in striatal (mainly caudate) connectivity with cognitive control regions, including dorsolateral and anterior pFC. Therefore, strong control networks may be relevant for implementing a new routine but may not benefit its automatization. We also show that habit tendency (assessed with an outcome-devaluation task), conscientiousness, and daily life regularity were positively related to routine automatization. This translational study moves the field of habit research forward by relating self-reported routine automatization to individual differences in performance on an experimental habit measure and to brain connectivity.

Changes in global functional network properties predict individual differences in habit formation

Prior evidence suggests that sensorimotor regions play a crucial role in habit formation. Yet, whether and how their global functional network properties might contribute to a more comprehensive characterization of habit formation still remains unclear. Capitalizing on advances in Elastic Net regression and predictive modeling, we examined whether learning-related functional connectivity alterations distributed across the whole brain could predict individual habit strength. Using the leave-one-subject-out cross-validation strategy, we found that the habit strength score of the novel unseen subjects could be successfully predicted. We further characterized the contribution of both, individual large-scale networks and individual brain regions by calculating their predictive weights. This highlighted the pivotal role of functional connectivity changes involving the sensorimotor network and the cingulo-opercular network in subject-specific habit strength prediction. These results contribute to the understanding the neural basis of human habit formation by demonstrating the importance of global functional network properties especially also for predicting the observable behavioral expression of habits.

Sensitivity to outcome devaluation in operant tasks is better predicted by food restriction level than reinforcement training schedule in mice

Behavioral strategies are often classified based on whether reinforcement is controlled by the value of the reinforcer. Value-sensitive behaviors, in which animals update their actions when reinforcer value is changed, are classified as goal-directed; conversely, value-insensitive actions, where behavior remains consistent when the reinforcer is removed or devalued, are considered habitual. Understanding the features of operant training that bias behavioral control toward either strategy is essential to understanding the cognitive and neuronal processes on which they rely. Using basic reinforcement principles, behavior can be biased toward relying on either process: random ratio (RR) schedules are thought to promote the formation of goal-directed behaviors while random intervals (RI) promote habitual control. However, how the schedule-specific features of these task structures relate to external factors to influence behavior is not well understood. Using male and female mice on distinct food restriction levels, we trained each group on RR schedules with responses-per-reinforcer rates matched to their RI counterparts to control for differences in reinforcement rate. We determined that food restriction level has a stronger effect on the behavior of mice following RR schedules than mice following RI schedules and that food restriction better predicted sensitivity to outcome devaluation than training schedule. Our results support the idea the relationships between RR or RI schedules with goal-directed or habitual behaviors, respectively, are more nuanced than previously appreciated and suggest that an animal's engagement in a task must be accounted for, together with the structure of reinforcement schedules, to appropriately interpret the cognitive underpinnings of behavior.

Does stress consistently favor habits over goal-directed behaviors? Data from two preregistered exact replication studies

Instrumental learning is controlled by two distinct parallel systems: goal-directed (action-outcome) and habitual (stimulus-response) processes. Seminal research by Schwabe and Wolf (2009, 2010) has demonstrated that stress renders behavior more habitual by decreasing goal-directed control. More recent studies yielded equivocal evidence for a stress-induced shift towards habitual responding, yet these studies used different paradigms to evaluate instrumental learning or used different stressors. Here, we performed exact replications of the original studies by exposing participants to an acute stressor either before (cf. Schwabe and Wolf, 2009) or directly after (cf. Schwabe and Wolf, 2010) an instrumental learning phase in which they had learned that distinct actions led to distinct, rewarding food outcomes (i.e., instrumental learning). Then, following an outcome devaluation phase in which one of the food outcomes was consumed until participants were satiated, action-outcome associations were tested in extinction. Despite successful instrumental learning and outcome devaluation and increased subjective and physiological stress levels following stress exposure, the stress and no-stress groups in both replication studies responded indifferently to valued and devalued outcomes. That is, non-stressed participants failed to demonstrate goal-directed behavioral control, thereby rendering the critical test of a shift from goal-directed to habitual control in the stress group inapt. Several reasons for these replication failures are discussed, including the rather indiscriminate devaluation of outcomes that may have contributed to indifferent responding during extinction, which emphasize the need to further our understanding of the boundary conditions in research aimed at demonstrating a stress-induced shift towards habitual control.

Behavioral construction of the future

OBJECTIVE

To examine Howard Rachlin's hypothetical model of molar choice as a tactic for resisting addictions and to explore how some of its then-radical components can be developed to account for nonphysical and far future rewards.

METHOD

The history of Rachlin's long dialog with the present author about molar choice is reviewed. The possible implications are described of both authors' proposal that behavior can depend entirely on reward.

RESULTS

Molar choice entails bringing wider-and thus further future-contingencies to bear on current choices. The two authors proposed mechanisms with different foci, which they respectively called teleological behaviorism and intertemporal bargaining. Laboratory results have been modest, but supplementary demonstrations by thought experiments and brain imaging are described. Both proposals have left open how the value of distant outcomes, such as sobriety, savings, and healthy aging, survives temporal discounting enough to compete with present motivational pressures. In contradiction to Rachlin, but following his suggestion that reward is behavior, it is deduced that some reward must be endogenous rather than secondary to external primary rewards. Endogenous reward is proposed as a fiat currency that can function only to the extent that it is protected from inflation by some kind of uniqueness (singularity). Such uniqueness can be provided by personal disciplines for testing reality, but also by extraneous factors such as needs, coincidences, and biases.

CONCLUSIONS

Rachlin's teleological behaviorism is a valuable hypothesis, but limited by its ruling out of nonexternal rewards and of intrapersonal self-prediction, both of them useful for understanding nonsubstance addictions. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Interaction between habits as action sequences and goal-directed behavior under time pressure

Human behavior consists in large parts of action sequences that are often repeated in mostly the same way. Through extensive repetition, sequential responses become automatic or habitual, but our environment often confronts us with events to which we have to react flexibly and in a goal-directed manner. To assess how implicitly learned action sequences interfere with goal-directed control, we developed a novel behavioral paradigm in which we combined action sequence learning through repetition with a goal-directed task component. So-called dual-target trials require the goal-directed selection of the response with the highest reward probability in a fast succession of trials with short response deadlines. Importantly, the response primed by the learned action sequence is sometimes different from that required by the goal-directed task. As expected, we found that participants learned the action sequence through repetition, as evidenced by reduced reaction times (RT) and error rates (ER), while still acting in a goal-directed manner in dual-target trials. Specifically, we found that the learned action sequence biased choices in the goal-directed task toward the sequential response, and this effect was more pronounced the better individuals had learned the sequence. Our novel task may help shed light on the acquisition of automatic behavioral patterns and habits through extensive repetition, allows to assess positive features of habitual behavior (e.g., increased response speed and reduced error rates), and importantly also the interaction of habitual and goal-directed behaviors under time pressure.

Impaired belief updating and devaluation in adult women with bulimia nervosa

Recent models of bulimia nervosa (BN) propose that binge-purge episodes ultimately become automatic in response to cues and insensitive to negative outcomes. Here, we examined whether women with BN show alterations in instrumental learning and devaluation sensitivity using traditional and computational modeling analyses of behavioral data. Adult women with BN (n = 30) and group-matched healthy controls (n = 31) completed a task in which they first learned stimulus-response-outcome associations. Then, participants were required to repeatedly adjust their responses in a "baseline test", when different sets of stimuli were explicitly devalued, and in a "slips-of-action test", when outcomes instead of stimuli were devalued. The BN group showed intact behavioral sensitivity to outcome devaluation during the slips-of-action test, but showed difficulty overriding previously learned stimulus-response associations on the baseline test. Results from a Bayesian learner model indicated that this impaired performance could be accounted for by a slower pace of belief updating when a new set of previously learned responses had to be inhibited (p = 0.036). Worse performance and a slower belief update in the baseline test were each associated with more frequent binge eating (p = 0.012) and purging (p = 0.002). Our findings suggest that BN diagnosis and severity are associated with deficits in flexibly updating beliefs to withhold previously learned responses to cues. Additional research is needed to determine whether this impaired ability to adjust behavior is responsible for maintaining automatic and persistent binge eating and purging in response to internal and environmental cues.

How to pin a compulsive behavior down: A systematic review and conceptual synthesis of compulsivity-sensitive items in measures of behavioral addiction

Experimental models identify the transition from choice to compulsivity as the main mechanism underlying addiction. In behavioral addictions research, however, the adjective compulsive is used to describe virtually any kind of excessive or dysregulated behavior, which hinders the connection between experimental and clinical models. In this systematic review, we adopted a preliminary definition of compulsive behavior based on previous theoretical work. Subsequently, a systematic review following PRISMA guidelines was conducted (a) to identify the validated instruments, currently used in behavioral addictions research, that include items that are sensitive (intendedly or not) to compulsivity, and (b) to categorize those items into differentiable operationalizations of compulsivity. Six operationalizations of compulsivity emerged from item content analysis: 1. Automatic or habitual behavior occurring in absence of conscious instrumental goals; 2. Behavior insensitive to negative consequences despite conscious awareness of them; 3. Overwhelming urge or desire that impels the individual to initiate the activity and jeopardizes control attempts; 4. Bingeing, or inability to stop or interrupt the activity once initiated, resulting in an episode substantially longer or more intense than intended; 5. Attentional capture and cognitive hijacking; and 6. Inflexible rules, stereotyped behaviors, and rituals related to task completion or execution. Subsequently, a list of 15 representative items per operationalization was elaborated for independent assessment and identification of delimitation problems. A high degree of agreement was reached in assessing them as instantiating compulsivity, as well as in their assignment to the corresponding categories. However, many of them were also considered overinclusive, i.e., uncapable of distinguishing compulsivity from value-based momentary choice. To increase their discriminative value, items in future compulsivity scales should be refined to explicitly mention disconnection between behavior and declarative goals. Further research on factorial structure of a pool of items derived from these operational definitions is warranted. Such a factorial structure could be used as an intermediate link between specific behavioral items and explanatory psychobiological, learning, and cognitive mechanisms.

Increased Habit Frequency in the Daily Lives of Patients with Acute Anorexia Nervosa

Strict eating routines and frequent rigid behavior patterns are commonly observed in patients with anorexia nervosa (AN). A recent theory proposes that while these behaviors may have been reinforced initially, they later become habitual. To date, however, research has been overly focused on eating-disorder (ED)-related habits. Over the course of seven days, we applied an ecological momentary assessment (EMA) to investigate the habit frequency and strength of ED-specific (food intake) and ED-unspecific (hygiene) habits in the daily lives of a sample of n = 57 AN and n = 57 healthy controls (HC). The results of the hierarchical models revealed that habits were significantly more likely in patients compared with HC for both categories, independently. Furthermore, a lower body mass index (BMI) was associated with increased habit frequency in AN. Our study strengthens the habit theory of AN by showing the relevance of habits beyond ED-specific behavioral domains. This also supports the development of innovative therapeutic interventions targeting habitual behavior in EDs.

Making habits measurable beyond what they are not: A focus on associative dual-process models

Habits are the subject of intense international research. Under the associative dual-process model the outcome devaluation paradigm has been used extensively to classify behaviours as being either goal-directed (sensitive to shifts in the value of associated outcomes) or habitual (triggered by stimuli without anticipation of consequences). This has proven to be a useful framework for studying the neurobiology of habit and relevance of habits in clinical psychopathology. However, in recent years issues have been raised about this rather narrow definition of habits in comparison to habitual behaviour experienced in the real world. Specifically, defining habits as the absence of goal-directed control, the very specific set-ups required to demonstrate habit experimentally and the lack of direct evidence for habits as stimulus-response behaviours are viewed as problematic. In this review paper we address key critiques that have been raised about habit research within the framework of the associative dual-process model. We then highlight novel research approaches studying different features of habits with methods that expand beyond traditional paradigms.

An fMRI meta-analysis of the role of the striatum in everyday-life vs laboratory-developed habits

The dorsolateral striatum plays a critical role in the acquisition and expression of stimulus-response habits that are learned in experimental laboratories. Here, we use meta-analytic procedures to contrast the neural circuits activated by laboratory-acquired habits with those activated by stimulus-response behaviours acquired in everyday-life. We confirmed that newly learned habits rely more on the anterior putamen with activation extending into caudate and nucleus accumbens. Motor and associative components of everyday-life habits were identified. We found that motor-dominant stimulus-response associations developed outside the laboratory primarily engaged posterior dorsal putamen, supplementary motor area (SMA) and cerebellum. Importantly, associative components were also represented in the posterior putamen. Thus, common neural representations for both naturalistic and laboratory-based habits were found in the left posterior and right anterior putamen. These findings suggest a partial common striatal substrate for habitual actions that are performed predominantly by stimulus-response associations represented in the posterior striatum. The overlapping neural substrates for laboratory and everyday-life habits supports the use of both methods for the analysis of habitual behaviour.

Habitual or hyper-controlled behavior: OCD symptoms and explicit sequence learning

BACKGROUND AND OBJECTIVES

This study examined whether ritualistic behaviors characteristic of obsessive-compulsive disorder (OCD) are a product of dysfunctional goal-directed behavior leading to habitual behavior (Gillan & Robbins, 2014). We used an explicit motor sequence learning task to investigate the repetition of chunked action sequences across the OC spectrum. As sequential motor behavior is practiced, action movements appear to get bundled together, and the initial movement of the sequence activates the entire sequence, leaving it relatively insensitive to change. Therefore, compulsive behavior in OCD may be a result of failing to inhibit the full activation of an extensively learned action sequence.

METHODS

Fifty-seven participants across the range of OCD symptoms practiced one sequence and then tested on a novel sequence in which one of the middle movements was omitted. Optimal performance for the new sequence required goal-directed inhibition of the original sequence and goal-directed execution of the new sequence instead. To manipulate activation of goal-directed behavior, we added a dual-task condition with a competing auditory tonal N-Back task. Data were analyzed using mixed-effects models.

RESULTS

Although we did observe expected learning patterns during learning of the original sequence, slower reaction times for the new sequences, and higher errors in the dual-task condition, performance was not significantly related to either obsessive-compulsive symptoms or distress symptoms.

LIMITATIONS

The current study used an analog sample; replication in a treatment seeking sample is warranted.

CONCLUSIONS

These findings challenge the goal-directed dysfunction model of OCD.

Threat reversal learning and avoidance habits in generalised anxiety disorder

Avoidance and heightened responses to perceived threats are key features of anxiety disorders. These disorders are characterised by inflexibility in dynamically updating behavioural and physiological responses to aversively conditioned cues or environmental contexts which are no longer objectively threatening, often manifesting in perseverative avoidance. However, less is known about how anxiety disorders might differ in adjusting to threat and safety shifts in the environment or how idiosyncratic avoidance responses are learned and persist. Twenty-eight patients with generalised anxiety disorder (GAD), without DSM co-morbidities, and 27 matched healthy controls were administered two previously established paradigms: Pavlovian threat reversal and shock avoidance habits through overtraining (assessed following devaluation with measures of perseverative responding). For both tasks we used subjective report scales and skin conductance responses (SCR). In the Pavlovian threat reversal task, patients with GAD showed a significantly overall higher SCR as well as a reduced differential SCR response compared to controls in the early but not late reversal phase. During the test of habitual avoidance responding, GAD patients did not differ from controls in task performance, habitual active avoidance responses during devaluation, or corresponding SCR during trials, but showed a trend toward more abstract confirmatory subjective justifications for continued avoidance following the task. GAD patients exhibited significantly greater skin conductance responses to signals of threat than controls, but did not exhibit the major deficits in reversal and safety signal learning shown previously by patients with OCD. Moreover, this patient group, again unlike OCD patients, did not show evidence of altered active avoidance learning or enhanced instrumental avoidance habits. Overall, these findings indicate no deficits in instrumental active avoidance or persistent avoidance habits, despite enhanced responses to Pavlovian threat cues in GAD. They suggest that GAD is characterised by passive, and not excessively rigid, avoidance styles.

Characterizing cerebral metabolite profiles in anorexia and bulimia nervosa and their associations with habitual behavior

Anorexia nervosa (AN) and bulimia nervosa (BN) are associated with altered brain structure and function, as well as increased habitual behavior. This neurobehavioral profile may implicate neurochemical changes in the pathogenesis of these illnesses. Altered glutamate, myo-inositol and N-acetyl aspartate (NAA) concentrations are reported in restrictive AN, yet whether these extend to binge-eating disorders, or relate to habitual traits in affected individuals, remains unknown. We therefore used single-voxel proton magnetic resonance spectroscopy to measure glutamate, myo-inositol, and NAA in the right inferior lateral prefrontal cortex and the right occipital cortex of 85 women [n = 22 AN (binge-eating/purging subtype; AN-BP), n = 33 BN, n = 30 controls]. To index habitual behavior, participants performed an instrumental learning task and completed the Creature of Habit Scale. Women with AN-BP, but not BN, had reduced myo-inositol and NAA concentrations relative to controls in both regions. Although patient groups had intact instrumental learning task performance, both groups reported increased routine behaviors compared to controls, and automaticity was related to reduced prefrontal glutamate and NAA participants with AN-BP. Our findings extend previous reports of reduced myo-inositol and NAA levels in restrictive AN to AN-BP, which may reflect disrupted axonal-glial signaling. Although we found inconsistent support for increased habitual behavior in AN-BP and BN, we identified preliminary associations between prefrontal metabolites and automaticity in AN-BP. These results provide further evidence of unique neurobiological profiles across binge-eating disorders.

Enhanced habit formation in Tourette patients explained by shortcut modulation in a hierarchical cortico-basal ganglia model

Devaluation protocols reveal that Tourette patients show an increased propensity to habitual behaviors as they continue to respond to devalued outcomes in a cognitive stimulus-response-outcome association task. We use a neuro-computational model of hierarchically organized cortico-basal ganglia-thalamo-cortical loops to shed more light on habit formation and its alteration in Tourette patients. In our model, habitual behavior emerges from cortico-thalamic shortcut connections, where enhanced habit formation can be linked to faster plasticity in the shortcut or to a stronger feedback from the shortcut to the basal ganglia. We explore two major hypotheses of Tourette pathophysiology-local striatal disinhibition and increased dopaminergic modulation of striatal medium spiny neurons-as causes for altered shortcut activation. Both model changes altered shortcut functioning and resulted in higher rates of responses towards devalued outcomes, similar to what is observed in Tourette patients. We recommend future experimental neuroscientific studies to locate shortcuts between cortico-basal ganglia-thalamo-cortical loops in the human brain and study their potential role in health and disease.

The Habitual Tendencies Questionnaire: A tool for psychometric individual differences research

Habits are automatic responses to learned stimuli or contextual cues that are insensitive to goals. Although habits may allow for automated behaviours that increase efficiency in our daily lives, an over-reliance on habits has been suggested to contribute to disorders such as obsessive-compulsive disorder (OCD). There are currently few established measures of individual differences in habitual tendencies. To fill this gap, the present study generated and validated a novel 11-item scale, the Habitual Tendencies Questionnaire (HTQ), to measure individual differences in habitual tendencies in the general population. In Study 1, factor analysis revealed three underlying subcomponents of the HTQ: Compulsivity, Preference for Regularity, and Aversion to Novelty, with Compulsivity showing the strongest association with subclinical OCD symptomatology. Study 2 validated the HTQ and replicated the findings of Study 1 in a larger sample, and explored relationships with other personality traits. The results emphasise the importance of measuring individual variation in habitual thinking styles, illustrating that different facets of habitual tendencies may contribute to diverse behavioural and clinical outcomes. The present investigation provides a new, reliable way of measuring habitual tendencies and has important implications for future explorations into the nature of individual differences from a dimensional perspective to psychiatry.

Decreased putamen activation in balancing goal-directed and habitual behavior in binge eating disorder

Acute stress is associated with a shift from goal-directed to habitual behavior. This stress-induced preference for habitual behavior has been suggested as a potential mechanism by which binge eating disorder (BED) patients succumb to eating large amounts of high-caloric foods in an uncontrolled manner (i.e., binge episodes). While in healthy subjects the balance between goal-directed and habitual behavior is subserved by the anterior cingulate cortex (ACC), insular cortex, orbitofrontal cortex (OFC), anterior caudate nucleus, and posterior putamen, the brain mechanism that underlies this (possibly amplified) stress-induced behavioral shift in BED patients is currently unknown. In the current study, 76 participants (38 BED, 38 healthy controls (HCs)) learned six stimulus-response-outcome associations in a well-established instrumental learning task. Subsequently, three outcomes were selectively devalued, after which participants underwent either a stress induction procedure (Maastricht Acute Stress Test; MAST) or a no-stress control procedure. Next, the balance between goal-directed and habitual behavior was assessed during functional magnetic resonance imaging. Findings show that the balance between goal-directed and habitual behavior was associated with activity in the ACC, insula, and OFC in no-stress HCs. Although stress and BED did not modulate the balance between goal-directed and habitual behavior, BED participants displayed a smaller difference in putamen activation between trials probing goal-directed and habitual behavior compared with HCs when using a ROI approach. We conclude that putamen activity differences between BED and HC could reflect changes in monitoring of response accuracy or reward value, albeit perhaps not sufficiently to induce a measurable shift from goal-directed to habitual behavior. Future research could clarify potential boundary conditions of stress-induced shifts in instrumental behavior in BED patients.

Aversive motivation and cognitive control

Aversive motivation plays a prominent role in driving individuals to exert cognitive control. However, the complexity of behavioral responses attributed to aversive incentives creates significant challenges for developing a clear understanding of the neural mechanisms of this motivation-control interaction. We review the animal learning, systems neuroscience, and computational literatures to highlight the importance of experimental paradigms that incorporate both motivational context manipulations and mixed motivational components (e.g., bundling of appetitive and aversive incentives). Specifically, we postulate that to understand aversive incentive effects on cognitive control allocation, a critical contextual factor is whether such incentives are associated with negative reinforcement or punishment. We further illustrate how the inclusion of mixed motivational components in experimental paradigms enables increased precision in the measurement of aversive influences on cognitive control. A sharpened experimental and theoretical focus regarding the manipulation and assessment of distinct motivational dimensions promises to advance understanding of the neural, monoaminergic, and computational mechanisms that underlie the interaction of motivation and cognitive control.

On the interrelation between alcohol addiction-like behaviors in rats

RATIONALE

Alcohol use disorder (AUD) is a complex, heterogeneous disorder that only occurs in a minority of alcohol users. Various behavioral constructs, including excessive intake, habit formation, motivation for alcohol and resistance to punishment have been implicated in AUD, but their interrelatedness is unclear.

OBJECTIVE

The aim of this study was therefore to explore the relation between these AUD-associated behavioral constructs in rats. We hypothesised that a subpopulation of animals could be identified that, based on these measures, display consistent AUD-like behavior.

METHODS

Lister Hooded rats (n = 47) were characterised for alcohol consumption, habit formation, motivation for alcohol and quinine-adulterated alcohol consumption. The interrelation between these measures was evaluated through correlation and cluster analyses. In addition, addiction severity scores were computed using different combinations of the behavioral measures, to assess the consistency of the AUD-like subpopulation.

RESULTS

We found that the data was uniformly distributed, as there was no significant tendency of the behavioral measures to cluster in the dataset. On the basis of multiple ranked addiction severity scores, five animals (~ 11%) were classified as displaying AUD-like behavior. The composition of the remaining subpopulation of animals with the highest addiction severity score (9 rats; ~ 19%) varied, depending on the combination of measures included.

CONCLUSION

Consistent AUD-like behavior was detected in a small proportion of alcohol drinking rats. Alcohol consumption, habit formation, motivation for alcohol and punishment resistance contribute in varying degrees to the AUD-like phenotype across the population. These findings emphasise the importance of considering the heterogeneity of AUD-like behavior.

Effort, success, and side of lesion determine arm choice in individuals with chronic stroke

In neurotypical individuals, arm choice in reaching movements depends on expected biomechanical effort, expected success, and a handedness bias. Following a stroke, does arm choice change to account for the decreased motor performance, or does it follow a preinjury habitual preference pattern? Participants with mild-to-moderate chronic stroke who were right-handed before stroke performed reaching movements in both spontaneous and forced-choice blocks, under no-time, medium-time, and fast-time constraint conditions designed to modulate reaching success. Mixed-effects logistic regression models of arm choice revealed that expected effort predicted choices. However, expected success only strongly predicted choice in left-hemiparetic individuals. In addition, reaction times decreased in left-hemiparetic individuals between the no-time and the fast-time constraint conditions but showed no changes in right-hemiparetic individuals. Finally, arm choice in the no-time constraint condition correlated with a clinical measure of spontaneous arm use for right-, but not for left-hemiparetic individuals. Our results are consistent with the view that right-hemiparetic individuals show a habitual pattern of arm choice for reaching movements relatively independent of failures. In contrast, left-hemiparetic individuals appear to choose their paretic left arm more optimally: that is, if a movement with the paretic arm is predicted to be not successful in the upcoming movement, the nonparetic right arm is chosen instead.NEW & NOTEWORTHY Although we are seldom aware of it, we constantly make decisions to use one arm or the other in daily activities. Here, we studied whether these decisions change following stroke. Our results show that effort, success, and side of lesion determine arm choice in a reaching task: whereas left-paretic individuals modified their arm choice in response to failures in reaching the target, right-paretic individuals showed a pattern of choice independent of failures.

Determining the effects of training duration on the behavioral expression of habitual control in humans: a multilaboratory investigation

It has been suggested that there are two distinct and parallel mechanisms for controlling instrumental behavior in mammals: goal-directed actions and habits. To gain an understanding of how these two systems interact to control behavior, it is essential to characterize the mechanisms by which the balance between these systems is influenced by experience. Studies in rodents have shown that the amount of training governs the relative expression of these two systems: Behavior is goal-directed following moderate training, but the more extensively an instrumental action is trained, the more it becomes habitual. It is less clear whether humans exhibit similar training effects on the expression of goal-directed and habitual behavior, as human studies have reported contradictory findings. To tackle these contradictory findings, we formed a consortium, where four laboratories undertook a preregistered experimental induction of habits by manipulating the amount of training. There was no statistical evidence for a main effect of the amount of training on the formation and expression of habits. However, exploratory analyses suggest a moderating effect of the affective component of stress on the impact of training over habit expression. Participants who were lower in affective stress appeared to be initially goal-directed, but became habitual with increased training, whereas participants who were high in affective stress were already habitual even after moderate training, thereby manifesting insensitivity to overtraining effects. Our findings highlight the importance of the role of moderating variables such as individual differences in stress and anxiety when studying the experimental induction of habits in humans.

Maintained goal-directed control with overtraining on ratio schedules

It is thought that goal-directed control of actions weakens or becomes masked by habits over time. We tested the opposing hypothesis that goal-directed control becomes stronger over time, and that this growth is modulated by the overall action-outcome contiguity. Despite group differences in action-outcome contiguity early in training, rats trained under random and fixed ratio schedules showed equivalent goal-directed control of lever pressing that appeared to grow over time. We confirmed that goal-directed control was maintained after extended training under another type of ratio schedule-continuous reinforcement-using specific satiety and taste aversion devaluation methods. These results add to the growing literature showing that extensive training does not reliably weaken goal-directed control and that it may strengthen it, or at least maintain it.

Examining the effect of depressive symptoms on habit formation and habit-breaking

BACKGROUND AND OBJECTIVES

Dysfunction in reward processing is a hallmark feature of depression. In the context of reinforcement learning, previous research has linked depression with reliance on simple habit-driven ('model-free') learning strategies over more complex, goal-directed ('model-based') strategies. However, the relationship between depression and habit-breaking remains an under-explored research area. The current study sought to bridge this gap by investigating the effect of depressive symptoms on habit formation and habit-breaking under monetary and social feedback conditions. Additionally, we examined whether spontaneous eyeblink rate (EBR), an indirect marker for striatal dopamine levels, would modulate such effects.

METHODS

Depressive symptoms were operationalized using self-report measures. To examine differences in habit formation and habit breaking, undergraduate participants (N = 156) completed a two-stage reinforcement learning task with a devaluation procedure using either monetary or social feedback.

RESULTS

Regression results showed that in the monetary feedback condition, spontaneous EBR moderated the relationship between depressive symptoms and model-free strategies; individuals with more depressive symptomatology and high EBR (higher dopamine levels) exhibited increased reliance on model-free strategies. Depressive symptoms negatively predicted devaluation sensitivity, indicative of difficulty in habit-breaking, in both monetary and social feedback contexts.

LIMITATIONS

Social feedback relied on fixed feedback rather than real-time peer evaluations; depressive symptoms were measured using self-report rather than diagnostic criteria for Major Depressive Disorder; dopaminergic functioning was measured using EBR rather than PET imaging; potential confounds were not controlled for.

CONCLUSIONS

These findings have implications for identifying altered patterns of habit formation and deficits in habit-breaking among those experiencing depressive symptoms.

Assigning the right credit to the wrong action: compulsivity in the general population is associated with augmented outcome-irrelevant value-based learning

Compulsive behavior is enacted under a belief that a specific act controls the likelihood of an undesired future event. Compulsive behaviors are widespread in the general population despite having no causal relationship with events they aspire to influence. In the current study, we tested whether there is an increased tendency to assign value to aspects of a task that do not predict an outcome (i.e., outcome-irrelevant learning) among individuals with compulsive tendencies. We studied 514 healthy individuals who completed self-report compulsivity, anxiety, depression, and schizotypal measurements, and a well-established reinforcement-learning task (i.e., the two-step task). As expected, we found a positive relationship between compulsivity and outcome-irrelevant learning. Specifically, individuals who reported having stronger compulsive tendencies (e.g., washing, checking, grooming) also tended to assign value to response keys and stimuli locations that did not predict an outcome. Controlling for overall goal-directed abilities and the co-occurrence of anxious, depressive, or schizotypal tendencies did not impact these associations. These findings indicate that outcome-irrelevant learning processes may contribute to the expression of compulsivity in a general population setting. We highlight the need for future research on the formation of non-veridical action-outcome associations as a factor related to the occurrence and maintenance of compulsive behavior.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.