Acute stress facilitates habitual behavior in female rats

Instrumental behavior can reflect the influence of goal-directed and habitual systems. Contemporary research suggests that stress may facilitate control by the habitual system under conditions where the behavior would otherwise reflect control by the goal-directed system. However, it is unclear how stress modulates the influence of these systems on instrumental responding to achieve this effect, particularly in females. Here, we examine whether a mild psychogenic stressor experienced before acquisition training (Experiment 1), or prior to the test of expression (Experiment 2) would influence goal-directed and habitual control of instrumental responding in female rats. In both experiments, rats acquired an instrumental nose-poke response for a sucrose reward. This was followed by a reinforcer devaluation phase in which half the rats in Stressed and Non-Stressed conditions received pairings of the sucrose pellet with illness induced by lithium chloride until they rejected the pellet when offered. The remaining rats received a control treatment consisting of pellets and illness on separate days (Unpaired). Control by goal-directed and habitual systems was evaluated in a subsequent nonreinforced test of nose poking. The results of Experiment 1 indicated that the Non-Stressed Paired group reduced nose-poking compared to the Unpaired controls, identifying the response as goal directed, whereas the Stressed Paired and Unpaired groups made a similar number of nose pokes identifying the response as habitual despite a similar amount of training. Results from Experiment 2 indicated habitual control of nose-poke responding was present when stress was experienced just prior to the test. Collectively, these data suggest that stress may facilitate habitual control by altering the relative influence of goal-directed and habitual processes underpinning instrumental behavior. These results may be clinically relevant for understanding the contributions of stress to dysregulated instrumental behavior in compulsive pathologies.

Early-life seizures alter habit behavior formation and fronto-striatal circuit dynamics

Obsessive compulsive disorder (OCD) can occur comorbidly with epilepsy; both are complex, disruptive disorders that lower quality of life. Both OCD and epilepsy are disorders of hyperexcitable circuits, but it is unclear whether common circuit pathology may underlie the co-occurrence of these two neuropsychiatric disorders. Here, we induced early-life seizures (ELS) in rats to examine habit formation as a model for compulsive behaviors. Compulsive, repetitive behaviors in OCD utilize the same circuitry as habit formation. We hypothesized that rats with ELS could be more susceptible to habit formation than littermate controls, and that altered behavior would correspond to altered signaling in fronto-striatal circuits that underlie decision-making and action initiation. Here, we show instead that rats with ELS were significantly less likely to form habit behaviors compared with control rats. This behavioral difference corresponded with significant alterations to temporal coordination within and between brain regions that underpin the action to habit transition: 1) phase coherence between the lateral orbitofrontal cortex and dorsomedial striatum (DMS) and 2) theta-gamma coupling within DMS. Finally, we used cortical electrical stimulation as a model of transcranial magnetic stimulation (TMS) to show that temporal coordination of fronto-striatal circuits in control and ELS rats are differentially susceptible to potentiating and suppressive stimulation, suggesting that altered underlying circuit physiology may lead to altered response to therapeutic interventions such as TMS.

Long-term effects of THC exposure on reward learning and motivated behavior in adolescent and adult male rats

RATIONALE

The endocannabinoid system makes critical contributions to reward processing, motivation, and behavioral control. Repeated exposure to THC or other cannabinoid drugs can cause persistent adaptions in the endocannabinoid system and associated neural circuitry. It remains unclear how such treatments affect the way rewards are processed and pursued.

OBJECTIVE AND METHODS

We examined if repeated THC exposure (5 mg/kg/day for 14 days) during adolescence or adulthood led to long-term changes in rats' capacity to flexibly encode and use action-outcome associations for goal-directed decision making. Effects on hedonic feeding and progressive ratio responding were also assessed.

RESULTS

THC exposure had no effect on rats' ability to flexibly select actions following reward devaluation. However, instrumental contingency degradation learning, which involves avoiding an action that is unnecessary for reward delivery, was augmented in rats with a history of adult but not adolescent THC exposure. THC-exposed rats also displayed more vigorous instrumental behavior in this study, suggesting a motivational enhancement. A separate experiment found that while THC exposure had no effect on hedonic feeding behavior, it increased rats' willingness to work for food on a progressive ratio schedule, an effect that was more pronounced when THC was administered to adults. Adolescent and adult THC exposure had opposing effects on the CB1 receptor dependence of progressive ratio performance, decreasing and increasing sensitivity to rimonabant-induced behavioral suppression, respectively.

CONCLUSIONS

Our findings reveal that exposure to a translationally relevant THC exposure regimen induces long-lasting, age-dependent alterations in cognitive and motivational processes that regulate the pursuit of rewards.

Animal models of action control and cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) has historically been considered a motor disorder induced by a loss of dopaminergic neurons in the substantia nigra pars compacta. More recently, it has been recognized to have significant non-motor symptoms, most prominently cognitive symptoms associated with a dysexecutive syndrome. It is common in the literature to see motor and cognitive symptoms treated separately and, indeed, there has been a general call for specialized treatment of the latter, particularly in the more severe cases of PD with mild cognitive impairment and dementia. Animal studies have similarly been developed to model the motor or non-motor symptoms. Nevertheless, considerable research has established that segregating consideration of cognition from the precursors to motor movement, particularly movement associated with goal-directed action, is difficult if not impossible. Indeed, on some contemporary views cognition is embodied in action control, something that is particularly prevalent in theory and evidence relating to the integration of goal-directed and habitual control processes. The current paper addresses these issues within the literature detailing animal models of cognitive dysfunction in PD and their neural and neurochemical bases. Generally, studies using animal models of PD provide some of the clearest evidence for the integration of these action control processes at multiple levels of analysis and imply that consideration of this integrative process may have significant benefits for developing new approaches to the treatment of PD.

Context, attention, and the switch between habit and goal-direction in behavior

This article reviews recent findings from the author’s laboratory that may provide new insights into how habits are made and broken. Habits are extensively practiced behaviors that are automatically evoked by antecedent cues and performed without their goal (or reinforcer) “in mind.” Goal-directed actions, in contrast, are instrumental behaviors that are performed because their goal is remembered and valued. New results suggest that actions may transition to habit after extended practice when conditions encourage reduced attention to the behavior. Consistent with theories of attention and learning, a behavior may command less attention (and become habitual) as its reinforcer becomes well-predicted by cues in the environment; habit learning is prevented if presentation of the reinforcer is uncertain. Other results suggest that habits are not permanent, and that goal-direction can be restored by several environmental manipulations, including exposure to unexpected reinforcers or context change. Habits are more context-dependent than goal-directed actions are. Habit learning causes retroactive interference in a way that is reminiscent of extinction: It inhibits, but does not erase, goal-direction in a context-dependent way. The findings have implications for the understanding of habitual and goal-directed control of behavior as well as disordered behaviors like addictions.

Unexpected food outcomes can return a habit to goal-directed action

Three experiments examined the return of a habitual instrumental response to the status of goal-directed action. In all experiments, rats received extensive training in which lever pressing was reinforced with food pellets on a random-interval schedule of reinforcement. In Experiment 1, the extensively-trained response was not affected by conditioning a taste aversion to the reinforcer, and was therefore considered a habit. However, if the response had earned a new and unexpected food pellet during the final training session, the response was affected by taste aversion conditioning to the (first) reinforcer, and had thus been converted to a goal-directed action. In Experiment 3, 30 min of prefeeding with an irrelevant food pellet immediately before the test also converted a habit back to action, as judged by the taste-aversion devaluation method. That result was consistent with difficulty in finding evidence of habit with the sensory-specific satiety method after extensive instrumental training (Experiment 2). The results suggest that an instrumental behavior's status as a habit is not permanent, and that a habit can be returned to action status by associating it with a surprising reinforcer (Experiment 1) or by giving the animal an unexpected prefeeding immediately prior to the action/habit test (Experiment 3).

A comparison of simple movement behaviors across three different devices

Reaching trajectories have provided a unique tool to observe changes in internal cognitive decisions. Furthermore, technological advances have made devices for measuring reach movements more accessible and researchers have recognized that various populations including children, elderly populations, and non-human primates can easily execute simple movements as responses. As a result, devices such as a three-dimensional (3D) reach tracker, a stylus, or a computer-mouse have been increasingly utilized to study cognitive processes. However, although the specific type of tracking device that a researcher uses may impact behavior due to the constraints it places on movements, most researchers in these fields are unaware of this potential issue. Here, we examined the potential behavioral impact of using each of these three devices. To induce re-directed movements that mimic the movements that often occur following changes in cognitive states, we used a double-step task in which displacement of an initial target location requires participants to quickly re-direct their movement. We found that reach movement parameters were largely comparable across the three devices. However, hand movements measured by a 3D reach tracker showed earlier reach initiation latencies (relative to stylus movements) and more curved movement trajectories (relative to both mouse and stylus movements). Reach movements were also re-directed following target displacement more rapidly. Thus, 3D reach trackers may be ideal for observing fast, subtle changes in internal decision-making processes compared to other devices. Taken together, this study provides a useful reference for comparing and implementing reaching studies to examine human cognition.

Affect-driven impulsivity impairs human action control and selection, as measured through Pavlovian instrumental transfer and outcome devaluation

Two experiments were designed to study the role of emotional impulsiveness in action control and selection, involving healthy young women participants. In Experiment 1, the effects of both outcome devaluation and Pavlovian-to-instrumental transfer (PIT) were assessed on instrumental responding. In Experiment 2, we further explored the effect of outcome devaluation on outcome-specific PIT. The role of emotional impulsivity, specifically negative urgency (NU), was also evaluated in both experiments using a self-reported measure (UPPS-P scale, Spanish short version). Experiment 1 showed both outcome devaluation and outcome-specific PIT effects, which were positively inter-correlated and negatively correlated with scores in NU. Experiment 2 found an effect of outcome devaluation on outcome-specific PIT, which was negatively correlated with scores on NU. These results highlight the relevance of considering individual differences in affect-driven impulsivity, specifically NU, when addressing failures in action control and selection (proneness to habit). Moreover, these findings suggest that, at least with the procedure used in these experiments, outcome-specific PIT may be based on a goal-directed process that is under the participant's control.

Subtle temporal delays of mothers' responses affect imitation learning in children: Mother-child interaction study

The relationship between temporal contiguity of mothers' teaching behaviors and children's imitation learning was investigated. Participants (2-year-old children) observed their mothers' demonstration of using novel toys over a double television system under live and delayed conditions. The dyads normally interacted in the live condition, whereas they interacted with a 1-s time delay inserted between the children's actions and mothers' responses in the delayed condition. Then, the children were tested with identical toys. Results indicated that children's smiling responses and imitation performances were significantly decreased in the delayed condition compared with the live condition, although mothers' teaching approach did not differ between conditions. These results suggest that a subtle temporal delay in mothers' responses could affect young children's imitation learning.

Motivational sensitivity of outcome-response priming: Experimental research and theoretical models

Outcome-response (O-R) priming is at the core of various associative theories of human intentional action. This is a simple and parsimonious mechanism by which activation of outcome representations (e.g. thinking about the light coming on) leads to activation of the associated motor patterns required to achieve it (e.g. pushing the light switch). In the current manuscript, we review the evidence for such O-R associative links demonstrated by converging (yet until now, separate) strands of research. While there is a wealth of evidence that both the perceptual and motivational properties of an outcome can be encoded in the O-R association and mediate O-R priming, we critically examine the integration of these mechanisms and the conditions under which motivational factors constrain the sensory O-R priming effect. We discuss the clinical relevance of this O-R priming mechanism, whether it can satisfactorily account for human goal-directed behaviour, and the implications for theories of human action control.

Impaired anticipatory vision and visuomotor coordination affects action planning and execution in children with hemiplegic cerebral palsy

BACKGROUND

Action-planning and execution deficits in children with hemiplegic cerebral palsy (HCP) are potentially due to deficits in the integration of sensory information, such as vision, with motor output.

AIMS

To determine differences in anticipatory visual patterns in children with HCP compared to typically developing (TD) children, and to assess visuomotor coordination in children with HCP.

METHODS AND PROCEDURES

We included 13 children with HCP (Age = 6.8 + 2.9 yrs) and 15 TD children (Age = 5.8 + 1.1 yrs). The experimental task used in this study is a valid action-planning task, which consisted of initially reaching and grasping an object placed at a fixed position, followed by placing the object in a random target position. Visual patterns were recorded using a head-mounted eye-tracker system and arm movements were recorded using motion capture (120 Hz).

OUTCOMES AND RESULTS

Children with HCP had delayed anticipatory gaze time and longer latency than TD children during the planning and execution phases. Children with HCP also had a higher frequency of gaze shifts, longer reaction times (RT) and movement times (MT) than TD children.

CONCLUSIONS AND IMPLICATIONS

Children with HCP may have deficits in anticipatory vision, which potentially affected planning and executing a goal-directed action. Therapeutic interventions focusing on improving visuomotor coordination may improve the motor performance in children with HCP.

Inactivation of prelimbic and infralimbic cortex respectively affects minimally-trained and extensively-trained goal-directed actions

Several studies have examined a role for the prelimbic cortex (PL) and infralimbic cortex (IL) in free operant behavior. The general conclusion has been that PL controls goal-directed actions (instrumental behaviors that are sensitive to reinforcer devaluation) whereas IL controls habits (instrumental behaviors that are not sensitive to reinforcer devaluation). To further examine the involvement of these regions in the expression of instrumental behavior, we first implanted male rats with bilateral guide cannulae into their PL, then trained two responses to produce a sucrose pellet reinforcer, R1 and R2, each in a distinct context. R1 received extensive training and R2 received minimal training. Rats then received lithium chloride injections either paired or unpaired with sucrose pellets in both contexts until paired rats rejected all pellets. Following acquisition, in Experiment 1, rats received either an infusion of saline or baclofen/muscimol into the PL and were tested (in extinction) on both R1 and R2. In vehicle controls, both responses were goal-directed actions, as indicated by their sensitivity to reinforcer devaluation. PL inactivation decreased expression of the minimally-trained action without affecting expression of the extensively-trained action. Experiment 2 utilized the same experimental design but with IL inactivation at test. The extensively-trained response was again a goal-directed action. However, now expression of the extensively-trained goal-directed action was suppressed by IL inactivation. The overall pattern of results suggests that the PL is involved in expression of minimally trained goal-directed behavior while the IL is involved in expression of extensively trained goal-directed behavior. This implies that the PL does not control all types of actions and the IL can control some types of actions. These results expand upon the traditional view that the PL controls action while the IL controls habit.

Is the extrastriate body area part of the dorsal visuomotor stream?

The extrastriate body area (EBA) processes visual information about body parts, and it is considered one among a series of category-specific perceptual modules distributed across the occipito-temporal cortex. However, recent evidence raises the possibility that EBA might also provide an interface between perception and action, linking the ventral and dorsal streams of visual information processing. Here, we assess anatomical evidence supporting this possibility. We localise EBA in individual subjects using a perceptual task and compare the characteristics of its functional and structural connectivity to those of two perceptual areas, the lateral occipital complex (LOC) and the fusiform body area (FBA), separately for each hemisphere. We apply complementary analyses of resting-state fMRI and diffusion-weighted MRI data in a group of healthy right-handed human subjects (N = 31). Functional and structural connectivity profiles indicate that EBA interacts more strongly with dorsal-stream regions compared to other portions of the occipito-temporal cortex involved in processing body parts (FBA) and object identification (LOC). These findings provide anatomical ground for a revision of the functional role of EBA. Building on a number of recent observations, we suggest that EBA contributes to planning goal-directed actions, possibly by specifying a desired postural configuration to parieto-frontal areas involved in computing movement parameters.

'Unwilling' versus 'unable': Tonkean macaques' understanding of human goal-directed actions

The present study investigated the understanding of goal-directed actions in Tonkean macaques (Macaca tonkeana) using the unwilling versus unable paradigm, previously used in several species. Subjects were tested in three experimental conditions that varied according to the goal-directed actions of a human actor. In the "unwilling" condition, the actor was capable of giving the subject food but unwilling to do it; in the "unable" condition, she was willing to give food but was unable to do it because of a physical barrier; and in the "distracted" condition, she was occupied by manipulating a pebble instead of food. We report for the first time that Tonkean macaques, like capuchins, chimpanzees and human infants, behaved differently across these experimental conditions. They attempted to grasp food in the actor's hand significantly more and displayed more threats in the presence of an unwilling actor rather than an unable or a distracted one. Inversely, they begged significantly more and displayed more frustration behaviors facing a distracted and unable experimenter rather than an unwilling one. These results suggest that Tonkean macaques understand human goal-directed actions by predicting whether they were likely to obtain food merely based on movements, cue and motor intentions reading and understanding of physical constraints.

Neuroscience of learning and memory for addiction medicine: from habit formation to memory reconsolidation

Identifying effective pharmacological treatments for addictive disorders has remained an elusive goal. Many different classes of drugs have shown some efficacy in preclinical models, but the number of effective clinical therapeutics has remained stubbornly low. The persistence of drug use and the high frequency of relapse is at least partly attributable to the enduring ability of environmental stimuli associated with drug use to maintain behavioral patterns of drug use and induce craving during abstinence. We propose that stimuli associated with drug use exert such powerful control over behavior through the development of abnormally strong memories, and their ability to initiate subconscious sequences of motor actions (habits) that promote uncontrolled drug use. In this chapter, we will review the evidence suggesting that drugs of abuse strengthen associations with cues in the environment and facilitate habit formation. We will also discuss potential mechanisms for disrupting memories associated with drug use to help improve treatments for addiction.

Limits of ideomotor action-outcome acquisition

Ideomotor theory proposes that goal-directed action emerges from the implicit, incidental acquisition of bi-directional associations between actions and their outcomes. In line with this idea, a simple two-stage priming paradigm has provided evidence that presentation of outcomes primes previously associated actions. In the current study we compare the standard priming paradigm with two actions and two unique outcomes (Experiment 1) with two more complex, but otherwise identical versions (Experiment 2: two vs. four actions with four outcomes). Our results show stronger evidence of action-outcome learning in the simple compared to the more complex versions. We suggest that, when using the classic two-stage paradigm, action-outcome acquisition is limited to just a few action-outcome associations that can be concurrently learned-at least if learning is not supported by discriminative stimuli and outcomes are not salient or motivationally relevant. This article is part of a Special Issue entitled SI: Prediction and Attention.

Goal-directed EEG activity evoked by discriminative stimuli in reinforcement learning

In reinforcement learning (RL), discriminative stimuli (S) allow agents to anticipate the value of a future outcome, and the response that will produce that outcome. We examined this processing by recording EEG locked to S during RL. Incentive value of outcomes and predictive value of S were manipulated, allowing us to discriminate between outcome-related and response-related activity. S predicting the correct response differed from nonpredictive S in the P2. S paired with high-value outcomes differed from those paired with low-value outcomes in a frontocentral positivity and in the P3b. A slow negativity then distinguished between predictive and nonpredictive S. These results suggest that, first, attention prioritizes detection of informative S. Activation of mental representations of these informative S then retrieves representations of outcomes, which in turn retrieve representations of responses that previously produced those outcomes.

Dorsal and ventral streams: the distinct role of striatal subregions in the acquisition and performance of goal-directed actions

Considerable evidence suggests that distinct neural processes mediate the acquisition and performance of goal-directed instrumental actions. Whereas a cortical-dorsomedial striatal circuit appears critical for the acquisition of goal-directed actions, a cortical-ventral striatal circuit appears to mediate instrumental performance, particularly the motivational control of performance. Here we review evidence that these distinct mechanisms of learning and performance constitute two distinct 'streams' controlling instrumental conditioning. From this perspective, the regulation of the interaction between these 'streams' becomes a matter of considerable importance. We describe evidence that the basolateral amygdala, which is heavily interconnected with both the dorsal and ventral subregions of the striatum, coordinates this interaction providing input to the final common path to action as a critical component of the limbic-motor interface.