Control over a stressor involves the posterior dorsal striatum and the act/outcome circuit

Making and breaking habits: Revisiting the definitions and behavioral factors that influence habits in animals

Habits have garnered significant interest in studies of associative learning and maladaptive behavior. However, habit research has faced scrutiny and challenges related to the definitions and methods. Differences in the conceptualizations of habits between animal and human studies create difficulties for translational research. Here, we review the definitions and commonly used methods for studying habits in animals and humans and discuss potential alternative ways to assess habits, such as automaticity. To better understand habits, we then focus on the behavioral factors that have been shown to make or break habits in animals, as well as potential mechanisms underlying the influence of these factors. We discuss the evidence that habitual and goal-directed systems learn in parallel and that they seem to interact in competitive and cooperative manners. Finally, we draw parallels between habitual responding and compulsive drug seeking in animals to delineate the similarities and differences in these behaviors.

Prior experience with behavioral control over stress facilitates social dominance

Dominance status has extensive effects on physical and mental health, and an individual's relative position can be shaped by experiential factors. A variety of considerations suggest that the experience of behavioral control over stressors should produce winning in dominance tests and that winning should blunt the impact of later stressors, as does prior control. To investigate the interplay between competitive success and stressor control, we first examined the impact of stressor controllability on subsequent performance in a warm spot competition test modified for rats. Prior experience of controllable, but not physically identical uncontrollable, stress increased later effortful behavior and occupation of the warm spot. Controllable stress subjects consistently ranked higher than did uncontrollable stress subjects. Pharmacological inactivation of the prelimbic (PL) cortex during behavioral control prevented later facilitation of dominance. Next, we explored whether repeated winning experiences produced later resistance against the typical sequelae of uncontrollable stress. To establish dominance status, triads of rats were given five sessions of warm spot competition. The development of stable dominance was prevented by reversible inactivation of the PL or NMDA receptor blockade in the dorsomedial striatum. Stable winning blunted the later stress-induced increase in dorsal raphe nucleus serotonergic activity, as well as prevented uncontrollable stress-induced social avoidance. In contrast, endocrine and neuroimmune responses to uncontrollable stress were unaffected, indicating a selective impact of prior dominance. Together, these data demonstrate that instrumental control over stress promotes later dominance, but also reveal that winning experiences buffer against the neural and behavioral outcomes of future adversity.

Punishment resistance for cocaine is associated with inflexible habits in rats

Addiction is characterized by continued drug use despite negative consequences. In an animal model, a subset of rats continues to self-administer cocaine despite footshock consequences, showing punishment resistance. We sought to test the hypothesis that punishment resistance arises from failure to exert goal-directed control over habitual cocaine seeking. While habits are not inherently permanent or maladaptive, continued use of habits under conditions that should encourage goal-directed control makes them maladaptive and inflexible. We trained male and female Sprague Dawley rats on a seeking-taking chained schedule of cocaine self-administration (2 h/day). We then exposed them to 4 days of punishment testing, in which footshock (0.4 mA, 0.3 s) was delivered randomly on one-third of trials, immediately following completion of seeking and prior to extension of the taking lever. Before and after punishment testing (4 days pre-punishment and ≥4 days post-punishment), we assessed whether cocaine seeking was goal-directed or habitual using outcome devaluation via cocaine satiety. We found that punishment resistance was associated with continued use of habits, whereas punishment sensitivity was associated with increased goal-directed control. Although punishment resistance was not predicted by habitual responding pre-punishment, it was associated with habitual responding post-punishment. In parallel studies of food self-administration, we similarly observed that punishment resistance was associated with habitual responding post-punishment but not pre-punishment. These findings indicate that punishment resistance is related to habits that have become inflexible and persist under conditions that should encourage a transition to goal-directed behavior.

Prior experience with behavioral control over stress facilitates social dominance

Dominance status has extensive effects on physical and mental health, and an individual's relative position can be shaped by experiential factors. A variety of considerations suggest that the experience of behavioral control over stressors should produce winning in dominance tests and that winning should blunt the impact of later stressors, as does prior control. To investigate the interplay between competitive success and stressor control, we first examined the impact of stressor controllability on subsequent performance in a warm spot competition test modified for rats. Prior experience of controllable, but not physically identical uncontrollable, stress increased later effortful behavior and occupation of the warm spot. Controllable stress subjects consistently ranked higher than did uncontrollable stress subjects. Pharmacological inactivation of the prelimbic (PL) cortex during behavioral control prevented later facilitation of dominance. Next, we explored whether repeated winning experiences produced later resistance against the typical sequelae of uncontrollable stress. To establish dominance status, triads of rats were given five sessions of warm spot competition. Reversible inactivation of the PL or NMDA receptor blockade in the dorsomedial striatum led to a long-term reduction in social rank. Stable dominance blunted the later stress-induced increase in dorsal raphe nucleus serotonergic activity, as well as prevented stress-induced social avoidance. In contrast, endocrine and neuroimmune responses to uncontrollable stress were unaffected, indicating a selective impact of prior dominance. Together, these data demonstrate that instrumental control over stress promotes later dominance, but also reveal that winning experiences buffer against the neural and behavioral outcomes of future adversity.

From helplessness to controllability: toward a neuroscience of resilience

"Learned helplessness" refers to debilitating outcomes, such as passivity and increased fear, that follow an uncontrollable adverse event, but do not when that event is controllable. The original explanation argued that when events are uncontrollable the animal learns that outcomes are independent of its behavior, and that this is the active ingredient in producing the effects. Controllable adverse events, in contrast, fail to produce these outcomes because they lack the active uncontrollability element. Recent work on the neural basis of helplessness, however, takes the opposite view. Prolonged exposure to aversive stimulation per se produces the debilitation by potent activation of serotonergic neurons in the brainstem dorsal raphe nucleus. Debilitation is prevented with an instrumental controlling response, which activates prefrontal circuitry detecting control and subsequently blunting the dorsal raphe nucleus response. Furthermore, learning control alters the prefrontal response to future adverse events, thereby preventing debilitation and producing long-term resiliency. The general implications of these neuroscience findings may apply to psychological therapy and prevention, in particular by suggesting the importance of cognitions and control, rather than habits of control.

Prefrontal cortex neurons in adult rats exposed to early life stress fail to appropriately signal the consequences of motivated actions

Early life stress (ELS) can set the stage for susceptibility to cognitive and emotional dysfunction in adulthood by disrupting typical neural development. The prefrontal cortex (PFC) continues to mature during early life, making this region particularly vulnerable to disruption for animals who experience ELS. Despite this, the effects of ELS experience on in vivo PFC function in awake and behaving adult animals are currently poorly understood. To assess this, we employed an instrumental conflict task to assess how hungry adult rats, either ELS (wet bedding) or unstressed Controls, were able to flexibly alter their motivation for food reward seeking (lever presses) in situations that were either threatening or safe. During this task, in vivo electrophysiological recordings (both single unit and local field potentials [LFPs]) were made in the rats' ventral-medial PFC (vmPFC). We found that ELS rats were less motivated to lever press for rewards than Controls in the threat situations during repeated extinction sessions. In recordings taken during this suppression task, Control vmPFC neurons displayed reliable differences between motivated actions, such as between rewarded and unrewarded presses, but ELS neurons failed to differentiate these action-outcome differences. We also found differences in task-related LFP activity between groups; in particular, prior ELS experience appears to induce abnormal changes in low-frequency oscillations during shock-associated threat stimuli prior to presses, as well as diminished higher-frequency oscillations following rewarded presses. Collectively, we demonstrate that ELS experience produces persistent impairment in motivational regulation that is associated with significant changes in in vivo PFC signals. Specifically, ELS-experienced adults fail to appropriately update motivated action strategies under threat conditions, and likewise fail to appropriately monitor and update action/outcome relationships in motivated behavior. These ELS-related changes may therefore lay the foundation for heightened susceptibility to mental-health disorders in adults such as substance abuse and post-traumatic stress disorder.

Elevated prefrontal dopamine interferes with the stress-buffering properties of behavioral control in female rats

Stress-linked disorders are more prevalent in women than in men and differ in their clinical presentation. Thus, investigating sex differences in factors that promote susceptibility or resilience to stress outcomes, and the circuit elements that mediate their effects, is important. In male rats, instrumental control over stressors engages a corticostriatal system involving the prelimbic cortex (PL) and dorsomedial striatum (DMS) that prevent many of the sequelae of stress exposure. Interestingly, control does not buffer against stress outcomes in females, and here, we provide evidence that the instrumental controlling response in females is supported instead by the dorsolateral striatum (DLS). Additionally, we used in vivo microdialysis, fluorescent in situ hybridization, and receptor subtype pharmacology to examine the contribution of prefrontal dopamine (DA) to the differential impact of behavioral control. Although both sexes preferentially expressed D1 receptor mRNA in PL GABAergic neurons, there were robust sex differences in the dynamic properties of prefrontal DA during controllable stress. Behavioral control potently attenuated stress-induced DA efflux in males, but not females, who showed a sustained DA increase throughout the entire stress session. Importantly, PL D1 receptor blockade (SCH 23390) shifted the proportion of striatal activity from the DLS to the DMS in females and produced the protective effects of behavioral control. These findings suggest a sex-selective mechanism in which elevated DA in the PL biases instrumental responding towards prefrontal-independent striatal circuitry, thereby eliminating the protective impact of coping with stress.

An evaluation of the rat intestinal monoamine biogeography days following exposure to acute stress

Stress-induced abnormalities in gut monoamine levels (e.g., serotonin, dopamine, norepinephrine) have been linked to gastrointestinal (GI) dysfunction, as well as the worsening of symptoms in GI disorders. However, the influence of stress on changes across the entire intestinal monoamine biogeography has not been well-characterized, especially in the days following stress exposure. Therefore, the aim of this study was to comprehensively assess changes to monoamine neurochemical signatures across the entire rat intestinal tract days after exposure to an acute stressor. To the end, adult male F344 rats were subjected to an episode of unpredictable tail shocks (acute stress) or left undisturbed. Forty-eight hours later rats were euthanized either following a 12 h period of fasting or 30 min of food access to evaluate neurochemical profiles during the peri- and early postprandial periods. Monoamine-related neurochemicals were measured via UHPLC in regions of the small intestine (duodenum, jejunum, ileum), large intestine (cecum, proximal colon, distal colon), cecal contents, fecal contents, and liver. The results suggest a relatively wide-spread increase in measures of serotonin activity across intestinal regions can be observed 48 h after exposure to acute stress, however some evidence was found supporting localized differences in serotonin metabolization. Moreover, acute stress exposure reduced catecholamine-related neurochemical concentrations most notably in the ileum, and to a lesser extent in the cecal contents. Next, stress-related fecal serotonin concentrations were consistent with intestinal profiles. However, fecal dopamine was elevated in association with stress, which did not parallel findings in any other intestinal area. Finally, stress exposure and the food access period together only had minor effects on intestinal monoamine profiles. Taken together, these data suggest nuanced differences in monoaminergic profiles exist across intestinal regions the days following exposure to an acute stressor, highlighting the importance of assessments that consider the entire intestinal tract biogeography when investigating stress-related biological outcomes that may be relevant to GI pathophysiology.

The effect of stress and exercise on the learning performance of horses

Domestic horses are widely used for physically demanding activities but the effect of exercise on their learning abilities has not been explored. Horses are also frequently exposed to stressors that may affect their learning. Stress and exercise result in the release of glucocorticoids, noradrenaline and other neurotransmitters that can influence learning. It is not currently possible to directly measure concentrations of neurotransmitters in the brains of behaving horses, however the inference of neurobiological processes from peripheral markers have been widely used in studies of human cognition. We assigned 41 horses to either ridden exercise, uncontrollable stress or inactivity and evaluated their acquisition of an industry-style aversive instrumental learning task. Exercised horses achieved the learning criterion in the fewest number of trials compared to the stressed and inactive horses whose performance did not differ. The exercised horses’ salivary cortisol concentrations decreased during learning whereas the concentrations of the other groups increased. Spearman’s correlations revealed that horses with the highest cortisol concentrations required the most trials to reach the criterion. We present novel data that exercise prior to learning may enhance the acquisition of learning in horses. Conversely, activities that expose horses to uncontrollable stressors causing strong cortisol release may impair learning. It is proposed that these effects may be due to the influence of neurotransmitters such as cortisol and noradrenaline on brain regions responsible for learning.

Environmental certainty influences the neural systems regulating responses to threat and stress

Flexible calibration of threat responding in accordance with the environment is an adaptive process that allows an animal to avoid harm while also maintaining engagement of other goal-directed actions. This calibration process, referred to as threat response regulation, requires an animal to calculate the probability that a given encounter will result in a threat so they can respond accordingly. Here we review the neural correlates of two highly studied forms of threat response suppression: extinction and safety conditioning. We focus on how relative levels of certainty or uncertainty in the surrounding environment alter the acquisition and application of these processes. We also discuss evidence indicating altered threat response regulation following stress exposure, including enhanced fear conditioning, and disrupted extinction and safety conditioning. To conclude, we discuss research using an animal model of coping that examines the impact of stressor controllability on threat responding, highlighting the potential for previous experiences with control, or other forms of coping, to protect against the effects of future adversity.

Neurocomputational mechanism of controllability inference under a multi-agent setting

Controllability perception significantly influences motivated behavior and emotion and requires an estimation of one’s influence on an environment. Previous studies have shown that an agent can infer controllability by observing contingency between one’s own action and outcome if there are no other outcome-relevant agents in an environment. However, if there are multiple agents who can influence the outcome, estimation of one’s genuine controllability requires exclusion of other agents’ possible influence. Here, we first investigated a computational and neural mechanism of controllability inference in a multi-agent setting. Our novel multi-agent Bayesian controllability inference model showed that other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which can be explained as a Bayesian inference. Model-based functional MRI analyses showed that multi-agent Bayesian controllability inference recruits the temporoparietal junction (TPJ) and striatum. Then, this inferred controllability information was leveraged to increase motivated behavior in the vmPFC. These results generalize the previously known role of the striatum and vmPFC in single-agent controllability to multi-agent controllability, and this generalized role requires the TPJ in addition to the striatum of single-agent controllability to integrate both self- and other-related information. Finally, we identified an innate positive bias toward the self during the multi-agent controllability inference, which facilitated behavioral adaptation under volatile controllability. Furthermore, low positive bias and high negative bias were associated with increased daily feelings of guilt. Our results provide a mechanism of how our sense of controllability fluctuates due to other people in our lives, which might be related to social learned helplessness and depression.

How we perceive controllability over an outcome if there are multiple other agents who can simultaneously influence that outcome? Previous ‘single-agent’ studies showed that an agents’ inferred controllability depends on contingency between its own action and following outcome and this inference involves striatum. Here, we show that in a multi-agent setting, other people’s action-outcome contingency information is integrated with one’s own action-outcome contingency to infer controllability, which was explained as a biased Bayesian inference. Notably, bias in inference played an adaptive role under volatile controllability and was associated with a perception of guilt. Striatum and temporoparietal junction (TPJ) were involved in this multi-agent Bayesian controllability inference and this controllability information was leveraged to increase motivated behavior in the vmPFC. Our results first provide a neurocomputational mechanism of multi-agent controllability inference.

Frontostriatal Projections Regulate Innate Avoidance Behavior

The dorsomedial prefrontal cortex (dmPFC) has been linked to avoidance and decision-making under conflict, key neural computations altered in anxiety disorders. However, the heterogeneity of prefrontal projections has obscured identification of specific top-down projections involved. While the dmPFC-amygdala circuit has long been implicated in controlling reflexive fear responses, recent work suggests that dmPFC-dorsomedial striatum (DMS) projections may be more important for regulating avoidance. Using fiber photometry recordings in both male and female mice during the elevated zero maze task, we show heightened neural activity in frontostriatal but not frontoamygdalar projection neurons during exploration of the anxiogenic open arms. Additionally, using optogenetics, we demonstrate that this frontostriatal projection preferentially excites postsynaptic D1 receptor-expressing neurons in the DMS and causally controls innate avoidance behavior. These results support a model for prefrontal control of defensive behavior in which the dmPFC-amygdala projection controls reflexive fear behavior and the dmPFC-striatum projection controls anxious avoidance behavior.SIGNIFICANCE STATEMENT The medial prefrontal cortex has been extensively linked to several behavioral symptom domains related to anxiety disorders, with much of the work centered around reflexive fear responses. Comparatively little is known at the mechanistic level about anxious avoidance behavior, a core feature across anxiety disorders. Recent work has suggested that the striatum may be an important hub for regulating avoidance behaviors. Our work uses optical circuit dissection techniques to identify a specific corticostriatal circuit involved in encoding and controlling avoidance behavior. Identifying neural circuits for avoidance will enable the development of more targeted symptom-specific treatments for anxiety disorders.

Functional networks activated by controllable and uncontrollable stress in male and female rats

The ability of an individual to reduce the intensity, duration or frequency of a stressor is a critical determinant of the consequences of that stressor on physiology and behavior. To expand our understanding of the brain networks engaged during controllable and uncontrollable stress and to identify sex differences, we used functional connectivity analyses of the immediate early gene product Fos in male and female rats exposed to either controllable or uncontrollable tail shocks. Twenty-eight regions of interest (ROI) were selected from the structures previously evinced to be responsible for stress response, action-outcome learning, or sexual dimorphism. We found that connectivity across these structures was strongest in female rats without control while weaker connectivity was evident in male rats with control over stress. Interestingly, this pattern correlates with known behavioral sex differences where stressor controllability leads to resilience in male but not female rats. Graph theoretical analysis identified several structures important to networks under specific conditions. In sum, the findings suggest that control over stress reshapes functional connectivity.

Data Triangulation in Consumer Neuroscience: Integrating Functional Neuroimaging With Meta-Analyses, Psychometrics, and Behavioral Data

This article reviews a wide range of functional magnetic resonance imaging (fMRI) studies conducted in the field of consumer neuroscience to (1) highlight common interpretative approaches of neuroimaging data (i.e., forward inference and reverse inference), (2) discuss potential interpretative issues associated with these approaches, and (3) provide a framework that employs a multi-method approach aimed to possibly raise the explanatory power and, thus, the validity of functional neuroimaging research in consumer neuroscience. Based on this framework, we argue that the validity of fMRI studies can be improved by the triangulation of (1) careful design of neuroimaging studies and analyses of data, (2) meta-analyses, and (3) the integration of psychometric and behavioral data with neuroimaging data. Guidelines on when and how to employ triangulation methods on neuroimaging data are included. Moreover, we also included discussions on practices and research directions that validate fMRI studies in consumer neuroscience beyond data triangulation.

Activity of a vmPFC-DRN Pathway Corresponds With Resistance to Acute Social Defeat Stress

The ventromedial prefrontal cortex (vmPFC) plays a critical role in stress resilience through top-down inhibition of key stress-sensitive limbic and hindbrain structures, including the dorsal raphe nucleus (DRN). In a model of experience-dependent stress resistance, socially dominant Syrian hamsters display fewer signs of anxiety following acute social defeat when compared to subordinate or control counterparts. Further, dominants activate vmPFC neurons to a greater degree during stress than do subordinates and become stress-vulnerable following pharmacological inhibition of the vmPFC. Dominants also display fewer stress-activated DRN neurons than subordinates do, suggesting that dominance experience gates activation of vmPFC neurons that inhibit the DRN during social defeat stress. To test whether social dominance alters stress-induced activity of a vmPFC-DRN pathway, we injected a retrograde tracer, cholera toxin B (CTB), into the DRN of dominant, subordinate, and control hamsters and used a dual-label immunohistochemical approach to identify vmPFC neurons co-labeled with CTB and the defeat-induced expression of an immediate early gene, cFos. Results indicate that dominant hamsters display more cFos+ and dual-labeled cells in layers V/VI of infralimbic and prelimbic subregions of the vmPFC compared to other animals. Furthermore, vmPFC-DRN activation corresponded directly with proactive behavioral strategies during defeat, which is indicative of stress resilience. Together, results suggest that recruiting the vmPFC-DRN pathway during acute stress corresponds with resistance to the effects of social defeat in dominant hamsters. Overall, these findings indicate that a monosynaptic vmPFC-DRN pathway can be engaged in an experience-dependent manner, which has implications for behavioral interventions aimed at alleviating stress-related psychopathologies.

Sex differences in resilience: Experiential factors and their mechanisms

Adverse life events can lead to stable changes in brain structure and function and are considered primary sources of risk for post-traumatic stress disorder, depression and other neuropsychiatric disorders. However, most individuals do not develop these conditions following exposure to traumatic experiences, and research efforts have identified a number of experiential factors associated with an individual's ability to withstand, adapt to and facilitate recovery from adversity. While multiple animal models of stress resilience exist, so that the detailed biological mechanisms can be explored, studies have been disproportionately conducted in male subjects even though the prevalence and presentation of stress-linked disorders differ between sexes. This review focuses on (a) the mechanisms by which experiential factors (behavioral control over a stressor, exercise) reduce the impact of adverse events as studied in males; (b) whether other manipulations (ketamine) that buffer against stress-induced sequelae engage the same circuit features; and (c) whether these processes operate similarly in females. We argue that investigation of experiential factors that produce resistance/resilience rather than vulnerability to adversity will generate a unique set of biological mechanisms that potentially underlie sex differences in mood disorders.

Controllability affects endocrine response of adolescent male rats to stress as well as impulsivity and behavioral flexibility during adulthood

Exposure to stress during adolescence exerts a long-term impact on behavior and might contribute to the development of several neuropsychiatric disorders. In adults, control over stress has been found to protect from the negative consequences of stress, but the influence of controllability at early ages has not been extensively studied. Here, we evaluated in a rodent model the effects of repeated exposure in adolescent male rats to controllable versus uncontrollable foot-shock stress (CST or UST, respectively). Rats were assigned to three groups: non-stress (stress-naïve), CST (exposed to 8 sessions of a two-way shuttle active avoidance task over a period of 22 days) and UST (receiving the same amount of shocks as CST, regardless of their actual behavior). During adulthood, different cohorts were tested in several tasks evaluating inhibitory control and cognitive flexibility: 5-choice serial reaction time, delay-discounting, gambling test and probabilistic reversal learning. Results showed that the hypothalamic-pituitary-adrenal response to the first shock session was similar in CST and UST animals, but the response to the 8^th session was lower in CST animals. In adulthood, the UST animals presented impaired motor (but not cognitive) impulsivity and more perseverative behavior. The behavioral effects of UST were associated with increased number of D2 dopamine receptors in dorsomedial striatum, but not in other striatal regions. In summary, UST exposure during adolescence induced long-term impairments in impulsivity and compulsivity, whereas CST had only minor effects. These data support a critical role of stress uncontrollability on the long-lasting consequences of stress, as a risk factor for mental illnesses.

A Reward-Based Framework of Perceived Control

Perceived control can be broadly defined as the belief in one's ability to exert control over situations or events. It has long been known that perceived control is a major contributor toward mental and physical health as well as a strong predictor of achievements in life. However, one issue that limits a mechanistic understanding of perceived control is the heterogeneity of how the term is defined in models in psychology and neuroscience, and used in experimental settings across a wide spectrum of studies. Here, we propose a framework for studying perceived control by integrating the ideas from traditionally separate work on perceived control. Specifically, we discuss key properties of perceived control from a reward-based framework, including choice opportunity, instrumental contingency, and success/reward rate. We argue that these separate reward-related processes are integral to fostering an enhanced perception of control and influencing an individual's behavior and well-being. We draw on select studies to elucidate how these reward-related elements are implicated separately and collectively in the investigation of perceived control. We highlight the role of dopamine within corticostriatal pathways shared by reward-related processes and perceived control. Finally, through the lens of this reward-based framework of perceived control, we consider the implications of perceived control in clinical deficits and how these insights could help us better understand psychopathology and treatment options.

Individual differences in blink rate modulate the effect of instrumental control on subsequent Pavlovian responding

RATIONALE

Pavlovian conditioned responses to cues that signal threat are rapidly acquired and tend to persist over time. However, recent research suggests that the ability to actively avoid or exert control over an anticipated threat can diminish the subsequent expression of Pavlovian responses. Studies in animal models suggest that active avoidance behavior and its consequences may be mediated by dopaminergic function. In the present study, we sought to replicate the finding that active control over threat can attenuate subsequent Pavlovian responding in humans and conducted exploratory analyses testing whether individual differences in blink rate, a putative index of dopaminergic function, might modulate this effect.

METHODS

Participants underwent Pavlovian aversive conditioning, followed immediately by one of two conditions. In the active avoidance condition, participants had the opportunity to actively prevent the occurrence of an anticipated shock, whereas in a yoked extinction condition, participants passively observed the cessation of shocks, but with no ability to influence their occurrence. The following day, the conditioned stimuli were presented without shock, but both groups of participants had no opportunity to employ active instrumental responses. Blink rate was measured throughout the task, and skin conductance responses served as our index of Pavlovian conditioned responding.

RESULTS

Consistent with our previous findings, we observed that the group that could actively avoid the shock on day 1 exhibited attenuated recovery of Pavlovian conditioned responses. Further, we found that individuals in the active avoidance group with higher blink rates exhibited a more robust attenuation of spontaneous recovery.

CONCLUSION

This finding suggests that individual variation in dopaminergic function may modulate the efficacy with which active avoidance strategies can attenuate reactive Pavlovian responses.

Behavioral and neural mechanisms underlying habitual and compulsive drug seeking

Addiction is characterized by compulsive drug use despite negative consequences. Here we review studies that indicate that compulsive drug use, and in particular punishment resistance in animal models of addiction, is related to impaired cortical control over habitual behavior. In humans and animals, instrumental behavior is supported by goal-directed and habitual systems that rely on distinct corticostriatal networks. Chronic exposure to addictive drugs or stress has been shown to bias instrumental response strategies toward habit learning, and impair prefrontal cortical (PFC) control over responding. Moreover, recent work has implicated prelimbic PFC hypofunction in the punishment resistance that has been observed in a subset of animals with an extended history of cocaine self-administration. This may be related to a broader role for prelimbic PFC in mediating adaptive responding and behavioral flexibility, including exerting goal-directed control over behavior. We hypothesize that impaired cortical control and reduced flexibility between habitual and goal-directed systems may be critically involved in the development of maladaptive, compulsive drug use.

Prefrontal endocannabinoids, stress controllability and resilience: A hypothesis

Stressor exposure is a predisposing risk factor for many psychiatric conditions such as PTSD and depression. However, stressors do not influence all individuals equally and in response to an identical stressor some individuals may be vulnerable while others are resilient. While various biological and behavioral factors contribute to vulnerability versus resilience, an individual's degree of control over the stressor is among the most potent. Even with only one experience with control over stress, behavioral control has been shown to have acute and long-lasting stress-mitigating effects. This suggests that control both blunts the response to acute stress and prepares the subject to be resilient to future stressors. In this review, we first summarize the evidence which suggests the ventromedial prefrontal cortex (vmPFC) is a critical component of stressor controllability circuits and a locus of neuroplasticity supporting the acute and long-lasting consequences of control. We next review the central endocannabinoid (eCB) system as a possible mediator of short and long-term synaptic transmission in the vmPFC, and offer a hypothesis whereby eCBs regulate vmPFC circuits engaged when a subject has control over stress and may contribute to the encoding of acute stress coping into long lasting stressor resilience.

Behavioural and neural sequelae of stressor exposure are not modulated by controllability in females

The degree of behavioural control that an organism has over a stressor is a potent modulator of the stressor's impact; controllable stressors produce none of the neurochemical and behavioural sequelae that occur if the stressor is uncontrollable. Research demonstrating the importance of control and the neural mechanisms responsible has been conducted almost entirely with male rats. It is unknown if behavioural control is stress blunting in females, and whether or not a similar resilience circuitry is engaged. Female rats were exposed to controllable, yoked uncontrollable or no tailshock. In separate experiments, behavioural (juvenile social exploration, fear and shuttle box escape) and neurochemical (activation of dorsal raphe serotonin and dorsal raphe-projecting prelimbic neurons) outcomes, which are sensitive to the dimension of control in males, were assessed. Despite successful acquisition of the controlling response, behavioural control did not mitigate dorsal raphe serotonergic activation and behavioural outcomes induced by tailshock, as it does in males. Moreover, behavioural control failed to selectively engage prelimbic cells that project to the dorsal raphe as in males. Pharmacological activation of the prelimbic cortex restored the stress-buffering effects of control. Collectively, the data demonstrate stressor controllability phenomena are absent in females and that the protective prelimbic circuitry is present but not engaged. Reduced benefit from coping responses may represent a novel approach for understanding differential sex prevalence in stress-related psychiatric disorders.

Impairments in action-outcome learning in schizophrenia

Learning the causal relation between actions and their outcomes (AO learning) is critical for goal-directed behavior when actions are guided by desire for the outcome. This can be contrasted with habits that are acquired by reinforcement and primed by prevailing stimuli, in which causal learning plays no part. Recently, we demonstrated that goal-directed actions are impaired in schizophrenia; however, whether this deficit exists alongside impairments in habit or reinforcement learning is unknown. The present study distinguished deficits in causal learning from reinforcement learning in schizophrenia. We tested people with schizophrenia (SZ, n = 25) and healthy adults (HA, n = 25) in a vending machine task. Participants learned two action-outcome contingencies (e.g., push left to get a chocolate M&M, push right to get a cracker), and they also learned one contingency was degraded by delivery of noncontingent outcomes (e.g., free M&Ms), as well as changes in value by outcome devaluation. Both groups learned the best action to obtain rewards; however, SZ did not distinguish the more causal action when one AO contingency was degraded. Moreover, action selection in SZ was insensitive to changes in outcome value unless feedback was provided, and this was related to the deficit in AO learning. The failure to encode the causal relation between action and outcome in schizophrenia occurred without any apparent deficit in reinforcement learning. This implies that poor goal-directed behavior in schizophrenia cannot be explained by a more primary deficit in reward learning such as insensitivity to reward value or reward prediction errors.

Brain c-fos expression patterns induced by emotional stressors differing in nature and intensity

Regardless of its particular nature, emotional stressors appear to elicit a widespread and roughly similar brain activation pattern as evaluated by c-fos expression. However, their behavioral and physiological consequences may strongly differ. Here we addressed in adult male rats the contribution of the intensity and the particular nature of stressors by comparing, in a set of brain areas, the number of c-fos expressing neurons in response to open-field, cat odor or immobilization on boards (IMO). These are qualitatively different stressors that are known to differ in terms of intensity, as evaluated by biological markers. In the present study, plasma levels of the adrenocorticotropic hormone (ACTH) demonstrated that intensity increases in the following order: open-field, cat odor and IMO. Four different c-fos activation patterns emerged among all areas studied: (i) positive relationship with intensity (posterior-dorsal medial amygdala, dorsomedial hypothalamus, lateral septum ventral and paraventricular nucleus of the hypothalamus), (ii) negative relationship with intensity (cingulate cortex 1, posterior insular cortex, dorsal striatum, nucleus accumbens and some subdivisions of the hippocampal formation); (iii) activation not dependent on the intensity of the stressor (prelimbic and infralimbic cortex and lateral and basolateral amygdala); and (iv) activation specifically associated with cat odor (ventromedial amygdala and ventromedial hypothalamus). Histone 3 phosphorylation at serine 10, another neuronal activation marker, corroborated c-fos results. Summarizing, deepest analysis of the brain activation pattern elicit by emotional stressor indicated that, in spite of activating similar areas, each stressor possess their own brain activation signature, mediated mainly by qualitative aspects but also by intensity.

Inhibition of a Descending Prefrontal Circuit Prevents Ketamine-Induced Stress Resilience in Females

Stress is a potent etiological factor in the onset of major depressive disorder and posttraumatic stress disorder (PTSD). Therefore, significant efforts have been made to identify factors that produce resilience to the outcomes of a later stressor, in hopes of preventing untoward clinical outcomes. The NMDA receptor antagonist ketamine has recently emerged as a prophylactic capable of preventing neurochemical and behavioral outcomes of a future stressor. Despite promising results of preclinical studies performed in male rats, the effects of proactive ketamine in female rats remains unknown. This is alarming given that stress-related disorders affect females at nearly twice the rate of males. Here we explore the prophylactic effects of ketamine on stress-induced anxiety-like behavior and the neural circuit-level processes that mediate these effects in female rats. Ketamine given one week prior to an uncontrollable stressor (inescapable tailshock; IS) reduced typical stress-induced activation of the serotonergic (5-HT) dorsal raphe nucleus (DRN) and eliminated DRN-dependent juvenile social exploration (JSE) deficits 24 h after the stressor. Proactive ketamine altered prelimbic cortex (PL) neural ensembles so that a later experience with IS now activated these cells, which it ordinarily would not. Ketamine acutely activated a PL to DRN (PL-DRN) circuit and inhibition of this circuit with Designer Receptors Exclusively Activated by Designer Drugs (DREADDs) at the time of IS one week later prevented stress prophylaxis, suggesting that persistent changes in PL-DRN circuit activity are responsible, at least in part, for mediating long-term effects associated with ketamine.

New tools for understanding coping and resilience

In humans, many of the factors determining vulnerability and resilience to the impact of an adverse event revolve around coping factors. This mini-review focuses on the neural mechanisms by which coping reduces the impact of adverse events, as studied in an animal model, and discusses some of the challenges of linking neural circuit activity with stressor outcome. We highlight several approaches for probing circuit function with cell-type and pathway-specificity that overcome some of the limitations of traditional neuroscience techniques and will likely yield a more detailed and comprehensive understanding of how the brain regulates stress-responsive structures when coping behaviors are engaged.

Learned helplessness at fifty: Insights from neuroscience

Learned helplessness, the failure to escape shock induced by uncontrollable aversive events, was discovered half a century ago. Seligman and Maier (1967) theorized that animals learned that outcomes were independent of their responses-that nothing they did mattered-and that this learning undermined trying to escape. The mechanism of learned helplessness is now very well-charted biologically, and the original theory got it backward. Passivity in response to shock is not learned. It is the default, unlearned response to prolonged aversive events and it is mediated by the serotonergic activity of the dorsal raphe nucleus, which in turn inhibits escape. This passivity can be overcome by learning control, with the activity of the medial prefrontal cortex, which subserves the detection of control leading to the automatic inhibition of the dorsal raphe nucleus. So animals learn that they can control aversive events, but the passive failure to learn to escape is an unlearned reaction to prolonged aversive stimulation. In addition, alterations of the ventromedial prefrontal cortex-dorsal raphe pathway can come to subserve the expectation of control. We speculate that default passivity and the compensating detection and expectation of control may have substantial implications for how to treat depression. (PsycINFO Database Record

Agency and the Calibration of Motivated Behavior

The controllability of positive or negative environmental events has long been recognized as a critical factor determining their impact on an organism. In studies across species, controllable and uncontrollable reinforcement have been found to yield divergent effects on subsequent behavior. Here we present a model of the organizing influence of control, or a lack thereof, on the behavioral repertoire. We propose that individuals derive a generalizable estimate of agency from controllable and uncontrollable outcomes, which serves to calibrate their behavioral strategies in a manner that is most likely to be adaptive given their prior experience.

Posterior insular cortex is necessary for conditioned inhibition of fear

Veridical detection of safety versus danger is critical to survival. Learned signals for safety inhibit fear, and so when presented, reduce fear responses produced by danger signals. This phenomenon is termed conditioned inhibition of fear. Here, we report that CS+/CS- fear discrimination conditioning over 5 days in rats leads the CS- to become a conditioned inhibitor of fear, as measured by the classic tests of conditioned inhibition: summation and retardation of subsequent fear acquisition. We then show that NMDA-receptor antagonist AP5 injected to posterior insular cortex (IC) before training completely prevented the acquisition of a conditioned fear inhibitor, while intra-AP5 to anterior and medial IC had no effect. To determine if the IC contributes to the recall of learned fear inhibition, injections of the GABAA agonist muscimol were made to posterior IC before a summation test. This resulted in fear inhibition per se, which obscured inference to the effect of IC inactivation with recall of the safety cue. Control experiments sought to determine if the role of the IC in conditioned inhibition learning could be reduced to simpler fear discrimination function, but fear discrimination and recall were unaffected by AP5 or muscimol, respectively, in the posterior IC. These data implicate a role of posterior IC in the learning of conditioned fear inhibitors.

Previous Ketamine Produces an Enduring Blockade of Neurochemical and Behavioral Effects of Uncontrollable Stress

Recent interest in the antidepressant and anti-stress effects of subanesthetic doses of ketamine, an NMDA receptor antagonist, has identified mechanisms whereby ketamine reverses the effect of stress, but little is known regarding the prophylactic effect ketamine might have on future stressors. Here we investigate the prophylactic effect of ketamine against neurochemical and behavioral changes that follow inescapable, uncontrollable tail shocks (ISs) in Sprague Dawley rats. IS induces increased anxiety, which is dependent on activation of serotonergic (5-HT) dorsal raphe nucleus (DRN) neurons that project to the basolateral amygdala (BLA). Ketamine (10 mg/kg, i.p.) administered 2 h, 1 week, or 2 weeks before IS prevented the increased extracellular levels of 5-HT in the BLA typically produced by IS. In addition, ketamine administered at these time points blocked the decreased juvenile social investigation produced by IS. Microinjection of ketamine into the prelimbic (PL) region of the medial prefrontal cortex duplicated the effects of systemic ketamine, and, conversely, systemic ketamine effects were prevented by pharmacological inhibition of the PL. Although IS does not activate DRN-projecting neurons from the PL, IS did so after ketamine, suggesting that the prophylactic effect of ketamine is a result of altered functioning of this projection.

SIGNIFICANCE STATEMENT

The reported data show that systemic ketamine, given up to 2 weeks before a stressor, blunts behavioral and neurochemical effects of the stressor. The study also advances understanding of the mechanisms involved and suggests that ketamine acts at the prelimbic cortex to sensitize neurons that project to and inhibit the DRN.

Cortico-Basal Ganglia Circuit Function in Psychiatric Disease

Circuit dysfunction models of psychiatric disease posit that pathological behavior results from abnormal patterns of electrical activity in specific cells and circuits in the brain. Many psychiatric disorders are associated with abnormal activity in the prefrontal cortex and in the basal ganglia, a set of subcortical nuclei implicated in cognitive and motor control. Here we discuss the role of the basal ganglia and connected prefrontal regions in the etiology and treatment of obsessive-compulsive disorder, anxiety, and depression, emphasizing mechanistic work in rodent behavioral models to dissect causal cortico-basal ganglia circuits underlying discrete behavioral symptom domains relevant to these complex disorders.

Behavioral control blunts reactions to contemporaneous and future adverse events: medial prefrontal cortex plasticity and a corticostriatal network

It has been known for many years that the ability to exert behavioral control over an adverse event blunts the behavioral and neurochemical impact of the event. More recently, it has become clear that the experience of behavioral control over adverse events also produces enduring changes that reduce the effects of subsequent negative events, even if they are uncontrollable and quite different from the original event controlled. This review focuses on the mechanism by which control both limits the impact of the stressor being experienced and produces enduring, trans-situational "immunization". The evidence will suggest that control is detected by a corticostriatal circuit involving the ventral medial prefrontal cortex (mPFC) and the posterior dorsomedial striatum (DMS). Once control is detected, other mPFC neurons that project to stress-responsive brainstem (dorsal raphe nucleus, DRN) and limbic (amygdala) structures exert top-down inhibitory control over the activation of these structures that is produced by the adverse event. These structures, such as the DRN and amygdala, in turn regulate the proximate mediators of the behavioral and physiological responses produced by adverse events, and so control blunts these responses. Importantly, the joint occurrence of control and adverse events seems to produce enduring plastic changes in the top-down inhibitory mPFC system such that this system is now activated by later adverse events even if they are uncontrollable, thereby reducing the impact of these events. Other issues are discussed that include a) whether other processes such as safety signals and exercise, that lead to resistance/resilience, also use the mPFC circuitry or do so in other ways; b) whether control has similar effects and neural mediation in humans, and c) the relationship of this work to clinical phenomena.

Learned stressor resistance requires extracellular signal-regulated kinase in the prefrontal cortex

Behaviorally controllable stressors confer protection from the neurochemical and behavioral consequences of future uncontrollable stressors, a phenomenon termed “behavioral immunization”. Recent data implicate protein synthesis within the ventromedial prefrontal cortex (mPFC) as critical to behavioral immunization. Adult, male Sprague-Dawley rats were exposed to a series of controllable tailshocks and 1 week later to uncontrollable tailshocks, followed 24 h later by social exploration and shuttlebox escape tests. To test the involvement of N-methyl-D-aspartate receptors (NMDARs) and the extracellular signal-regulated kinase (ERK) cascade in behavioral immunization, either D-AP5 or the MEK inhibitor U0126 was injected to the prelimbic (PL) or infralimbic (IL) mPFC prior to controllable stress exposure. Phosphorylated ERK and P70S6K, regulators of transcription and translation, were quantified by Western blot or immunohistochemistry after controllable or uncontrollable tailshocks. Prior controllable stress prevented the social exploration and shuttlebox performance deficits caused by the later uncontrollable stressor, and this effect was blocked by injections of D-AP5 into mPFC. A significant increase in phosphorylated ERK1 and ERK2, but not P70S6K, occurred within the PL and IL in rats exposed to controllable stress, but not to uncontrollable stress. However, U0126 only prevented behavioral immunization when injected to the PL. We provide evidence that NMDAR and ERK dependent signaling within the PL region is required for behavioral immunization, a learned form of stressor resistance.