Translational studies of goal-directed action as a framework for classifying deficits across psychiatric disorders

Common neural dysfunction of economic decision-making across psychiatric conditions

Adaptive decision-making, which is often impaired in various psychiatric conditions, is essential for well-being. Recent evidence has indicated that decision-making capacity in multiple tasks could be accounted for by latent dimensions, enlightening the question of whether there is a common disruption of brain networks in economic decision-making across psychiatric conditions. Here, we addressed the issue by combining activation/lesion network mapping analyses with a transdiagnostic brain imaging meta-analysis. Our findings indicate that there were transdiagnostic alterations in the thalamus and ventral striatum during the decision or outcome stage of decision-making. The identified regions represent key nodes in a large-scale network, which is composed of multiple heterogeneous brain regions and plays a causal role in motivational functioning. The findings suggest that disturbances in the network associated with emotion- and reward-related processing play a key role in dysfunctions of decision-making observed in various psychiatric conditions. This study provides the first meta-analytic evidence of common neural alterations linked to deficits in economic decision-making.

Arbitration between model-free and model-based control is not affected by transient changes in tonic serotonin levels

BACKGROUND

Serotonin has been suggested to modulate decision-making by influencing the arbitration between model-based and model-free control. Disruptions in these control mechanisms are involved in mental disorders such as drug dependence or obsessive-compulsive disorder. While previous reports indicate that lower brain serotonin levels reduce model-based control, it remains unknown whether increases in serotonergic availability might thus increase model-based control. Moreover, the mediating neural mechanisms have not been studied yet.

AIM

The first aim of this study was to investigate whether increased/decreased tonic serotonin levels affect the arbitration between model-free and model-based control. Second, we aimed to identify the underlying neural processes.

METHODS

We employed a sequential two-stage Markov decision-task and measured brain responses during functional magnetic resonance imaging in 98 participants in a randomized, double-blind cross-over within-subject design. To investigate the influence of serotonin on the balance between model-free and model-based control, we used a tryptophan intervention with three intervention levels (loading, balanced, depletion). We hypothesized that model-based behaviour would increase with higher serotonin levels.

RESULTS

We found evidence that neither model-free nor model-based control were affected by changes in tonic serotonin levels. Furthermore, our tryptophan intervention did not elicit relevant changes in Blood-Oxygenation-Level Dependent activity.

A dual-pathway architecture enables chronic stress to promote habit formation

Chronic stress can change how we learn and, thus, how we make decisions by promoting the formation of inflexible, potentially maladaptive, habits. Here we investigated the neuronal circuit mechanisms that enable this. Using a multifaceted approach in male and female mice, we reveal a dual pathway, amygdala-striatal, neuronal circuit architecture by which a recent history of chronic stress shapes learning to disrupt flexible goal-directed behavior in favor of inflexible habits. Chronic stress inhibits activity of basolateral amygdala projections to the dorsomedial striatum to impede the action-outcome learning that supports flexible, goal-directed decisions. Stress also increases activity in direct central amygdala projections to the dorsomedial striatum to promote the formation of rigid, inflexible habits. Thus, stress exerts opposing effects on two amygdala-striatal pathways to promote premature habit formation. These data provide neuronal circuit insights into how chronic stress shapes learning and decision making, and help understand how stress can lead to the disrupted decision making and pathological habits that characterize substance use disorders and other psychiatric conditions.

Antagonism of D2 receptors via raclopride ameliorates amphetamine-induced associative learning deficits in male mice

Dopamine levels in the dorsomedial striatum (DMS) are highly dynamic and are thought to underly the encoding of action-outcome associations. Although it is known that amphetamine disrupts the learning that is required for goal-directed action, the role of D1 and D2 receptors in this process has not been established. In this study, we examined the role of D1 and D2 receptor antagonists on learning in response to amphetamine. We used the outcome-specific devaluation task to examine goal-directed action in male C57BL6/J mice treated systemically with either a D1 antagonist (SCH-23990; 0.01 mg/kg) or a D2 antagonist (raclopride; 0.5 mg/kg) and then administered amphetamine (1 mg/kg). The mice were injected repeatedly throughout the instrumental training phase of the task to assess the impact on the learning of action-outcomes, and the subsequent choice test assessing performance of goal-directed action was conducted drug free. Effects of chronic drug administration on locomotor behaviour was assessed before and after the choice test. Treatment during learning with either amphetamine, or the D1 or D2 antagonists, impaired the subsequent performance of goal-directed action. The amphetamine-induced impairment in goal-directed action was reversed in mice treated with raclopride, but not when treated with SCH-23990. By contrast, amphetamine-induced hyperactivity was reversed in mice treated with SCH-23990, but not in mice treated with raclopride. Taken together, these data support the role of a balance of dopamine receptor signalling after amphetamine treatment. While overall D1 receptor availability is necessary to promote learning, in a state of elevated dopamine, modifying D2 receptor function can ameliorate learning deficits.

The Motivational Determinants of Human Action, Their Neural Bases and Functional Impact in Adolescents With Obsessive-Compulsive Disorder

Background

Establishing the motivational influences on human action is essential for understanding choice and decision making in health and disease. Here we used tests of value-based decision making, manipulating both predicted and experienced reward values to assess the motivational control of goal-directed action in healthy adolescents and those with obsessive-compulsive disorder (OCD).

Methods

After instrumental training on a two action-two outcome probabilistic task, adolescents (n = 21) underwent Pavlovian conditioning using distinct stimuli predicting either the instrumental outcomes, a third outcome, or nothing. We then assessed functional magnetic resonance imaging during choice tests in which we varied the predicted value, using specific and general Pavlovian-instrumental transfer, and the experienced value, using outcome devaluation. To establish functional significance, we tested a matched cohort of adolescents with OCD (n = 20).

Results

In healthy adolescents, both predicted and experienced values influenced the performance of goal-directed actions, mediated by distinct orbitofrontal-striatal circuits involving the lateral orbitofrontal cortex (OFC) and medial OFC, respectively. However, in adolescents with OCD, choice was insensitive to changes in either predicted or experienced values. These impairments were related to hypoactivity in the lateral OFC and hyperactivity in the medial OFC during specific Pavlovian-instrumental transfer and hypoactivity in the anterior prefrontal cortex, caudate nucleus, and their connectivity in the devaluation test.

Conclusions

We found that predicted and experienced values exerted a potent influence on the performance of goal-directed actions in adolescents via distinct orbitofrontal- and prefrontal-striatal circuits. Furthermore, the influence of these motivational processes was severely blunted in OCD, as was the functional segregation of circuits involving medial and lateral OFC, producing dysregulated action control.

Exploring expected reward and efficacy in enhancing cognitive control in patients with depression

BACKGROUND

Depression is associated with impairments in cognitive control. Considering the lack of mechanistic models accounting for cognitive control deficits in depression, the expected value of control (EVC) theory offers a mechanistic view for allocating cognitive control emphasizing motivational components (efficacy, value). Efficacy refers to the possibility that an effort leads to a special outcome and reward refers to the value (amount) associated with the outcome. This study aimed to examine the role of the EVC in depression.

METHOD

This study used a within-between-subject design. Participants with depression (n = 36) and healthy controls (n = 31) completed a clinical diagnostic interview, the Beck Depression Inventory-II, the General Health Questionnaire-12, and a computer-based incentivized Stroop Color-Word Paradigm in which levels of efficacy (high vs. low) and the amount of rewards (high vs. low) were presented as cues before target stimuli.

RESULTS

We found significant interaction effects of group × efficacy and efficacy × reward in terms of reaction time in the Stroop Paradigm. Follow-up analyses indicated the Depressed group were significantly slower than Controls on high efficacy trials, but the two groups did not differ significantly on low efficacy trials. Additionally, on high efficacy trials, reward did not influence performance, but on low efficacy trials, high reward improved performance in both groups.

LIMITATION

Lack of neurological measures and eye tracking techniques.

CONCLUSION

Overall, our findings suggest that reward and efficacy may jointly improve cognitive control allocation and highlight the need for further research examining EVC theory as a mechanistic account of cognitive control deficits in depression.

Causal role of the dorsolateral prefrontal cortex in modulating the balance between Pavlovian and instrumental systems in the punishment domain

Previous literature suggests that a balance between Pavlovian and instrumental decision-making systems is critical for optimal decision-making. Pavlovian bias (i.e., approach toward reward-predictive stimuli and avoid punishment-predictive stimuli) often contrasts with the instrumental response. Although recent neuroimaging studies have identified brain regions that may be related to Pavlovian bias, including the dorsolateral prefrontal cortex (dlPFC), it is unclear whether a causal relationship exists. Therefore, we investigated whether upregulation of the dlPFC using transcranial current direct stimulation (tDCS) would reduce Pavlovian bias. In this double-blind study, participants were assigned to the anodal or the sham group; they received stimulation over the right dlPFC for 3 successive days. On the last day, participants performed a reinforcement learning task known as the orthogonalized go/no-go task; this was used to assess each participant's degree of Pavlovian bias in reward and punishment domains. We used computational modeling and hierarchical Bayesian analysis to estimate model parameters reflecting latent cognitive processes, including Pavlovian bias, go bias, and choice randomness. Several computational models were compared; the model with separate Pavlovian bias parameters for reward and punishment domains demonstrated the best model fit. When using a behavioral index of Pavlovian bias, the anodal group showed significantly lower Pavlovian bias in the punishment domain, but not in the reward domain, compared with the sham group. In addition, computational modeling showed that Pavlovian bias parameter in the punishment domain was lower in the anodal group than in the sham group, which is consistent with the behavioral findings. The anodal group also showed a lower go bias and choice randomness, compared with the sham group. These findings suggest that anodal tDCS may lead to behavioral suppression or change in Pavlovian bias in the punishment domain, which will help to improve comprehension of the causal neural mechanism.

Modulation of ventromedial orbitofrontal cortical glutamatergic activity affects the explore-exploit balance and influences value-based decision-making

The balance between exploration and exploitation is essential for decision-making. The present study investigated the role of ventromedial orbitofrontal cortex (vmOFC) glutamate neurons in mediating value-based decision-making by first using optogenetics to manipulate vmOFC glutamate activity in rats during a probabilistic reversal learning (PRL) task. Rats that received vmOFC activation during informative feedback completed fewer reversals and exhibited reduced reward sensitivity relative to rats. Analysis with a Q-learning computational model revealed that increased vmOFC activity did not affect the learning rate but instead promoted maladaptive exploration. By contrast, vmOFC inhibition increased the number of completed reversals and increased exploitative behavior. In a separate group of animals, calcium activity of vmOFC glutamate neurons was recorded using fiber photometry. Complementing our results above, we found that suppression of vmOFC activity during the latter part of rewarded trials was associated with improved PRL performance, greater win-stay responding and selecting the correct choice on the next trial. These data demonstrate that excessive vmOFC activity during reward feedback disrupted value-based decision-making by increasing the maladaptive exploration of lower-valued options. Our findings support the premise that pharmacological interventions that normalize aberrant vmOFC glutamate activity during reward feedback processing may attenuate deficits in value-based decision-making.

UBE3A expression during early postnatal brain development is required for proper dorsomedial striatal maturation

Angelman syndrome (AS) is a severe neurodevelopmental disorder (NDD) caused by loss of functional ubiquitin protein ligase E3A (UBE3A). Previous studies showed that UBE3A plays an important role in the first postnatal weeks of mouse brain development, but its precise role is unknown. Since impaired striatal maturation has been implicated in several mouse models for NDDs, we studied the importance of UBE3A in striatal maturation. We used inducible Ube3a mouse models to investigate the maturation of medium spiny neurons (MSNs) from dorsomedial striatum. MSNs of mutant mice matured properly till postnatal day 15 (P15) but remained hyperexcitable with fewer excitatory synaptic events at later ages, indicative of stalled striatal maturation in Ube3a mice. Reinstatement of UBE3A expression at P21 fully restored MSN excitability but only partially restored synaptic transmission and the operant conditioning behavioral phenotype. Gene reinstatement at P70 failed to rescue both electrophysiological and behavioral phenotypes. In contrast, deletion of Ube3a after normal brain development did not result in these electrophysiological and behavioral phenotypes. This study emphasizes the role of UBE3A in striatal maturation and the importance of early postnatal reinstatement of UBE3A expression to obtain a full rescue of behavioral phenotypes associated with striatal function in AS.

Depression mediated the relationships between precentral-subcortical causal links and motor recovery in spinal cord injury patients

Depression after brain damage may impede the motivation and consequently influence the motor recovery after spinal cord injury (SCI); however, the neural mechanism underlying the psychological effects remains unclear. This study aimed to examine the casual connectivity changes of the emotion-motivation-motor circuit and the potential mediating effects of depression on motor recovery after SCI. Using the resting-state functional magnetic resonance imaging data of 35 SCI patients (24 good recoverers, GR and 11 poor recoverers, PR) and 32 healthy controls (HC), the results from the conditional Granger causality (GC) analysis demonstrated that the GR group exhibited sparser emotion-motivation-motor GC network compared with the HC and PR groups, though the in-/out-degrees of the emotion subnetwork and the motor subnetwork were relatively balanced in the HC and GR group. The PR group showed significantly inhibitory causal links from amygdala to supplementary motor area and from precentral gyrus to nucleus accumbens compared with GR group. Further mediation analysis revealed the indirect effect of the 2 causal connections on motor function recovery via depression severity. Our findings provide further evidence of abnormal causal connectivity in emotion-motivation-motor circuit in SCI patients and highlight the importance of emotion intervention for motor function recovery after SCI.

Two-stage reinforcement learning task predicts psychological traits

External sources of information influence human actions. However, psychological traits (PTs), considered internal variables, also play a crucial role in decision making. PTs are stable across time and contexts and define the set of behavioral repertoires that individuals express. Here, we explored how multiple metrics of adaptive behavior under uncertainty related to several PTs. Participants solved a reversal-learning task with volatile contingencies, from which we characterized a detailed behavioral profile based on their response sequences. We then tested the relationship between this multimetric behavioral profile and scores obtained from self-report psychological questionnaires. The PT measurements were based on the Hierarchical Taxonomy Of Psychopathology (HiTOP) model. By using multiple linear regression models (MLRMs), we found that the learning curves predicted important differences in the PTs and task response times. We confirmed the significance of these relationships by using random permutations of the predictors of the MLRM. Therefore, the behavioral profile configurations predicted the PTs and served as a "fingerprint" to identify participants with a high certainty level. We discuss briefly how this characterization and approach could contribute to better nosological classifications.

The positive valence system, adaptive behaviour and the origins of reward

Although the hey-day of motivation as an area of study is long past, the issues with which motivational theorists grappled have not grown less important: i.e. the development of deterministic explanations for the particular tuning of the nervous system to specific changes in the internal and external environment and the organisation of adaptive behavioural responses to those changes. Here, we briefly elaborate these issues in describing the structure and function of the 'positive valence system'. We describe the origins of adaptive behaviour in an ascending arousal system, sensitive to peripheral regulatory changes, that modulates and activates various central motivational states. Associations between these motivational states and sensory inputs underlie evaluative conditioning and generate the representation of the 'unconditioned' stimuli fundamental to Pavlovian conditioning. As a consequence, associations with these stimuli can generate Pavlovian conditioned responses through the motivational control of stimulus event associations with sensory and affective components of the valence system to elicit conditioned orienting, consummatory and preparatory responses, particularly the affective responses reflecting Pavlovian excitation and inhibition, arousal and reinforcement, the latter used to control the formation of habits. These affective processes also provoke emotional responses, allowing the externalisation of positive valence in hedonic experience to generate the goal or reward values that mediate goal-directed action. Together these processes form the positive valence system, ensure the maintenance of adaptive behaviour and, through the association of sensory events and emotional responses through consummatory experience, provide the origins of reward.

Anxiety-related activity of ventral hippocampal interneurons

Anxiety is an aversive mood reflecting the anticipation of potential threats. The ventral hippocampus (vH) is a key brain region involved in the genesis of anxiety responses. Recent studies have shown that anxiety is mediated by the activation of vH pyramidal neurons targeting various limbic structures. Throughout the cortex, the activity of pyramidal neurons is controlled by GABA-releasing inhibitory interneurons and the GABAergic system represents an important target of anxiolytic drugs. However, how the activity of vH inhibitory interneurons is related to different anxiety behaviours has not been investigated so far. Here, we integrated in vivo electrophysiology with behavioural phenotyping of distinct anxiety exploration behaviours in rats. We showed that pyramidal neurons and interneurons of the vH are selectively active when animals explore specific compartments of the elevated-plus-maze (EPM), an anxiety task for rodents. Moreover, rats with prior goal-related experience exhibited low-anxiety exploratory behaviour and showed a larger trajectory-related activity of vH interneurons during EPM exploration compared to high anxiety rats. Finally, in low anxiety rats, trajectory-related vH interneurons exhibited opposite activity to pyramidal neurons specifically in the open arms (i.e. more anxiogenic) of the EPM. Our results suggest that vH inhibitory micro-circuits could act as critical elements underlying different anxiety states.

Learned helplessness and its relevance for psychological suffering: a new perspective illustrated with attachment problems, burn-out, and fatigue complaints

We develop a new perspective on various forms of psychological suffering - including attachment issues, burn-out, and fatigue complaints - by drawing on the construct of learned helplessness. We conceptualise learned helplessness in operant terms as the behavioural effects of a lack of reinforcement and in goal-directed terms as the dysregulation of goal-directed behaviour. Our central claim is that if one fails to reach a goal (e.g. the goal to secure a job), then not only this goal but also other related goals (e.g. the goal to maintain social relationships) may lose their motivating effects. The similarity relation between goal stimuli can therefore shed light on how failure in one life domain can come to affect various other life domains. We detail the relation between our proposal and existing theories and discuss new research and clinical directions.

Chemogenetic Disconnection between the Orbitofrontal Cortex and the Rostromedial Caudate Nucleus Disrupts Motivational Control of Goal-Directed Action

The orbitofrontal cortex (OFC) and its major downstream target within the basal ganglia-the rostromedial caudate nucleus (rmCD)-are involved in reward-value processing and goal-directed behavior. However, a causal contribution of the pathway linking these two structures to goal-directed behavior has not been established. Using the chemogenetic technology of designer receptors exclusively activated by designer drugs with a crossed inactivation design, we functionally and reversibly disrupted interactions between the OFC and rmCD in two male macaque monkeys. We injected an adeno-associated virus vector expressing an inhibitory designer receptor, hM4Di, into the OFC and contralateral rmCD, the expression of which was visualized in vivo by positron emission tomography and confirmed by postmortem immunohistochemistry. Functional disconnection of the OFC and rmCD resulted in a significant and reproducible loss of sensitivity to the cued reward value for goal-directed action. This decreased sensitivity was most prominent when monkeys had accumulated a certain amount of reward. These results provide causal evidence that the interaction between the OFC and the rmCD is needed for motivational control of action on the basis of the relative reward value and internal drive. This finding extends the current understanding of the physiological basis of psychiatric disorders in which goal-directed behavior is affected, such as obsessive-compulsive disorder.SIGNIFICANCE STATEMENT In daily life, we routinely adjust the speed and accuracy of our actions on the basis of the value of expected reward. Abnormalities in these kinds of motivational adjustments might be related to behaviors seen in psychiatric disorders such as obsessive-compulsive disorder. In the current study, we show that the connection from the orbitofrontal cortex to the rostromedial caudate nucleus is essential for motivational control of action in monkeys. This finding expands our knowledge about how the primate brain controls motivation and behavior and provides a particular insight into disorders like obsessive-compulsive disorder in which altered connectivity between the orbitofrontal cortex and the striatum has been implicated.

The Neural Bases of Action-Outcome Learning in Humans

From an associative perspective the acquisition of new goal-directed actions requires the encoding of specific action-outcome (AO) associations and, therefore, sensitivity to the validity of an action as a predictor of a specific outcome relative to other events. Although competitive architectures have been proposed within associative learning theory to achieve this kind of identity-based selection, whether and how these architectures are implemented by the brain is still a matter of conjecture. To investigate this issue, we trained human participants to encode various AO associations while undergoing functional neuroimaging (fMRI). We then degraded one AO contingency by increasing the probability of the outcome in the absence of its associated action while keeping other AO contingencies intact. We found that this treatment selectively reduced performance of the degraded action. Furthermore, when a signal predicted the unpaired outcome, performance of the action was restored, suggesting that the degradation effect reflects competition between the action and the context for prediction of the specific outcome. We used a Kalman filter to model the contribution of different causal variables to AO learning and found that activity in the medial prefrontal cortex (mPFC) and the dorsal anterior cingulate cortex (dACC) tracked changes in the association of the action and context, respectively, with regard to the specific outcome. Furthermore, we found the mPFC participated in a network with the striatum and posterior parietal cortex to segregate the influence of the various competing predictors to establish specific AO associations.SIGNIFICANCE STATEMENT Humans and other animals learn the consequences of their actions, allowing them to control their environment in a goal-directed manner. Nevertheless, it is unknown how we parse environmental causes from the effects of our own actions to establish these specific action-outcome (AO) relationships. Here, we show that the brain learns the causal structure of the environment by segregating the unique influence of actions from other causes in the medial prefrontal and anterior cingulate cortices and, through a network of structures, including the caudate nucleus and posterior parietal cortex, establishes the distinct causal relationships from which specific AO associations are formed.

A novel giant non-cholinergic striatal interneuron restricted to the ventrolateral striatum coexpresses Kv3.3 potassium channel, parvalbumin, and the vesicular GABA transporter

The striatum is the main input structure of the basal ganglia. Distinct striatal subfields are involved in voluntary movement generation and cognitive and emotional tasks, but little is known about the morphological and molecular differences of striatal subregions. The ventrolateral subfield of the striatum (VLS) is the orofacial projection field of the sensorimotor cortex and is involved in the development of orofacial dyskinesias, involuntary chewing-like movements that often accompany long-term neuroleptic treatment. The biological basis for this particular vulnerability of the VLS is not known. Potassium channels are known to be strategically localized within the striatum. In search of possible molecular correlates of the specific vulnerability of the VLS, we analyzed the expression of voltage-gated potassium channels in rodent and primate brains using qPCR, in situ hybridization, and immunocytochemical single and double staining. Here we describe a novel, giant, non-cholinergic interneuron within the VLS. This neuron coexpresses the vesicular GABA transporter, the calcium-binding protein parvalbumin (PV), and the Kv3.3 potassium channel subunit. This novel neuron is much larger than PV neurons in other striatal regions, displays characteristic electrophysiological properties, and, most importantly, is restricted to the VLS. Consequently, the giant striatal Kv3.3-expressing PV neuron may link compromised Kv3 channel function and VLS-based orofacial dyskinesias.

Behavioral Engagement With Playable Objects Resolves Stress-Induced Adaptive Changes by Reshaping the Reward System

BACKGROUND

The reward system regulates motivated behavior, and repeated practice of specific motivated behavior might conversely modify the reward system. However, the detailed mechanisms by which they reciprocally regulate each other are not clearly understood.

METHODS

Mice subjected to chronic restraint stress show long-lasting depressive-like behavior, which is rescued by continual engagement with playable objects. A series of molecular, pharmacological, genetic, and behavioral analyses, combined with microarray, liquid chromatography, and chemogenetic tools, are used to investigate the neural mechanisms of antidepressive effects of playable objects.

RESULTS

Here, we show that repeated restraint induces dopamine surges into the nucleus accumbens-lateral shell (NAc-lSh), which cause upregulation of the neuropeptide PACAP in the NAc-lSh. As repeated stress is continued, the dopamine surge by stressors is adaptively suppressed without restoring PACAP upregulation, and the resulting enhanced PACAP inputs from NAc-lSh neurons to the ventral pallidum facilitate depressive-like behaviors. Continual engagement with playable objects in mice subjected to chronic stress remediates reduced dopamine response to new stressors, enhanced PACAP upregulation, and depressive-like behaviors. Overactivation of dopamine D₁ receptors over the action of D₂ receptors in the NAc-lSh promotes depressive-like behaviors. Conversely, inhibition of D₁ receptors or PACAP upregulation in the NAc-lSh confers resilience to chronic stress-induced depressive-like behaviors. Histochemical and chemogenetic analyses reveal that engagement with playable objects produces antidepressive effects by reshaping the ventral tegmental area-to-NAc-lSh and NAc-lSh-to-ventral pallidum circuits.

CONCLUSIONS

These results suggest that behavioral engagement with playable objects remediates depressive-like behaviors by resolving stress-induced maladaptive changes in the reward system.

Mindful Awareness Promotes Resilience: Buffered Links Among Childhood Sexual Abuse Severity, Goal-Directed Emotion Dysregulation, and Psychopathology

Objectives

Childhood sexual abuse is linked to long-term consequences, including depression and anxiety in adulthood. Although considerable progress has been made to understand mechanisms that may account for this relation, such as emotion dysregulation, less attention has been given to protective factors that may mitigate it. One such protective factor might be mindful awareness. Those who act with awareness in daily living tend to engage in healthy emotion regulation skills when faced with stressors and experience less depression and anxiety. In the current study, we aimed to replicate the positive associations among childhood sexual abuse severity, emotion dysregulation, and psychopathology across time, and also identify a personal strength-in this case, mindful awareness-that might mitigate these effects.

Methods

Participants were 491 women recruited from the community who completed self-report assessments at three time points over a 32-month period.

Results

A series of moderated mediation models revealed childhood sexual abuse severity predicted later reports of depression and anxiety symptoms through greater emotion dysregulation in the form of difficulties engaging in goal-directed behaviors. As expected, mindful awareness weakened the relation between goal-directed emotion dysregulation and symptoms of depression and anxiety, such that greater levels of mindful awareness fully buffered these effects.

Conclusions

Through a better understanding of natural resiliency processes among survivors, we can ultimately encourage continued examination of what might be effective additions to existing treatments for the mental health consequences of trauma and adversity.

Animal models of action control and cognitive dysfunction in Parkinson's disease

Parkinson's disease (PD) has historically been considered a motor disorder induced by a loss of dopaminergic neurons in the substantia nigra pars compacta. More recently, it has been recognized to have significant non-motor symptoms, most prominently cognitive symptoms associated with a dysexecutive syndrome. It is common in the literature to see motor and cognitive symptoms treated separately and, indeed, there has been a general call for specialized treatment of the latter, particularly in the more severe cases of PD with mild cognitive impairment and dementia. Animal studies have similarly been developed to model the motor or non-motor symptoms. Nevertheless, considerable research has established that segregating consideration of cognition from the precursors to motor movement, particularly movement associated with goal-directed action, is difficult if not impossible. Indeed, on some contemporary views cognition is embodied in action control, something that is particularly prevalent in theory and evidence relating to the integration of goal-directed and habitual control processes. The current paper addresses these issues within the literature detailing animal models of cognitive dysfunction in PD and their neural and neurochemical bases. Generally, studies using animal models of PD provide some of the clearest evidence for the integration of these action control processes at multiple levels of analysis and imply that consideration of this integrative process may have significant benefits for developing new approaches to the treatment of PD.

A multi-pronged investigation of option generation using depression, PET and modafinil

Option generation is a critical process in decision making, but previous studies have largely focused on choices between options given by a researcher. Consequently, how we self-generate options for behaviour remain poorly understood. Here, we investigated option generation in major depressive disorder and how dopamine might modulate this process, as well as the effects of modafinil (a putative cognitive enhancer) on option generation in healthy individuals. We first compared differences in self-generated options between healthy non-depressed adults [n = 44, age = 26.3 years (SD 5.9)] and patients with major depressive disorder [n = 54, age = 24.8 years (SD 7.4)]. In the second study, a subset of depressed individuals [n = 22, age = 25.6 years (SD 7.8)] underwent PET scans with 11C-raclopride to examine the relationships between dopamine D2/D3 receptor availability and individual differences in option generation. Finally, a randomized, double-blind, placebo-controlled, three-way crossover study of modafinil (100 mg and 200 mg), was conducted in an independent sample of healthy people [n = 19, age = 23.2 years (SD 4.8)] to compare option generation under different doses of this drug. The first study revealed that patients with major depressive disorder produced significantly fewer options [t(96) = 2.68, P = 0.009, Cohen's d = 0.54], albeit with greater uniqueness [t(96) = -2.54, P = 0.01, Cohen's d = 0.52], on the option generation task compared to healthy controls. In the second study, we found that 11C-raclopride binding potential in the putamen was negatively correlated with fluency (r = -0.69, P = 0.001) but positively associated with uniqueness (r = 0.59, P = 0.007). Hence, depressed individuals with higher densities of unoccupied putamen D2/D3 receptors in the putamen generated fewer but more unique options, whereas patients with lower D2/D3 receptor availability were likely to produce a larger number of similar options. Finally, healthy participants were less unique [F(2,36) = 3.32, P = 0.048, partial η2 = 0.16] and diverse [F(2,36) = 4.31, P = 0.021, partial η2 = 0.19] after taking 200 mg versus 100 mg and 0 mg of modafinil, while fluency increased linearly with dosage at a trend level [F(1,18) = 4.11, P = 0.058, partial η2 = 0.19]. Our results show, for the first time, that option generation is affected in clinical depression and that dopaminergic activity in the putamen of patients with major depressive disorder may play a key role in the self-generation of options. Modafinil was also found to influence option generation in healthy people by reducing the creativity of options produced.

The prediction-error hypothesis of schizophrenia: new data point to circuit-specific changes in dopamine activity

Schizophrenia is a severe psychiatric disorder affecting 21 million people worldwide. People with schizophrenia suffer from symptoms including psychosis and delusions, apathy, anhedonia, and cognitive deficits. Strikingly, schizophrenia is characterised by a learning paradox involving difficulties learning from rewarding events, whilst simultaneously 'overlearning' about irrelevant or neutral information. While dysfunction in dopaminergic signalling has long been linked to the pathophysiology of schizophrenia, a cohesive framework that accounts for this learning paradox remains elusive. Recently, there has been an explosion of new research investigating how dopamine contributes to reinforcement learning, which illustrates that midbrain dopamine contributes in complex ways to reinforcement learning, not previously envisioned. This new data brings new possibilities for how dopamine signalling contributes to the symptomatology of schizophrenia. Building on recent work, we present a new neural framework for how we might envision specific dopamine circuits contributing to this learning paradox in schizophrenia in the context of models of reinforcement learning. Further, we discuss avenues of preclinical research with the use of cutting-edge neuroscience techniques where aspects of this model may be tested. Ultimately, it is hoped that this review will spur to action more research utilising specific reinforcement learning paradigms in preclinical models of schizophrenia, to reconcile seemingly disparate symptomatology and develop more efficient therapeutics.

Prefrontal cortex and depression

The prefrontal cortex (PFC) has emerged as one of the regions most consistently impaired in major depressive disorder (MDD). Although functional and structural PFC abnormalities have been reported in both individuals with current MDD as well as those at increased vulnerability to MDD, this information has not translated into better treatment and prevention strategies. Here, we argue that dissecting depressive phenotypes into biologically more tractable dimensions - negative processing biases, anhedonia, despair-like behavior (learned helplessness) - affords unique opportunities for integrating clinical findings with mechanistic evidence emerging from preclinical models relevant to depression, and thereby promises to improve our understanding of MDD. To this end, we review and integrate clinical and preclinical literature pertinent to these core phenotypes, while emphasizing a systems-level approach, treatment effects, and whether specific PFC abnormalities are causes or consequences of MDD. In addition, we discuss several key issues linked to cross-species translation, including functional brain homology across species, the importance of dissecting neural pathways underlying specific functional domains that can be fruitfully probed across species, and the experimental approaches that best ensure translatability. Future directions and clinical implications of this burgeoning literature are discussed.

Neural Circuitry of Salience and Reward Processing in Psychosis

The processing of salient and rewarding stimuli is integral to engaging our attention, stimulating anticipation for future events, and driving goal-directed behaviors. Widespread impairments in these processes are observed in psychosis, which may be associated with worse functional outcomes or mechanistically linked to the development of symptoms. Here, we summarize the current knowledge of behavioral and functional neuroimaging in salience, prediction error, and reward. Although each is a specific process, they are situated in multiple feedback and feedforward systems integral to decision making and cognition more generally. We argue that the origin of salience and reward processing dysfunctions may be centered in the subcortex during the earliest stages of psychosis, with cortical abnormalities being initially more spared but becoming more prominent in established psychotic illness/schizophrenia. The neural circuits underpinning salience and reward processing may provide targets for delaying or preventing progressive behavioral and neurobiological decline.

Chronic Distress in Male Mice Impairs Motivation Compromising Both Effort and Reward Processing With Altered Anterior Insular Cortex and Basolateral Amygdala Neural Activation

In humans and mammals, effort-based decision-making for monetary or food rewards paradigms contributes to the study of adaptive goal-directed behaviours acquired through reinforcement learning. Chronic distress modelled by repeated exposure to glucocorticoids in rodents induces suboptimal decision-making under uncertainty by impinging on instrumental acquisition and prompting negative valence behaviours. In order to further disentangle the motivational tenets of adaptive decision-making, this study addressed the consequences of enduring distress on relevant effort and reward-processing dimensions. Experimentally, appetitive and consummatory components of motivation were evaluated in adult C57BL/6JRj male mice experiencing chronic distress induced by oral corticosterone (CORT), using multiple complementary discrete behavioural tests. Behavioural data (from novelty suppressed feeding, operant effort-based choice, free feeding, and sucrose preference tasks) collectively show that behavioural initiation, effort allocation, and hedonic appreciation and valuation are altered in mice exposed to several weeks of oral CORT treatment. Additionally, data analysis from FosB immunohistochemical processing of postmortem brain samples highlights CORT-dependent dampening of neural activation in the anterior insular cortex (aIC) and basolateral amygdala (BLA), key telencephalic brain regions involved in appetitive and consummatory motivational processing. Combined, these results suggest that chronic distress-induced irregular aIC and BLA neural activations with reduced effort production and attenuated reward value processing during reinforcement-based instrumental learning could result in maladaptive decision-making under uncertainty. The current study further illustrates how effort and reward processing contribute to adjust the motivational threshold triggering goal-directed behaviours in versatile environments.

Pharmacologically induced N-methyl-D-aspartate receptor hypofunction impairs goal-directed food seeking in rats

AIM

Acute N-methyl-D-aspartate (NMDA) receptor antagonism is an important pharmacological animal model of schizophrenia. In previous studies, schizophrenia patients show impaired goal-directed behavior in an outcome-specific devaluation procedure. In this study, we investigated whether the rat model of the NMDA receptor blockade also showed altered goal-directed behavior in a satiety-induced outcome devaluation paradigm.

METHODS

In experiments 1 and 2, we aimed to establish the satiety-induced outcome devaluation test using sucrose and lipid rewards in operant conditioning and free consumption paradigms. In experiment 3, we tested the effect of MK-801 (0.1 mg/kg, i.p.) on outcome-specific devaluation.

RESULTS

Experiments 1 and 2 demonstrated that 1-h ad libitum food consumption is sufficient to induce outcome-specific devaluation in both lever-press and free consumption tests in rats. Experiment 3 showed that the administration of MK-801 impaired satiety-induced devaluation in the lever-press test but not in the subsequent free consumption test.

CONCLUSIONS

Our results suggest that acute pharmacological NMDA receptor antagonism in rats is a useful animal model for impaired goal-directed behavior in schizophrenia.

Neural correlates of extrinsic and intrinsic outcome processing during learning in individuals with TBI: a pilot investigation

Outcome processing, the ability to learn from feedback, is an important component of adaptive behavior and rehabilitation. Evidence from healthy adults implicates the striatum and dopamine in outcome processing. Animal research shows that damage to dopaminergic pathways in the brain can lead to a disruption of dopamine tone and transmission. Such evidence thus suggests that persons with TBI experience deficits in outcome processing. However, no research has directly investigated outcome processing and associated neural mechanisms in TBI. Here, we examine outcome processing in individuals with TBI during learning. Given that TBI negatively impacts striatal and dopaminergic systems, we hypothesize that individuals with TBI exhibit deficits in learning from outcomes. To test this hypothesis, individuals with moderate-to-severe TBI and healthy adults were presented with a declarative paired-associate word learning task. Outcomes indicating performance accuracy were presented immediately during task performance and in the form of either monetary or performance-based feedback. Two types of feedback provided the opportunity to test whether extrinsic and intrinsic motivational aspects of outcome presentation play a role during learning and outcome processing. Our results show that individuals with TBI exhibited impaired learning from feedback compared to healthy participants. Additionally, individuals with TBI exhibited increased activation in the striatum during outcome processing. The results of this study suggest that outcome processing and learning from immediate outcomes is impaired in individuals with TBI and might be related to inefficient use of neural resources during task performance as reflected by increased activation of the striatum.

White matter microstructural differences in underweight adolescents with anorexia nervosa and a preliminary longitudinal investigation of change following short-term weight restoration

PURPOSE

Anorexia nervosa (AN) affects approximately 2.9% of females and has the highest mortality rate among all psychiatric disorders. Despite several advances, the neurobiology of this disorder is still not well understood. Several studies have reported abnormalities in the white matter, but it is not know if these are disease-related or secondary to undernutrition. This study aimed to further our understanding of white matter pathology using diffusion-weighted imaging in underweight adolescents with AN, and to examine changes occurring after short-term weight restoration.

METHODS

Analyses were conducted on diffusion-weighted imaging from 24 female adolescents with AN and 17 age- and gender-matched healthy controls (HC), aged 14-19 years. Groups were compared on fractional anisotropy (FA), mean diffusivity (MD), axial diffusivity (AD) and radial diffusivity (RD) using tract-based spatial statistics analysis and DTI measures were correlated with eating disorder examination questionnaire (EDE-Q) subscales and body mass index (BMI). Preliminary repeated-measure analyses were also conducted on eight participants after short-term weight restoration (median 41 days).

RESULTS

Widespread increases in MD of up to 9% were found in underweight AN relative to HC, particularly in the corpus callosum. This was associated with both increased AD and RD, suggestive of dys- or de-myelination. There were no significant group differences in FA, and no significant correlations between DTI measures, BMI or EDE-Q subscale score. Weight restoration therapy significantly reduced MD, to levels significantly lower than HC, but did not consistently alter FA across individuals.

CONCLUSIONS

White matter microstructure is significantly altered in female adolescents with AN, with preliminary longitudinal data suggesting that it may be reversible with short-term weight restoration.

LEVEL OF EVIDENCE

Level III: evidence obtained from well-designed cohort or case-control analytic studies.

Depression in the Direct Pathway of the Dorsomedial Striatum Permits the Formation of Habitual Action

In order to achieve optimal outcomes in an ever-changing environment, humans and animals generally manage their action control via either goal-directed action or habitual action. These two action strategies are thought to be encoded in distinct parallel circuits in the dorsal striatum, specifically, the posterior dorsomedial striatum (DMS) and the dorsolateral striatum (DLS), respectively. The striatum is primarily composed of two subtypes of medium spiny neurons (MSNs): the direct-pathway striatonigral and the indirect-pathway striatopallidal MSNs. MSN-subtype-specific synaptic plasticity in the DMS and the DLS has been revealed to underlie goal-directed action and habitual action, respectively. However, whether any MSN-subtype-specific synaptic plasticity in the DMS is associated with habitual action, and if so, whether the synaptic plasticity affects the formation of habitual action, are not known. This study demonstrates that postsynaptic depression in the excitatory synapses of the direct-pathway striatonigral MSNs in the DMS is formed after habit learning. Moreover, chemogenetically rescuing this depression compromises the acquisition, but not the expression, of habitual action. These findings reveal that an MSN-subtype-specific synaptic plasticity in the DMS affects habitual action and suggest that plasticity in the DMS as well as in the DLS contributes to the formation of habitual action.

How predictive learning influences choice: Evidence for a GPCR-based memory process necessary for Pavlovian-instrumental transfer

Predictive learning endows stimuli with the capacity to signal both the sensory-specific and general motivational properties of their associated rewards or outcomes. These two signals can be distinguished behaviorally by their influence on the selection and performance of instrumental actions, respectively. This review focuses on how sensory-specific predictive learning guides choice between actions that earn otherwise equally desirable outcomes. We describe evidence that outcome-specific predictive learning is encoded in the basolateral amygdala and drives the accumulation of delta-opioid receptors on the surface of cholinergic interneurons located in the nucleus accumbens shell. This accumulation constitutes a novel form of cellular memory, not for outcome-specific predictive learning per se but for the selection of, and choice between, future instrumental actions. We describe recent evidence regarding the cascade of events necessary for the formation and expression of this cellular memory and point to open questions for future research into this process. Beyond these mechanistic considerations, the discovery of this new form of memory is consistent with recent evidence suggesting that intracellular rather than synaptic changes can mediate learning-related plasticity to modify brain circuitry to prepare for future significant events.

Disruption of Nrxn1α within excitatory forebrain circuits drives value-based dysfunction

Goal-directed behaviors are essential for normal function and significantly impaired in neuropsychiatric disorders. Despite extensive associations between genetic mutations and these disorders, the molecular contributions to goal-directed dysfunction remain unclear. We examined mice with constitutive and brain region-specific mutations in Neurexin1α, a neuropsychiatric disease-associated synaptic molecule, in value-based choice paradigms. We found Neurexin1α knockouts exhibited reduced selection of beneficial outcomes and impaired avoidance of costlier options. Reinforcement modeling suggested that this was driven by deficits in updating and representation of value. Disruption of Neurexin1α within telencephalic excitatory projection neurons, but not thalamic neurons, recapitulated choice abnormalities of global Neurexin1α knockouts. Furthermore, this selective forebrain excitatory knockout of Neurexin1α perturbed value-modulated neural signals within striatum, a central node in feedback-based reinforcement learning. By relating deficits in value-based decision-making to region-specific Nrxn1α disruption and changes in value-modulated neural activity, we reveal potential neural substrates for the pathophysiology of neuropsychiatric disease-associated cognitive dysfunction.

Using internal memory representations in associative learning to study hallucination-like phenomenon

Studies of Pavlovian conditioning have enriched our understanding of how relations among events can adaptively guide behavior through the formation and use of internal mental representations. In this review, we illustrate how internal representations flexibly integrate new updated information in reinforcer revaluation to influence relationships to impact actions and outcomes. We highlight representation-mediated learning to show the similarities in properties and functions between internally generated and directly activated representations, and how normal perception of internal representations could contribute to hallucinations. Converging evidence emerges from recent behavioral and neural activation studies using animal models of schizophrenia as well as clinical studies in patients to support increased tendencies in these populations to evoke internal representations from prior associative experience that approximate hallucination-like percepts. The heightened propensity is dependent on dopaminergic activation which is known to be sensitive to hippocampal overexcitability, a condition that has been observed in patients with psychosis. This presents a network that overlaps with cognitive neural circuits and offers a fresh approach for the development of therapeutic interventions targeting psychosis.

Relating goal-directed behaviour to grazing in persons with obesity with and without eating disorder features

BACKGROUND

Both obesity and eating disorders (ED) have been associated with reductions in purposeful, flexible goal-directed behaviour, and with an overreliance on more rigid habitual behaviour. It is currently unknown whether grazing, an eating style which is common in both conditions, is related to goal-directed behaviour. The current study therefore aimed to relate grazing to goal-directed behaviour in a group of participants with obesity with and without ED features, compared to a healthy-weight control group.

METHODS

Participants (N = 87; 67.8% women, mean age 28.57 years), of whom 19 had obesity and significant eating disorder features, 25 had obesity but without marked eating disorder features, and 43 were age- and sex-matched healthy-weight controls, completed two instrumental learning tasks assessing action-outcome contingency sensitivity and devaluation sensitivity, as well as demographic and eating disorder-related questionnaires. Gamma and Ordinary Least Squares regressions were performed to examine the effect of group and grazing on goal-directed behaviour.

RESULTS

Lower action-outcome contingency sensitivity was found in the group with obesity and with eating disorder features than in the group with obesity but without eating disorder features or in healthy controls. No group differences in devaluation sensitivity were found. A small but significant relationship was found between grazing severity and contingency sensitivity in the group with obesity and eating disorder features, such that increasing grazing severity was associated with less diminished contingency sensitivity.

CONCLUSIONS

There is some indication that in persons with obesity and eating disorder features instrumental behaviour is less flexible and adaptive; furthermore, within this group grazing may represent a goal-directed behaviour, despite unhelpful long-term implications of grazing.

Cumulative Effects of Social Stress on Reward-Guided Actions and Prefrontal Cortical Activity

BACKGROUND

When exposed to chronic social stress, animals display behavioral changes that are relevant to depressive-like phenotypes. However, the cascading relationship between incremental stress exposure and neural dysfunctions over time remains incompletely understood.

METHODS

We characterized the longitudinal effect of social defeat on goal-directed actions and prefrontal cortical activity in mice using a novel head-fixed sucrose preference task and two-photon calcium imaging.

RESULTS

Behaviorally, stress-induced loss of reward sensitivity intensifies over days. Motivational anhedonia, the failure to translate positive reinforcements into future actions, requires multiple sessions of stress exposure to become fully established. For neural activity, individual layer 2/3 pyramidal neurons in the cingulate and medial secondary motor subregions of the medial prefrontal cortex have heterogeneous responses to stress. Changes in ensemble activity differ significantly between susceptible and resilient mice after the first defeat session and continue to diverge following successive stress episodes before reaching persistent abnormal levels.

CONCLUSIONS

Collectively, these results demonstrate that the cumulative impact of an ethologically relevant stress can be observed at the level of cellular activity of individual prefrontal neurons. The distinct neural responses associated with resilience versus susceptibility suggests the hypothesis that the negative impact of social stress is neutralized in resilient animals, in part through an adaptive reorganization of prefrontal cortical activity.

Linking Grazing to Inhibition and Goal-Directed Behaviour in Obesity With and Without Eating Disorder Features

Recent models of obesity and eating behaviour have implicated both automatic responding to food-related cues and executive functioning in driving dietary choice. This study aimed to relate grazing severity to high weight with and without significant eating disorder features via the effects of inhibition and degree of goal-directed behaviour, in persons with obesity with and without significant eating disorder features compared to healthy controls. Forty-four participants with obesity (43.1% endorsing marked eating disorder features), and 43 healthy-weight age- and sex-matched participants (N = 87; 67.8% female, age = 28.57 (8.70; 18.18–58.34) years, BMI = 29.18 (7.80; 18.65–51.95) kg/m2) completed demographic and eating disorder-related questionnaires, a neuropsychological task of inhibition and an instrumental decision-making task. Bootstrapped serial mediation analyses were performed to examine the effect of group on grazing via goal-directed behaviour and inhibition. While significant differences existed between the groups in terms of inhibition, goal-directed behaviour and grazing severity, the effect of group on grazing severity was not found to be mediated by the degree of behavioural goal-directedness and inhibition. Therefore, :in persons with obesity with or without eating disorder symptoms, putative relationships between a reduced inhibitory profile and/or behaviour that is less flexible and goal-directed and eating behaviours such as grazing, remain unclear.

Subcortical Dopamine and Cognition in Schizophrenia: Looking Beyond Psychosis in Preclinical Models

Schizophrenia is characterized by positive, negative and cognitive symptoms. All current antipsychotic treatments feature dopamine-receptor antagonism that is relatively effective at addressing the psychotic (positive) symptoms of schizophrenia. However, there is no clear evidence that these medications improve the negative or cognitive symptoms, which are the greatest predictors of functional outcomes. One of the most robust pathophysiological observations in patients with schizophrenia is increased subcortical dopamine neurotransmission, primarily in the associative striatum. This brain area has an important role in a range of cognitive processes. Dopamine is also known to play a major part in regulating a number of cognitive functions impaired in schizophrenia but much of this research has been focused on cortical dopamine. Emerging research highlights the strong influence subcortical dopamine has on a range of cognitive domains, including attention, reward learning, goal-directed action and decision-making. Nonetheless, the precise role of the associative striatum in the cognitive impairments observed in schizophrenia remains poorly understood, presenting an opportunity to revisit its contribution to schizophrenia. Without a better understanding of the mechanisms underlying cognitive dysfunction, treatment development remains at a standstill. For this reason, improved preclinical animal models are needed if we are to understand the complex relationship between subcortical dopamine and cognition. A range of new techniques are facillitating the discrete manipulation of dopaminergic neurotransmission and measurements of cognitive performance, which can be investigated using a variety of sensitive translatable tasks. This has the potential to aid the successful incorporation of recent clinical research to address the lack of treatment strategies for cognitive symptoms in schizophrenia. This review will give an overview on the current state of research focused on subcortical dopamine and cognition in the context of schizophrenia research. We also discuss future strategies and approaches aimed at improving the translational outcomes for the treatment of cognitive deficits in schizophrenia.

Neural substrates of habit

Active reward pursuit is supported by the balance between the cognitive and habitual control of behavior. The cognitive, goal-directed strategy relies on the prospective evaluation of anticipated consequences, which allows behavior to readily adapt when circumstances change. Repetition of successful actions promotes less cognitively taxing habits, in which behavior is automatically executed without prospective consideration. Disruption in either of these behavioral regulatory systems contributes to the symptoms that underlie many psychiatric disorders. Here, I review recently identified neural substrates, at multiple neural levels, that contribute to habits and outline gaps in knowledge that must be addressed to fully understand the neural mechanisms of behavioral control.

Involvement of the rodent prelimbic and medial orbitofrontal cortices in goal-directed action: A brief review

Goal-directed action refers to selecting behaviors based on the expectation that they will be reinforced with desirable outcomes. It is typically conceptualized as opposing habit-based behaviors, which are instead supported by stimulus-response associations and insensitive to consequences. The prelimbic prefrontal cortex (PL) is positioned along the medial wall of the rodent prefrontal cortex. It is indispensable for action-outcome-driven (goal-directed) behavior, consolidating action-outcome relationships and linking contextual information with instrumental behavior. In this brief review, we will discuss the growing list of molecular factors involved in PL function. Ventral to the PL is the medial orbitofrontal cortex (mOFC). We will also summarize emerging evidence from rodents (complementing existing literature describing humans) that it too is involved in action-outcome conditioning. We describe experiments using procedures that quantify responding based on reward value, the likelihood of reinforcement, or effort requirements, touching also on experiments assessing food consumption more generally. We synthesize these findings with the argument that the mOFC is essential to goal-directed action when outcome value information is not immediately observable and must be recalled and inferred.

A sensorimotor control framework for understanding emotional communication and regulation

Our research team was asked to consider the relationship of the neuroscience of sensorimotor control to the language of emotions and feelings. Actions are the principal means for the communication of emotions and feelings in both humans and other animals, and the allostatic mechanisms controlling action also apply to the regulation of emotional states by the self and others. We consider how motor control of hierarchically organised, feedback-based, goal-directed action has evolved in humans, within a context of consciousness, appraisal and cultural learning, to serve emotions and feelings. In our linguistic analysis, we found that many emotion and feelings words could be assigned to stages in the sensorimotor learning process, but the assignment was often arbitrary. The embodied nature of emotional communication means that action words are frequently used, but that the meanings or senses of the word depend on its contextual use, just as the relationship of an action to an emotion is also contextually dependent.

Reward presentation reduces on-task fatigue in traumatic brain injury

While cognitive fatigue is experienced by up to 80% of individuals with traumatic brain injury (TBI), little is known about its neural underpinnings. We previously hypothesized that presentation of rewarding outcomes leads to cognitive fatigue reduction and activation of the striatum, a brain region shown to be associated with cognitive fatigue in clinical populations and processing of rewarding outcomes. We have demonstrated this in individuals with multiple sclerosis. Here, we tested this hypothesis in individuals with TBI. Twenty-one individuals with TBI and 24 healthy participants underwent functional magnetic resonance imaging. Participants performed a task during which they were presented with 1) the Outcome condition where they were exposed to monetary rewards, and 2) the No Outcome condition that served as the control condition and was not associated with monetary rewards. In accordance with our hypothesis, results showed that attainment of rewarding outcomes leads to cognitive fatigue reduction in individuals with TBI, as well as activation of the striatum. Specifically, we observed a significant group by condition interaction on fatigue scores driven by the TBI group reporting lower levels of fatigue after the Outcome condition. fMRI data revealed a significant main-effect of condition in regions previously implicated in outcome processing, while a significant group by condition interaction was observed in the left ventral striatum as revealed by a priori region of interest analysis. Results suggest that a salient motivator can significantly reduce fatigue and that outcome presentation leads to increased activation of the ventral striatum in TBI. These findings can inform the development of future non-pharmacological cognitive fatigue treatment methods and contribute to the growing body of evidence showing the association between cognitive fatigue and the striatum.

Glucocorticoid-sensitive ventral hippocampal-orbitofrontal cortical connections support goal-directed action - Curt Richter Award Paper 2019

In an ever-changing and often ambiguous environment, organisms must use previously learned associations between antecedents and outcomes to predict future associations and make optimal choices. Chronic stress can impair one's ability to flexibly adjust behaviors when environmental contingencies change, particularly in cases of early-life stress. In mice, exposure to elevated levels of the primary stress hormone, corticosterone (CORT), during early adolescence is sufficient to impair response-outcome decision making later in life, biasing response strategies towards inflexible habits. Nevertheless, neurobiological mechanisms are still being defined. Here, we report that exposure to excess CORT in adolescence causes a loss of dendritic spines on excitatory pyramidal neurons in the lateral, but not medial, orbital prefrontal cortex (loPFC) of mice, and spine loss correlates with the severity of habit biases in adulthood. Excess CORT also reduces the presence of ventral hippocampal (vHC) axon terminals in the loPFC. To identify functional consequences, we inactivated vHC→loPFC projections in typical healthy mice during a period when mice must update response-outcome expectations to optimally acquire food reinforcers. Inactivation impaired the animals' subsequent ability to sustainably choose actions based on likely outcomes, causing them to defer to habit-based response strategies. Thus, vHC→loPFC projections are necessary for response-outcome expectancy updating and a target of excess glucocorticoids during early-life development. Their degradation is likely involved in long-term biases towards habit-based behaviors following glucocorticoid excess in adolescence.

Habit Expression and Disruption as a Function of Attention-Deficit/Hyperactivity Disorder Symptomology

Attention-deficit/hyperactivity disorder (ADHD) is associated with neurobehavioral reward system dysfunctions that pose debilitating impairments in adaptive decision-making. A candidate mechanism for such anomalies in ADHD may be a compromise in the control of motivated behaviors. Thus, demonstrating and restoring potential motivational control irregularities may serve significant clinical benefit. The motivational control of action guides goal-directed behaviors that are driven by outcome value, and habits that are inflexibly cue-triggered. We examined whether ADHD symptomology within the general population is linked to habitual control, and whether a motivation-based manipulation can break well-learned habits. We obtained symptom severity scores from 106 participants and administered a Go/NoGo task that capitalizes on familiar, well-learned associations (green-Go and red-NoGo) to demonstrate outcome-insensitivity when compared to newly learned Go/NoGo associations. We tested for outcome-insensitive habits by changing the Go and NoGo contingencies, such that Go signals became NoGo signals and vice versa. We found that generally, participants responded less accurately when green and red stimuli were mapped to color-response contingencies that were incongruent with daily experiences, whereas novel Go/NoGo stimuli evoked similar accuracy regardless of color-response mappings. Thus, our Go/NoGo task successfully elicited outcome-insensitive habits (i.e., persistent responses to familiar stimuli without regard for consequences); however, this effect was independent of ADHD symptomology. Nevertheless, we found an association between hyperactivity and congruent Go response latency, suggesting heightened pre-potency to perform habitual Go actions as hyperactivity increases. To examine habit disruption, participants returned to the lab and underwent the familiar version of the Go/NoGo task, but were given mid-experiment performance tracking information and a monetary incentive prior to contingency change. We found that this motivational boost via dual feedback prevented the incongruency-related accuracy impairment, effectively breaking the habit, albeit independent of ADHD symptomology. Our findings present only a modest link between ADHD symptomology and motivational control, which may be due to compensatory mechanisms in ADHD driving goal-directed control, or our task's potential insensitivity to individual differences in ADHD symptomology. Further investigations may be crucial for determining whether ADHD is related to motivational impairments.

Motivation and cognitive control in depression

Depression is linked to deficits in cognitive control and a host of other cognitive impairments arise as a consequence of these deficits. Despite of their important role in depression, there are no mechanistic models of cognitive control deficits in depression. In this paper we propose how these deficits can emerge from the interaction between motivational and cognitive processes. We review depression-related impairments in key components of motivation along with new cognitive neuroscience models that focus on the role of motivation in the decision-making about cognitive control allocation. Based on this review we propose a unifying framework which connects motivational and cognitive control deficits in depression. This framework is rooted in computational models of cognitive control and offers a mechanistic understanding of cognitive control deficits in depression.

Prediction and control of operant behavior: What you see is not all there is

Prediction and control of operant behavior are major goals of behavior analysis. We suggest that achieving these goals can benefit from doing more than identifying the three-term contingency between the behavior, its setting stimulus, and its consequences. Basic research now underscores the idea that prediction and control require consideration of the behavior's history. As one example, if an operant is a goal-directed action, it is controlled by the current value of the reinforcer, as illustrated by the so-called reinforcer devaluation effect. In contrast, if the behavior is a habit, it occurs automatically, without regard to the reinforcer's value, as illustrated by its insensitivity to the reinforcer devaluation effect. History variables that distinguish actions and habits include the extent of their prior practice and their schedule of reinforcement. Other operants can appear to have very low or zero strength. However, if the behavior has reached that level through extinction or punishment, it may precipitously increase in strength by changing the context, allowing time to pass, presenting the reinforcer contingently or noncontingently, or extinguishing an alternative behavior. Behaviors that are not suppressed by extinction or punishment are not affected the same way. When predicting the strength of an operant behavior, what you see is not all there is. The behavior's history counts.

A Reward-Based Framework of Perceived Control

Perceived control can be broadly defined as the belief in one's ability to exert control over situations or events. It has long been known that perceived control is a major contributor toward mental and physical health as well as a strong predictor of achievements in life. However, one issue that limits a mechanistic understanding of perceived control is the heterogeneity of how the term is defined in models in psychology and neuroscience, and used in experimental settings across a wide spectrum of studies. Here, we propose a framework for studying perceived control by integrating the ideas from traditionally separate work on perceived control. Specifically, we discuss key properties of perceived control from a reward-based framework, including choice opportunity, instrumental contingency, and success/reward rate. We argue that these separate reward-related processes are integral to fostering an enhanced perception of control and influencing an individual's behavior and well-being. We draw on select studies to elucidate how these reward-related elements are implicated separately and collectively in the investigation of perceived control. We highlight the role of dopamine within corticostriatal pathways shared by reward-related processes and perceived control. Finally, through the lens of this reward-based framework of perceived control, we consider the implications of perceived control in clinical deficits and how these insights could help us better understand psychopathology and treatment options.

Behavioral Paradigms to Probe Individual Mouse Differences in Value-Based Decision Making

Value-based decision making relies on distributed neural systems that weigh the benefits of actions against the cost required to obtain a given outcome. Perturbations of these systems are thought to underlie abnormalities in action selection seen across many neuropsychiatric disorders. Genetic tools in mice provide a promising opportunity to explore the cellular components of these systems and their molecular foundations. However, few tasks have been designed that robustly characterize how individual mice integrate differential reward benefits and cost in their selection of actions. Here we present a forced-choice, two-alternative task in which each option is associated with a specific reward outcome, and unique operant contingency. We employed global and individual trial measures to assess the choice patterns and behavioral flexibility of mice in response to differing "choice benefits" (modeled as varying reward magnitude ratios) and different modalities of "choice cost" (modeled as either increasing repetitive motor output to obtain reward or increased delay to reward delivery). We demonstrate that (1) mouse choice is highly sensitive to the relative benefit of outcomes; (2) choice costs are heavily discounted in environments with large discrepancies in relative reward; (3) divergent cost modalities are differentially integrated into action selection; (4) individual mouse sensitivity to reward benefit is correlated with sensitivity to reward costs. These paradigms reveal stable individual animal differences in value-based action selection, thereby providing a foundation for interrogating the neural circuit and molecular pathophysiology of goal-directed dysfunction.

Prefrontal cortical trkB, glucocorticoids, and their interactions in stress and developmental contexts

The tropomyosin/tyrosine receptor kinase B (trkB) and glucocorticoid receptor (GR) regulate neuron structure and function and the hormonal stress response. Meanwhile, disruption of trkB and GR activity (e.g., by chronic stress) can perturb neuronal morphology in cortico-limbic regions implicated in stressor-related illnesses like depression. Further, several of the short- and long-term neurobehavioral consequences of stress depend on the developmental timing and context of stressor exposure. We review how the levels and activities of trkB and GR in the prefrontal cortex (PFC) change during development, interact, are modulated by stress, and are implicated in depression. We review evidence that trkB- and GR-mediated signaling events impact the density and morphology of dendritic spines, the primary sites of excitatory synapses in the brain, highlighting effects in adolescents when possible. Finally, we review the role of neurotrophin and glucocorticoid systems in stress-related metaplasticity. We argue that better understanding the long-term effects of developmental stressors on PFC trkB, GR, and related factors may yield insights into risk for chronic, remitting depression and related neuropsychiatric illnesses.

Habits Are Negatively Regulated by Histone Deacetylase 3 in the Dorsal Striatum

BACKGROUND

Optimal behavior and decision making result from a balance of control between two strategies, one cognitive/goal-directed and one habitual. These systems are known to rely on the anatomically distinct dorsomedial and dorsolateral striatum, respectively. However, the transcriptional regulatory mechanisms required to learn and transition between these strategies are unknown. Here we examined the role of one chromatin-based transcriptional regulator, histone modification via histone deacetylases (HDACs), in this process.

METHODS

We combined procedures that diagnose behavioral strategy in rats with pharmacological and viral-mediated HDAC manipulations, chromatin immunoprecipitation, and messenger RNA quantification.

RESULTS

The results indicate that dorsal striatal HDAC3 activity constrains habit formation. Systemic HDAC inhibition following instrumental (lever press → reward) conditioning increased histone acetylation throughout the dorsal striatum and accelerated habitual control of behavior. HDAC3 was removed from the promoters of key learning-related genes in the dorsal striatum as habits formed with overtraining and with posttraining HDAC inhibition. Decreasing HDAC3 function, either by selective pharmacological inhibition or by expression of dominant-negative mutated HDAC3, in either the dorsolateral striatum or the dorsomedial striatum accelerated habit formation, while HDAC3 overexpression in either region prevented habit.

CONCLUSIONS

These results challenge the strict dissociation between dorsomedial striatum and dorsolateral striatum function in goal-directed versus habitual behavioral control and identify dorsostriatal HDAC3 as a critical molecular directive of the transition to habit. Because this transition is disrupted in many neurodegenerative and psychiatric diseases, these data suggest a potential molecular mechanism for the negative behavioral symptoms of these conditions and a target for therapeutic intervention.