Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence

Impaired Flexible Reward-Based Decision-Making in Binge Eating Disorder: Evidence from Computational Modeling and Functional Neuroimaging

Despite its clinical relevance and the recent recognition as a diagnostic category in the DSM-5, binge eating disorder (BED) has rarely been investigated from a cognitive neuroscientific perspective targeting a more precise neurocognitive profiling of the disorder. BED patients suffer from a lack of behavioral control during recurrent binge eating episodes and thus fail to adapt their behavior in the face of negative consequences, eg, high risk for obesity. To examine impairments in flexible reward-based decision-making, we exposed BED patients (n=22) and matched healthy individuals (n=22) to a reward-guided decision-making task during functional resonance imaging (fMRI). Performing fMRI analysis informed via computational modeling of choice behavior, we were able to identify specific signatures of altered decision-making in BED. On the behavioral level, we observed impaired behavioral adaptation in BED, which was due to enhanced switching behavior, a putative deficit in striking a balance between exploration and exploitation appropriately. This was accompanied by diminished activation related to exploratory decisions in the anterior insula/ventro-lateral prefrontal cortex. Moreover, although so-called model-free reward prediction errors remained intact, representation of ventro-medial prefrontal learning signatures, incorporating inference on unchosen options, was reduced in BED, which was associated with successful decision-making in the task. On the basis of a computational psychiatry account, the presented findings contribute to defining a neurocognitive phenotype of BED.

Deficits in reinforcement learning but no link to apathy in patients with schizophrenia

Negative symptoms in schizophrenia have been linked to selective reinforcement learning deficits in the context of gains combined with intact loss-avoidance learning. Fundamental mechanisms of reinforcement learning and choice are prediction error signaling and the precise representation of reward value for future decisions. It is unclear which of these mechanisms contribute to the impairments in learning from positive outcomes observed in schizophrenia. A recent study suggested that patients with severe apathy symptoms show deficits in the representation of expected value. Considering the fundamental relevance for the understanding of these symptoms, we aimed to assess the stability of these findings across studies. Sixty-four patients with schizophrenia and 19 healthy control participants performed a probabilistic reward learning task. They had to associate stimuli with gain or loss-avoidance. In a transfer phase participants indicated valuation of the previously learned stimuli by choosing among them. Patients demonstrated an overall impairment in learning compared to healthy controls. No effects of apathy symptoms on task indices were observed. However, patients with schizophrenia learned better in the context of loss-avoidance than in the context of gain. Earlier findings were thus partially replicated. Further studies are needed to clarify the mechanistic link between negative symptoms and reinforcement learning.

Striatal dopamine, reward, and decision making in schizophrenia

Elevated striatal dopamine function is one of the best-established findings in schizophrenia. In this review, we discuss causes and consequences of this striata! dopamine alteration. We first summarize earlier findings regarding striatal reward processing and anticipation using functional neuroimaging. Secondly, we present a series of recent studies that are exemplary for a particular research approach: a combination of theory-driven reinforcement learning and decision-making tasks in combination with computational modeling and functional neuroimaging. We discuss why this approach represents a promising tool to understand underlying mechanisms of symptom dimensions by dissecting the contribution of multiple behavioral control systems working in parallel. We also discuss how it can advance our understanding of the neurobiological implementation of such functions. Thirdly, we review evidence regarding the topography of dopamine dysfunction within the striatum. Finally, we present conclusions and outline important aspects to be considered in future studies.

An Integrative Perspective on the Role of Dopamine in Schizophrenia

We propose that schizophrenia involves a combination of decreased phasic dopamine responses for relevant stimuli and increased spontaneous phasic dopamine release. Using insights from computational reinforcement-learning models and basic-science studies of the dopamine system, we show that each of these two disturbances contributes to a specific symptom domain and explains a large set of experimental findings associated with that domain. Reduced phasic responses for relevant stimuli help to explain negative symptoms and provide a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with negative symptoms: reduced learning from rewards; blunted activation of the ventral striatum, midbrain, and other limbic regions for rewards and positive prediction errors; blunted activation of the ventral striatum during reward anticipation; blunted autonomic responding for relevant stimuli; blunted neural activation for aversive outcomes and aversive prediction errors; reduced willingness to expend effort for rewards; and psychomotor slowing. Increased spontaneous phasic dopamine release helps to explain positive symptoms and provides a unified explanation for the following experimental findings in schizophrenia, most of which have been shown to correlate with positive symptoms: aberrant learning for neutral cues (assessed with behavioral and autonomic responses), and aberrant, increased activation of the ventral striatum, midbrain, and other limbic regions for neutral cues, neutral outcomes, and neutral prediction errors. Taken together, then, these two disturbances explain many findings in schizophrenia. We review evidence supporting their co-occurrence and consider their differential implications for the treatment of positive and negative symptoms.

Examining associations between psychosis risk, social anhedonia, and performance of striatum-related behavioral tasks

Both psychosis and anhedonia have been associated to some extent with striatal functioning. The current study examined whether either psychosis risk or social anhedonia was associated with performance on 3 tasks related to striatal functioning. Psychosis risk participants had extremely elevated Perceptual Aberration/Magical Ideation (PerMag) scores (n = 69), with 43% of psychosis risk participants also having semistructured interview-assessed psychotic-like experiences which further heightens their risk of psychotic disorder (Chapman, Chapman, Kwapil, Eckblad, & Zinser, 1994). Compared with both extremely elevated social anhedonia (n = 60) and control (n = 68) groups, the PerMag group exhibited poorer performance on 2 of the striatum-related tasks, the Weather Prediction Task (WPT) and the Learned Irrelevance Paradigm, but not on Finger Tapping. In addition, PerMag participants with psychotic-like experiences were especially impaired on the WPT. Overall, this study arguably provides the first evidence that psychosis risk but not social anhedonia is associated with performance on the WPT, a task thought to be strongly associated with activation in the associative striatum, and also suggests that the WPT might be especially useful as a behavioral measure of psychosis risk.

Ecological momentary assessment of negative symptoms in schizophrenia: Relationships to effort-based decision making and reinforcement learning

Negative symptoms are a core clinical feature of schizophrenia, but conceptual and methodological problems with current instruments can make their assessment challenging. One hypothesis is that current symptom assessments may be influenced by impairments in memory and may not be fully reflective of actual functioning outside of the laboratory. The present study sought to investigate the validity of assessing negative symptoms using ecological momentary assessment (EMA). Participants with schizophrenia (N = 31) completed electronic questionnaires on smartphones 4 times a day for 1 week. Participants also completed effort-based decision making and reinforcement learning (RL) tasks to assess the relationship between EMA and laboratory measures, which tap into negative symptom relevant domains. Hierarchical linear modeling analyses revealed that clinician-rated and self-report measures of negative symptoms were significantly related to negative symptoms assessed via EMA. However, working memory moderated the relationship between EMA and retrospective measures of negative symptoms, such that there was a stronger relationship between EMA and retrospective negative symptom measures among individuals with better working memory. The authors also found that negative symptoms assessed via EMA were related to poor performance on the effort task, whereas clinician-rated symptoms and self-reports were not. Further, they found that negative symptoms were related to poorer performance on learning reward contingencies. The findings suggest that negative symptoms can be assessed through EMA and that working memory impairments frequently seen in schizophrenia may affect recall of symptoms. Moreover, these findings suggest the importance of examining the relationship between laboratory tasks and symptoms assessed during daily life. (PsycINFO Database Record

Motivational Context Modulates Prediction Error Response in Schizophrenia

BACKGROUND

Recent findings demonstrate that patients with schizophrenia are worse at learning to predict rewards than losses, suggesting that motivational context modulates learning in this disease. However, these findings derive from studies in patients treated with antipsychotic medications, D2 receptor antagonists that may interfere with the neural systems that underlie motivation and learning. Thus, it remains unknown how motivational context affects learning in schizophrenia, separate from the effects of medication.

METHODS

To examine the impact of motivational context on learning in schizophrenia, we tested 16 unmedicated patients with schizophrenia and 23 matched controls on a probabilistic learning task while they underwent functional magnetic resonance imaging (fMRI) under 2 conditions: one in which they pursued rewards, and one in which they avoided losses. Computational models were used to derive trial-by-trial prediction error responses to feedback.

RESULTS

Patients performed worse than controls on the learning task overall, but there were no behavioral effects of condition. FMRI revealed an attenuated prediction error response in patients in the medial prefrontal cortex, striatum, and medial temporal lobe when learning to predict rewards, but not when learning to avoid losses.

CONCLUSIONS

Patients with schizophrenia showed differences in learning-related brain activity when learning to predict rewards, but not when learning to avoid losses. Together with prior work, these results suggest that motivational deficits related to learning in schizophrenia are characteristic of the disease and not solely a result of antipsychotic treatment.

Mild Reinforcement Learning Deficits in Patients With First-Episode Psychosis

Numerous studies have identified reinforcement learning (RL) deficits in schizophrenia. Most have focused on chronic patients with longstanding antipsychotic treatment, however, and studies of RL in early-illness patients have produced mixed results, particularly regarding gradual/procedural learning. No study has directly contrasted both rapid and gradual RL in first-episode psychosis (FEP) samples. We examined probabilistic RL in 34 FEP patients and 36 controls, using Go/NoGo (GNG) and Gain vs Loss-Avoidance (GLA) paradigms. Our results were mixed, with FEP patients exhibiting greater impairment in the ability to use positive, as opposed to negative, feedback to drive rapid RL on the GLA, but not the GNG. By contrast, patients and controls showed similar improvement across the acquisition. Finally, we found no significant between-group differences in the postacquisition expression of value-based preference in both tasks. Negative symptoms were modestly associated with RL measures, while the overall bias to engage in Go-responding correlated significantly with psychosis severity in FEP patients, consistent with striatal hyperdopaminergia. Taken together, FEP patients demonstrated more circumscribed RL impairments than previous studies have documented in chronic samples, possibly reflecting differential symptom profiles between first-episode and chronic samples. Our finding of relatively preserved gradual/procedural RL, in briefly medicated FEP patients, might suggest spared or restored basal ganglia function. Our findings of preserved abilities to use representations of expected value to guide decision making, and our mixed results regarding rapid RL, may reflect a lesser degree of prefrontal cortical functional impairment in FEP than in chronic samples. Further longitudinal research, in larger samples, is required.

Intact Ventral Striatal Prediction Error Signaling in Medicated Schizophrenia Patients

BACKGROUND

Midbrain dopaminergic neurons code a computational quantity, reward prediction error (RPE), which has been causally related to learning. Recently, this insight has been leveraged to link phenomenological and biological levels of understanding in psychiatric disorders, such as schizophrenia. However, results have been mixed, possibly due to small sample sizes. Here we present results from two studies with relatively large Ns to assess VS RPE in schizophrenia.

METHODS

In the current study we analyzed data from two independent studies, involving a total of 87 chronic medicated schizophrenia patients and 61 controls. Subjects completed a probabilistic reinforcement-learning task in conjunction with fMRI scanning. We fit each participant's choice behavior to a Q-learning model and derived trial-wise RPEs. We then modeled BOLD signal data with parametric regressor functions using these values to determine whether patient and control groups differed in prediction-error-related BOLD signal modulations.

RESULTS

Both groups demonstrated robust VS RPE BOLD activations. Interestingly, these BOLD activation patterns did not differ between groups in either study. This was true when we included all participants in the analysis, as well as when we excluded participants whose data was not sufficiently fit by the models.

CONCLUSIONS

These data demonstrate the utility of computational methods in isolating/testing underlying mechanisms of interest in psychiatric disorders. Importantly, similar VS RPE signal encoding across groups suggests that this mechanism does not drive task deficits in these patients. Deficits may instead stem from aberrant prefrontal/parietal circuits associated with maintenance and selection of goal-relevant information.

Probabilistic Reinforcement Learning in Patients With Schizophrenia: Relationships to Anhedonia and Avolition

BACKGROUND

Anhedonia (a reduced experience of pleasure) and avolition (a reduction in goal-directed activity) are common features of schizophrenia that have substantial effects on functional outcome, but are poorly understood and treated. Here, we examined whether alterations in reinforcement learning may contribute to these symptoms in schizophrenia by impairing the translation of reward information into goal-directed action.

METHODS

38 stable outpatients with schizophrenia or schizoaffective disorder and 37 healthy controls underwent fMRI during a probabilistic stimulus selection reinforcement learning task with dissociated choice- and feedback-related activation, followed by a behavioral transfer task allowing separate assessment of learning from positive versus negative outcomes. A Q-learning algorithm was used to examine functional activation relating to prediction error at the time of feedback and to expected value at the time of choice.

RESULTS

Behavioral results suggested a reduction in learning from positive feedback in patients; however, this reduction was unrelated to anhedonia/avolition severity. On fMRI analysis, prediction error-related activation at the time of feedback was highly similar between patients and controls. During early learning, patients activated regions in the cognitive control network to a lesser extent than controls. Correlation analyses revealed reduced responses to positive feedback in dorsolateral prefrontal cortex and caudate among those patients higher in anhedonia/avolition.

CONCLUSIONS

Together, these results suggest that anhedonia/avolition are as strongly related to cortical learning or higher-level processes involved in goal-directed behavior such as effort computation and planning as to striatally mediated learning mechanisms.

Reduced model-based decision-making in schizophrenia

Individuals with schizophrenia have a diminished ability to use reward history to adaptively guide behavior. However, tasks traditionally used to assess such deficits often rely on multiple cognitive and neural processes, leaving etiology unresolved. In the current study, we adopted recent computational formalisms of reinforcement learning to distinguish between model-based and model-free decision-making in hopes of specifying mechanisms associated with reinforcement-learning dysfunction in schizophrenia. Under this framework, decision-making is model-free to the extent that it relies solely on prior reward history, and model-based if it relies on prospective information such as motivational state, future consequences, and the likelihood of obtaining various outcomes. Model-based and model-free decision-making was assessed in 33 schizophrenia patients and 30 controls using a 2-stage 2-alternative forced choice task previously demonstrated to discern individual differences in reliance on the 2 forms of reinforcement-learning. We show that, compared with controls, schizophrenia patients demonstrate decreased reliance on model-based decision-making. Further, parameter estimates of model-based behavior correlate positively with IQ and working memory measures, suggesting that model-based deficits seen in schizophrenia may be partially explained by higher-order cognitive deficits. These findings demonstrate specific reinforcement-learning and decision-making deficits and thereby provide valuable insights for understanding disordered behavior in schizophrenia. (PsycINFO Database Record

Probability and magnitude evaluation in schizophrenia

Alterations in reinforcement learning and decision making in schizophrenia have been linked with orbitofrontal cortex (OFC) dysfunction, a region critical for weighing reward magnitude in the calculation of expected value (EV). However, much of this work has used complex tasks that require combined learning and EV calculation. Here we used a simple "Roulette" task that examined the calculation of EV directly through a combination of text and/or pictorial representation of reward probability and magnitude. Forty-four people with schizophrenia and 30 controls were recruited. Patients were less sensitive to adjustments in a parameter combining probability and magnitude into one EV construct. Breaking down the construct into independent contributions of probability and magnitude, we found that negative symptoms were associated with magnitude sensitivity. This is consistent with the hypothesized role of OFC in actively representing magnitude and the notion that negative symptoms may involve a failure to appropriately estimate and use future reward magnitude to guide decision making.

Neural correlates of reward processing in adults with 22q11 deletion syndrome

BACKGROUND

22q11.2 deletion syndrome (22q11DS) is caused by a microdeletion on chromosome 22q11.2 and associated with an increased risk to develop psychosis. The gene coding for catechol-O-methyl-transferase (COMT) is located at the deleted region, resulting in disrupted dopaminergic neurotransmission in 22q11DS, which may contribute to the increased vulnerability for psychosis. A dysfunctional motivational reward system is considered one of the salient features in psychosis and thought to be related to abnormal dopaminergic neurotransmission. The functional anatomy of the brain reward circuitry has not yet been investigated in 22q11DS.

METHODS

This study aims to investigate neural activity during anticipation of reward and loss in adult patients with 22q11DS. We measured blood-oxygen-level dependent (BOLD) activity in 16 patients with 22q11DS and 12 healthy controls during a monetary incentive delay task using a 3T Philips Intera MRI system. Data were analysed using SPM8.

RESULTS

During anticipation of reward, the 22q11DS group alone displayed significant activation in bilateral middle frontal and temporal brain regions. Compared to healthy controls, significantly less activation in bilateral cingulate gyrus extending to premotor, primary motor and somatosensory areas was found. During anticipation of loss, the 22q11DS group displayed activity in the left middle frontal gyrus and anterior cingulate cortex, and relative to controls, they showed reduced brain activation in bilateral (pre)cuneus and left posterior cingulate. Within the 22q11DS group, COMT Val hemizygotes displayed more activation compared to Met hemizygotes in right posterior cingulate and bilateral parietal regions during anticipation of reward. During anticipation of loss, COMT Met hemizygotes compared to Val hemizygotes showed more activation in bilateral insula, striatum and left anterior cingulate.

CONCLUSIONS

This is the first study to investigate reward processing in 22q11DS. Our preliminary results suggest that people with 22q11DS engage a fronto-temporal neural network. Compared to healthy controls, people with 22q11DS primarily displayed reduced activity in medial frontal regions during reward anticipation. COMT hemizygosity affects responsivity of the reward system in this condition. Alterations in reward processing partly underlain by the dopamine system may play a role in susceptibility for psychosis in 22q11DS.

Computational psychiatry as a bridge from neuroscience to clinical applications

Translating advances in neuroscience into benefits for patients with mental illness presents enormous challenges because it involves both the most complex organ, the brain, and its interaction with a similarly complex environment. Dealing with such complexities demands powerful techniques. Computational psychiatry combines multiple levels and types of computation with multiple types of data in an effort to improve understanding, prediction and treatment of mental illness. Computational psychiatry, broadly defined, encompasses two complementary approaches: data driven and theory driven. Data-driven approaches apply machine-learning methods to high-dimensional data to improve classification of disease, predict treatment outcomes or improve treatment selection. These approaches are generally agnostic as to the underlying mechanisms. Theory-driven approaches, in contrast, use models that instantiate prior knowledge of, or explicit hypotheses about, such mechanisms, possibly at multiple levels of analysis and abstraction. We review recent advances in both approaches, with an emphasis on clinical applications, and highlight the utility of combining them.

Probabilistic Reversal Learning in Schizophrenia: Stability of Deficits and Potential Causal Mechanisms

Although individuals with schizophrenia show impaired feedback-driven learning on probabilistic reversal learning (PRL) tasks, the specific factors that contribute to these deficits remain unknown. Recent work has suggested several potential causes including neurocognitive impairments, clinical symptoms, and specific types of feedback-related errors. To examine this issue, we administered a PRL task to 126 stable schizophrenia outpatients and 72 matched controls, and patients were retested 4 weeks later. The task involved an initial probabilistic discrimination learning phase and subsequent reversal phases in which subjects had to adjust their responses to sudden shifts in the reinforcement contingencies. Patients showed poorer performance than controls for both the initial discrimination and reversal learning phases of the task, and performance overall showed good test-retest reliability among patients. A subgroup analysis of patients (n = 64) and controls (n = 49) with good initial discrimination learning revealed no between-group differences in reversal learning, indicating that the patients who were able to achieve all of the initial probabilistic discriminations were not impaired in reversal learning. Regarding potential contributors to impaired discrimination learning, several factors were associated with poor PRL, including higher levels of neurocognitive impairment, poor learning from both positive and negative feedback, and higher levels of indiscriminate response shifting. The results suggest that poor PRL performance in schizophrenia can be the product of multiple mechanisms.

Abnormal Frontostriatal Activity During Unexpected Reward Receipt in Depression and Schizophrenia: Relationship to Anhedonia

Alterations in reward processes may underlie motivational and anhedonic symptoms in depression and schizophrenia. However it remains unclear whether these alterations are disorder-specific or shared, and whether they clearly relate to symptom generation or not. We studied brain responses to unexpected rewards during a simulated slot-machine game in 24 patients with depression, 21 patients with schizophrenia, and 21 healthy controls using functional magnetic resonance imaging. We investigated relationships between brain activation, task-related motivation, and questionnaire rated anhedonia. There was reduced activation in the orbitofrontal cortex, ventral striatum, inferior temporal gyrus, and occipital cortex in both depression and schizophrenia in comparison with healthy participants during receipt of unexpected reward. In the medial prefrontal cortex both patient groups showed reduced activation, with activation significantly more abnormal in schizophrenia than depression. Anterior cingulate and medial frontal cortical activation predicted task-related motivation, which in turn predicted anhedonia severity in schizophrenia. Our findings provide evidence for overlapping hypofunction in ventral striatal and orbitofrontal regions in depression and schizophrenia during unexpected reward receipt, and for a relationship between unexpected reward processing in the medial prefrontal cortex and the generation of motivational states.

Study of positive and negative feedback sensitivity in psychosis using the Wisconsin Card Sorting Test

BACKGROUND

Schizophrenia involves marked motivational and learning deficits that may reflect abnormalities in reward processing. The purpose of this study was to examine positive and negative feedback sensitivity in schizophrenia using computational modeling derived from the Wisconsin Card Sorting Test (WCST). We also aimed to explore feedback sensitivity in a sample with bipolar disorder.

METHODS

Eighty-three individuals with schizophrenia and 27 with bipolar disorder were included. Demographic, clinical and cognitive outcomes, together with the WCST, were considered in both samples. Computational modeling was performed using the R syntax to calculate 3 parameters based on trial-by-trial execution on the WCST: reward sensitivity (R), punishment sensitivity (P), and choice consistency (D). The associations between outcome variables and the parameters were investigated.

RESULTS

Positive and negative sensitivity showed deficits, but P parameter was clearly diminished in schizophrenia. Cognitive variables, age, and symptoms were associated with R, P, and D parameters in schizophrenia. The sample with bipolar disorder would show cognitive deficits and feedback abnormalities to a lesser extent than individuals with schizophrenia.

CONCLUSION

Negative feedback sensitivity demonstrated greater deficit in both samples. Idiosyncratic cognitive requirements in the WCST might introduce confusion when supposing model-free reinforcement learning. Negative symptoms of schizophrenia were related to lower feedback sensitivity and less goal-directed patterns of choice.

Feasibility of PRIME: A Cognitive Neuroscience-Informed Mobile App Intervention to Enhance Motivated Behavior and Improve Quality of Life in Recent Onset Schizophrenia

Background

Despite improvements in treating psychosis, schizophrenia remains a chronic and debilitating disorder that affects approximately 1% of the US population and costs society more than depression, dementia, and other medical illnesses across most of the lifespan. Improving functioning early in the course of illness could have significant implications for long-term outcome of individuals with schizophrenia. Yet, current gold-standard treatments do not lead to clinically meaningful improvements in outcome, partly due to the inherent challenges of treating a population with significant cognitive and motivational impairments. The rise of technology presents an opportunity to develop novel treatments that may circumvent the motivational and cognitive challenges observed in schizophrenia.

Objective

The purpose of this study was two-fold: (1) to evaluate the feasibility and acceptability of implementing a Personalized Real-Time Intervention for Motivation Enhancement (PRIME), a mobile app intervention designed to target reward-processing impairments, enhance motivation, and thereby improve quality of life in recent onset schizophrenia, and (2) evaluate the empirical benefits of using an iterative, user-centered design (UCD) process.

Methods

We conducted two design workshops with 15 key stakeholders, followed by a series of in-depth interviews in collaboration with IDEO, a design and innovation firm. The UCD approach ultimately resulted in the first iteration of PRIME, which was evaluated by 10 RO participants. Results from the Stage 1 participants were then used to guide the next iteration that is currently being evaluated in an ongoing RCT. Participants in both phases were encouraged to use the app daily with a minimum frequency of 1/week over a 12-week period.

Results

The UCD process resulted in the following feature set: (1) delivery of text message (short message service, SMS)-based motivational coaching from trained therapists, (2) individualized goal setting in prognostically important psychosocial domains, (3) social networking via direct peer-to-peer messaging, and (4) community “moments feed” to capture and reinforce rewarding experiences and goal achievements. Users preferred an experience that highlighted several of the principles of self-determination theory, including the desire for more control of their future (autonomy and competence) and an approach that helps them improve existing relationships (relatedness). IDEO, also recommended an approach that was casual, friendly, and nonstigmatizing, which is in line with the recovery model of psychosis. After 12-weeks of using PRIME, participants used the app, on average, every other day, were actively engaged with its various features each time they logged in and retention and satisfaction was high (20/20, 100% retention, high satisfaction ratings). The iterative design process lead to a 2- to 3-fold increase in engagement from Stage 1 to Stage 2 in almost each aspect of the platform.

Conclusions

These results indicate that the neuroscience-informed mobile app, PRIME, is a feasible and acceptable intervention for young people with schizophrenia.

Reduction of Pavlovian Bias in Schizophrenia: Enhanced Effects in Clozapine-Administered Patients

The negative symptoms of schizophrenia (SZ) are associated with a pattern of reinforcement learning (RL) deficits likely related to degraded representations of reward values. However, the RL tasks used to date have required active responses to both reward and punishing stimuli. Pavlovian biases have been shown to affect performance on these tasks through invigoration of action to reward and inhibition of action to punishment, and may be partially responsible for the effects found in patients. Forty-five patients with schizophrenia and 30 demographically-matched controls completed a four-stimulus reinforcement learning task that crossed action ("Go" or "NoGo") and the valence of the optimal outcome (reward or punishment-avoidance), such that all combinations of action and outcome valence were tested. Behaviour was modelled using a six-parameter RL model and EEG was simultaneously recorded. Patients demonstrated a reduction in Pavlovian performance bias that was evident in a reduced Go bias across the full group. In a subset of patients administered clozapine, the reduction in Pavlovian bias was enhanced. The reduction in Pavlovian bias in SZ patients was accompanied by feedback processing differences at the time of the P3a component. The reduced Pavlovian bias in patients is suggested to be due to reduced fidelity in the communication between striatal regions and frontal cortex. It may also partially account for previous findings of poorer "Go-learning" in schizophrenia where "Go" responses or Pavlovian consistent responses are required for optimal performance. An attenuated P3a component dynamic in patients is consistent with a view that deficits in operant learning are due to impairments in adaptively using feedback to update representations of stimulus value.

The doxastic shear pin: delusions as errors of learning and memory

We reconsider delusions in terms of a "doxastic shear pin", a mechanism that errs so as to prevent the destruction of the machine (brain) and permit continued function (in an attenuated capacity). Delusions may disable flexible (but energetically expensive) inference. With each recall, delusions may be reinforced further and rendered resistant to contradiction. We aim to respond to deficit accounts of delusions - that delusions are only a problem without any benefit - by considering delusion formation and maintenance in terms of predictive coding. We posit that brains conform to a simple computational principle: to minimize prediction error (the mismatch between prior top-down expectation and current bottom-up input) across hierarchies of brain regions and psychological representation. Recent data suggest that delusions may form in the absence of constraining top-down expectations. Then, once formed, they become new priors that motivate other beliefs, perceptions, and actions by providing strong (sometimes overriding) top-down expectation. We argue that delusions form when the shear-pin breaks, permitting continued engagement with an overwhelming world, and ongoing function in the face of paralyzing difficulty. This crucial role should not be ignored when we treat delusions: we need to consider how a person will function in the world without them..

Frontal-striatum dysfunction during reward processing: Relationships to amotivation in schizophrenia

Schizophrenia is characterized by deficits of context processing, thought to be related to dorsolateral prefrontal cortex (DLPFC) impairment. Despite emerging evidence suggesting a crucial role of the DLPFC in integrating reward and goal information, we do not know whether individuals with schizophrenia can represent and integrate reward-related context information to modulate cognitive control. To address this question, 36 individuals with schizophrenia (n = 29) or schizoaffective disorder (n = 7) and 27 healthy controls performed a variant of a response conflict task (Padmala & Pessoa, 2011) during fMRI scanning, in both baseline and reward conditions, with monetary incentives on some reward trials. We used a mixed state-item design that allowed us to examine both sustained and transient reward effects on cognitive control. Different from predictions about impaired DLPFC function in schizophrenia, we found an intact pattern of increased sustained DLPFC activity during reward versus baseline blocks in individuals with schizophrenia at a group level but blunted sustained activations in the putamen. Contrary to our predictions, individuals with schizophrenia showed blunted cue-related activations in several regions of the basal ganglia responding to reward-predicting cues. Importantly, as predicted, individual differences in anhedonia/amotivation symptoms severity were significantly associated with reduced sustained DLPFC activation in the same region that showed overall increased activity as a function of reward. These results suggest that individual differences in motivational impairments in schizophrenia may be related to dysfunction of the DLPFC and striatum in motivationally salient situations.

Distinct Reward Properties are Encoded via Corticostriatal Interactions

The striatum serves as a critical brain region for reward processing. Yet, understanding the link between striatum and reward presents a challenge because rewards are composed of multiple properties. Notably, affective properties modulate emotion while informative properties help obtain future rewards. We approached this problem by emphasizing affective and informative reward properties within two independent guessing games. We found that both reward properties evoked activation within the nucleus accumbens, a subregion of the striatum. Striatal responses to informative, but not affective, reward properties predicted subsequent utilization of information for obtaining monetary reward. We hypothesized that activation of the striatum may be necessary but not sufficient to encode distinct reward properties. To investigate this possibility, we examined whether affective and informative reward properties were differentially encoded in corticostriatal interactions. Strikingly, we found that the striatum exhibited dissociable connectivity patterns with the ventrolateral prefrontal cortex, with increasing connectivity for affective reward properties and decreasing connectivity for informative reward properties. Our results demonstrate that affective and informative reward properties are encoded via corticostriatal interactions. These findings highlight how corticostriatal systems contribute to reward processing, potentially advancing models linking striatal activation to behavior.

Charting the landscape of priority problems in psychiatry, part 1: classification and diagnosis

Contemporary psychiatry faces major challenges. Its syndrome-based disease classification is not based on mechanisms and does not guide treatment, which largely depends on trial and error. The development of therapies is hindered by ignorance of potential beneficiary patient subgroups. Neuroscientific and genetics research have yet to affect disease definitions or contribute to clinical decision making. In this challenging setting, what should psychiatric research focus on? In two companion papers, we present a list of problems nominated by clinicians and researchers from different disciplines as candidates for future scientific investigation of mental disorders. These problems are loosely grouped into challenges concerning nosology and diagnosis (this Personal View) and problems related to pathogenesis and aetiology (in the companion Personal View). Motivated by successful examples in other disciplines, particularly the list of Hilbert's problems in mathematics, this subjective and eclectic list of priority problems is intended for psychiatric researchers, helping to re-focus existing research and providing perspectives for future psychiatric science.

Avolition in schizophrenia is associated with reduced willingness to expend effort for reward on a Progressive Ratio task

The current study examined whether effort-cost computation was associated with negative symptoms of schizophrenia (SZ). Participants included outpatients diagnosed with SZ (n=27) and demographically matched healthy controls (n=32) who completed a Progressive Ratio task that required incrementally greater amounts of physical effort to obtain monetary reward. Breakpoint, the point at which participants was no longer willing to exert effort for a certain reward value, was examined as an index of effort-cost computation. There were no group differences in breakpoint for low, medium, or high value rewards on the Progressive Ratio task. However, lower breakpoint scores were associated with greater severity of avolition and anhedonia symptoms in SZ patients. Findings provide further evidence that impaired effort-cost computation is linked to motivational abnormalities in SZ.

Computational Psychiatry: towards a mathematically informed understanding of mental illness

Computational Psychiatry aims to describe the relationship between the brain's neurobiology, its environment and mental symptoms in computational terms. In so doing, it may improve psychiatric classification and the diagnosis and treatment of mental illness. It can unite many levels of description in a mechanistic and rigorous fashion, while avoiding biological reductionism and artificial categorisation. We describe how computational models of cognition can infer the current state of the environment and weigh up future actions, and how these models provide new perspectives on two example disorders, depression and schizophrenia. Reinforcement learning describes how the brain can choose and value courses of actions according to their long-term future value. Some depressive symptoms may result from aberrant valuations, which could arise from prior beliefs about the loss of agency ('helplessness'), or from an inability to inhibit the mental exploration of aversive events. Predictive coding explains how the brain might perform Bayesian inference about the state of its environment by combining sensory data with prior beliefs, each weighted according to their certainty (or precision). Several cortical abnormalities in schizophrenia might reduce precision at higher levels of the inferential hierarchy, biasing inference towards sensory data and away from prior beliefs. We discuss whether striatal hyperdopaminergia might have an adaptive function in this context, and also how reinforcement learning and incentive salience models may shed light on the disorder. Finally, we review some of Computational Psychiatry's applications to neurological disorders, such as Parkinson's disease, and some pitfalls to avoid when applying its methods.

Effort cost computation in schizophrenia: a commentary on the recent literature

The cognitive and affective factors implicated in the motivational impairments seen in many people with schizophrenia remain poorly understood. Many research groups have done studies in the past 2 years examining the role of effort-cost computations driven by the hypothesis that overestimation of the cost of effort involved in volitional behavior might underlie the reduction in goal-directed behavior seen in some people with schizophrenia. The goal of this review is to assess the available evidence and the interpretative ambiguities that remain to be addressed by further studies. There is a clear preponderance of evidence suggesting that people with schizophrenia demonstrate altered effort allocation by failing to make high-effort response choices to maximize reward. The evidence relating altered effort allocation to the severity of negative symptoms is mixed. It remains for future work to determine the precise mechanisms implicated in altered effort allocation with two prominent possibilities: that patients 1) overestimate the cost of effort or 2) underestimate the value of potential awards. Other mechanisms that need to be investigated include the potential contributions of other impairments associated with the illness that increase the cost of effort. Furthermore, it is possible that accurate value representations fail to invigorate behavior. Although questions remain, evidence available to date suggests that the study of cost/benefit decision making may shed new light on the motivational impairments seen in many people with schizophrenia.

Medial-Frontal Stimulation Enhances Learning in Schizophrenia by Restoring Prediction Error Signaling

Posterror learning, associated with medial-frontal cortical recruitment in healthy subjects, is compromised in neuropsychiatric disorders. Here we report novel evidence for the mechanisms underlying learning dysfunctions in schizophrenia. We show that, by noninvasively passing direct current through human medial-frontal cortex, we could enhance the event-related potential related to learning from mistakes (i.e., the error-related negativity), a putative index of prediction error signaling in the brain. Following this causal manipulation of brain activity, the patients learned a new task at a rate that was indistinguishable from healthy individuals. Moreover, the severity of delusions interacted with the efficacy of the stimulation to improve learning. Our results demonstrate a causal link between disrupted prediction error signaling and inefficient learning in schizophrenia. These findings also demonstrate the feasibility of nonpharmacological interventions to address cognitive deficits in neuropsychiatric disorders.

SIGNIFICANCE STATEMENT

When there is a difference between what we expect to happen and what we actually experience, our brains generate a prediction error signal, so that we can map stimuli to responses and predict outcomes accurately. Theories of schizophrenia implicate abnormal prediction error signaling in the cognitive deficits of the disorder. Here, we combine noninvasive brain stimulation with large-scale electrophysiological recordings to establish a causal link between faulty prediction error signaling and learning deficits in schizophrenia. We show that it is possible to improve learning rate, as well as the neural signature of prediction error signaling, in patients to a level quantitatively indistinguishable from that of healthy subjects. The results provide mechanistic insight into schizophrenia pathophysiology and suggest a future therapy for this condition.

Cognitive effort avoidance and detection in people with schizophrenia

Many people with schizophrenia exhibit avolition, a difficulty initiating and maintaining goal-directed behavior, considered to be a key negative symptom of the disorder. Recent evidence indicates that patients with higher levels of negative symptoms differ from healthy controls in showing an exaggerated cost of the physical effort needed to obtain a potential reward. We examined whether patients show an exaggerated avoidance of cognitive effort, using the demand selection task developed by Kool, McGuire, Rosen, and Botvinick (Journal of Experimental Psychology. General, 139, 665-682, 2010). A total of 83 people with schizophrenia or schizoaffective disorder and 71 healthy volunteers participated in three experiments where instructions varied. In the standard task (Experiment 1), neither controls nor patients showed expected cognitive demand avoidance. With enhanced instructions (Experiment 2), controls demonstrated greater demand avoidance than patients. In Experiment 3, patients showed nonsignificant reductions in demand avoidance, relative to controls. In a control experiment, patients showed significantly reduced ability to detect the effort demands associated with different response alternatives. In both groups, the ability to detect effort demands was associated with increased effort avoidance. In both groups, increased cognitive effort avoidance was associated with higher IQ and general neuropsychological ability. No significant correlations between demand avoidance and negative symptom severity were observed. Thus, it appears that individual differences in general intellectual ability and effort detection are related to cognitive effort avoidance and likely account for the subtle reduction in effort avoidance observed in schizophrenia.

Identifying differences between depressed adolescent suicide ideators and attempters

BACKGROUND

Adolescent depression and suicide are pressing public health concerns, and identifying key differences among suicide ideators and attempters is critical. The goal of the current study is to test whether depressed adolescent suicide attempters report greater anhedonia severity and exhibit aberrant effort-cost computations in the face of uncertainty.

METHODS

Depressed adolescents (n=101) ages 13-19 years were administered structured clinical interviews to assess current mental health disorders and a history of suicidality (suicide ideators=55, suicide attempters=46). Then, participants completed self-report instruments assessing symptoms of suicidal ideation, depression, anhedonia, and anxiety as well as a computerized effort-cost computation task.

RESULTS

Compared with depressed adolescent suicide ideators, attempters report greater anhedonia severity, even after concurrently controlling for symptoms of suicidal ideation, depression, and anxiety. Additionally, when completing the effort-cost computation task, suicide attempters are less likely to pursue the difficult, high value option when outcomes are uncertain. Follow-up, trial-level analyses of effort-cost computations suggest that receipt of reward does not influence future decision-making among suicide attempters, however, suicide ideators exhibit a win-stay approach when receiving rewards on previous trials.

LIMITATIONS

Findings should be considered in light of limitations including a modest sample size, which limits generalizability, and the cross-sectional design.

CONCLUSIONS

Depressed adolescent suicide attempters are characterized by greater anhedonia severity, which may impair the ability to integrate previous rewarding experiences to inform future decisions. Taken together, this may generate a feeling of powerlessness that contributes to increased suicidality and a needless loss of life.

Measuring anhedonia: impaired ability to pursue, experience, and learn about reward

Ribot’s (1896) long standing definition of anhedonia as “the inability to experience pleasure” has been challenged recently following progress in affective neuroscience. In particular, accumulating evidence suggests that reward consists of multiple subcomponents of wanting, liking and learning, as initially outlined by Berridge and Robinson (2003), and these processes have been proposed to relate to appetitive, consummatory and satiety phases of a pleasure cycle. Building on this work, we recently proposed to reconceptualize anhedonia as “impairments in the ability to pursue, experience, and/or learn about pleasure, which is often, but not always accessible to conscious awareness.” (Rømer Thomsen et al., 2015). This framework is in line with Treadway and Zald’s (2011) proposal to differentiate between motivational and consummatory types of anhedonia, and stresses the need to combine traditional self-report measures with behavioral measures or procedures. In time, this approach may lead to improved clinical assessment and treatment. In line with our reconceptualization, increasing evidence suggests that reward processing deficits are not restricted to impaired hedonic impact in major psychiatric disorders. Successful translations of animal models have led to strong evidence of impairments in the ability to pursue and learn about reward in psychiatric disorders such as major depressive disorder, schizophrenia, and addiction. It is of high importance that we continue to systematically target impairments in all phases of reward processing across disorders using behavioral testing in combination with neuroimaging techniques. This in turn has implications for diagnosis and treatment, and is essential for the purposes of identifying the underlying neurobiological mechanisms. Here I review recent progress in the development and application of behavioral procedures that measure subcomponents of anhedonia across relevant patient groups, and discuss methodological caveats as well as implications for assessment and treatment.

Probabilistic reward- and punishment-based learning in opioid addiction: Experimental and computational data

Addiction is the continuation of a habit in spite of negative consequences. A vast literature gives evidence that this poor decision-making behavior in individuals addicted to drugs also generalizes to laboratory decision making tasks, suggesting that the impairment in decision-making is not limited to decisions about taking drugs. In the current experiment, opioid-addicted individuals and matched controls with no history of illicit drug use were administered a probabilistic classification task that embeds both reward-based and punishment-based learning trials, and a computational model of decision making was applied to understand the mechanisms describing individuals' performance on the task. Although behavioral results showed that opioid-addicted individuals performed as well as controls on both reward- and punishment-based learning, the modeling results suggested subtle differences in how decisions were made between the two groups. Specifically, the opioid-addicted group showed decreased tendency to repeat prior responses, meaning that they were more likely to "chase reward" when expectancies were violated, whereas controls were more likely to stick with a previously-successful response rule, despite occasional expectancy violations. This tendency to chase short-term reward, potentially at the expense of developing rules that maximize reward over the long term, may be a contributing factor to opioid addiction. Further work is indicated to better understand whether this tendency arises as a result of brain changes in the wake of continued opioid use/abuse, or might be a pre-existing factor that may contribute to risk for addiction.

Probing the Dynamic Updating of Value in Schizophrenia Using a Sensory-Specific Satiety Paradigm

It has been proposed that both positive and negative symptoms in schizophrenia (SZ) may derive, at least in part, from a disrupted ability to accurately and flexibly represent the value of stimuli and actions. To assess relationships between dimensions of psychopathology in SZ, and the tendency to devalue food stimuli, on which subjects were fed to satiety, we administered a sensory-specific satiety (SSS) paradigm to 42 SZ patients and 44 controls. In each of 2 sessions, subjects received 16 0.7-ml squirts of each of 2 rewarding foods and 32 squirts of a control solution, using syringes. In between the 2 sessions, each subject was instructed to drink one of the foods until he/she felt "full, but not uncomfortable." At 10 regular intervals, interspersed throughout the 2 sessions, subjects rated each liquid for pleasantness, using a Likert-type scale. Mann-Whitney U-tests revealed group differences in SSS effects. Within-group tests revealed that, while controls showed an effect of satiety that was sensory specific, patients showed an effect of satiety that was not, devaluing the sated and unsated foods similarly. In SZ patients, we observed correlations between the magnitude of SSS effects and measures of both positive and negative symptoms. We argue that the ability to flexibly and rapidly update representations of the value of stimuli and actions figures critically in the ability of patients with psychotic illness to process salient events and adaptively engage in goal-directed behavior.

Abnormal context-reward associations in an immune-mediated neurodevelopmental mouse model with relevance to schizophrenia

Impairments in central reward processing constitute an important aspect of the negative symptoms of schizophrenia. Despite its clinical relevance, the etiology of deficient reward processing in schizophrenia remains largely unknown. Here, we used an epidemiologically informed mouse model of schizophrenia to explore the effects of prenatal immune activation on reward-related functions. The model is based on maternal administration of the viral mimic PolyI:C and has been developed in relation to the epidemiological evidence demonstrating enhanced risk of schizophrenia and related disorders following prenatal maternal infection. We show that prenatal immune activation induces selective deficits in the expression (but not acquisition) of conditioned place preference for a natural reward (sucrose) without changing hedonic or neophobic responses to the reward. On the other hand, prenatal immune activation led to enhanced place preference for the psychostimulant drug cocaine, while it attenuated the locomotor reaction to the drug. The prenatal exposure did not alter negative reinforcement learning as assessed using a contextual fear conditioning paradigm. Our findings suggest that the nature of reward-related abnormalities following prenatal immune challenge depends on the specificity of the reward (natural reward vs drug of abuse) as well as on the valence domain (positive vs negative reinforcement learning). Moreover, our data indicate that reward abnormalities emerging in prenatally immune-challenged offspring may, at least in part, stem from an inability to retrieve previously established context-reward associations and to integrate such information for appropriate goal-directed behavior.

Improving temporal cognition by enhancing motivation

Increasing motivation can positively impact cognitive performance. Here we employed a cognitive timing task that allows us to detect changes in cognitive performance that are not influenced by general activity or arousal factors such as the speed or persistence of responding. This approach allowed us to manipulate motivation using three different methods; molecular/genetic, behavioral and pharmacological. Increased striatal D2Rs resulted in deficits in temporal discrimination. Switching off the transgene improved motivation in earlier studies, and here partially rescued the temporal discrimination deficit. To manipulate motivation behaviorally, we altered reward magnitude and found that increasing reward magnitude improved timing in control mice and partially rescued timing in the transgenic mice. Lastly, we manipulated motivation pharmacologically using a functionally selective 5-HT2C receptor ligand, SB242084, which we previously found to increase incentive motivation. SB242084 improved temporal discrimination in both control and transgenic mice. Thus, while there is a general intuitive belief that motivation can affect cognition, we here provide a direct demonstration that enhancing motivation, in a variety of ways, can be an effective strategy for enhancing temporal cognition. Understanding the interaction of motivation and cognition is of clinical significance since many psychiatric disorders are characterized by deficits in both domains.

Abnormalities in personal space and parietal-frontal function in schizophrenia

Schizophrenia is associated with subtle abnormalities in day-to-day social behaviors, including a tendency in some patients to “keep their distance” from others in physical space. The neural basis of this abnormality, and related changes in social functioning, is unknown. Here we examined, in schizophrenic patients and healthy control subjects, the functioning of a parietal–frontal network involved in monitoring the space immediately surrounding the body (“personal space”). Using fMRI, we found that one region of this network, the dorsal intraparietal sulcus (DIPS), was hyper-responsive in schizophrenic patients to face stimuli appearing to move towards the subjects, intruding into personal space. This hyper-responsivity was predicted both by the size of personal space (which was abnormally elevated in the schizophrenia group) and the severity of negative symptoms. In contrast, in a second study, the activity of two lower-level visual areas that send information to DIPS (the fusiform face area and middle temporal area) was normal in schizophrenia. Together, these findings suggest that changes in parietal–frontal networks that support the sensory-guided initiation of behavior, including actions occurring in the space surrounding the body, contribute to social dysfunction and negative symptoms in schizophrenia.

•

A parietal–frontal network in the primate brain monitors the space near the body.

•

“Personal space”, a person's “comfort zone”, is influenced by this network.

•

Patients with schizophrenia show an abnormal enlargement of personal space.

•

This enlargement correlates with negative symptom levels.

•

This enlargement also correlates with parietal responses to personal space intrusions.

Experiential pleasure deficits in different stages of schizophrenia

Prior research has found dampened anticipatory pleasure but relatively intact consummatory pleasure in people with first-episode and more chronic schizophrenia, but no study has examined anticipatory and consummatory pleasure across the schizophrenia spectrum. To confirm the factor structure of the Chinese version of the Temporal Experience Pleasure Scale (TEPS), which measures four components of anhedonia, we recruited 364 people with schizophrenia for confirmatory factor analysis. To examine anhedonia in people across the schizophrenia spectrum, we recruited people with first-episode (n=76) and chronic schizophrenia (n=45), people with schizotypal traits (n=210), first-degree relatives (n=45) of people with schizophrenia and healthy controls. Deficit in abstract anticipatory pleasure appeared to be most severe in people with chronic schizophrenia, while dampened abstract consummatory pleasure was observed in people with schizotypal personality features and in people with chronic schizophrenia. In addition, both abstract anticipatory and abstract consummatory pleasure were negatively correlated with negative schizotypal personality features and schizophrenia symptoms. Our results suggest that deficits in anticipatory pleasure are present across the schizophrenia spectrum, particularly in the abstract domain.

Working memory contributions to reinforcement learning impairments in schizophrenia

Previous research has shown that patients with schizophrenia are impaired in reinforcement learning tasks. However, behavioral learning curves in such tasks originate from the interaction of multiple neural processes, including the basal ganglia- and dopamine-dependent reinforcement learning (RL) system, but also prefrontal cortex-dependent cognitive strategies involving working memory (WM). Thus, it is unclear which specific system induces impairments in schizophrenia. We recently developed a task and computational model allowing us to separately assess the roles of RL (slow, cumulative learning) mechanisms versus WM (fast but capacity-limited) mechanisms in healthy adult human subjects. Here, we used this task to assess patients' specific sources of impairments in learning. In 15 separate blocks, subjects learned to pick one of three actions for stimuli. The number of stimuli to learn in each block varied from two to six, allowing us to separate influences of capacity-limited WM from the incremental RL system. As expected, both patients (n = 49) and healthy controls (n = 36) showed effects of set size and delay between stimulus repetitions, confirming the presence of working memory effects. Patients performed significantly worse than controls overall, but computational model fits and behavioral analyses indicate that these deficits could be entirely accounted for by changes in WM parameters (capacity and reliability), whereas RL processes were spared. These results suggest that the working memory system contributes strongly to learning impairments in schizophrenia.

Orbitofrontal cortex mediates the differential impact of signaled-reward probability on discrimination accuracy

Orbitofrontal cortex (OFC) function is critical to decision making and behavior based on the value of expected outcomes. While some of the roles the OFC plays in value computations and behavior have been identified, the role of the OFC in modulating cognitive resources based on reward expectancy has not been explored. Here we assessed the involvement of OFC in the interaction between motivation and attention. We tested mice in a sustained-attention task in which explicitly signaling the probability of reward differentially modulates discrimination accuracy. Using pharmacogenetic methods, we generated mice in which neuronal activity in the OFC could be transiently and reversibly inhibited during performance of our signaled-probability task. We found that inhibiting OFC neuronal activity abolished the ability of reward-associated cues to differentially impact accuracy of sustained-attention performance. This failure to modulate attention occurred despite evidence that mice still processed the differential value of the reward-associated cues. These data indicate that OFC function is critical for the ability of a reward-related signal to impact other cognitive and decision-making processes and begin to delineate the neural circuitry involved in the interaction between motivation and attention.

The rat's not for turning: Dissociating the psychological components of cognitive inflexibility

•

Non-rewarded or irrelevant prior associations are important for flexible responding.

•

Associations of reward and non-reward in reversal learning are neurally dissociable.

•

Disruption of prior irrelevant or rewarded associations cause pathological deficits.

•

Experimental paradigms of cognitive flexibility can be improved to aid translation.

Executive function is commonly assessed by assays of cognitive flexibility such as reversal learning and attentional set-shifting. Disrupted performance in these assays, apparent in many neuropsychiatric disorders, is frequently interpreted as inability to overcome prior associations with reward.

However, non-rewarded or irrelevant associations may be of considerable importance in both discrimination learning and cognitive flexibility. Non-rewarded associations can have greater influence on choice behaviour than rewarded associations in discrimination learning. Pathology-related deficits in cognitive flexibility can produce selective disruptions to both the processing of irrelevant associations and associations with reward. Genetic and pharmacological animal models demonstrate that modulation of reversal learning may result from alterations in either rewarded or non-rewarded associations.

Successful performance in assays of cognitive flexibility can therefore depend on a combination of rewarded, non-rewarded, and irrelevant associations derived from previous learning, accounting for some inconsistencies observed in the literature. Taking this combination into account may increase the validity of animal models and may also reveal pathology-specific differences in problem solving and executive function.

Motivation and effort in individuals with social anhedonia

It has been proposed that anhedonia may, in part, reflect difficulties in reward processing and effortful decision making. The current study aimed to replicate previous findings of effortful decision making deficits associated with elevated anhedonia and expand upon these findings by investigating whether these decision making deficits are specific to elevated social anhedonia or are also associated with elevated positive schizotypy characteristics. The current study compared controls (n=40) to individuals elevated on social anhedonia (n=30), and individuals elevated on perceptual aberration/magical ideation (n=30) on the Effort Expenditure for Rewards Task (EEfRT). Across groups, participants chose a higher proportion of hard tasks with increasing probability of reward and reward magnitude, demonstrating sensitivity to probability and reward values. Contrary to our expectations, when the probability of reward was most uncertain (50% probability), at low and medium reward values, the social anhedonia group demonstrated more effortful decision making than either individuals high in positive schizotypy or controls. The positive schizotypy group only differed from controls (making less effortful choices than controls) when reward probability was lowest (12%) and the magnitude of reward was the smallest. Our results suggest that social anhedonia is related to intact motivation and effort for monetary rewards, but that individuals with this characteristic display a unique and perhaps inefficient pattern of effort allocation when the probability of reward is most uncertain. Future research is needed to better understand effortful decision making and the processing of reward across a range of individual difference characteristics.

The impact of motivation on cognitive performance in an animal model of the negative and cognitive symptoms of schizophrenia

Interactions between motivation and cognition are implicated in producing functional impairments and poor quality of life in psychiatric patients. This interaction, however, is not well understood at either the behavioral or neural level. We developed a procedure for mice in which a cognitive measure, sustained attention, is modulated by a motivationally relevant signal that predicts reward probability on a trial-by-trial basis. Using this paradigm, we tested the interaction between motivation and cognition in mice that model the increased striatal D2 receptor activity observed in schizophrenia patients (D2R-OE mice). In control mice, attention was modulated by signaled-reward probability. In D2R-OE mice, however, attention was not modulated by reward-related cues. This impairment was not due to any global deficits in attention or maintenance of the trial-specific information in working memory. Turning off the transgene in D2R-OE mice rescued the motivational modulation of attention. These results indicate that deficits in motivation impair the ability to use reward-related cues to recruit attention and that improving motivation improves functional cognitive performance. These results further suggest that addressing motivational impairments in patients is critical to achieving substantive cognitive and functional gains.

Ventral tegmental area cholinergic mechanisms mediate behavioral responses in the forced swim test

Recent studies revealed a causal link between ventral tegmental area (VTA) phasic dopamine (DA) activity and pro-depressive and antidepressant-like behavioral responses in rodent models of depression. Cholinergic activity in the VTA has been demonstrated to regulate phasic DA activity, but the role of VTA cholinergic mechanisms in depression-related behavior is unclear. The goal of this study was to determine whether pharmacological manipulation of VTA cholinergic activity altered behavioral responding in the forced swim test (FST) in rats. Here, male Sprague-Dawley rats received systemic or VTA-specific administration of the acetylcholinesterase inhibitor, physostigmine (systemic; 0.06 or 0.125mg/kg, intra-cranial; 1 or 2μg/side), the muscarinic acetylcholine receptor (AChR) antagonist scopolamine (2.4 or 24μg/side), or the nicotinic AChR antagonist mecamylamine (3 or 30μg/side), prior to the FST test session. In control experiments, locomotor activity was also examined following systemic and intra-cranial administration of cholinergic drugs. Physostigmine administration, either systemically or directly into the VTA, significantly increased immobility time in FST, whereas physostigmine infusion into a dorsal control site did not alter immobility time. In contrast, VTA infusion of either scopolamine or mecamylamine decreased immobility time, consistent with an antidepressant-like effect. Finally, the VTA physostigmine-induced increase in immobility was blocked by co-administration with scopolamine, but unaltered by co-administration with mecamylamine. These data show that enhancing VTA cholinergic tone and blocking VTA AChRs has opposing effects in FST. Together, the findings provide evidence for a role of VTA cholinergic mechanisms in behavioral responses in FST.

Reconceptualizing anhedonia: novel perspectives on balancing the pleasure networks in the human brain

Anhedonia, the lack of pleasure, has been shown to be a critical feature of a range of psychiatric disorders. Yet, it is currently measured primarily through subjective self-reports and as such has been difficult to submit to rigorous scientific analysis. New insights from affective neuroscience hold considerable promise in improving our understanding of anhedonia and for providing useful objective behavioral measures to complement traditional self-report measures, potentially leading to better diagnoses and novel treatments. Here, we review the state-of-the-art of hedonia research and specifically the established mechanisms of wanting, liking, and learning. Based on this framework we propose to conceptualize anhedonia as impairments in some or all of these processes, thereby departing from the longstanding view of anhedonia as solely reduced subjective experience of pleasure. We discuss how deficits in each of the reward components can lead to different expressions, or subtypes, of anhedonia affording novel ways of measurement. Specifically, we review evidence suggesting that patients suffering from depression and schizophrenia show impairments in wanting and learning, while some aspects of conscious liking seem surprisingly intact. Furthermore, the evidence suggests that anhedonia is heterogeneous across psychiatric disorders, depending on which parts of the pleasure networks are most affected. This in turn has implications for diagnosis and treatment of anhedonia.

Bridging Levels of Understanding in Schizophrenia Through Computational Modeling

Schizophrenia is an illness with a remarkably complex symptom presentation that has thus far been out of reach of neuroscientific explanation. This presents a fundamental problem for developing better treatments that target specific symptoms or root causes. One promising path forward is the incorporation of computational neuroscience, which provides a way to formalize experimental observations and, in turn, make theoretical predictions for subsequent studies. We review three complementary approaches: (a) biophysically based models developed to test cellular-level and synaptic hypotheses, (b) connectionist models that give insight into large-scale neural-system-level disturbances in schizophrenia, and (c) models that provide a formalism for observations of complex behavioral deficits, such as negative symptoms. We argue that harnessing all of these modeling approaches represents a productive approach for better understanding schizophrenia. We discuss how blending these approaches can allow the field to progress toward a more comprehensive understanding of schizophrenia and its treatment.

Reduced susceptibility to confirmation bias in schizophrenia

Patients with schizophrenia (SZ) show cognitive impairments on a wide range of tasks, with clear deficiencies in tasks reliant on prefrontal cortex function and less consistently observed impairments in tasks recruiting the striatum. This study leverages tasks hypothesized to differentially recruit these neural structures to assess relative deficiencies of each. Forty-eight patients and 38 controls completed two reinforcement learning tasks hypothesized to interrogate prefrontal and striatal functions and their interaction. In each task, participants learned reward discriminations by trial and error and were tested on novel stimulus combinations to assess learned values. In the task putatively assessing fronto-striatal interaction, participants were (inaccurately) instructed that one of the stimuli was valuable. Consistent with prior reports and a model of confirmation bias, this manipulation resulted in overvaluation of the instructed stimulus after its true value had been experienced. Patients showed less susceptibility to this confirmation bias effect than did controls. In the choice bias task hypothesized to more purely assess striatal function, biases in endogenously and exogenously chosen actions were assessed. No group differences were observed. In the subset of participants who showed learning in both tasks, larger group differences were observed in the confirmation bias task than in the choice bias task. In the confirmation bias task, patients also showed impairment in the task conditions with no prior instruction. This deficit was most readily observed on the most deterministic discriminations. Taken together, these results suggest impairments in fronto-striatal interaction in SZ, rather than in striatal function per se.

Assessing motivation orientations in schizophrenia: Scale development and validation

Motivation deficits are common in several disorders including schizophrenia, and are an important factor in both functioning and treatment adherence. Self-Determination Theory (SDT), a leading macro-theory of motivation, has contributed a number of insights into how motivation is impaired in schizophrenia. Nonetheless, self-report measures of motivation appropriate for people with severe mental illness (including those that emphasize SDT) are generally lacking in the literature. To fill this gap, we adapted and abbreviated the well-validated General Causality Orientation Scale for use with people with schizophrenia and with other severe mental disorders (GCOS-clinical populations; GCOS-CP). In Study 1, we tested the similarity of our measure to the existing GCOS (using a college sample) and then validated this new measure in a schizophrenia and healthy control sample (Study 2). Results from Study 1 (N=360) indicated that the GCOS-CP was psychometrically similar to the original GCOS and provided good convergent and discriminant validity. In Study 2, the GCOS-CP was given to individuals with (N=44) and without schizophrenia (N=42). In line with both laboratory-based and observer-based research, people with schizophrenia showed lower motivational autonomy and higher impersonal/amotivated orientations. Additional applications of the GCOS-CP are discussed.

Motivational Deficits in Schizophrenia and the Representation of Expected Value

Motivational deficits (avolition and anhedonia) have historically been considered important negative symptoms of schizophrenia (SZ). Numerous studies have attempted to identify the neural substrates of avolition and anhedonia in schizophrenia , but these studies have not produced much agreement. Deficits in various aspects of reinforcement processing have been observed in individuals with schizophrenia, but it is not exactly clear which of these deficits actually engender motivational impairments in SZ. The purpose of this chapter is to examine how various reinforcement-related behavioral and neural signals could contribute to motivational impairments in both schizophrenia and psychiatric illness, in general. In particular, we describe different aspects of the concept of expected value (EV), such as the distinction between the EV of stimuli and the expected value of actions, the acquisition of value versus the estimation of value, and the discounting of value as a consequence of time or effort required. We conclude that avolition and anhedonia in SZ are most commonly tied to aberrant signals for expected value, in the context of learning. We discuss implications for further research on the neural substrates of motivational impairments in psychiatric illness.

Mechanisms Underlying Motivational Deficits in Psychopathology: Similarities and Differences in Depression and Schizophrenia

Motivational and hedonic impairments are core aspects of a variety of types of psychopathology. These impairments cut across diagnostic categories and may be critical to understanding major aspects of the functional impairments accompanying psychopathology. Given the centrality of motivational and hedonic systems to psychopathology, the Research Domain Criteria (RDoC) initiative includes a "positive valence" systems domain that outlines a number of constructs that may be key to understanding the nature and mechanisms of motivational and hedonic impairments in psychopathology. These component constructs include initial responsiveness to reward, reward anticipation or expectancy, incentive or reinforcement learning, effort valuation, and action selection. Here, we review behavioral and neuroimaging studies providing evidence for impairments in these constructs in individuals with psychosis versus in individuals with depressive pathology. There are important differences in the nature of reward-related and hedonic deficits associated with psychosis versus depression that have major implications for our understanding of etiology and treatment development. In particular, the literature strongly suggests the presence of impairments in in-the-moment hedonics or "liking" in individuals with depressive pathology, particularly among those who experience anhedonia. Such deficits may propagate forward and contribute to impairments in other constructs that are dependent on hedonic responses, such as anticipation, learning, effort, and action selection. Such hedonic impairments could reflect alterations in dopamine and/or opioid signaling in the striatum related to depression or specifically to anhedonia in depressed populations. In contrast, the literature points to relatively intact in-the-moment hedonic processing in psychosis, but provides much evidence for impairments in other components involved in translating reward to action selection. Particularly, individuals with schizophrenia exhibit altered reward prediction and associated striatal and prefrontal activation, impaired reward learning, and impaired reward-modulated action selection.