Dopamine Modulates Adaptive Prediction Error Coding in the Human Midbrain and Striatum

Valence-dependent dopaminergic modulation during reversal learning in Parkinson's disease: A neurocomputational approach

Reinforcement learning, crucial for behavior in dynamic environments, is driven by rewards and punishments, modulated by dopamine (DA) changes. This study explores the dopaminergic system's influence on learning, particularly in Parkinson's disease (PD), where medication leads to impaired adaptability. Highlighting the role of tonic DA in signaling the valence of actions, this research investigates how DA affects response vigor and decision-making in PD. DA not only influences reward and punishment learning but also indicates the cognitive effort level and risk propensity in actions, which are essential for understanding and managing PD symptoms. In this work, we adapt our existing neurocomputational model of basal ganglia (BG) to simulate two reversal learning tasks proposed by Cools et al. We first optimized a Hebb rule for both probabilistic and deterministic reversal learning, conducted a sensitivity analysis (SA) on parameters related to DA effect, and compared performances between three groups: PD-ON, PD-OFF, and control subjects. In our deterministic task simulation, we explored switch error rates after unexpected task switches and found a U-shaped relationship between tonic DA levels and switch error frequency. Through SA, we classify these three groups. Then, assuming that the valence of the stimulus affects the tonic levels of DA, we were able to reproduce the results by Cools et al. As for the probabilistic task simulation, our results are in line with clinical data, showing similar trends with PD-ON, characterized by higher tonic DA levels that are correlated with increased difficulty in both acquisition and reversal tasks. Our study proposes a new hypothesis: valence, signaled by tonic DA levels, influences learning in PD, confirming the uncorrelation between phasic and tonic DA changes. This hypothesis challenges existing paradigms and opens new avenues for understanding cognitive processes in PD, particularly in reversal learning tasks.

Computational phenotyping of aberrant belief updating in individuals with schizotypal traits and schizophrenia

BACKGROUND

Psychotic experiences are thought to emerge from various interrelated patterns of disrupted belief updating, such as overestimating the reliability of sensory information and misjudging task volatility. Yet, these substrates have never been jointly addressed under one computational framework and it is not clear to what degree they reflect trait-like computational patterns.

METHODS

We introduced a novel hierarchical Bayesian model that describes how individuals simultaneously update their beliefs about the task volatility and noise in observation. We applied this model to data from a modified Predictive inference task in a test-retest study with healthy volunteers (N=45, 4 sessions) and examined the relationship between model parameters and schizotypal traits in a larger online sample (N = 437) and in a cohort of patients with schizophrenia (N = 100).

RESULTS

The interclass correlations were moderate to high for model parameters and excellent for averaged belief trajectories and precision-weighted learning rates estimated through hierarchical Bayesian inference. We found that uncertainty about the task volatility was related to schizotypal traits and to positive symptoms in patients, when learning to gain rewards. In contrast, negative symptoms in patients were associated with more rigid beliefs about observational noise, when learning to avoid losses.

CONCLUSION

These findings suggest that individuals with schizotypal traits across the psychosis continuum are less likely to learn or utilize higher-order statistical regularities of the environment and showcase the potential of clinically relevant computational phenotypes for differentiating symptom groups in a transdiagnostic manner.

The neurobiology of aesthetic chills: How bodily sensations shape emotional experiences

The phenomenon of aesthetic chills-shivers and goosebumps associated with either rewarding or threatening stimuli-offers a unique window into the brain basis of conscious reward because of their universal nature and simultaneous subjective and physical counterparts. Elucidating the neural mechanisms underlying aesthetic chills can reveal fundamental insights about emotion, consciousness, and the embodied mind. What is the precise timing and mechanism of bodily feedback in emotional experience? How are conscious feelings and motivations generated from interoceptive predictions? What is the role of uncertainty and precision signaling in shaping emotions? How does the brain distinguish and balance processing of rewards versus threats? We review neuroimaging evidence and highlight key questions for understanding how bodily sensations shape conscious feelings. This research stands to advance models of brain-body interactions shaping affect and may lead to novel nonpharmacological interventions for disorders of motivation and pleasure.

Risk-Taking Is Associated with Decreased Subjective Value Signals and Increased Prediction Error Signals in the Hot Columbia Card Task

It remains a pressing concern to understand how neural computations relate to risky decisions. However, most observations of brain-behavior relationships in the risk-taking domain lack a rigorous computational basis or fail to emulate of the dynamic, sequential nature of real-life risky decision-making. Recent advances emphasize the role of neural prediction error (PE) signals. We modeled, according to prospect theory, the choices of n = 43 human participants (33 females, 10 males) performing an EEG version of the hot Columbia Card Task, featuring rounds of sequential decisions between stopping (safe option) and continuing with increasing odds of a high loss (risky option). Single-trial regression EEG analyses yielded a subjective value signal at centroparietal (300-700 ms) and frontocentral (>800 ms) electrodes and in the delta band, as well as PE signals tied to the feedback-related negativity, P3a, and P3b, and in the theta band. Higher risk preference (total number of risky choices) was linked to attenuated subjective value signals but increased PE signals. Higher P3-like activity associated with the most positive PE in each round predicted stopping in the present round but not risk-taking in the subsequent round. Our findings indicate that decreased representation of decision values and increased sensitivity to winning despite low odds (positive PE) facilitate risky choices at the subject level. Strong neural responses when gains are least expected (the most positive PE on each round) adaptively contribute to safer choices at the trial-by-trial level but do not affect risky choice at the round-by-round level.

Reinforcement feedback impairs locomotor adaptation and retention

Introduction

Locomotor adaptation is a motor learning process used to alter spatiotemporal elements of walking that are driven by prediction errors, a discrepancy between the expected and actual outcomes of our actions. Sensory and reward prediction errors are two different types of prediction errors that can facilitate locomotor adaptation. Reward and punishment feedback generate reward prediction errors but have demonstrated mixed effects on upper extremity motor learning, with punishment enhancing adaptation, and reward supporting motor memory. However, an in-depth behavioral analysis of these distinct forms of feedback is sparse in locomotor tasks.

Methods

For this study, three groups of healthy young adults were divided into distinct feedback groups [Supervised, Reward, Punishment] and performed a novel locomotor adaptation task where each participant adapted their knee flexion to 30 degrees greater than baseline, guided by visual supervised or reinforcement feedback (Adaptation). Participants were then asked to recall the new walking pattern without feedback (Retention) and after a washout period with feedback restored (Savings).

Results

We found that all groups learned the adaptation task with external feedback. However, contrary to our initial hypothesis, enhancing sensory feedback with a visual representation of the knee angle (Supervised) accelerated the rate of learning and short-term retention in comparison to monetary reinforcement feedback. Reward and Punishment displayed similar rates of adaptation, short-term retention, and savings, suggesting both types of reinforcement feedback work similarly in locomotor adaptation. Moreover, all feedback enhanced the aftereffect of locomotor task indicating changes to implicit learning.

Discussion

These results demonstrate the multi-faceted nature of reinforcement feedback on locomotor adaptation and demonstrate the possible different neural substrates that underly reward and sensory prediction errors during different motor tasks.

The Placebo Effect in Psychosis: Why It Matters and How to Measure It

Psychosis is characterized by unusual percepts and beliefs in the form of hallucinations and delusions. Antipsychotic medication, the primary treatment for psychosis, is often ineffective and accompanied by severe side effects, but research has not identified an effective alternative in several decades. One reason that clinical trials fail is that patients with psychosis tend to show a significant therapeutic response to inert control treatments, known as the placebo effect, which makes it difficult to distinguish drug effects from placebo effects. Conversely, in clinical practice, a strong placebo effect may be useful because it could enhance the overall treatment response. Identifying factors that predict large placebo effects could improve the future outlook of psychosis treatment. Biomarkers of the placebo effect have already been suggested in pain and depression, but not in psychosis. Quantifying markers of the placebo effect would have the potential to predict placebo effects in psychosis clinical trials. Furthermore, the placebo effect and psychosis may represent a shared neurocognitive mechanism in which prior beliefs are weighted against new sensory information to make inferences about reality. Examining this overlap could reveal new insights into the mechanisms underlying psychosis and indicate novel treatment targets. We provide a narrative review of the importance of the placebo effect in psychosis and propose a novel method to assess it.

Liraglutide restores impaired associative learning in individuals with obesity

Survival under selective pressure is driven by the ability of our brain to use sensory information to our advantage to control physiological needs. To that end, neural circuits receive and integrate external environmental cues and internal metabolic signals to form learned sensory associations, consequently motivating and adapting our behaviour. The dopaminergic midbrain plays a crucial role in learning adaptive behaviour and is particularly sensitive to peripheral metabolic signals, including intestinal peptides, such as glucagon-like peptide 1 (GLP-1). In a single-blinded, randomized, controlled, crossover basic human functional magnetic resonance imaging study relying on a computational model of the adaptive learning process underlying behavioural responses, we show that adaptive learning is reduced when metabolic sensing is impaired in obesity, as indexed by reduced insulin sensitivity (participants: N = 30 with normal insulin sensitivity; N = 24 with impaired insulin sensitivity). Treatment with the GLP-1 receptor agonist liraglutide normalizes impaired learning of sensory associations in men and women with obesity. Collectively, our findings reveal that GLP-1 receptor activation modulates associative learning in people with obesity via its central effects within the mesoaccumbens pathway. These findings provide evidence for how metabolic signals can act as neuromodulators to adapt our behaviour to our body's internal state and how GLP-1 receptor agonists work in clinics.

Blocking D2/D3 dopamine receptors in male participants increases volatility of beliefs when learning to trust others

The ability to learn about other people is crucial for human social functioning. Dopamine has been proposed to regulate the precision of beliefs, but direct behavioural evidence of this is lacking. In this study, we investigate how a high dose of the D2/D3 dopamine receptor antagonist sulpiride impacts learning about other people's prosocial attitudes in a repeated Trust game. Using a Bayesian model of belief updating, we show that in a sample of 76 male participants sulpiride increases the volatility of beliefs, which leads to higher precision weights on prediction errors. This effect is driven by participants with genetically conferred higher dopamine availability (Taq1a polymorphism) and remains even after controlling for working memory performance. Higher precision weights are reflected in higher reciprocal behaviour in the repeated Trust game but not in single-round Trust games. Our data provide evidence that the D2 receptors are pivotal in regulating prediction error-driven belief updating in a social context.

Clinical response to treatment with a partial dopamine agonist is related to changes in reward processing

Aberrant neuronal coding of reward processing has been linked to psychosis. It remains unresolved how treatment with a partial dopamine agonist affects reward processing, and whether treatment affects reward processing differently in patients responding and not responding to treatment. Here, 33 antipsychotic-naïve psychosis patients and 33 matched healthy controls underwent functional magnetic resonance imaging before and after patients received aripiprazole monotherapy for six weeks. Processing of motivational salient events and negative outcome evaluation (NOE) was examined using a monetary incentive delay task. Psychopathology was assessed with the Positive and Negative Syndrome Scale, and responders were identified by having ≥30% reduction in positive symptoms (N=21). At baseline, patients displayed an increased NOE signal in the caudate and dorsolateral prefrontal cortex compared to healthy controls. In the caudate, the NOE signal was normalized at follow-up, and normalization was driven by responders. In responders only, there was a significant improvement in the motivational salience signal in the caudate at follow-up. Motivational salience and NOE signals in the caudate may be associated with a dopaminergic mechanism in patients characterized as responders which may not be the case in non-responders. Likewise, non-dopaminergic mechanism may underly abnormal NOE processing in dorsolateral prefrontal cortex.

Insular and Striatal Correlates of Uncertain Risky Reward Pursuit in Schizophrenia

BACKGROUND AND HYPOTHESIS

Risk-taking in specific contexts can be beneficial, leading to rewarding outcomes. Schizophrenia is associated with disadvantageous decision-making, as subjects pursue uncertain risky rewards less than controls. However, it is unclear whether this behavior is associated with more risk sensitivity or less reward incentivization. Matching on demographics and intelligence quotient (IQ), we determined whether risk-taking was more associated with brain activation in regions affiliated with risk evaluation or reward processing.

STUDY DESIGN

Subjects (30 schizophrenia/schizoaffective disorder, 30 controls) completed a modified, fMRI Balloon Analogue Risk Task. Brain activation was modeled during decisions to pursue risky rewards and parametrically modeled according to risk level.

STUDY RESULTS

The schizophrenia group exhibited less risky-reward pursuit despite previous adverse outcomes (Average Explosions; F(1,59) = 4.06, P = .048) but the comparable point at which risk-taking was volitionally discontinued (Adjusted Pumps; F(1,59) = 2.65, P = .11). Less activation was found in schizophrenia via whole brain and region of interest (ROI) analyses in the right (F(1,59) = 14.91, P < 0.001) and left (F(1,59) = 16.34, P < 0.001) nucleus accumbens (NAcc) during decisions to pursue rewards relative to riskiness. Risk-taking correlated with IQ in schizophrenia, but not controls. Path analyses of average ROI activation revealed less statistically determined influence of anterior insula upon dorsal anterior cingulate bilaterally (left: χ2 = 12.73, P < .001; right: χ2 = 9.54, P = .002) during risky reward pursuit in schizophrenia.

CONCLUSIONS

NAcc activation in schizophrenia varied less according to the relative riskiness of uncertain rewards compared to controls, suggesting aberrations in reward processing. The lack of activation differences in other regions suggests similar risk evaluation. Less insular influence on the anterior cingulate may relate to attenuated salience attribution or inability for risk-related brain region collaboration to sufficiently perceive situational risk.

l-DOPA and oxytocin influence the neurocomputational mechanisms of self-benefitting and prosocial reinforcement learning

Humans learn through reinforcement, particularly when outcomes are unexpected. Recent research suggests similar mechanisms drive how we learn to benefit other people, that is, how we learn to be prosocial. Yet the neurochemical mechanisms underlying such prosocial computations remain poorly understood. Here, we investigated whether pharmacological manipulation of oxytocin and dopamine influence the neurocomputational mechanisms underlying self-benefitting and prosocial reinforcement learning. Using a double-blind placebo-controlled cross-over design, we administered intranasal oxytocin (24 IU), dopamine precursor l-DOPA (100 mg + 25 mg carbidopa), or placebo over three sessions. Participants performed a probabilistic reinforcement learning task with potential rewards for themselves, another participant, or no one, during functional magnetic resonance imaging. Computational models of reinforcement learning were used to calculate prediction errors (PEs) and learning rates. Participants behavior was best explained by a model with different learning rates for each recipient, but these were unaffected by either drug. On the neural level, however, both drugs blunted PE signaling in the ventral striatum and led to negative signaling of PEs in the anterior mid-cingulate cortex, dorsolateral prefrontal cortex, inferior parietal gyrus, and precentral gyrus, compared to placebo, and regardless of recipient. Oxytocin (versus placebo) administration was additionally associated with opposing tracking of self-benefitting versus prosocial PEs in dorsal anterior cingulate cortex, insula and superior temporal gyrus. These findings suggest that both l-DOPA and oxytocin induce a context-independent shift from positive towards negative tracking of PEs during learning. Moreover, oxytocin may have opposing effects on PE signaling when learning to benefit oneself versus another.

Habitual daily intake of a sweet and fatty snack modulates reward processing in humans

Western diets rich in fat and sugar promote excess calorie intake and weight gain; however, the underlying mechanisms are unclear. Despite a well-documented association between obesity and altered brain dopamine function, it remains elusive whether these alterations are (1) pre-existing, increasing the individual susceptibility to weight gain, (2) secondary to obesity, or (3) directly attributable to repeated exposure to western diet. To close this gap, we performed a randomized, controlled study (NCT05574660) with normal-weight participants exposed to a high-fat/high-sugar snack or a low-fat/low-sugar snack for 8 weeks in addition to their regular diet. The high-fat/high-sugar intervention decreased the preference for low-fat food while increasing brain response to food and associative learning independent of food cues or reward. These alterations were independent of changes in body weight and metabolic parameters, indicating a direct effect of high-fat, high-sugar foods on neurobehavioral adaptations that may increase the risk for overeating and weight gain.

Pre-clinical evidence that methylphenidate increases motivation and/or reward preference to search for high value rewards

Methylphenidate is a stimulant used to treat attention deficit and hyperactivity disorder (ADHD). In the last decade, illicit use of methylphenidate has increased among healthy young adults, who consume the drug under the assumption that it will improve cognitive performance. However, the studies that aimed to assess the methylphenidate effects on memory are not consistent. Here, we tested whether the effect of methylphenidate on a spatial memory task can be explained as a motivational and/or a reward effect. We tested the effects of acute and chronic i.p. administration of 0.3, 1 or 3 mg/kg of methylphenidate on motivation, learning and memory by using the 8-arm radial maze task. Adult male Wistar rats learned that 3 of the 8 arms of the maze were consistently baited with 1, 3, or 6 sucrose pellets, and the number of entries and reentries into reinforced and non-reinforced arms of the maze were scored. Neither acute nor chronic (20 days) methylphenidate treatment affected the number of entries in the non-baited arms. However, chronic, but not acute, 1-3 mg/kg methylphenidate increased the number of reentries in the higher reward arms, which suggests a motivational/rewarding effect rather than a working memory deficit. In agreement with this hypothesis, the methylphenidate treatment also decreased the approach latency to the higher reward arms, increased the approach latency to the low reward arm, and increased the time spent in the high, but not low, reward arm. These findings suggest that methylphenidate may act more as a motivational enhancer rather than a cognitive enhancer in healthy people.

Belief Updating in Subclinical and Clinical Delusions

Background and Hypothesis

Current frameworks propose that delusions result from aberrant belief updating due to altered prediction error (PE) signaling and misestimation of environmental volatility. We aimed to investigate whether behavioral and neural signatures of belief updating are specifically related to the presence of delusions or generally associated with manifest schizophrenia.

Methods

Our cross-sectional design includes human participants (n[female/male] = 66[25/41]), stratified into four groups: healthy participants with minimal (n = 22) or strong delusional-like ideation (n = 18), and participants with diagnosed schizophrenia with minimal (n = 13) or strong delusions (n = 13), resulting in a 2 × 2 design, which allows to test for the effects of delusion and diagnosis. Participants performed a reversal learning task with stable and volatile task contingencies during fMRI scanning. We formalized learning with a hierarchical Gaussian filter model and conducted model-based fMRI analysis regarding beliefs of outcome uncertainty and volatility, precision-weighted PEs of the outcome- and the volatility-belief.

Results

Patients with schizophrenia as compared to healthy controls showed lower accuracy and heightened choice switching, while delusional ideation did not affect these measures. Participants with delusions showed increased precision-weighted PE-related neural activation in fronto-striatal regions. People with diagnosed schizophrenia overestimated environmental volatility and showed an attenuated neural representation of volatility in the anterior insula, medial frontal and angular gyrus.

Conclusions

Delusional beliefs are associated with altered striatal PE-signals. Juxtaposing, the potentially unsettling belief that the environment is constantly changing and weaker neural encoding of this subjective volatility seems to be associated with manifest schizophrenia, but not with the presence of delusional ideation.

The catecholamine precursor Tyrosine reduces autonomic arousal and decreases decision thresholds in reinforcement learning and temporal discounting

Supplementation with the catecholamine precursor L-Tyrosine might enhance cognitive performance, but overall findings are mixed. Here, we investigate the effect of a single dose of tyrosine (2g) vs. placebo on two catecholamine-dependent trans-diagnostic traits: model-based control during reinforcement learning (2-step task) and temporal discounting, using a double-blind, placebo-controlled, within-subject design (n = 28 healthy male participants). We leveraged drift diffusion models in a hierarchical Bayesian framework to jointly model participants' choices and response times (RTS) in both tasks. Furthermore, comprehensive autonomic monitoring (heart rate, heart rate variability, pupillometry, spontaneous eye blink rate) was performed both pre- and post-supplementation, to explore potential physiological effects of supplementation. Across tasks, tyrosine consistently reduced participants' RTs without deteriorating task-performance. Diffusion modeling linked this effect to attenuated decision-thresholds in both tasks and further revealed increased model-based control (2-step task) and (if anything) attenuated temporal discounting. On the physiological level, participants' pupil dilation was predictive of the individual degree of temporal discounting. Tyrosine supplementation reduced physiological arousal as revealed by increases in pupil dilation variability and reductions in heart rate. Supplementation-related changes in physiological arousal predicted individual changes in temporal discounting. Our findings provide first evidence that tyrosine supplementation might impact psychophysiological parameters, and suggest that modeling approaches based on sequential sampling models can yield novel insights into latent cognitive processes modulated by amino-acid supplementation.

Adaptive coding occurs in object categorization and may not be associated with schizotypal personality traits

Processing more likely inputs with higher sensitivity (adaptive coding) enables the brain to represent the large range of inputs coming in from the world. Healthy individuals high in schizotypy show reduced adaptive coding in the reward domain but it is an open question whether these deficits extend to non-motivational domains, such as object categorization. Here, we develop a novel variant of a classic task to test range adaptation for face/house categorization in healthy participants on the psychosis spectrum. In each trial of this task, participants decide whether a presented image is a face or a house. Images vary on a face-house continuum and appear in both wide and narrow range blocks. The wide range block includes most of the face-house continuum (2.50-97.5% face), while the narrow range blocks limit inputs to a smaller section of the continuum (27.5-72.5% face). Adaptive coding corresponds to better performance for the overlapping smaller section of the continuum in the narrow range than in the wide range block. We find that participants show efficient use of the range in this task, with more accurate responses in the overlapping section for the narrow range blocks relative to the wide range blocks. However, we find little evidence that range adaptation in our object categorization task is reduced in healthy individuals scoring high on schizotypy. Thus, reduced range adaptation may not be a domain-general feature of schizotypy.

Uncertainty-guided learning with scaled prediction errors in the basal ganglia

To accurately predict rewards associated with states or actions, the variability of observations has to be taken into account. In particular, when the observations are noisy, the individual rewards should have less influence on tracking of average reward, and the estimate of the mean reward should be updated to a smaller extent after each observation. However, it is not known how the magnitude of the observation noise might be tracked and used to control prediction updates in the brain reward system. Here, we introduce a new model that uses simple, tractable learning rules that track the mean and standard deviation of reward, and leverages prediction errors scaled by uncertainty as the central feedback signal. We show that the new model has an advantage over conventional reinforcement learning models in a value tracking task, and approaches a theoretic limit of performance provided by the Kalman filter. Further, we propose a possible biological implementation of the model in the basal ganglia circuit. In the proposed network, dopaminergic neurons encode reward prediction errors scaled by standard deviation of rewards. We show that such scaling may arise if the striatal neurons learn the standard deviation of rewards and modulate the activity of dopaminergic neurons. The model is consistent with experimental findings concerning dopamine prediction error scaling relative to reward magnitude, and with many features of striatal plasticity. Our results span across the levels of implementation, algorithm, and computation, and might have important implications for understanding the dopaminergic prediction error signal and its relation to adaptive and effective learning.

Phasic Dopamine Changes and Hebbian Mechanisms during Probabilistic Reversal Learning in Striatal Circuits: A Computational Study

Cognitive flexibility is essential to modify our behavior in a non-stationary environment and is often explored by reversal learning tasks. The basal ganglia (BG) dopaminergic system, under a top-down control of the pre-frontal cortex, is known to be involved in flexible action selection through reinforcement learning. However, how adaptive dopamine changes regulate this process and learning mechanisms for training the striatal synapses remain open questions. The current study uses a neurocomputational model of the BG, based on dopamine-dependent direct (Go) and indirect (NoGo) pathways, to investigate reinforcement learning in a probabilistic environment through a task that associates different stimuli to different actions. Here, we investigated: the efficacy of several versions of the Hebb rule, based on covariance between pre- and post-synaptic neurons, as well as the required control in phasic dopamine changes crucial to achieving a proper reversal learning. Furthermore, an original mechanism for modulating the phasic dopamine changes is proposed, assuming that the expected reward probability is coded by the activity of the winner Go neuron before a reward/punishment takes place. Simulations show that this original formulation for an automatic phasic dopamine control allows the achievement of a good flexible reversal even in difficult conditions. The current outcomes may contribute to understanding the mechanisms for active control of dopamine changes during flexible behavior. In perspective, it may be applied in neuropsychiatric or neurological disorders, such as Parkinson’s or schizophrenia, in which reinforcement learning is impaired.

Dopaminergic challenge dissociates learning from primary versus secondary sources of information

Some theories of human cultural evolution posit that humans have social-specific learning mechanisms that are adaptive specialisations moulded by natural selection to cope with the pressures of group living. However, the existence of neurochemical pathways that are specialised for learning from social information and individual experience is widely debated. Cognitive neuroscientific studies present mixed evidence for social-specific learning mechanisms: some studies find dissociable neural correlates for social and individual learning, whereas others find the same brain areas and, dopamine-mediated, computations involved in both. Here, we demonstrate that, like individual learning, social learning is modulated by the dopamine D2 receptor antagonist haloperidol when social information is the primary learning source, but not when it comprises a secondary, additional element. Two groups (total N = 43) completed a decision-making task which required primary learning, from own experience, and secondary learning from an additional source. For one group, the primary source was social, and secondary was individual; for the other group this was reversed. Haloperidol affected primary learning irrespective of social/individual nature, with no effect on learning from the secondary source. Thus, we illustrate that dopaminergic mechanisms underpinning learning can be dissociated along a primary-secondary but not a social-individual axis. These results resolve conflict in the literature and support an expanding field showing that, rather than being specialised for particular inputs, neurochemical pathways in the human brain can process both social and non-social cues and arbitrate between the two depending upon which cue is primarily relevant for the task at hand.

Acute stress blunts prediction error signals in the dorsal striatum during reinforcement learning

Acute stress is pervasive in everyday modern life and is thought to affect how people make choices and learn from them. Reinforcement learning, which implicates learning from the unexpected rewarding and punishing outcomes of our choices (i.e., prediction errors), is critical for adjusted behaviour and seems to be affected by acute stress. However, the neural mechanisms by which acute stress disrupts this type of learning are still poorly understood. Here, we investigate whether and how acute stress blunts neural signalling of prediction errors during reinforcement learning using model-based functional magnetic resonance imaging. Male participants completed a well-established reinforcement-learning task involving monetary gains and losses whilst under stress and control conditions. Acute stress impaired participants’ (n = 23) behavioural performance towards obtaining monetary gains (p < 0.001), but not towards avoiding losses (p = 0.57). Importantly, acute stress blunted signalling of prediction errors during gain and loss trials in the dorsal striatum (p = 0.040) — with subsidiary analyses suggesting that acute stress preferentially blunted signalling of positive prediction errors. Our results thus reveal a neurocomputational mechanism by which acute stress may impair reward learning.

Range-Adaptive Value Representation in Different Stages of Schizophrenia: A Proof of Concept Study

Amotivation is related to value representation. A comprehensive account of amotivation requires a mechanistic understanding of how the brain exploits external information to represent value. To achieve maximal value discriminability, brain valuation system will dynamically adapt its coding sensitivity to the range of values available in any given condition, so-called range adaptive coding. We administered an experimental task to 30 patients with chronic schizophrenia (C-SCZ), 30 first-episode schizophrenia (FE-SCZ), 34 individuals with high social anhedonia (HSoA), and their paired controls to assess range adaptation ability. C-SCZ patients exhibited over-adaptation and their performances were negatively correlated with avolition symptoms and positive symptoms and positively correlated with blunted-affect symptoms and self-reported consummatory interpersonal pleasure scores, though the results were non-significant. FE-SCZ patients exhibited reduced adaptation, which was significantly and negatively correlated with avolition symptoms and positively correlated with the overall proportion of choosing to exert more effort. Although HSoA participants exhibited comparable range adaptation to controls, their performances were significantly and negatively correlated with the proportion of choosing to exert more effort under the lowest value condition. Our results suggest that different stages of schizophrenia spectrum showed distinct range adaptation patterns. Range adaptation impairments may index a possible underlying mechanism for amotivation symptoms in FE-SCZ and more complicated and pervasive effects on clinical symptoms in C-SCZ.

Association of Brain Reward Response With Body Mass Index and Ventral Striatal-Hypothalamic Circuitry Among Young Women With Eating Disorders

IMPORTANCE

Eating disorders are severe psychiatric disorders; however, disease models that cross subtypes and integrate behavior and neurobiologic factors are lacking.

OBJECTIVE

To assess brain response during unexpected receipt or omission of a salient sweet stimulus across a large sample of individuals with eating disorders and healthy controls and test for evidence of whether this brain response is associated with the ventral striatal-hypothalamic circuitry, which has been associated with food intake control, and whether salient stimulus response and eating disorder related behaviors are associated.

DESIGN, SETTING, AND PARTICIPANTS

In this cross-sectional functional brain imaging study, young adults across the eating disorder spectrum were matched with healthy controls at a university brain imaging facility and eating disorder treatment program. During a sucrose taste classic conditioning paradigm, violations of learned associations between conditioned visual and unconditioned taste stimuli evoked the dopamine-related prediction error. Dynamic effective connectivity during expected sweet taste receipt was studied to investigate hierarchical brain activation between food intake relevant brain regions. The study was conducted from June 2014 to November 2019. Data were analyzed from December 2019 to February 2020.

MAIN OUTCOMES AND MEASURES

Prediction error brain reward response across insula and striatum; dynamic effective connectivity between hypothalamus and ventral striatum; and demographic and behavior variables and their correlations with prediction error brain response and connectivity edge coefficients.

RESULTS

Of 317 female participants (197 with eating disorders and 120 healthy controls), the mean (SD) age was 23.8 (5.6) years and mean (SD) body mass index was 20.8 (5.4). Prediction error response was elevated in participants with anorexia nervosa (Wilks λ, 0.843; P = .001) and in participants with eating disorders inversely correlated with body mass index (left nucleus accumbens: r = -0.291; 95% CI, -0.413 to -0.167; P < .001; right dorsal anterior insula: r = -0.228; 95% CI, -0.366 to -0.089; P = .001), eating disorder inventory-3 binge eating tendency (left nucleus accumbens: r = -0.207; 95% CI, -0.333 to -0.073; P = .004; right dorsal anterior insula: r = -0.220; 95% CI, -0.354 to -0.073; P = .002), and trait anxiety (left nucleus accumbens: r = -0.148; 95% CI, -0.288 to -0.003; P = .04; right dorsal anterior insula: r = -0.221; 95% CI, -0.357 to -0.076; P = .002). Ventral striatal to hypothalamus directed connectivity was positively correlated with ventral striatal prediction error in eating disorders (r = 0.189; 95% CI, 0.045-0.324; P = .01) and negatively correlated with feeling out of control after eating (right side: r = -0.328; 95% CI, -0.480 to -0.164; P < .001; left side: r = -0.297; 95% CI, -0.439 to -0.142; P = .001).

CONCLUSIONS AND RELEVANCE

The results of this cross-sectional imaging study support that body mass index modulates prediction error and food intake control circuitry in the brain. Once altered, this circuitry may reinforce eating disorder behaviors when paired with behavioral traits associated with overeating or undereating.

Precision and the Bayesian brain

Scientific thinking about the minds of humans and other animals has been transformed by the idea that the brain is Bayesian. A cornerstone of this idea is that agents set the balance between prior knowledge and incoming evidence based on how reliable or 'precise' these different sources of information are - lending the most weight to that which is most reliable. This concept of precision has crept into several branches of cognitive science and is a lynchpin of emerging ideas in computational psychiatry - where unusual beliefs or experiences are explained as abnormalities in how the brain estimates precision. But what precisely is precision? In this Primer we explain how precision has found its way into classic and contemporary models of perception, learning, self-awareness, and social interaction. We also chart how ideas around precision are beginning to change in radical ways, meaning we must get more precise about how precision works.

Anhedonia as a central factor in depression: Neural mechanisms revealed from preclinical to clinical evidence

Anhedonia is one of the core symptoms of major depressive disorder (MDD), which is often inadequately treated by traditional antidepressants. The modern framework of anhedonia extends the definition from impaired consummatory pleasure or interest in rewards to a broad spectrum of deficits that impact functions such as reward anticipation, approach motivation, effort expenditure, reward valuation, expectation, and reward-cue association learning. Substantial preclinical and clinical research has explored the neural basis of reward deficits in the context of depression, and has implicated mesocorticolimbic reward circuitry comprising the nucleus accumbens, ventral pallidum, ventral tegmental area, amygdala, hippocampus, anterior cingulate, insula, orbitofrontal cortex, and other prefrontal cortex regions. Dopamine modulates several reward facets including anticipation, motivation, effort, and learning. As well, serotonin, norepinephrine, opioids, glutamate, Gamma aminobutyric acid (GABA), and acetylcholine are also involved in anhedonia, and medications targeting these systems may also potentially normalize reward processing in depression. Unfortunately, whereas reward anticipation and reward outcome are extensively explored by both preclinical and clinical studies, translational gaps remain in reward motivation, effort, valuation, and learning, where clinical neuroimaging studies are in the early stages. This review aims to synthesize the neurobiological mechanisms underlying anhedonia in MDD uncovered by preclinical and clinical research. The translational difficulties in studying the neural basis of reward are also discussed.

Single-Dimensional Human Brain Signals for Two-Dimensional Economic Choice Options

Rewarding choice options typically contain multiple components, but neural signals in single brain voxels are scalar and primarily vary up or down. In a previous study, we had designed reward bundles that contained the same two milkshakes with independently set amounts; we had used psychophysics and rigorous economic concepts to estimate two-dimensional choice indifference curves (ICs) that represented revealed stochastic preferences for these bundles in a systematic, integrated manner. All bundles on the same ICs were equally revealed preferred (and thus had same utility, as inferred from choice indifference); bundles on higher ICs (higher utility) were preferred to bundles on lower ICs (lower utility). In the current study, we used the established behavior for testing with functional magnetic resonance imaging (fMRI). We now demonstrate neural responses in reward-related brain structures of human female and male participants, including striatum, midbrain, and medial orbitofrontal cortex (mid-OFC) that followed the characteristic pattern of ICs: similar responses along ICs (same utility despite different bundle composition), but monotonic change across ICs (different utility). Thus, these brain structures integrated multiple reward components into a scalar signal, well beyond the known subjective value coding of single-component rewards.SIGNIFICANCE STATEMENT Rewards have several components, like the taste and size of an apple, but it is unclear how each component contributes to the overall value of the reward. While choice indifference curves (ICs) of economic theory provide behavioral approaches to this question, it is unclear whether brain responses capture the preference and utility integrated from multiple components. We report activations in striatum, midbrain, and orbitofrontal cortex (OFC) that follow choice ICs representing behavioral preferences over and above variations of individual reward components. In addition, the concept-driven approach encourages future studies on natural, multicomponent rewards that are prone to irrational choice of normal and brain-damaged individuals.

The effect of risperidone on reward-related brain activity is robust to drug-induced vascular changes

Dopamine (DA) mediated brain activity is intimately linked to reward‐driven cerebral responses, while aberrant reward processing has been implicated in several psychiatric disorders. fMRI has been a valuable tool in understanding the mechanism by which DA modulators alter reward‐driven responses and how they may exert their therapeutic effect. However, the potential effects of a pharmacological compound on aspects of neurovascular coupling may cloud the interpretability of the BOLD contrast. Here, we assess the effects of risperidone on reward driven BOLD signals produced by reward anticipation and outcome, while attempting to control for potential drug effects on regional cerebral blood flow (CBF) and cerebrovascular reactivity (CVR). Healthy male volunteers (n = 21) each received a single oral dose of either 0.5 mg, 2 mg of risperidone or placebo in a double‐blind, placebo‐controlled, randomised, three‐period cross‐over study design. Participants underwent fMRI scanning while performing the widely used Monetary Incentive Delay (MID) task to assess drug impact on reward function. Measures of CBF (Arterial Spin Labelling) and breath‐hold challenge induced BOLD signal changes (as a proxy for CVR) were also acquired and included as covariates. Risperidone produced divergent, dose‐dependent effects on separate phases of reward processing, even after controlling for potential nonneuronal influences on the BOLD signal. These data suggest the D2 antagonist risperidone has a wide‐ranging influence on DA‐mediated reward function independent of nonneuronal factors. We also illustrate that assessment of potential vascular confounds on the BOLD signal may be advantageous when investigating CNS drug action and advocate for the inclusion of these additional measures into future study designs.

Dopamine, Prediction Error and Beyond

A large body of work has linked dopaminergic signaling to learning and reward processing. It stresses the role of dopamine in reward prediction error signaling, a key neural signal that allows us to learn from past experiences, and that facilitates optimal choice behavior. Latterly, it has become clear that dopamine does not merely code prediction error size but also signals the difference between the expected value of rewards, and the value of rewards actually received, which is obtained through the integration of reward attributes such as the type, amount, probability and delay. More recent work has posited a role of dopamine in learning beyond rewards. These theories suggest that dopamine codes absolute or unsigned prediction errors, playing a key role in how the brain models associative regularities within its environment, while incorporating critical information about the reliability of those regularities. Work is emerging supporting this perspective and, it has inspired theoretical models of how certain forms of mental pathology may emerge in relation to dopamine function. Such pathology is frequently related to disturbed inferences leading to altered internal models of the environment. Thus, it is critical to understand the role of dopamine in error-related learning and inference.

Precision weighting of cortical unsigned prediction error signals benefits learning, is mediated by dopamine, and is impaired in psychosis

Recent theories of cortical function construe the brain as performing hierarchical Bayesian inference. According to these theories, the precision of prediction errors plays a key role in learning and decision-making, is controlled by dopamine and contributes to the pathogenesis of psychosis. To test these hypotheses, we studied learning with variable outcome-precision in healthy individuals after dopaminergic modulation with a placebo, a dopamine receptor agonist bromocriptine or a dopamine receptor antagonist sulpiride (dopamine study n = 59) and in patients with early psychosis (psychosis study n = 74: 20 participants with first-episode psychosis, 30 healthy controls and 24 participants with at-risk mental state attenuated psychotic symptoms). Behavioural computational modelling indicated that precision weighting of prediction errors benefits learning in health and is impaired in psychosis. FMRI revealed coding of unsigned prediction errors, which signal surprise, relative to their precision in superior frontal cortex (replicated across studies, combined n = 133), which was perturbed by dopaminergic modulation, impaired in psychosis and associated with task performance and schizotypy (schizotypy correlation in 86 healthy volunteers). In contrast to our previous work, we did not observe significant precision-weighting of signed prediction errors, which signal valence, in the midbrain and ventral striatum in the healthy controls (or patients) in the psychosis study. We conclude that healthy people, but not patients with first-episode psychosis, take into account the precision of the environment when updating beliefs. Precision weighting of cortical prediction error signals is a key mechanism through which dopamine modulates inference and contributes to the pathogenesis of psychosis.

A continuum hypothesis of psychotomimetic rapid antidepressants

Ketamine, classical psychedelics and sleep deprivation are associated with rapid effects on depression. Interestingly, these interventions also have common psychotomimetic actions, mirroring aspects of psychosis such as an altered sense of self, perceptual distortions and distorted thinking. This raises the question whether these interventions might be acute antidepressants through the same mechanisms that underlie some of their psychotomimetic effects. That is, perhaps some symptoms of depression can be understood as occupying the opposite end of a spectrum where elements of psychosis can be found on the other side. This review aims at reviewing the evidence underlying a proposed continuum hypothesis of psychotomimetic rapid antidepressants, suggesting that a range of psychotomimetic interventions are also acute antidepressants as well as trying to explain these common features in a hierarchical predictive coding framework, where we hypothesise that these interventions share a common mechanism by increasing the flexibility of prior expectations. Neurobiological mechanisms at play and the role of different neuromodulatory systems affected by these interventions and their role in controlling the precision of prior expectations and new sensory evidence will be reviewed. The proposed hypothesis will also be discussed in relation to other existing theories of antidepressants. We also suggest a number of novel experiments to test the hypothesis and highlight research areas that could provide further insights, in the hope to better understand the acute antidepressant properties of these interventions.

Acute stress impairs reward learning in men

Acute stress is ubiquitous in everyday life, but the extent to which acute stress affects how people learn from the outcomes of their choices is still poorly understood. Here, we investigate how acute stress impacts reward and punishment learning in men using a reinforcement-learning task. Sixty-two male participants performed the task whilst under stress and control conditions. We observed that acute stress impaired participants' choice performance towards monetary gains, but not losses. To unravel the mechanism(s) underlying such impairment, we fitted a reinforcement-learning model to participants' trial-by-trial choices. Computational modeling indicated that under acute stress participants learned more slowly from positive prediction errors - when the outcomes were better than expected - consistent with stress-induced dopamine disruptions. Such mechanistic understanding of how acute stress impairs reward learning is particularly important given the pervasiveness of stress in our daily life and the impact that stress can have on our wellbeing and mental health.

Using pharmacological manipulations to study the role of dopamine in human reward functioning: A review of studies in healthy adults

Dopamine (DA) plays a key role in reward processing and is implicated in psychological disorders such as depression, substance use, and schizophrenia. The role of DA in reward processing is an area of highly active research. One approach to this question is drug challenge studies with drugs known to alter DA function. These studies provide good experimental control and can be performed in parallel in laboratory animals and humans. This review aimed to summarize results of studies using pharmacological manipulations of DA in healthy adults. 'Reward' is a complex process, so we separated 'phases' of reward, including anticipation, evaluation of cost and benefits of upcoming reward, execution of actions to obtain reward, pleasure in response to receiving a reward, and reward learning. Results indicated that i) DAergic drugs have different effects on different phases of reward; ii) the relationship between DA and reward functioning appears unlikely to be linear; iii) our ability to detect the effects of DAergic drugs varies depending on whether subjective, behavioral, imaging measures are used.

The Role of the Human Auditory Corticostriatal Network in Speech Learning

We establish a mechanistic account of how the mature human brain functionally reorganizes to acquire and represent new speech sounds. Native speakers of English learned to categorize Mandarin lexical tone categories produced by multiple talkers using trial-by-trial feedback. We hypothesized that the corticostriatal system is a key intermediary in mediating temporal lobe plasticity and the acquisition of new speech categories in adulthood. We conducted a functional magnetic resonance imaging experiment in which participants underwent a sound-to-category mapping task. Diffusion tensor imaging data were collected, and probabilistic fiber tracking analysis was employed to assay the auditory corticostriatal pathways. Multivariate pattern analysis showed that talker-invariant novel tone category representations emerged in the left superior temporal gyrus (LSTG) within a few hundred training trials. Univariate analysis showed that the putamen, a subregion of the striatum, was sensitive to positive feedback in correctly categorized trials. With learning, functional coupling between the putamen and LSTG increased during error processing. Furthermore, fiber tractography demonstrated robust structural connectivity between the feedback-sensitive striatal regions and the LSTG regions that represent the newly learned tone categories. Our convergent findings highlight a critical role for the auditory corticostriatal circuitry in mediating the acquisition of new speech categories.

Effects of serotonin and dopamine depletion on neural prediction computations during social learning

We have previously shown that individuals with high depression scores demonstrate impaired behavioral and neural responses during social learning. Given that depression is associated with altered dopamine (DA) and serotonin (5-HT) functioning, the current study aimed to elucidate the role of these neurotransmitters in the social learning process using a dietary depletion manipulation. In a double-blind design, 70 healthy volunteers were randomly allocated to a 5-HT depletion (N = 24), DA depletion (N = 24), or placebo (N = 22) group. Participants performed a social learning task during fMRI scanning, as part of which they learned associations between name cues and rewarding (happy faces) or aversive (fearful faces) social outcomes. Behaviorally, 5-HT depleted subjects demonstrated impaired social reward learning compared to placebo controls, with a marginal effect in the same direction in the DA depletion group. On the neural level, computational modeling-based fMRI analyses revealed that 5-HT depletion altered social reward prediction signals in the insula, temporal lobe, and prefrontal cortex, while DA depletion affected social reward prediction encoding only in the prefrontal cortex. These results indicate that 5-HT depletion impairs learning from social rewards, on both the behavioral and the neural level, while DA depletion has a less extensive effect. Interestingly, the behavioral and neural responses observed after 5-HT depletion in the current study closely resemble our previous findings in individuals with high depression scores using the same task. It may thus be the case that decreased 5-HT levels contribute to social learning deficits in depression.

A low dosage of the dopamine D2-receptor antagonist sulpiride affects effort allocation for reward regardless of trait extraversion

Dopamine (DA) is known to be involved in various aspects of reward processing and goal-directed behavior. The present preregistered study aims at directly accessing the causal influence of DA activity on reward motivation in humans, while also accounting for trait extraversion. Therefore, we examined the effect of a single dose of the DA D2 receptor antagonist sulpiride (200 mg) on effort allocation in a modified version of the Effort-Expenditure for Reward Task (EEfRT). Based on its presumably DA increasing action, we expected the low dose of sulpiride to increase participants' willingness to allocate effort during the modified EEfRT relative to placebo, especially in trials with low probability of reward attainment. Further, we expected a moderating effect of trait extraversion on the effects of sulpiride. Two hundred and three healthy male participants were tested in a randomized, double-blind between-subjects design. Contrary to our expectations, sulpiride reduced the average number of clicks within the modified EEfRT and did not interact with reward attributes, suggesting a more global and not reward-specific effect of sulpiride. Furthermore, trait extraversion did not moderate the effect of sulpiride. Our results provide initial support for the validity of the modified version of the EEfRT, suggesting a possible inhibiting effect of a low dose of sulpiride on approach motivation regardless of trait extraversion. However, given the mixed pattern of findings and the possible confounding role of motoric abilities, further studies examining these effects are clearly warranted.

Striatal Dopamine and Reward Prediction Error Signaling in Unmedicated Schizophrenia Patients

Increased striatal dopamine synthesis capacity has consistently been reported in patients with schizophrenia. However, the mechanism translating this into behavior and symptoms remains unclear. It has been proposed that heightened striatal dopamine may blunt dopaminergic reward prediction error signaling during reinforcement learning. In this study, we investigated striatal dopamine synthesis capacity, reward prediction errors, and their association in unmedicated schizophrenia patients (n = 19) and healthy controls (n = 23). They took part in FDOPA-PET and underwent functional magnetic resonance imaging (fMRI) scanning, where they performed a reversal-learning paradigm. The groups were compared regarding dopamine synthesis capacity (Kicer), fMRI neural prediction error signals, and the correlation of both. Patients did not differ from controls with respect to striatal Kicer. Taking into account, comorbid alcohol abuse revealed that patients without such abuse showed elevated Kicer in the associative striatum, while those with abuse did not differ from controls. Comparing all patients to controls, patients performed worse during reversal learning and displayed reduced prediction error signaling in the ventral striatum. In controls, Kicer in the limbic striatum correlated with higher reward prediction error signaling, while there was no significant association in patients. Kicer in the associative striatum correlated with higher positive symptoms and blunted reward prediction error signaling was associated with negative symptoms. Our results suggest a dissociation between striatal subregions and symptom domains, with elevated dopamine synthesis capacity in the associative striatum contributing to positive symptoms while blunted prediction error signaling in the ventral striatum related to negative symptoms.

Dose-dependent effects of estrogen on prediction error related neural activity in the nucleus accumbens of healthy young women

RATIONALE

Whereas the effect of the sex steroid 17-beta-estradiol (E2) on dopaminergic (DA) transmission in the nucleus accumbens (NAc) is well evidenced in female rats, studies in humans are inconsistent. Moreover, linear and inverted u-shaped dose response curves have been observed for E2's effects on hippocampal plasticity, but the shape of dose response curves for E2's effects on the NAc is much less characterized.

OBJECTIVES

Investigation of dose response curves for E2's effects on DA-related neural activity in the human NAc.

METHODS

Placebo or E2 valerate in doses of 2, 4, 6 or 12 mg was orally administered to 125 naturally cycling young women during the low-hormone menstruation phase on two consecutive days using a randomized, double-blinded design. The E2 treatment regimen induced a wide range of E2 levels, from physiological (2- and 4-mg groups; equivalent to cycle peak) to supraphysiological levels (6- and 12-mg groups; equivalent to early pregnancy). This made it possible to study different dose response functions for E2's effects on NAc activity. During E2 peak, participants performed a well-established reversal learning paradigm. We used trial-wise prediction errors (PE) estimated via a computational reinforcement learning model as a proxy for dopaminergic activity. Linear and quadratic regression analyses predicting PE-related NAc activity from salivary E2 levels were calculated.

RESULTS

There was a positive linear relationship between PE-associated NAc activity and salivary E2 increases.

CONCLUSIONS

The randomized, placebo-controlled elevation of E2 levels stimulates NAc activity in the human brain, likely mediated by dopaminergic processes.

A roadmap to integrate astrocytes into Systems Neuroscience

Systems neuroscience is still mainly a neuronal field, despite the plethora of evidence supporting the fact that astrocytes modulate local neural circuits, networks, and complex behaviors. In this article, we sought to identify which types of studies are necessary to establish whether astrocytes, beyond their well-documented homeostatic and metabolic functions, perform computations implementing mathematical algorithms that sub-serve coding and higher-brain functions. First, we reviewed Systems-like studies that include astrocytes in order to identify computational operations that these cells may perform, using Ca2+ transients as their encoding language. The analysis suggests that astrocytes may carry out canonical computations in a time scale of subseconds to seconds in sensory processing, neuromodulation, brain state, memory formation, fear, and complex homeostatic reflexes. Next, we propose a list of actions to gain insight into the outstanding question of which variables are encoded by such computations. The application of statistical analyses based on machine learning, such as dimensionality reduction and decoding in the context of complex behaviors, combined with connectomics of astrocyte-neuronal circuits, is, in our view, fundamental undertakings. We also discuss technical and analytical approaches to study neuronal and astrocytic populations simultaneously, and the inclusion of astrocytes in advanced modeling of neural circuits, as well as in theories currently under exploration such as predictive coding and energy-efficient coding. Clarifying the relationship between astrocytic Ca2+ and brain coding may represent a leap forward toward novel approaches in the study of astrocytes in health and disease.

Catecholaminergic modulation of meta-learning

The remarkable expedience of human learning is thought to be underpinned by meta-learning, whereby slow accumulative learning processes are rapidly adjusted to the current learning environment. To date, the neurobiological implementation of meta-learning remains unclear. A burgeoning literature argues for an important role for the catecholamines dopamine and noradrenaline in meta-learning. Here, we tested the hypothesis that enhancing catecholamine function modulates the ability to optimise a meta-learning parameter (learning rate) as a function of environmental volatility. 102 participants completed a task which required learning in stable phases, where the probability of reinforcement was constant, and volatile phases, where probabilities changed every 10-30 trials. The catecholamine transporter blocker methylphenidate enhanced participants' ability to adapt learning rate: Under methylphenidate, compared with placebo, participants exhibited higher learning rates in volatile relative to stable phases. Furthermore, this effect was significant only with respect to direct learning based on the participants' own experience, there was no significant effect on inferred-value learning where stimulus values had to be inferred. These data demonstrate a causal link between catecholaminergic modulation and the adjustment of the meta-learning parameter learning rate.

Blindness, Psychosis, and the Visual Construction of the World

The relationship between visual loss and psychosis is complex: congenital visual loss appears to be protective against the development of a psychotic disorder, particularly schizophrenia. In later life, however, visual deprivation or visual loss can give rise to hallucinosis, disorders of visual insight such as blindsight or Anton syndrome, or, in the context of neurodegenerative disorders, more complex psychotic presentations. We draw on a computational psychiatric approach to consider the foundational role of vision in the construction of representations of the world and the effects of visual loss at different developmental stages. Using a Bayesian prediction error minimization model, we describe how congenital visual loss may be protective against the development of the kind of computational deficits postulated to underlie schizophrenia, by increasing the precision (and consequent stability) of higher-level (including supramodal) priors, focusing on visual loss-induced changes in NMDA receptor structure and function as a possible mechanistic substrate. In simple terms, we argue that when people cannot see from birth, they rely more heavily on the context they extract from the other senses, and the resulting model of the world is more impervious to the false inferences, made in the face of inevitably noisy perceptual input, that characterize schizophrenia. We show how a Bayesian prediction error minimization framework can also explain the relationship between later visual loss and other psychotic symptoms, as well as the effects of visual deprivation and hallucinogenic drugs, and outline experimentally testable hypotheses generated by this approach.

Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals

Individuals from across the psychosis spectrum display impairments in reinforcement learning. In some individuals, these deficits may result from aberrations in reward prediction error (RPE) signaling, conveyed by dopaminergic projections to the ventral striatum (VS). However, there is mounting evidence that VS RPE signals are relatively intact in medicated people with schizophrenia (PSZ). We hypothesized that, in PSZ, reinforcement learning deficits often are not related to RPE signaling per se but rather their impact on learning and behavior (i.e., learning rate modulation), due to dysfunction in anterior cingulate and dorsomedial prefrontal cortex (dmPFC). Twenty-six PSZ and 23 healthy volunteers completed a probabilistic reinforcement learning paradigm with occasional, sudden, shifts in contingencies. Using computational modeling, we found evidence of an impairment in trial-wise learning rate modulation (α) in PSZ before and after a reinforcement contingency shift, expressed most in PSZ with more severe motivational deficits. In a subsample of 22 PSZ and 22 healthy volunteers, we found little evidence for between-group differences in VS RPE and dmPFC learning rate signals, as measured with fMRI. However, a follow-up psychophysiological interaction analysis revealed decreased dmPFC-VS connectivity concurrent with learning rate modulation, most prominently in individuals with the most severe motivational deficits. These findings point to an impairment in learning rate modulation in PSZ, leading to a reduced ability to adjust task behavior in response to unexpected outcomes. At the level of the brain, learning rate modulation deficits may be associated with decreased involvement of the dmPFC within a greater RL network.

Neurodevelopment and Cognitive Impairment in Parents and Progeny of Perinatal Dietary Protein Deficiency Models

There is an absolute dependence of the concept of development on supply of adequately balanced nutrients especially during the perinatal age which is critical to development. Therefore, an upgraded nutrition is specially required during gestation and lactation, as this is the critical period of neurodevelopment. This study sought to investigate the effect of protein deficiency during gestation (F0) and lactation through to adolescence on neurological functions of subsequent (F1 and F2) generations, establishing the possible consequential mechanistic association. Rats in four groups were fed different rations of protein diets (PD) as formulated: 21% PD, 10% PD, 5% PD and control diet (standard rat chow, containing 16-18% protein), from adolescent through to gestation and lactation, next generations were weaned to the maternal diet group. Neurobehavioral studies (which include; Surface righting reflex, Negative geotaxis, Learning and Memory tests), brain oxidative stress and quantification of serotonin and dopamine levels in the brain were conducted. Result shows significantly altered neurobehavior, reflected in the reduction of reflex response and postural reaction score at P ≤ 0.05. There was also a transgenerational cognitive impairment of brain function in the F-generations, following perinatal protein malnutrition as shown in the Y-maze result, measuring spatial memory and Morris water maze result (cognition), providing a background for the observed sensorimotor response. The significant increase in dopamine level, decrease in the antioxidant capacity of the protein deficient brain groups are consistent with significantly altered serotonin system, critical to neurodevelopment and functional activities of learning and memory. Therefore, persistent early life protein deficiency mediates dysfunction in neurodevelopment and this involves life-long changes in key neurotransmitters and the brain redox status underlying the neurobehavioral display.

Dopamine blockade impairs the exploration-exploitation trade-off in rats

In a volatile environment where rewards are uncertain, successful performance requires a delicate balance between exploitation of the best option and exploration of alternative choices. It has theoretically been proposed that dopamine contributes to the control of this exploration-exploitation trade-off, specifically that the higher the level of tonic dopamine, the more exploitation is favored. We demonstrate here that there is a formal relationship between the rescaling of dopamine positive reward prediction errors and the exploration-exploitation trade-off in simple non-stationary multi-armed bandit tasks. We further show in rats performing such a task that systemically antagonizing dopamine receptors greatly increases the number of random choices without affecting learning capacities. Simulations and comparison of a set of different computational models (an extended Q-learning model, a directed exploration model, and a meta-learning model) fitted on each individual confirm that, independently of the model, decreasing dopaminergic activity does not affect learning rate but is equivalent to an increase in random exploration rate. This study shows that dopamine could adapt the exploration-exploitation trade-off in decision-making when facing changing environmental contingencies.

Expected Value of Reward Predicts Episodic Memory for Incidentally Learnt Reward-Item Associations

In this paper, we draw connections between reward processing and cognition by behaviourally testing the implications of neurobiological theories of reward processing on memory. Single-cell neurophysiology in non-human primates and imaging work in humans suggests that the dopaminergic reward system responds to different components of reward: expected value; outcome or prediction error; and uncertainty of reward (Schultz et al., 2008). The literature on both incidental and motivated learning has focused on understanding how expected value and outcome—linked to increased activity in the reward system—lead to consolidation-related memory enhancements. In the current study, we additionally investigate the impact of reward uncertainty on human memory. The contribution of reward uncertainty—the spread of the reward probability distribution irrespective of the magnitude—has not been previously examined. To examine the effects of uncertainty on memory, a word-learning task was introduced, along with a surprise delayed recognition memory test. Using Bayesian model selection, we found evidence only for expected value as a predictor of memory performance. Our findings suggest that reward uncertainty does not enhance memory for individual items. This supports emerging evidence that an effect of uncertainty on memory is only observed in high compared to low risk environments.

Dopamine dysregulation hypothesis: the common basis for motivational anhedonia in major depressive disorder and schizophrenia?

Abnormalities in reward processing are crucial symptoms of major depressive disorder (MDD) and schizophrenia (SCH). Recent neuroscientific findings regarding MDD have led to conclusions about two different symptoms related to reward processing: motivational and consummatory anhedonia, corresponding, respectively, to impaired motivation to obtain rewards ('wanting'), and diminished satisfaction from consuming them ('liking'). One can ask: which of these is common for MDD and SCH. In our review of the latest neuroscientific studies, we show that MDD and SCH do not share consummatory anhedonia, as SCH patients usually have unaltered liking. Therefore, we investigated whether motivational anhedonia is the common symptom across MDD and SCH. With regard to the similarities and differences between the neural mechanisms of MDD and SCH, here we expand the current knowledge of motivation deficits and present the common underlying mechanism of motivational anhedonia - the dopamine dysregulation hypothesis - stating that any prolonged dysregulation in tonic dopamine signaling that exceeds the given equilibrium can lead to striatal dysfunction and motivational anhedonia. The implications for further research and treatment of MDD and SCH are also discussed.

Anomalous Perceptions and Beliefs Are Associated With Shifts Toward Different Types of Prior Knowledge in Perceptual Inference

Psychotic phenomena manifest in healthy and clinical populations as complex patterns of aberrant perceptions (hallucinations) and tenacious, irrational beliefs ( delusions). According to predictive processing accounts, hallucinations and delusions arise from atypicalities in the integration of prior knowledge with incoming sensory information. However, the computational details of these atypicalities and their specific phenomenological manifestations are not well characterized. We tested the hypothesis that hallucination-proneness arises from increased reliance on overly general application of prior knowledge in perceptual inference, generating percepts that readily capture the gist of the environment but inaccurately render its details. We separately probed the use of prior knowledge to perceive the gist vs the details of ambiguous images in a healthy population with varying degrees of hallucination- and delusion-proneness. We found that the use of prior knowledge varied with psychotic phenomena and their composition in terms of aberrant percepts vs aberrant beliefs. Consistent with previous findings, hallucination-proneness conferred an advantage using prior knowledge to perceive image gist but, contrary to predictions, did not confer disadvantage perceiving image details. Predominant hallucination-proneness actually conferred advantages perceiving both image gist and details, consistent with reliance on highly detailed perceptual knowledge. Delusion-proneness, and especially predominance of delusion-proneness over hallucination-proneness, conferred disadvantage perceiving image details but not image gist, though evidence of specific impairment of detail perception was preliminary. We suggest this is consistent with reliance on abstract, belief-like knowledge. We posit that phenomenological variability in psychotic experiences may be driven by variability in the type of knowledge observers rely upon to resolve perceptual ambiguity.

Deficits in context-dependent adaptive coding in early psychosis and healthy individuals with schizotypal personality traits

Adaptive coding of information is a fundamental principle of brain functioning. It allows for efficient representation over a large range of inputs and thereby alleviates the limited coding range of neurons. In the present study, we investigated for the first time potential alterations in context-dependent reward adaptation and its association with symptom dimensions in the schizophrenia spectrum. We studied 27 patients with first-episode psychosis, 26 individuals with schizotypal personality traits and 25 healthy controls. We used functional MRI in combination with a variant of the monetary incentive delay task and assessed adaptive reward coding in two reward conditions with different reward ranges. Compared to healthy controls, patients with first-episode psychosis and healthy individuals with schizotypal personality traits showed a deficit in increasing the blood oxygen level-dependent response slope in the right caudate for the low reward range compared to the high reward range. In other words, the two groups showed inefficient neural adaptation to the current reward context. In addition, we found impaired adaptive coding of reward in the caudate nucleus and putamen to be associated with total symptom severity across the schizophrenia spectrum. Symptom severity was more strongly associated with neural deficits in adaptive coding than with the neural coding of absolute reward outcomes. Deficits in adaptive coding were prominent across the schizophrenia spectrum and even detectable in unmedicated (healthy) individuals with schizotypal personality traits. Furthermore, the association between total symptom severity and impaired adaptive coding in the right caudate and putamen suggests a dimensional mechanism underlying imprecise neural adaptation. Our findings support the idea that impaired adaptive coding may be a general information-processing deficit explaining disturbances within the schizophrenia spectrum over and above a simple model of blunted absolute reward signals.

Enhanced transformation of incidentally learned knowledge into explicit memory by dopaminergic modulation

During incidental learning statistical regularities are extracted from the environment without the intention to learn. Acquired implicit memory of these regularities can affect behavior in the absence of awareness. However, conscious insight in the underlying regularities can also develop during learning. Such emergence of explicit memory is an important learning mechanism that is assumed to involve prediction errors in the striatum and to be dopamine-dependent. Here we directly tested this hypothesis by manipulating dopamine levels during incidental learning in a modified serial reaction time task (SRTT) featuring a hidden regular sequence of motor responses in a placebo-controlled between-group study. Awareness for the sequential regularity was subsequently assessed using cued generation and additionally verified using free recall. The results demonstrated that dopaminergic modulation nearly doubled the amount of explicit sequence knowledge emerged during learning in comparison to the placebo group. This strong effect clearly argues for a causal role of dopamine-dependent processing for the development of awareness for sequential regularities during learning.

Impaired reward prediction error encoding and striatal-midbrain connectivity in depression

Anhedonia (hyposensitivity to rewards) and negative bias (hypersensitivity to punishments) are core features of major depressive disorder (MDD), which could stem from abnormal reinforcement learning. Emerging evidence highlights blunted reward learning and reward prediction error (RPE) signaling in the striatum in MDD, although inconsistencies exist. Preclinical studies have clarified that ventral tegmental area (VTA) neurons encode RPE and habenular neurons encode punishment prediction error (PPE), which are then transmitted to the striatum and cortex to guide goal-directed behavior. However, few studies have probed striatal activation, and functional connectivity between VTA-striatum and VTA-habenula during reward and punishment learning respectively, in unmedicated MDD. To fill this gap, we acquired fMRI data from 25 unmedicated MDD and 26 healthy individuals during a monetary instrumental learning task and utilized a computational modeling approach to characterize underlying neural correlates of RPE and PPE. Relative to controls, MDD individuals showed impaired reward learning, blunted RPE signal in the striatum and overall reduced VTA-striatal connectivity to feedback. Critically, striatal RPE signal was increasingly blunted with more major depressive episodes (MDEs). No group differences emerged in PPE signals in the habenula and VTA or in connectivity between these regions. However, PPE signals in the habenula correlated positively with number of MDEs. These results highlight impaired reward learning, disrupted RPE signaling in the striatum (particularly among individuals with more lifetime MDEs) as well as reduced VTA-striatal connectivity in MDD. Collectively, these findings highlight reward-related learning deficits in MDD and their underlying pathophysiology.

Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis

Learning occurs when an outcome differs from expectations, generating a reward prediction error signal (RPE). The RPE signal has been hypothesized to simultaneously embody the valence of an outcome (better or worse than expected) and its surprise (how far from expectations). Nonetheless, growing evidence suggests that separate representations of the two RPE components exist in the human brain. Meta-analyses provide an opportunity to test this hypothesis and directly probe the extent to which the valence and surprise of the error signal are encoded in separate or overlapping networks. We carried out several meta-analyses on a large set of fMRI studies investigating the neural basis of RPE, locked at decision outcome. We identified two valence learning systems by pooling studies searching for differential neural activity in response to categorical positive-versus-negative outcomes. The first valence network (negative > positive) involved areas regulating alertness and switching behaviours such as the midcingulate cortex, the thalamus and the dorsolateral prefrontal cortex whereas the second valence network (positive > negative) encompassed regions of the human reward circuitry such as the ventral striatum and the ventromedial prefrontal cortex. We also found evidence of a largely distinct surprise-encoding network including the anterior cingulate cortex, anterior insula and dorsal striatum. Together with recent animal and electrophysiological evidence this meta-analysis points to a sequential and distributed encoding of different components of the RPE signal, with potentially distinct functional roles.