Neural signals encoding shifts in beliefs

Belief inference for hierarchical hidden states in spatial navigation

Uncertainty abounds in the real world, and in environments with multiple layers of unobservable hidden states, decision-making requires resolving uncertainties based on mutual inference. Focusing on a spatial navigation problem, we develop a Tiger maze task that involved simultaneously inferring the local hidden state and the global hidden state from probabilistically uncertain observation. We adopt a Bayesian computational approach by proposing a hierarchical inference model. Applying this to human task behaviour, alongside functional magnetic resonance brain imaging, allows us to separate the neural correlates associated with reinforcement and reassessment of belief in hidden states. The imaging results also suggest that different layers of uncertainty differentially involve the basal ganglia and dorsomedial prefrontal cortex, and that the regions responsible are organised along the rostral axis of these areas according to the type of inference and the level of abstraction of the hidden state, i.e. higher-order state inference involves more anterior parts.

People with a tobacco use disorder exhibit misaligned Bayesian belief updating by falsely attributing non-drug cues as worse predictors of positive outcomes compared to drug cues

Adaptive behaviours depend on dynamically updating internal representations of the world based on the ever-changing environmental contingencies. People with a substance use disorder (pSUD) show maladaptive behaviours with high persistence in drug-taking, despite severe negative consequences. We recently proposed a salience misattribution model for addiction (SMMA; Kalhan et al., 2021), arguing that pSUD have aberrations in their updating processes where drug cues are misattributed as strong predictors of positive outcomes, but weaker predictors of negative outcomes. We also argued that conversely, non-drug cues are misattributed as weak predictors of positive outcomes, but stronger predictors of negative outcomes. We tested these hypotheses using a multi-cue reversal learning task, with reversals in whether drug or non-drug cues are relevant in predicting the outcome (monetary win or loss). We show that people with a tobacco use disorder (pTUD), do form misaligned internal representations. We found that pTUD updated less towards learning the drug cue's relevance in predicting a loss. Further, when neither drug nor non-drug cue predicted a win, pTUD updated more towards the drug cue being relevant predictors of that win. Our Bayesian belief updating model revealed that pTUD had a low estimated likelihood of non-drug cues being predictors of wins, compared to drug cues, which drove the misaligned updating. Overall, several hypotheses of the SMMA were supported, but not all. Our results implicate that strengthening the non-drug cue association with positive outcomes may help restore the misaligned internal representation in pTUD, and offers a quantifiable, computational account of these updating processes.

Neural correlates of valence-dependent belief and value updating during uncertainty reduction: An fNIRS study

Selective use of new information is crucial for adaptive decision-making. Combining a gamble bidding task with assessing cortical responses using functional near-infrared spectroscopy (fNIRS), we investigated potential effects of information valence on behavioral and neural processes of belief and value updating during uncertainty reduction in young adults. By modeling changes in the participants' expressed subjective values using a Bayesian model, we dissociated processes of (i) updating beliefs about statistical properties of the gamble, (ii) updating values of a gamble based on new information about its winning probabilities, as well as (iii) expectancy violation. The results showed that participants used new information to update their beliefs and values about the gambles in a quasi-optimal manner, as reflected in the selective updating only in situations with reducible uncertainty. Furthermore, their updating was valence-dependent: information indicating an increase in winning probability was underweighted, whereas information about a decrease in winning probability was updated in good agreement with predictions of the Bayesian decision theory. Results of model-based and moderation analyses showed that this valence-dependent asymmetry was associated with a distinct contribution of expectancy violation, besides belief updating, to value updating after experiencing new positive information regarding winning probabilities. In line with the behavioral results, we replicated previous findings showing involvements of frontoparietal brain regions in the different components of updating. Furthermore, this study provided novel results suggesting a valence-dependent recruitment of brain regions. Individuals with stronger oxyhemoglobin responses during value updating was more in line with predictions of the Bayesian model while integrating new information that indicates an increase in winning probability. Taken together, this study provides first results showing expectancy violation as a contributing factor to sub-optimal valence-dependent updating during uncertainty reduction and suggests limitations of normative Bayesian decision theory.

First few seconds for flow: A comprehensive proposal of the neurobiology and neurodynamics of state onset

Flow is a cognitive state that manifests when there is complete attentional absorption while performing a task. Flow occurs when certain internal as well as external conditions are present, including intense concentration, a sense of control, feedback, and a balance between the challenge of the task and the relevant skillset. Phenomenologically, flow is accompanied by a loss of self-consciousness, seamless integration of action and awareness, and acute changes in time perception. Research has begun to uncover some of the neurophysiological correlates of flow, as well as some of the state's neuromodulatory processes. We comprehensively review this work and consider the neurodynamics of the onset of the state, considering large-scale brain networks, as well as dopaminergic, noradrenergic, and endocannabinoid systems. To accomplish this, we outline an evidence-based hypothetical situation, and consider the flow state in a broader context including other profound alterations in consciousness, such as the psychedelic state and the state of traumatic stress that can induce PTSD. We present a broad theoretical framework which may motivate future testable hypotheses.

Functional neuroimaging in psychiatry and the case for failing better

Psychiatric disorders encompass complex aberrations of cognition and affect and are among the most debilitating and poorly understood of any medical condition. Current treatments rely primarily on interventions that target brain function (drugs) or learning processes (psychotherapy). A mechanistic understanding of how these interventions mediate their therapeutic effects remains elusive. From the early 1990s, non-invasive functional neuroimaging, coupled with parallel developments in the cognitive neurosciences, seemed to signal a new era of neurobiologically grounded diagnosis and treatment in psychiatry. Yet, despite three decades of intense neuroimaging research, we still lack a neurobiological account for any psychiatric condition. Likewise, functional neuroimaging plays no role in clinical decision making. Here, we offer a critical commentary on this impasse and suggest how the field might fare better and deliver impactful neurobiological insights.

Beauty and Uncertainty as Transformative Factors: A Free Energy Principle Account of Aesthetic Diagnosis and Intervention in Gestalt Psychotherapy

Drawing from field theory, Gestalt therapy conceives psychological suffering and psychotherapy as two intentional field phenomena, where unprocessed and chaotic experiences seek the opportunity to emerge and be assimilated through the contact between the patient and the therapist (i.e., the intentionality of contacting). This therapeutic approach is based on the therapist’s aesthetic experience of his/her embodied presence in the flow of the healing process because (1) the perception of beauty can provide the therapist with feedback on the assimilation of unprocessed experiences; (2) the therapist’s attentional focus on intrinsic aesthetic diagnostic criteria can facilitate the modification of rigid psychopathological fields by supporting the openness to novel experiences. The aim of the present manuscript is to review recent evidence from psychophysiology, neuroaesthetic research, and neurocomputational models of cognition, such as the free energy principle (FEP), which support the notion of the therapeutic potential of aesthetic sensibility in Gestalt psychotherapy. Drawing from neuroimaging data, psychophysiology and recent neurocognitive accounts of aesthetic perception, we propose a novel interpretation of the sense of beauty as a self-generated reward motivating us to assimilate an ever-greater spectrum of sensory and affective states in our predictive representation of ourselves and the world and supporting the intentionality of contact. Expecting beauty, in the psychotherapeutic encounter, can help therapists tolerate uncertainty avoiding impulsive behaviours and to stay tuned to the process of change.

Circular inference predicts nonuniform overactivation and dysconnectivity in brain-wide connectomes

Schizophrenia is a severe mental disorder whose neural basis remains difficult to ascertain. Among the available pathophysiological theories, recent work has pointed towards subtle perturbations in the excitation-inhibition (E/I) balance within different neural circuits. Computational approaches have suggested interesting mechanisms that can account for both E/I imbalances and psychotic symptoms. Based on hierarchical neural networks propagating information through a message-passing algorithm, it was hypothesized that changes in the E/I ratio could cause a "circular belief propagation" in which bottom-up and top-down information reverberate. This circular inference (CI) was proposed to account for the clinical features of schizophrenia. Under this assumption, this paper examined the impact of CI on network dynamics in light of brain imaging findings related to psychosis. Using brain-inspired graphical models, we show that CI causes overconfidence and overactivation most specifically at the level of connector hubs (e.g., nodes with many connections allowing integration across networks). By also measuring functional connectivity in these graphs, we provide evidence that CI is able to predict specific changes in modularity known to be associated with schizophrenia. Altogether, these findings suggest that the CI framework may facilitate behavioral and neural research on the multifaceted nature of psychosis.

Spatiotemporal Signatures of Surprise Captured by Magnetoencephalography

Surprise and social influence are linked through several neuropsychological mechanisms. By garnering attention, causing arousal, and motivating engagement, surprise provides a context for effective or durable social influence. Attention to a surprising event motivates the formation of an explanation or updating of models, while high arousal experiences due to surprise promote memory formation. They both encourage engagement with the surprising event through efforts aimed at understanding the situation. By affecting the behavior of the individual or a social group via setting an attractive engagement context, surprise plays an important role in shaping personal and social change. Surprise is an outcome of the brain’s function in constantly anticipating the future of sensory inputs based on past experiences. When new sensory data is different from the brain’s predictions shaped by recent trends, distinct neural signals are generated to report this surprise. As a quantitative approach to modeling the generation of brain surprise, input stimuli containing surprising elements are employed in experiments such as oddball tasks during which brain activity is recorded. Although surprise has been well characterized in many studies, an information-theoretical model to describe and predict the surprise level of an external stimulus in the recorded MEG data has not been reported to date, and setting forth such a model is the main objective of this paper. Through mining trial-by-trial MEG data in an oddball task according to theoretical definitions of surprise, the proposed surprise decoding model employs the entire epoch of the brain response to a stimulus to measure surprise and assesses which collection of temporal/spatial components in the recorded data can provide optimal power for describing the brain’s surprise. We considered three different theoretical formulations for surprise assuming the brain acts as an ideal observer that calculates transition probabilities to estimate the generative distribution of the input. We found that middle temporal components and the right and left fronto-central regions offer the strongest power for decoding surprise. Our findings provide a practical and rigorous method for measuring the brain’s surprise, which can be employed in conjunction with behavioral data to evaluate the interactive and social effects of surprising events.

Neural mechanisms of credit assignment for inferred relationships in a structured world

Animals abstract compact representations of a task's structure, which supports accelerated learning and flexible behavior. Whether and how such abstracted representations may be used to assign credit for inferred, but unobserved, relationships in structured environments are unknown. We develop a hierarchical reversal-learning task and Bayesian learning model to assess the computational and neural mechanisms underlying how humans infer specific choice-outcome associations via structured knowledge. We find that the medial prefrontal cortex (mPFC) efficiently represents hierarchically related choice-outcome associations governed by the same latent cause, using a generalized code to assign credit for both experienced and inferred outcomes. Furthermore, the mPFC and lateral orbitofrontal cortex track the current "position" within a latent association space that generalizes over stimuli. Collectively, these findings demonstrate the importance of both tracking the current position in an abstracted task space and efficient, generalizable representations in the prefrontal cortex for supporting flexible learning and inference in structured environments.

Self-esteem depends on beliefs about the rate of change of social approval

A major challenge in understanding the neurobiological basis of psychiatric disorders is rigorously quantifying subjective metrics that lie at the core of mental illness, such as low self-esteem. Self-esteem can be conceptualized as a 'gauge of social approval' that increases in response to approval and decreases in response to disapproval. Computational studies have shown that learning signals that represent the difference between received and expected social approval drive changes in self-esteem. However, it is unclear whether self-esteem based on social approval should be understood as a value updated through associative learning, or as a belief about approval, updated by new evidence depending on how strongly it is held. Our results show that belief-based models explain self-esteem dynamics in response to social evaluation better than associative learning models. Importantly, they suggest that in the short term, self-esteem signals the direction and rate of change of one's beliefs about approval within a group, rather than one's social position.

Models of Dynamic Belief Updating in Psychosis-A Review Across Different Computational Approaches

To understand the dysfunctional mechanisms underlying maladaptive reasoning of psychosis, computational models of decision making have widely been applied over the past decade. Thereby, a particular focus has been on the degree to which beliefs are updated based on new evidence, expressed by the learning rate in computational models. Higher order beliefs about the stability of the environment can determine the attribution of meaningfulness to events that deviate from existing beliefs by interpreting these either as noise or as true systematic changes (volatility). Both, the inappropriate downplaying of important changes as noise (belief update too low) as well as the overly flexible adaptation to random events (belief update too high) were theoretically and empirically linked to symptoms of psychosis. Whereas models with fixed learning rates fail to adjust learning in reaction to dynamic changes, increasingly complex learning models have been adopted in samples with clinical and subclinical psychosis lately. These ranged from advanced reinforcement learning models, over fully Bayesian belief updating models to approximations of fully Bayesian models with hierarchical learning or change point detection algorithms. It remains difficult to draw comparisons across findings of learning alterations in psychosis modeled by different approaches e.g., the Hierarchical Gaussian Filter and change point detection. Therefore, this review aims to summarize and compare computational definitions and findings of dynamic belief updating without perceptual ambiguity in (sub)clinical psychosis across these different mathematical approaches. There was strong heterogeneity in tasks and samples. Overall, individuals with schizophrenia and delusion-proneness showed lower behavioral performance linked to failed differentiation between uninformative noise and environmental change. This was indicated by increased belief updating and an overestimation of volatility, which was associated with cognitive deficits. Correlational evidence for computational mechanisms and positive symptoms is still sparse and might diverge from the group finding of instable beliefs. Based on the reviewed studies, we highlight some aspects to be considered to advance the field with regard to task design, modeling approach, and inclusion of participants across the psychosis spectrum. Taken together, our review shows that computational psychiatry offers powerful tools to advance our mechanistic insights into the cognitive anatomy of psychotic experiences.

Interoception as modeling, allostasis as control

The brain regulates the body by anticipating its needs and attempting to meet them before they arise - a process called allostasis. Allostasis requires a model of the changing sensory conditions within the body, a process called interoception. In this paper, we examine how interoception may provide performance feedback for allostasis. We suggest studying allostasis in terms of control theory, reviewing control theory's applications to related issues in physiology, motor control, and decision making. We synthesize these by relating them to the important properties of allostatic regulation as a control problem. We then sketch a novel formalism for how the brain might perform allostatic control of the viscera by analogy to skeletomotor control, including a mathematical view on how interoception acts as performance feedback for allostasis. Finally, we suggest ways to test implications of our hypotheses.

Magic, Bayes and wows: A Bayesian account of magic tricks

Magic tricks have enjoyed an increasing interest by scientists. However, most research in magic focused on isolated aspects of it and a conceptual understanding of magic, encompassing its distinct components and varieties, is missing. Here, we present an account of magic within the theory of Bayesian predictive coding. We present the "wow" effect of magic as an increase in surprise evoked by the prediction error between expected and observed data. We take into account prior knowledge of the observer, attention, and (mis-)direction of perception and beliefs by the magician to bias the observer's predictions and present a simple example for the modelling of the evoked surprise. The role of misdirection is described as everything that aims to maximize the surprise a trick evokes by the generation of novel beliefs, the exploitation of background knowledge and attentional control of the incoming information. Understanding magic within Bayesian predictive coding allows unifying all aspects of magic tricks within one framework, making it tractable, comparable and unifiable with other models in psychology and neuroscience.

To be specific: The role of orbitofrontal cortex in signaling reward identity

The orbitofrontal cortex (OFC) plays a prominent role in signaling reward expectations. Two important features of rewards are their value (how good they are) and their specific identity (what they are). Whereas research on OFC has traditionally focused on reward value, recent findings point toward a pivotal role of reward identity in understanding OFC signaling and its contribution to behavior. Here, we review work in rodents, nonhuman primates, and humans on how the OFC represents expectations about the identity of rewards, and how these signals contribute to outcome-guided behavior. Moreover, we summarize recent findings suggesting that specific reward expectations in OFC are learned and updated by means of identity errors in the dopaminergic midbrain. We conclude by discussing how OFC encoding of specific rewards complements recent proposals that this region represents a cognitive map of relevant task states, which forms the basis for model-based behavior. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

A salience misattribution model for addictive-like behaviors

Adapting to the changing environment is a key component of optimal decision-making. Internal-models that accurately represent and selectively update from behaviorally relevant/salient stimuli may facilitate adaptive behaviors. Anterior cingulate cortex (ACC) and dopaminergic systems may produce these adaptive internal-models through selective updates from behaviorally relevant stimuli. Dysfunction of ACC and dopaminergic systems could therefore produce misaligned internal-models where updates are disproportionate to the salience of the cues. An aspect of addictive-like behaviors is reduced adaptation and, ACC and dopaminergic systems typically exhibit dysfunction in drug-dependents. We argue that ACC and dopaminergic dysfunction in dependents may produce misaligned internal-models such that drug-related stimuli are misattributed with a higher salience compared to non-drug related stimuli. Hence, drug-related rewarding stimuli generate over-weighted updates to the internal-model, while negative feedback and non-drug related rewarding stimuli generate down-weighted updates. This misaligned internal-model may therefore incorrectly reinforce maladaptive drug-related behaviors. We use the proposed framework to discuss ways behavior may be made more adaptive and how the framework may be supported or falsified experimentally.

Electroencephalographic correlates of temporal Bayesian belief updating and surprise

The brain predicts the timing of forthcoming events to optimize responses to them. Temporal predictions have been formalized in terms of the hazard function, which integrates prior beliefs on the likely timing of stimulus occurrence with information conveyed by the passage of time. However, how the human brain updates prior temporal beliefs is still elusive. Here we investigated electroencephalographic (EEG) signatures associated with Bayes-optimal updating of temporal beliefs. Given that updating usually occurs in response to surprising events, we sought to disentangle EEG correlates of updating from those associated with surprise. Twenty-six participants performed a temporal foreperiod task, which comprised a subset of surprising events not eliciting updating. EEG data were analyzed through a regression-based massive approach in the electrode and source space. Distinct late positive, centro-parietally distributed, event-related potentials (ERPs) were associated with surprise and belief updating in the electrode space. While surprise modulated the commonly observed P3b, updating was associated with a later and more sustained P3b-like waveform deflection. Results from source analyses revealed that neural encoding of surprise comprises neural activity in the cingulo-opercular network (CON) and parietal regions. These data provide evidence that temporal predictions are computed in a Bayesian manner, and that this is reflected in P3 modulations, akin to other cognitive domains. Overall, our study revealed that analyzing P3 modulations provides an important window into the Bayesian brain. Data and scripts are shared on OSF: https://osf.io/ckqa5/.

Neural surprise in somatosensory Bayesian learning

Tracking statistical regularities of the environment is important for shaping human behavior and perception. Evidence suggests that the brain learns environmental dependencies using Bayesian principles. However, much remains unknown about the employed algorithms, for somesthesis in particular. Here, we describe the cortical dynamics of the somatosensory learning system to investigate both the form of the generative model as well as its neural surprise signatures. Specifically, we recorded EEG data from 40 participants subjected to a somatosensory roving-stimulus paradigm and performed single-trial modeling across peri-stimulus time in both sensor and source space. Our Bayesian model selection procedure indicates that evoked potentials are best described by a non-hierarchical learning model that tracks transitions between observations using leaky integration. From around 70ms post-stimulus onset, secondary somatosensory cortices are found to represent confidence-corrected surprise as a measure of model inadequacy. Indications of Bayesian surprise encoding, reflecting model updating, are found in primary somatosensory cortex from around 140ms. This dissociation is compatible with the idea that early surprise signals may control subsequent model update rates. In sum, our findings support the hypothesis that early somatosensory processing reflects Bayesian perceptual learning and contribute to an understanding of its underlying mechanisms.

Our environment features statistical regularities, such as a drop of rain predicting imminent rainfall. Despite the importance for behavior and survival, much remains unknown about how these dependencies are learned, particularly for somatosensation. As surprise signalling about novel observations indicates a mismatch between one’s beliefs and the world, it has been hypothesized that surprise computation plays an important role in perceptual learning. By analyzing EEG data from human participants receiving sequences of tactile stimulation, we compare different formulations of surprise and investigate the employed underlying learning model. Our results indicate that the brain estimates transitions between observations. Furthermore, we identified different signatures of surprise computation and thereby provide a dissociation of the neural correlates of belief inadequacy and belief updating. Specifically, early surprise responses from around 70ms were found to signal the need for changes to the model, with encoding of its subsequent updating occurring from around 140ms. These results provide insights into how somatosensory surprise signals may contribute to the learning of environmental statistics.

Dopamine, Prediction Error and Beyond

A large body of work has linked dopaminergic signaling to learning and reward processing. It stresses the role of dopamine in reward prediction error signaling, a key neural signal that allows us to learn from past experiences, and that facilitates optimal choice behavior. Latterly, it has become clear that dopamine does not merely code prediction error size but also signals the difference between the expected value of rewards, and the value of rewards actually received, which is obtained through the integration of reward attributes such as the type, amount, probability and delay. More recent work has posited a role of dopamine in learning beyond rewards. These theories suggest that dopamine codes absolute or unsigned prediction errors, playing a key role in how the brain models associative regularities within its environment, while incorporating critical information about the reliability of those regularities. Work is emerging supporting this perspective and, it has inspired theoretical models of how certain forms of mental pathology may emerge in relation to dopamine function. Such pathology is frequently related to disturbed inferences leading to altered internal models of the environment. Thus, it is critical to understand the role of dopamine in error-related learning and inference.

A Bayesian brain model of adaptive behavior: an application to the Wisconsin Card Sorting Task

Adaptive behavior emerges through a dynamic interaction between cognitive agents and changing environmental demands. The investigation of information processing underlying adaptive behavior relies on controlled experimental settings in which individuals are asked to accomplish demanding tasks whereby a hidden regularity or an abstract rule has to be learned dynamically. Although performance in such tasks is considered as a proxy for measuring high-level cognitive processes, the standard approach consists in summarizing observed response patterns by simple heuristic scoring measures. With this work, we propose and validate a new computational Bayesian model accounting for individual performance in the Wisconsin Card Sorting Test (WCST), a renowned clinical tool to measure set-shifting and deficient inhibitory processes on the basis of environmental feedback. We formalize the interaction between the task's structure, the received feedback, and the agent's behavior by building a model of the information processing mechanisms used to infer the hidden rules of the task environment. Furthermore, we embed the new model within the mathematical framework of the Bayesian Brain Theory (BBT), according to which beliefs about hidden environmental states are dynamically updated following the logic of Bayesian inference. Our computational model maps distinct cognitive processes into separable, neurobiologically plausible, information-theoretic constructs underlying observed response patterns. We assess model identification and expressiveness in accounting for meaningful human performance through extensive simulation studies. We then validate the model on real behavioral data in order to highlight the utility of the proposed model in recovering cognitive dynamics at an individual level. We highlight the potentials of our model in decomposing adaptive behavior in the WCST into several information-theoretic metrics revealing the trial-by-trial unfolding of information processing by focusing on two exemplary individuals whose behavior is examined in depth. Finally, we focus on the theoretical implications of our computational model by discussing the mapping between BBT constructs and functional neuroanatomical correlates of task performance. We further discuss the empirical benefit of recovering the assumed dynamics of information processing for both clinical and research practices, such as neurological assessment and model-based neuroscience.

"Stopping for knowledge": The sense of beauty in the perception-action cycle

According to a millennial-old philosophical debate, aesthetic emotions have been connected to knowledge acquisition. Recent scientific evidence, collected across different disciplinary domains, confirms this link, but also reveals that motor inhibition plays a crucial role in the process. In this review, we discuss multidisciplinary results and propose an original account of aesthetic appreciation (the stopping for knowledge hypothesis) framed within the predictive coding theory. We discuss evidence showing that aesthetic emotions emerge in correspondence with an inhibition of motor behavior (i.e., minimizing action), promoting a simultaneous perceptual processing enhancement, at the level of sensory cortices (i.e., optimizing learning). Accordingly, we suggest that aesthetic appreciation may represent a hedonic feedback over learning progresses, motivating the individual to inhibit motor routines to seek further knowledge acquisition. Furthermore, the neuroimaging and neuropsychological studies we review reveal the presence of a strong association between aesthetic appreciation and the activation of the dopaminergic reward-related circuits. Finally, we propose a number of possible applications of the stopping for knowledge hypothesis in the clinical and education domains.

Impaired context-sensitive adjustment of behaviour in Parkinson's disease patients tested on and off medication: An fMRI study

The brain's sensitivity to and accentuation of unpredicted over predicted sensory signals plays a fundamental role in learning. According to recent theoretical models of the predictive coding framework, dopamine is responsible for balancing the interplay between bottom-up input and top-down predictions by controlling the precision of surprise signals that guide learning. Using functional MRI, we investigated whether patients with Parkinson's disease (PD) show impaired learning from prediction errors requiring either adaptation or stabilisation of current predictions. Moreover, we were interested in whether deficits in learning over a specific time scale would be accompanied by altered surprise responses in dopamine-related brain structures. To this end, twenty-one PD patients tested on and off dopaminergic medication and twenty-one healthy controls performed a digit prediction paradigm. During the task, violations of sequence-based predictions either signalled the need to update or to stabilise the current prediction and, thus, to react to them or ignore them, respectively. To investigate contextual adaptation to prediction errors, the probability (or its inverse, surprise) of the violations fluctuated across the experiment. When the probability of prediction errors over a specific time scale increased, healthy controls but not PD patients off medication became more flexible, i.e., error rates at violations requiring a motor response decreased in controls but increased in patients. On the neural level, this learning deficit in patients was accompanied by reduced signalling in the substantia nigra and the caudate nucleus. In contrast, differences between the groups regarding the probabilistic modulation of behaviour and neural responses were much less pronounced at prediction errors requiring only stabilisation but no adaptation. Interestingly, dopaminergic medication could neither improve learning from prediction errors nor restore the physiological, neurotypical pattern. Our findings point to a pivotal role of dysfunctions of the substantia nigra and caudate nucleus in deficits in learning from flexibility-demanding prediction errors in PD. Moreover, the data witness poor effects of dopaminergic medication on learning in PD.

Doubting what you already know: Uncertainty regarding state transitions is associated with obsessive compulsive symptoms

Obsessive compulsive (OC) symptoms involve excessive information gathering (e.g., checking, reassurance-seeking), and uncertainty about possible, often catastrophic, future events. Here we propose that these phenomena are the result of excessive uncertainty regarding state transitions (transition uncertainty): a computational impairment in Bayesian inference leading to a reduced ability to use the past to predict the present and future, and to oversensitivity to feedback (i.e. prediction errors). Using a computational model of Bayesian learning under uncertainty in a reversal learning task, we investigate the relationship between OC symptoms and transition uncertainty. Individuals high and low in OC symptoms performed a task in which they had to detect shifts (i.e. transitions) in cue-outcome contingencies. Modeling subjects' choices was used to estimate each individual participant's transition uncertainty and associated responses to feedback. We examined both an optimal observer model and an approximate Bayesian model in which participants were assumed to attend (and learn about) only one of several cues on each trial. Results suggested the participants were more likely to distribute attention across cues, in accordance with the optimal observer model. As hypothesized, participants with higher OC symptoms exhibited increased transition uncertainty, as well as a pattern of behavior potentially indicative of a difficulty in relying on learned contingencies, with no evidence for perseverative behavior. Increased transition uncertainty compromised these individuals' ability to predict ensuing feedback, rendering them more surprised by expected outcomes. However, no evidence for excessive belief updating was found. These results highlight a potential computational basis for OC symptoms and obsessive compulsive disorder (OCD). The fact the OC symptoms predicted a decreased reliance on the past rather than perseveration challenges preconceptions of OCD as a disorder of inflexibility. Our results have implications for the understanding of the neurocognitive processes leading to excessive uncertainty and distrust of past experiences in OCD.

Aesthetic appreciation of musical intervals enhances behavioural and neurophysiological indexes of attentional engagement and motor inhibition

From Kant to current perspectives in neuroaesthetics, the experience of beauty has been described as disinterested, i.e. focusing on the stimulus perceptual features while neglecting self-referred concerns. At a neurophysiological level, some indirect evidence suggests that disinterested aesthetic appreciation might be associated with attentional enhancement and inhibition of motor behaviour. To test this hypothesis, we performed three auditory-evoked potential experiments, employing consonant and dissonant two-note musical intervals. Twenty-two volunteers judged the beauty of intervals (Aesthetic Judgement task) or responded to them as fast as possible (Detection task). In a third Go-NoGo task, a different group of twenty-two participants had to refrain from responding when hearing intervals. Individual aesthetic judgements positively correlated with response times in the Detection task, with slower motor responses for more appreciated intervals. Electrophysiological indexes of attentional engagement (N1/P2) and motor inhibition (N2/P3) were enhanced for more appreciated intervals. These findings represent the first experimental evidence confirming the disinterested interest hypothesis and may have important applications in research areas studying the effects of stimulus features on learning and motor behaviour.

Beauty in mind: Aesthetic appreciation correlates with perceptual facilitation and attentional amplification

Neuroaesthetic research suggests that aesthetic appreciation results from the interaction between the object perceptual features and the perceiver's sensory processing dynamics. In the present study, we investigated the relationship between aesthetic appreciation and attentional modulation at a behavioural and psychophysiological level. In a first experiment, fifty-eight healthy participants performed a visual search task with abstract stimuli containing more or less natural spatial frequencies and subsequently were asked to give an aesthetic evaluation of the images. The results evidenced that response times were faster for more appreciated stimuli. In a second experiment, we recorded visual evoked potentials (VEPs) during exposure to the same stimuli. The results showed, only for more appreciated images, an enhancement in C1 and N1, P3 and N4 VEP components. Moreover, we found increased attention-related occipital alpha desynchronization for more appreciated images. We interpret these data as indicative of the existence of a correlation between aesthetic appreciation and perceptual processing enhancement, both at a behavioural and at a neurophysiological level.

Dopamine neuron ensembles signal the content of sensory prediction errors

Dopamine neurons respond to errors in predicting value-neutral sensory information. These data, combined with causal evidence that dopamine transients support sensory-based associative learning, suggest that the dopamine system signals a multidimensional prediction error. Yet such complexity is not evident in the activity of individual neurons or population averages. How then do downstream areas know what to learn in response to these signals? One possibility is that information about content is contained in the pattern of firing across many dopamine neurons. Consistent with this, here we show that the pattern of firing across a small group of dopamine neurons recorded in rats signals the identity of a mis-predicted sensory event. Further, this same information is reflected in the BOLD response elicited by sensory prediction errors in human midbrain. These data provide evidence that ensembles of dopamine neurons provide highly specific teaching signals, opening new possibilities for how this system might contribute to learning.

Statistical context dictates the relationship between feedback-related EEG signals and learning

Learning should be adjusted according to the surprise associated with observed outcomes but calibrated according to statistical context. For example, when occasional changepoints are expected, surprising outcomes should be weighted heavily to speed learning. In contrast, when uninformative outliers are expected to occur occasionally, surprising outcomes should be less influential. Here we dissociate surprising outcomes from the degree to which they demand learning using a predictive inference task and computational modeling. We show that the P300, a stimulus-locked electrophysiological response previously associated with adjustments in learning behavior, does so conditionally on the source of surprise. Larger P300 signals predicted greater learning in a changing context, but less learning in a context where surprise was indicative of a one-off outlier (oddball). Our results suggest that the P300 provides a surprise signal that is interpreted by downstream learning processes differentially according to statistical context in order to appropriately calibrate learning across complex environments.

Bayesian modeling of temporal expectations in the human brain

The brain predicts the timing of forthcoming events to optimize processes in response to them. Temporal predictions are driven by both our prior expectations on the likely timing of stimulus occurrence and the information conveyed by the passage of time. Specifically, such predictions can be described in terms of the hazard function, that is, the conditional probability that an event will occur, given it has not yet occurred. Events violating expectations cause surprise and often induce updating of prior expectations. While it is well-known that the brain is able to track the temporal hazard of event occurrence, the question of how prior temporal expectations are updated is still unsettled. Here we combined a Bayesian computational approach with brain imaging to map updating of temporal expectations in the human brain. Moreover, since updating is usually highly correlated with surprise, participants performed a task that allowed partially differentiating between the two processes. Results showed that updating and surprise differently modulated activity in areas belonging to two critical networks for cognitive control, the fronto-parietal (FPN) and the cingulo-opercular network (CON). Overall, these data provide a first computational characterization of the neural correlates associated with updating and surprise related to temporal expectation.

Sensory prediction errors in the human midbrain signal identity violations independent of perceptual distance

The firing of dopaminergic midbrain neurons is thought to reflect prediction errors (PE) that depend on the difference between the value of expected and received rewards. However, recent work has demonstrated that unexpected changes in value-neutral outcome features, such as identity, can evoke similar responses. It remains unclear whether the magnitude of these identity PEs scales with the perceptual dissimilarity of expected and received rewards, or whether they are independent of perceptual similarity. We used a Pavlovian transreinforcer reversal task to elicit identity PEs for value-matched food odor rewards, drawn from two perceptual categories (sweet, savory). Replicating previous findings, identity PEs were correlated with fMRI activity in midbrain, OFC, piriform cortex, and amygdala. However, the magnitude of identity PE responses was independent of the perceptual distance between expected and received outcomes, suggesting that identity comparisons underlying sensory PEs may occur in an abstract state space independent of straightforward sensory percepts.

Bayesian computational markers of relapse in methamphetamine dependence

Methamphetamine use disorder is associated with a high likelihood of relapse. Identifying robust predictors of relapse that have explanatory power is critical to develop secondary prevention based on a mechanistic understanding of relapse. Computational approaches have the potential to identify such predictive markers of psychiatric illness, with the advantage of providing a finer mechanistic explanation of the cognitive processes underlying psychiatric vulnerability.

In this study, sixty-two recently sober methamphetamine-dependent individuals were recruited from a 28-day inpatient treatment program, and completed a Stop Signal Task (SST) while undergoing functional magnetic resonance imaging (fMRI). These individuals were prospectively followed for 1 year and assessed for relapse to methamphetamine use. Thirty-three percent of followed participants reported relapse.

We found that neural activity associated with two types of Bayesian prediction error, i.e. the difference between actual and expected need to stop on a given trial, significantly differentiated those individuals who remained abstinent and those who relapsed. Specifically, relapsed individuals exhibited smaller neural activations to such Bayesian prediction errors relative to those individuals who remained abstinent in the left temporoparietal junction (Cohen's d = 0.91), the left inferior frontal gyrus (Cohen's d = 0.57), and left anterior insula (Cohen's d = 0.63). In contrast, abstinent and relapsed participants did not differ in neural activation to non-model based task contrasts or on various self-report clinical measures.

In conclusion, Bayesian cognitive models may help identify predictive biomarkers of relapse, while providing a computational explanation of belief processing and updating deficits in individuals with methamphetamine use disorder.

•

Methamphetamine-dependent individuals (MDI) face a high rate of relapse after treatment.

•

Can a Bayesian learning modeling and fMRI be used to identify markers of relapse?

•

MDI who relapsed within 1 year have smaller activation to Bayesian model-based prediction errors.

•

Such neural pattern was observed in left temporo-parietal junction, IFG, and anterior insula.

•

MDI more likely to relapse show weaker tracking of uncertainty and updating of their belief model.

Computational mechanisms of curiosity and goal-directed exploration

Successful behaviour depends on the right balance between maximising reward and soliciting information about the world. Here, we show how different types of information-gain emerge when casting behaviour as surprise minimisation. We present two distinct mechanisms for goal-directed exploration that express separable profiles of active sampling to reduce uncertainty. 'Hidden state' exploration motivates agents to sample unambiguous observations to accurately infer the (hidden) state of the world. Conversely, 'model parameter' exploration, compels agents to sample outcomes associated with high uncertainty, if they are informative for their representation of the task structure. We illustrate the emergence of these types of information-gain, termed active inference and active learning, and show how these forms of exploration induce distinct patterns of 'Bayes-optimal' behaviour. Our findings provide a computational framework for understanding how distinct levels of uncertainty systematically affect the exploration-exploitation trade-off in decision-making.

Older adults fail to form stable task representations during model-based reversal inference

Older adults struggle in dealing with changeable and uncertain environments across several cognitive domains. This has been attributed to difficulties in forming adequate task representations that help navigate uncertain environments. Here, we investigate how, in older adults, inadequate task representations impact on model-based reversal learning. We combined computational modeling and pupillometry during a novel model-based reversal learning task, which allowed us to isolate the relevance of task representations at feedback evaluation. We find that older adults overestimate the changeability of task states and consequently are less able to converge on unequivocal task representations through learning. Pupillometric measures and behavioral data show that these unreliable task representations in older adults manifest as a reduced ability to focus on feedback that is relevant for updating task representations, and as a reduced metacognitive awareness in the accuracy of their actions. Instead, the data suggested older adults' choice behavior was more consistent with a guidance by uninformative feedback properties such as outcome valence. Our study highlights that an inability to form adequate task representations may be a crucial factor underlying older adults' impaired model-based inference.

•

Older adults overestimate the changeability of task states in uncertain environments.

•

Unreliable task representations impact model updating in older adults.

•

Older adults focus on outcome valence rather than model updating during feedback.

•

Older adults show reduced metacognitive awareness of their decision-making accuracy.

•

Pupil diameter encodes model updating and not surprise in young and older adults.

Dopaminergic basis for signaling belief updates, but not surprise, and the link to paranoia

Distinguishing between meaningful and meaningless sensory information is fundamental to forming accurate representations of the world. Dopamine is thought to play a central role in processing the meaningful information content of observations, which motivates an agent to update their beliefs about the environment. However, direct evidence for dopamine's role in human belief updating is lacking. We addressed this question in healthy volunteers who performed a model-based fMRI task designed to separate the neural processing of meaningful and meaningless sensory information. We modeled participant behavior using a normative Bayesian observer model and used the magnitude of the model-derived belief update following an observation to quantify its meaningful information content. We also acquired PET imaging measures of dopamine function in the same subjects. We show that the magnitude of belief updates about task structure (meaningful information), but not pure sensory surprise (meaningless information), are encoded in midbrain and ventral striatum activity. Using PET we show that the neural encoding of meaningful information is negatively related to dopamine-2/3 receptor availability in the midbrain and dexamphetamine-induced dopamine release capacity in the striatum. Trial-by-trial analysis of task performance indicated that subclinical paranoid ideation is negatively related to behavioral sensitivity to observations carrying meaningful information about the task structure. The findings provide direct evidence implicating dopamine in model-based belief updating in humans and have implications for understating the pathophysiology of psychotic disorders where dopamine function is disrupted.

Active Inference, Belief Propagation, and the Bethe Approximation

When modeling goal-directed behavior in the presence of various sources of uncertainty, planning can be described as an inference process. A solution to the problem of planning as inference was previously proposed in the active inference framework in the form of an approximate inference scheme based on variational free energy. However, this approximate scheme was based on the mean-field approximation, which assumes statistical independence of hidden variables and is known to show overconfidence and may converge to local minima of the free energy. To better capture the spatiotemporal properties of an environment, we reformulated the approximate inference process using the so-called Bethe approximation. Importantly, the Bethe approximation allows for representation of pairwise statistical dependencies. Under these assumptions, the minimizer of the variational free energy corresponds to the belief propagation algorithm, commonly used in machine learning. To illustrate the differences between the mean-field approximation and the Bethe approximation, we have simulated agent behavior in a simple goal-reaching task with different types of uncertainties. Overall, the Bethe agent achieves higher success rates in reaching goal states. We relate the better performance of the Bethe agent to more accurate predictions about the consequences of its own actions. Consequently, active inference based on the Bethe approximation extends the application range of active inference to more complex behavioral tasks.

Identity prediction errors in the human midbrain update reward-identity expectations in the orbitofrontal cortex

There is general consensus that dopaminergic midbrain neurons signal reward prediction errors, computed as the difference between expected and received reward value. However, recent work in rodents shows that these neurons also respond to errors related to inferred value and sensory features, indicating an expanded role for dopamine beyond learning cached values. Here we utilize a transreinforcer reversal learning task and functional magnetic resonance imaging (fMRI) to test whether prediction error signals in the human midbrain are evoked when the expected identity of an appetitive food odor reward is violated, while leaving value matched. We found that midbrain fMRI responses to identity and value errors are correlated, suggesting a common neural origin for these error signals. Moreover, changes in reward-identity expectations, encoded in the orbitofrontal cortex (OFC), are directly related to midbrain activity, demonstrating that identity-based error signals in the midbrain support the formation of outcome identity expectations in OFC.

Responses in the dopaminergic midbrain are known to signal prediction errors for reward value. Here, the authors show that the human midbrain also encodes errors in predicted reward identity, and that these signals update expectations of reward identity in the orbitofrontal cortex.

Separate neural representations of prediction error valence and surprise: Evidence from an fMRI meta-analysis

Learning occurs when an outcome differs from expectations, generating a reward prediction error signal (RPE). The RPE signal has been hypothesized to simultaneously embody the valence of an outcome (better or worse than expected) and its surprise (how far from expectations). Nonetheless, growing evidence suggests that separate representations of the two RPE components exist in the human brain. Meta-analyses provide an opportunity to test this hypothesis and directly probe the extent to which the valence and surprise of the error signal are encoded in separate or overlapping networks. We carried out several meta-analyses on a large set of fMRI studies investigating the neural basis of RPE, locked at decision outcome. We identified two valence learning systems by pooling studies searching for differential neural activity in response to categorical positive-versus-negative outcomes. The first valence network (negative > positive) involved areas regulating alertness and switching behaviours such as the midcingulate cortex, the thalamus and the dorsolateral prefrontal cortex whereas the second valence network (positive > negative) encompassed regions of the human reward circuitry such as the ventral striatum and the ventromedial prefrontal cortex. We also found evidence of a largely distinct surprise-encoding network including the anterior cingulate cortex, anterior insula and dorsal striatum. Together with recent animal and electrophysiological evidence this meta-analysis points to a sequential and distributed encoding of different components of the RPE signal, with potentially distinct functional roles.

Hierarchical prediction errors in midbrain and septum during social learning

Social learning is fundamental to human interactions, yet its computational and physiological mechanisms are not well understood. One prominent open question concerns the role of neuromodulatory transmitters. We combined fMRI, computational modelling and genetics to address this question in two separate samples (N = 35, N = 47). Participants played a game requiring inference on an adviser’s intentions whose motivation to help or mislead changed over time. Our analyses suggest that hierarchically structured belief updates about current advice validity and the adviser’s trustworthiness, respectively, depend on different neuromodulatory systems. Low-level prediction errors (PEs) about advice accuracy not only activated regions known to support ‘theory of mind’, but also the dopaminergic midbrain. Furthermore, PE responses in ventral striatum were influenced by the Met/Val polymorphism of the Catechol-O-Methyltransferase (COMT) gene. By contrast, high-level PEs (‘expected uncertainty’) about the adviser’s fidelity activated the cholinergic septum. These findings, replicated in both samples, have important implications: They suggest that social learning rests on hierarchically related PEs encoded by midbrain and septum activity, respectively, in the same manner as other forms of learning under volatility. Furthermore, these hierarchical PEs may be broadcast by dopaminergic and cholinergic projections to induce plasticity specifically in cortical areas known to represent beliefs about others.

Optogenetic Blockade of Dopamine Transients Prevents Learning Induced by Changes in Reward Features

Prediction errors are critical for associative learning [1, 2]. Transient changes in dopamine neuron activity correlate with positive and negative reward prediction errors and can mimic their effects [3-15]. However, although causal studies show that dopamine transients of 1-2 s are sufficient to drive learning about reward, these studies do not address whether they are necessary (but see [11]). Further, the precise nature of this signal is not yet fully established. Although it has been equated with the cached-value error signal proposed to support model-free reinforcement learning, cached-value errors are typically confounded with errors in the prediction of reward features [16]. Here, we used optogenetic and transgenic approaches to prevent transient changes in midbrain dopamine neuron activity during the critical error-signaling period of two unblocking tasks. In one, learning was unblocked by increasing the number of rewards, a manipulation that induces errors in predicting both value and reward features. In another, learning was unblocked by switching from one to another equally valued reward, a manipulation that induces errors only in reward feature prediction. Preventing dopamine neurons in the ventral tegmental area from firing for 5 s beginning before and continuing until after the changes in reward prevented unblocking of learning in both tasks. A similar duration suppression did not induce extinction when delivered during an expected reward, indicating that it did not act independently as a negative prediction error. This result suggests that dopamine transients play a general role in error signaling rather than being restricted to only signaling errors in value.

Dopamine Neurons Respond to Errors in the Prediction of Sensory Features of Expected Rewards

Midbrain dopamine neurons have been proposed to signal prediction errors as defined in model-free reinforcement learning algorithms. While these algorithms have been extremely powerful in interpreting dopamine activity, these models do not register any error unless there is a difference between the value of what is predicted and what is received. Yet learning often occurs in response to changes in the unique features that characterize what is received, sometimes with no change in its value at all. Here, we show that classic error-signaling dopamine neurons also respond to changes in value-neutral sensory features of an expected reward. This suggests that dopamine neurons have access to a wider variety of information than contemplated by the models currently used to interpret their activity and that, while their firing may conform to predictions of these models in some cases, they are not restricted to signaling errors in the prediction of value.

A Bayesian perspective on delusions: Suggestions for modifying two reasoning tasks

BACKGROUND AND OBJECTIVES

There are a range of mechanistic explanations on the formation and maintenance of delusions. Within the Bayesian brain hypothesis, particularly within the framework of predictive coding models, delusions are seen as an aberrant inference process characterized by either a failure in sensory attenuation or an aberrant weighting of prior experience. Testing of these Bayesian decision theories requires measuring of both the patients' confidence in their beliefs and the confidence they assign new, incoming information. In the Bayesian framework we apply here, the former is referred to as the prior while the latter is usually called the data or likelihood.

METHODS AND RESULTS

This narrative review will commence by giving an introduction to the basic concept underlying the Bayesian decision theory approach to delusion. A consequence of crucial importance of this sketch is that it provides a measure for the persistence of a belief. Experimental tasks measuring these parameters are presented. Further, a modification of two standard reasoning tasks, the beads task and the evidence integration task, is proposed that permits testing the parameters from Bayesian decision theory.

LIMITATIONS

Patients differ from controls by the distress the delusions causes to them. The Bayesian Decision theory framework has no explicit parameter for distress.

CONCLUSIONS

A more detailed reporting of differences between patients with delusions is warranted.

Neural Mechanisms of Updating under Reducible and Irreducible Uncertainty

Adaptive decision making depends on an agent's ability to use environmental signals to reduce uncertainty. However, because of multiple types of uncertainty, agents must take into account not only the extent to which signals violate prior expectations but also whether uncertainty can be reduced in the first place. Here we studied how human brains of both sexes respond to signals under conditions of reducible and irreducible uncertainty. We show behaviorally that subjects' value updating was sensitive to the reducibility of uncertainty, and could be quantitatively characterized by a Bayesian model where agents ignore expectancy violations that do not update beliefs or values. Using fMRI, we found that neural processes underlying belief and value updating were separable from responses to expectancy violation, and that reducibility of uncertainty in value modulated connections from belief-updating regions to value-updating regions. Together, these results provide insights into how agents use knowledge about uncertainty to make better decisions while ignoring mere expectancy violation.SIGNIFICANCE STATEMENT To make good decisions, a person must observe the environment carefully, and use these observations to reduce uncertainty about consequences of actions. Importantly, uncertainty should not be reduced purely based on how surprising the observations are, particularly because in some cases uncertainty is not reducible. Here we show that the human brain indeed reduces uncertainty adaptively by taking into account the nature of uncertainty and ignoring mere surprise. Behaviorally, we show that human subjects reduce uncertainty in a quasioptimal Bayesian manner. Using fMRI, we characterize brain regions that may be involved in uncertainty reduction, as well as the network they constitute, and dissociate them from brain regions that respond to mere surprise.

Violations of Expectations As Matter for the Believing Process

For the purpose of this communication it is postulated that violation of expectation means a disturbing event or conflict interfering with a previously established mental state that affords a firm belief or confident feeling. According to this hypothesis a violation of an expectation contradicts predictions and intentions that have been attained on stored experiences, valuations, and actual mood. We will argue that the notion of belief as static or stable which is usually described by expressions such as "my belief" or "our general belief" has to be extended to accommodate the process of belief formation. The credition model emphasizes the procedural aspect of belief by which the "process of believing" becomes similar to other psychological processes. We will describe that the "violation of expectation" can be decoded from the credition perspective and has brain functional correlates.

Predictive Processing, Source Monitoring, and Psychosis

A comprehensive understanding of psychosis requires models that link multiple levels of explanation: the neurobiological, the cognitive, the subjective, and the social. Until we can bridge several explanatory gaps, it is difficult to explain how neurobiological perturbations can manifest in bizarre beliefs or hallucinations, or how trauma or social adversity can perturb lower-level brain processes. We propose that the predictive processing framework has much to offer in this respect. We show how this framework may underpin and complement source monitoring theories of delusions and hallucinations and how, when considered in terms of a dynamic and hierarchical system, it may provide a compelling model of several key clinical features of psychosis. We see little conflict between source monitoring theories and predictive coding. The former act as a higher-level description of a set of capacities, and the latter aims to provide a deeper account of how these and other capacities may emerge.

Processes of believing: Where do they come from? What are they good for?

Despite the long scholarly discourse in Western theology and philosophy on religion, spirituality, and faith, explanations of what a belief and what believing is are still lacking. Recently, cognitive neuroscience research addressed the human capacity of believing. We present evidence suggesting that believing is a human brain function which results in probabilistic representations with attributes of personal meaning and value and thereby guides individuals’ behavior. We propose that the same mental processes operating on narratives and rituals constitute belief systems in individuals and social groups. Our theoretical model of believing is suited to account for secular and non-secular belief formation.

Processes of believing: Where do they come from? What are they good for?

Despite the long scholarly discourse in Western theology and philosophy on religion, spirituality, and faith, explanations of what a belief and what believing is are still lacking. Recently, cognitive neuroscience research addressed the human capacity of believing. We present evidence suggesting that believing is a human brain function which results in probabilistic representations with attributes of personal meaning and value and thereby guides individuals' behavior. We propose that the same mental processes operating on narratives and rituals constitute belief systems in individuals and social groups. Our theoretical model of believing is suited to account for secular and non-secular belief formation.

Models of neuromodulation for computational psychiatry

Psychiatry faces fundamental challenges: based on a syndrome-based nosology, it presently lacks clinical tests to infer on disease processes that cause symptoms of individual patients and must resort to trial-and-error treatment strategies. These challenges have fueled the recent emergence of a novel field-computational psychiatry-that strives for mathematical models of disease processes at physiological and computational (information processing) levels. This review is motivated by one particular goal of computational psychiatry: the development of 'computational assays' that can be applied to behavioral or neuroimaging data from individual patients and support differential diagnosis and guiding patient-specific treatment. Because the majority of available pharmacotherapeutic approaches in psychiatry target neuromodulatory transmitters, models that infer (patho)physiological and (patho)computational actions of different neuromodulatory transmitters are of central interest for computational psychiatry. This article reviews the (many) outstanding questions on the computational roles of neuromodulators (dopamine, acetylcholine, serotonin, and noradrenaline), outlines available evidence, and discusses promises and pitfalls in translating these findings to clinical applications. WIREs Cogn Sci 2017, 8:e1420. doi: 10.1002/wcs.1420 For further resources related to this article, please visit the WIREs website.

Prior probabilities modulate cortical surprise responses: A study of event-related potentials

The human brain predicts events in its environment based on expectations, and unexpected events are surprising. When probabilistic contingencies in the environment are precisely instructed, the individual can form expectations based on quantitative probabilistic information ('inference-based learning'). In contrast, when probabilistic contingencies are imprecisely instructed, expectations are formed based on the individual's cumulative experience ('experience-based learning'). Here, we used the urn-ball paradigm to investigate how variations in prior probabilities and in the precision of information about these priors modulate choice behavior and event-related potential (ERP) correlates of surprise. In the urn-ball paradigm, participants are repeatedly forced to infer hidden states responsible for generating observable events, given small samples of factual observations. We manipulated prior probabilities of the states, and we rendered the priors calculable or incalculable, respectively. The analysis of choice behavior revealed that the tendency to consider prior probabilities when making decisions about hidden states was stronger when prior probabilities were calculable, at least in some of our participants. Surprise-related P3b amplitudes were observed in both the calculable and the incalculable prior probability condition. In contrast, calculability of prior probabilities modulated anteriorly distributed ERP amplitudes: when prior probabilities were calculable, surprising events elicited enhanced P3a amplitudes. However, when prior probabilities were incalculable, surprise was associated with enhanced N2 amplitudes. Furthermore, interindividual variability in reliance on prior probabilities was associated with attenuated P3b surprise responses under calculable in comparison to incalculable prior probabilities. Our results suggest two distinct neural systems for probabilistic learning that are recruited depending on contextual cues such as the precision of probabilistic information. Individuals with stronger tendencies to rely on calculable prior probabilities seem to have better adapted expectations at their disposal, as indicated by an attenuation of their P3b surprise responses when prior probabilities are calculable.

Two Anatomically and Computationally Distinct Learning Signals Predict Changes to Stimulus-Outcome Associations in Hippocampus

Complex cognitive processes require sophisticated local processing but also interactions between distant brain regions. It is therefore critical to be able to study distant interactions between local computations and the neural representations they act on. Here we report two anatomically and computationally distinct learning signals in lateral orbitofrontal cortex (lOFC) and the dopaminergic ventral midbrain (VM) that predict trial-by-trial changes to a basic internal model in hippocampus. To measure local computations during learning and their interaction with neural representations, we coupled computational fMRI with trial-by-trial fMRI suppression. We find that suppression in a medial temporal lobe network changes trial-by-trial in proportion to stimulus-outcome associations. During interleaved choice trials, we identify learning signals that relate to outcome type in lOFC and to reward value in VM. These intervening choice feedback signals predicted the subsequent change to hippocampal suppression, suggesting a convergence of signals that update the flexible representation of stimulus-outcome associations.