A bayesian foundation for individual learning under uncertainty

Pain reflects the informational value of nociceptive inputs

ABSTRACT

Pain perception and its modulation are fundamental to human learning and adaptive behavior. This study investigated the hypothesis that pain perception is tied to pain's learning function. Thirty-one participants performed a threat conditioning task where certain cues were associated with a possibility of receiving a painful electric shock. The cues that signaled potential pain or safety were regularly changed, requiring participants to continually establish new associations. Using computational models, we quantified participants' pain expectations and prediction errors throughout the task and assessed their relationship with pain perception and electrophysiological responses. Our findings suggest that subjective pain perception increases with prediction error, that is, when pain was unexpected. Prediction errors were also related to physiological nociceptive responses, including the amplitude of nociceptive flexion reflex and electroencephalography markers of cortical nociceptive processing (N1-P2-evoked potential and gamma-band power). In addition, higher pain expectations were related to increased late event-related potential responses and alpha/beta decreases in amplitude during cue presentation. These results further strengthen the idea of a crucial link between pain and learning and suggest that understanding the influence of learning mechanisms in pain modulation could help us understand when and why pain perception is modulated in health and disease.

Belief Updating, Childhood Maltreatment, and Paranoia in Schizophrenia-Spectrum Disorders

BACKGROUND AND HYPOTHESIS

Exposure to childhood maltreatment-a risk factor for psychosis is associated with paranoia-may impact one's beliefs about the world and how beliefs are updated. We hypothesized that increased exposure to childhood maltreatment is related to volatility-related belief updating, specifically higher expectations of volatility, and that these relationships are strongest for threat-related maltreatment. Additionally, we tested whether belief updating mediates the relationship between maltreatment and paranoia.

STUDY DESIGN

Belief updating was measured in 75 patients with schizophrenia-spectrum disorders and 76 nonpsychiatric controls using a 3-option probabilistic reversal learning (3PRL) task. A Hierarchical Gaussian Filter (HGF) was used to estimate computational parameters of belief updating, including prior expectations of volatility (μ03). The Childhood Trauma Questionnaire (CTQ) was used to assess cumulative maltreatment, threat, and deprivation exposure. Paranoia was measured using the Positive and Negative Syndrome Scale (PANSS) and the revised Green et al. Paranoid Thoughts Scale (R-GPTS).

RESULTS

Greater exposure to childhood maltreatment is associated with higher prior expectations of volatility in the whole sample and in individuals with schizophrenia-spectrum disorders. This was specific to threat-related maltreatment, rather than deprivation, in schizophrenia-spectrum disorders. Paranoia was associated with both exposure to childhood maltreatment and volatility priors, but we did not observe a significant indirect effect of volatility priors on the relationship between maltreatment and paranoia.

CONCLUSIONS

Our study suggests that individuals with schizophrenia-spectrum disorders who were exposed to threatening experiences during childhood expect their environment to be more volatile, potentially facilitating aberrant belief updating and conferring risk for paranoia.

Neurocomputational mechanisms involved in adaptation to fluctuating intentions of others

Humans frequently interact with agents whose intentions can fluctuate between competition and cooperation over time. It is unclear how the brain adapts to fluctuating intentions of others when the nature of the interactions (to cooperate or compete) is not explicitly and truthfully signaled. Here, we use model-based fMRI and a task in which participants thought they were playing with another player. In fact, they played with an algorithm that alternated without signaling between cooperative and competitive strategies. We show that a neurocomputational mechanism with arbitration between competitive and cooperative experts outperforms other learning models in predicting choice behavior. At the brain level, the fMRI results show that the ventral striatum and ventromedial prefrontal cortex track the difference of reliability between these experts. When attributing competitive intentions, we find increased coupling between these regions and a network that distinguishes prediction errors related to competition and cooperation. These findings provide a neurocomputational account of how the brain arbitrates dynamically between cooperative and competitive intentions when making adaptive social decisions.

Incorporating uncertainty within dynamic interoceptive learning

Introduction

Interoception, the perception of the internal state of the body, has been shown to be closely linked to emotions and mental health. Of particular interest are interoceptive learning processes that capture associations between environmental cues and body signals as a basis for making homeostatically relevant predictions about the future. One method of measuring respiratory interoceptive learning that has shown promising results is the Breathing Learning Task (BLT). While the original BLT required binary predictions regarding the presence or absence of an upcoming inspiratory resistance, here we extended this paradigm to capture continuous measures of prediction (un)certainty.

Methods

Sixteen healthy participants completed the continuous version of the BLT, where they were asked to predict the likelihood of breathing resistances on a continuous scale from 0.0 to 10.0. In order to explain participants' responses, a Rescorla-Wagner model of associative learning was combined with suitable observation models for continuous or binary predictions, respectively. For validation, we compared both models against corresponding null models and examined the correlation between observed and modeled predictions. The model was additionally extended to test whether learning rates differed according to stimuli valence. Finally, summary measures of prediction certainty as well as model estimates for learning rates were considered against interoceptive and mental health questionnaire measures.

Results

Our results demonstrated that the continuous model fits closely captured participant behavior using empirical data, and the binarised predictions showed excellent replicability compared to previously collected data. However, the model extension indicated that there were no significant differences between learning rates for negative (i.e. breathing resistance) and positive (i.e. no breathing resistance) stimuli. Finally, significant correlations were found between fatigue severity and both prediction certainty and learning rate, as well as between anxiety sensitivity and prediction certainty.

Discussion

These results demonstrate the utility of gathering enriched continuous prediction data in interoceptive learning tasks, and suggest that the updated BLT is a promising paradigm for future investigations into interoceptive learning and potential links to mental health.

A social inference model of idealization and devaluation

People often form polarized beliefs, imbuing objects (e.g., themselves or others) with unambiguously positive or negative qualities. In clinical settings, this is referred to as dichotomous thinking or "splitting" and is a feature of several psychiatric disorders. Here, we introduce a Bayesian model of splitting that parameterizes a tendency to rigidly categorize objects as either entirely "Bad" or "Good," rather than to flexibly learn dispositions along a continuous scale. Distinct from the previous descriptive theories, the model makes quantitative predictions about how dichotomous beliefs emerge and are updated in light of new information. Specifically, the model addresses how splitting is context-dependent, yet exhibits stability across time. A key model feature is that phases of devaluation and/or idealization are consolidated by rationally attributing counter-evidence to external factors. For example, when another person is idealized, their less-than-perfect behavior is attributed to unfavorable external circumstances. However, sufficient counter-evidence can trigger switches of polarity, producing bistable dynamics. We show that the model can be fitted to empirical data, to measure individual susceptibility to relational instability. For example, we find that a latent categorical belief that others are "Good" accounts for less changeable, and more certain, character impressions of benevolent as opposed to malevolent others among healthy participants. By comparison, character impressions made by participants with borderline personality disorder reveal significantly higher and more symmetric splitting. The generative framework proposed invites applications for modeling oscillatory relational and affective dynamics in psychotherapeutic contexts. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Moral learning and positive schizotypy: Social cognitive mechanisms in psychosis- proneness

BACKGROUND

Delusional thoughts such as paranoia and conspiratorial thinking reflect beliefs in others' intentions to do harm. Given the integral role of harm evaluation in moral cognition, a better understanding of how psychosis-prone individuals process others' moral characters may provide insights into social cognitive mechanisms of these types of delusions.

METHODS

An online sample of 293 participants was drawn from the general population, using Amazon Mechanical Turk. Participants performed a moral inference task, where they predicted and judged the binary choices of two fictitious agents ("good" or "bad") to impose harm under different levels of financial incentives. An investment game involving the same agents then examined participants' trust behavior. Psychosis-proneness was measured with the Multidimensional Schizotypy Scale Brief Edition.

RESULTS

A set of multiple regressions showed that positive schizotypy was associated with a lower yet more confident pre-experimental expectation of the agent's moral character, lower prediction accuracy of the agent's harm preferences, less belief revision, and undifferentiated perception of the good and bad agents' characters. Positive schizotypy was also related to higher expectations for reciprocity in the investment game, regardless of agent characters.

CONCLUSION

Our findings suggest that inflexible beliefs associated with psychosis-proneness extend beyond negative prior expectations, also reflecting difficulties in moral learning. The resulting undifferentiated moral impressions might contribute to undue suspicion of benevolent individuals and increased gullibility to malicious ones, potentially further strengthening conspiratorial beliefs.

The mechanisms underlying conditioning of phantom percepts differ between those with hallucinations and synesthesia

There are many different kinds of 'phantom' percepts but it is unknown whether they are united by common mechanisms. For example, synaesthesia (e.g., numbers evoking colour) and hallucinations appear conceptually and phenomenologically similar: both result in a percept that does not have an environmental correlate. Here, people with synaesthesia (n = 66) performed a conditioned hallucinations paradigm known to be sensitive to hallucination susceptibility, and we asked whether synaesthetes would show the same behavioural profile as hallucinators in this task. Repeated pairing of checkerboards with tones, and gratings with colours encourages the participant to draw on prior knowledge when asked to report on the presence of the difficult-to-detect target stimulus. Synaesthetes show increased modelled expectancies for the stimulus association across the board, resulting in a higher number of detections at all stimulus intensities. This is in contrast to the pattern observed in hallucinators, who weigh their prior beliefs more strongly than controls, giving rise to more conditioned hallucinations. Results indicate that fundamentally different perceptual processes may be at the core of these seemingly similar experiences.

Evidence for Reduced Sensory Precision and Increased Reliance on Priors in Hallucination-Prone Individuals in a General Population Sample

BACKGROUND

There is increasing evidence that people with hallucinations overweight perceptual beliefs relative to incoming sensory evidence. Past work demonstrating prior overweighting has used simple, nonlinguistic stimuli. However, auditory hallucinations in psychosis are often complex and linguistic. There may be an interaction between the type of auditory information being processed and its perceived quality in engendering hallucinations.

STUDY DESIGN

We administered a linguistic version of the conditioned hallucinations (CH) task to an online sample of 88 general population participants. Metrics related to hallucination-proneness, hallucination severity, stimulus thresholds, and stimulus detection rates were collected. Data were used to fit parameters of a Hierarchical Gaussian Filter (HGF) model of perceptual inference to determine how latent perceptual states influenced task behavior.

STUDY RESULTS

Replicating past results, higher CH rates were observed both in those with recent hallucinatory experiences as well as participants with high hallucination-proneness; CH rates were positively correlated with increased prior weighting; and increased prior weighting was related to hallucination severity. Unlike past results, participants with recent hallucinatory experiences as well as those with higher hallucination-proneness had higher stimulus thresholds, lower sensitivity to stimuli presented at the highest threshold, and had lower response confidence, consistent with lower precision of sensory evidence.

CONCLUSIONS

We replicate the finding that increased CH rates and recent hallucinations correlate with increased prior weighting using a linguistic version of the CH task. Results support a role for reduced sensory precision in the interplay between prior weighting and hallucination-proneness.

Learning environment-specific learning rates

People often have to switch back and forth between different environments that come with different problems and volatilities. While volatile environments require fast learning (i.e., high learning rates), stable environments call for lower learning rates. Previous studies have shown that people adapt their learning rates, but it remains unclear whether they can also learn about environment-specific learning rates, and instantaneously retrieve them when revisiting environments. Here, using optimality simulations and hierarchical Bayesian analyses across three experiments, we show that people can learn to use different learning rates when switching back and forth between two different environments. We even observe a signature of these environment-specific learning rates when the volatility of both environments is suddenly the same. We conclude that humans can flexibly adapt and learn to associate different learning rates to different environments, offering important insights for developing theories of meta-learning and context-specific control.

Evaluating the Influence of Musical and Monetary Rewards on Decision Making through Computational Modelling

A central question in behavioural neuroscience is how different rewards modulate learning. While the role of monetary rewards is well-studied in decision-making research, the influence of abstract rewards like music remains poorly understood. This study investigated the dissociable effects of these two reward types on decision making. Forty participants completed two decision-making tasks, each characterised by probabilistic associations between stimuli and rewards, with probabilities changing over time to reflect environmental volatility. In each task, choices were reinforced either by monetary outcomes (win/lose) or by the endings of musical melodies (consonant/dissonant). We applied the Hierarchical Gaussian Filter, a validated hierarchical Bayesian framework, to model learning under these two conditions. Bayesian statistics provided evidence for similar learning patterns across both reward types, suggesting individuals' similar adaptability. However, within the musical task, individual preferences for consonance over dissonance explained some aspects of learning. Specifically, correlation analyses indicated that participants more tolerant of dissonance behaved more stochastically in their belief-to-response mappings and were less likely to choose the response associated with the current prediction for a consonant ending, driven by higher volatility estimates. By contrast, participants averse to dissonance showed increased tonic volatility, leading to larger updates in reward tendency beliefs.

The empirical status of predictive coding and active inference

Research on predictive processing models has focused largely on two specific algorithmic theories: Predictive Coding for perception and Active Inference for decision-making. While these interconnected theories possess broad explanatory potential, they have only recently begun to receive direct empirical evaluation. Here, we review recent studies of Predictive Coding and Active Inference with a focus on evaluating the degree to which they are empirically supported. For Predictive Coding, we find that existing empirical evidence offers modest support. However, some positive results can also be explained by alternative feedforward (e.g., feature detection-based) models. For Active Inference, most empirical studies have focused on fitting these models to behavior as a means of identifying and explaining individual or group differences. While Active Inference models tend to explain behavioral data reasonably well, there has not been a focus on testing empirical validity of active inference theory per se, which would require formal comparison to other models (e.g., non-Bayesian or model-free reinforcement learning models). This review suggests that, while promising, a number of specific research directions are still necessary to evaluate the empirical adequacy and explanatory power of these algorithms.

Bayesian Decision-Making Under Uncertainty in Borderline Personality Disorder

Bayesian decision theory suggests that optimal decision-making should use and weigh prior beliefs with current information, according to their relative uncertainties. However, some characteristics of borderline personality disorder (BPD) patients, such as fast, drastic changes in the overall perception of themselves and others, suggest they may be underrelying on priors. Here, we investigated if BPD patients have a general deficit in relying on or combining prior with current information. We analyzed this by having BPD patients (n = 23) and healthy controls (n = 18) perform a coin-catching sensorimotor task with varying levels of prior and current information uncertainty. Our results indicate that BPD patients learned and used prior information and combined it with current information in a qualitatively Bayesian-like way. Our results show that, at least in a lower-level, nonsocial sensorimotor task, BPD patients can appropriately use both prior and current information, illustrating that potential deficits using priors may not be widespread or domain-general.

What's next?: Time is subjectively dilated not only for 'oddball' events, but also for events immediately after oddballs

Our experience of time is strikingly plastic: Depending on contextual factors, the same objective duration can seem to fly by or drag on. Perhaps the most direct demonstration of such subjective time dilation is the oddball effect: when seeing identical objects appear one after another, followed by an "oddball" (e.g., a disc that suddenly grows in size, in a sequence of otherwise static discs), observers experience this oddball as having lasted longer than its nonoddball counterparts. Despite extensive work on this phenomenon, a surprisingly foundational question remains unasked: What actually gets dilated? Beyond the oddball, are the objects just before (or just after) the oddball also dilated? As in previous studies, observers viewed sequences of colored discs, one of which could be the oddball-and subsequently reproduced the oddball's duration. Unlike previous studies, however, there were also critical trials in which observers instead reproduced the duration of the disc immediately before or after the oddball. A clear pattern emerged: oddball-induced time dilation extended to the post-oddball disc, but not the pre-oddball disc. Whence this temporal asymmetry? We suggest that an oddball's sudden appearance may induce uncertainty about what will happen next, heightening attention until after the uncertainty is resolved.

Aberrant Hierarchical Prediction Errors Are Associated With Transition to Psychosis: A Computational Single-Trial Analysis of the Mismatch Negativity

BACKGROUND

Mismatch negativity reductions are among the most reliable biomarkers for schizophrenia and have been associated with increased risk for conversion to psychosis in individuals who are at clinical high risk for psychosis (CHR-P). Here, we adopted a computational approach to develop a mechanistic model of mismatch negativity reductions in CHR-P individuals and patients early in the course of schizophrenia.

METHODS

Electroencephalography was recorded in 38 CHR-P individuals (15 converters), 19 patients early in the course of schizophrenia (≤5 years), and 44 healthy control participants during three different auditory oddball mismatch negativity paradigms including 10% duration, frequency, or double deviants, respectively. We modeled sensory learning with the hierarchical Gaussian filter and extracted precision-weighted prediction error trajectories from the model to assess how the expression of hierarchical prediction errors modulated electroencephalography amplitudes over sensor space and time.

RESULTS

Both low-level sensory and high-level volatility precision-weighted prediction errors were altered in CHR-P individuals and patients early in the course of schizophrenia compared with healthy control participants. Moreover, low-level precision-weighted prediction errors were significantly different in CHR-P individuals who later converted to psychosis compared with nonconverters.

CONCLUSIONS

Our results implicate altered processing of hierarchical prediction errors as a computational mechanism in early psychosis consistent with predictive coding accounts of psychosis. This computational model seems to capture pathophysiological mechanisms that are relevant to early psychosis and the risk for future psychosis in CHR-P individuals and may serve as predictive biomarkers and mechanistic targets for the development of novel treatments.

On the relationship between emotions and cognitive control: Evidence from an observational study on emotional priming Stroop task

Evidence is discordant regarding how emotional processing and cognitive control interact to shape behavior. This observational study sought to examine this interaction by looking at the distinction between proactive and reactive modes of control and how they relate to emotional processing. Seventy-four healthy participants performed an emotional priming Stroop task. On each trial, target stimuli of a spatial Stroop task were preceded by sad or neutral facial expressions, providing two emotional conditions. To manipulate the requirement of both proactive and reactive control, the proportion of congruent trials (PC) was varied at the list-wide (LWPC) and item-specific (ISPC) levels, respectively. We found that sad priming led to behavioral costs only in trials with low proactive and reactive cognitive control demands. Our findings suggest that emotional processing affects cognitive processes other than cognitive control in the Stroop task. Moreover, both proactive and reactive control modes seem effective in overcoming emotional interference of priming stimuli.

Modulation of prefrontal couplings by prior belief-related responses in ventromedial prefrontal cortex

Humans and other animals can maintain constant payoffs in an uncertain environment by steadily re-evaluating and flexibly adjusting current strategy, which largely depends on the interactions between the prefrontal cortex (PFC) and mediodorsal thalamus (MD). While the ventromedial PFC (vmPFC) represents the level of uncertainty (i.e., prior belief about external states), it remains unclear how the brain recruits the PFC-MD network to re-evaluate decision strategy based on the uncertainty. Here, we leverage non-linear dynamic causal modeling on fMRI data to test how prior belief-dependent activity in vmPFC gates the information flow in the PFC-MD network when individuals switch their decision strategy. We show that the prior belief-related responses in vmPFC had a modulatory influence on the connections from dorsolateral PFC (dlPFC) to both, lateral orbitofrontal (lOFC) and MD. Bayesian parameter averaging revealed that only the connection from the dlPFC to lOFC surpassed the significant threshold, which indicates that the weaker the prior belief, the less was the inhibitory influence of the vmPFC on the strength of effective connections from dlPFC to lOFC. These findings suggest that the vmPFC acts as a gatekeeper for the recruitment of processing resources to re-evaluate the decision strategy in situations of high uncertainty.

Neurocomputations on dual-brain signals underlie interpersonal prediction during a natural conversation

Prediction on the partner's speech plays a key role in a smooth conversation. However, previous studies on this issue have been majorly conducted at the single-brain rather than dual-brain level, leaving the interpersonal prediction hypothesis untested. To fill this gap, this study combined a neurocomputational modeling approach with a natural conversation paradigm in which two salespersons persuaded a customer to buy their product with their haemodynamic signals being collected using functional near-infrared spectroscopy hyperscanning. First, the results showed a cognitive hierarchy in a natural conversation, with the lower-level process (i.e., pragmatic representation of the persuasion) in the salesperson interacting with the higher-level process (i.e., value representation of the product) in the customer. Next, we found that the right dorsal lateral prefrontal cortex (rdlPFC) and temporoparietal junction (rTPJ) were associated with the representation of the product's value in the customer, while the right inferior frontal cortex (rIFC) was associated with the representation of the pragmatic processes in the salesperson. Finally, neurocomputational modeling results supported the prediction of the salesperson's lower-level brain activity based on the customer's higher-level brain activity. Moreover, the updating weight of the prediction model based on the neural computation between the rIFC of the salesperson and the rTPJ of the customer was closely associated with the interaction context, whereas that based on the rIFC-rdlPFC was not. In summary, these findings provide initial support for the interpersonal prediction hypothesis at the dual-brain level and reveal a hierarchy for the interpersonal prediction process.

Modeling face recognition in the predictive coding framework: A combined computational modeling and functional imaging study

The learning of new facial identities and the recognition of familiar faces are crucial processes for social interactions. Recently, a combined computational modeling and functional magnetic resonance imaging (fMRI) study used predictive coding as a biologically plausible framework to model face identity learning and to relate specific model parameters with brain activity (Apps and Tsakiris, Nat Commun 4, 2698, 2013). On the one hand, it was shown that behavioral responses on a two-option face recognition task could be predicted by the level of contextual and facial familiarity in a computational model derived from predictive-coding principles. On the other hand, brain activity in specific brain regions was associated with these parameters. More specifically, brain activity in the superior temporal sulcus (STS) varied with contextual familiarity, whereas activity in the fusiform face area (FFA) covaried with the prediction error parameter that updated facial familiarity. Literature combining fMRI assessments and computational modeling in humans still needs to be expanded. Furthermore, prior results are largely not replicated. The present study was, therefore, specifically set up to replicate these previous findings. Our results support the original findings in two critical aspects. First, on a group level, the behavioral responses were modeled best by the same computational model reported by the original authors. Second, we showed that estimates of these model parameters covary with brain activity in specific, face-sensitive brain regions. Our results thus provide further evidence that the functional properties of the face perception network conform to central principles of predictive coding. However, our study yielded diverging findings on specific computational model parameters reflected in brain activity. On the one hand, we did not find any evidence of a computational involvement of the STS. On the other hand, our results showed that activity in the right FFA was associated with multiple computational model parameters. Our data do not provide evidence for functional segregation between particular face-sensitive brain regions, as previously proposed.

Anxiety associated with perceived uncontrollable stress enhances expectations of environmental volatility and impairs reward learning

Unavoidable stress can lead to perceived lack of control and learned helplessness, a risk factor for depression. Avoiding punishment and gaining rewards involve updating the values of actions based on experience. Such updating is however useful only if action values are sufficiently stable, something that a lack of control may impair. We examined whether self-reported stress uncontrollability during the first wave of the COVID-19 pandemic predicted impaired reward-learning. In a preregistered study during the first-wave of the COVID-19 pandemic, we used self-reported measures of depression, anxiety, uncontrollable stress, and COVID-19 risk from 427 online participants to predict performance in a three-armed-bandit probabilistic reward learning task. As hypothesised, uncontrollable stress predicted impaired learning, and a greater proportion of probabilistic errors following negative feedback for correct choices, an effect mediated by state anxiety. A parameter from the best-fitting hidden Markov model that estimates expected beliefs that the identity of the optimal choice will shift across images, mediated effects of state anxiety on probabilistic errors and learning deficits. Our findings show that following uncontrollable stress, anxiety promotes an overly volatile representation of the reward-structure of uncertain environments, impairing reward attainment, which is a potential path to anhedonia in depression.

Subanaesthetic doses of ketamine reduce but do not eliminate predictive coding responses: implications for mechanisms of sensory disconnection

BACKGROUND

Sensory disconnection is a key feature of sleep and anaesthesia. We have proposed that predictive coding offers a framework for understanding the mechanisms of disconnection. Low doses of ketamine that do not induce disconnection should thus diminish predictive coding, but not abolish it.

METHODS

Ketamine was administered to 14 participants up to a blood concentration of 0.3 μg ml-1 Participants were played a series of tones comprising a roving oddball sequence while electroencephalography evoked response potentials were recorded. We fit a Bayesian observer model to the tone sequence, correlating neural activity with the prediction errors generated by the model using linear mixed effects models and cluster-based statistics.

RESULTS

Ketamine modulated prediction errors associated with the transition of one tone to the next (transitional probability), but not how often tones changed (environmental volatility), of the system. Transitional probability was reduced when blood concentrations of ketamine were increased to 0.2-0.3 μg ml-1 (96-208 ms, P=0.003); however, correlates of prediction error were still evident in the electroencephalogram (124-168 ms, P=0.003). Prediction errors related to environmental volatility were associated with electroencephalographic activity before ketamine (224-284 ms, P=0.028) and during 0.2-0.3 μg ml-1 ketamine (108-248 ms, P=0.003). At this subanaesthetic dose, ketamine did not exert a dose-dependent modulation of prediction error.

CONCLUSIONS

Subanaesthetic dosing of ketamine reduced correlates of predictive coding but did not eliminate them. Future studies should evaluate whether states of sensory disconnection, including anaesthetic doses of ketamine, are associated with a complete absence of predictive coding responses.

CLINICAL TRIAL REGISTRATION

NCT03284307.

Relative fluency (unfelt vs felt) in active inference

For a growing number of researchers, it is now accepted that the brain is a predictive organ that predicts the content of the sensorium and crucially the precision of-or confidence in-its own predictions. In order to predict the precision of its predictions, the brain has to infer the reliability of its own beliefs. This means that our brains have to recognise the precision of their predictions or, at least, their accuracy. In this paper, we argue that fluency is product of this recognition process. In short, to recognise fluency is to infer that we have a precise 'grip' on the unfolding processes that generate our sensations. More specifically, we propose that it is changes in fluency - from unfelt to felt - that are both recognised and realised when updating predictions about precision. Unfelt fluency orients attention to unpredicted sensations, while felt fluency supervenes on-and contextualises-unfelt fluency; thereby rendering certain attentional processes, phenomenologically opaque. As such, fluency underwrites the precision we place in our predictions and therefore acts upon our perceptual inferences. Hence, the causes of conscious subjective inference have unconscious perceptual precursors.

Testing predictive coding theories of autism spectrum disorder using models of active inference

Several competing neuro-computational theories of autism have emerged from predictive coding models of the brain. To disentangle their subtly different predictions about the nature of atypicalities in autistic perception, we performed computational modelling of two sensorimotor tasks: the predictive use of manual gripping forces during object lifting and anticipatory eye movements during a naturalistic interception task. In contrast to some accounts, we found no evidence of chronic atypicalities in the use of priors or weighting of sensory information during object lifting. Differences in prior beliefs, rates of belief updating, and the precision weighting of prediction errors were, however, observed for anticipatory eye movements. Most notably, we observed autism-related difficulties in flexibly adapting learning rates in response to environmental change (i.e., volatility). These findings suggest that atypical encoding of precision and context-sensitive adjustments provide a better explanation of autistic perception than generic attenuation of priors or persistently high precision prediction errors. Our results did not, however, support previous suggestions that autistic people perceive their environment to be persistently volatile.

Control dilemma: Evidence of the stability-flexibility trade-off

Cognitive control can be applied flexibly when task goals or environments change (i.e., cognitive flexibility), or stably to pursue a goal in the face of distraction (i.e., cognitive stability). Whether these seemingly contradictory characteristics have an inverse relationship has been controversial, as some studies have suggested a trade-off mechanism between cognitive flexibility and cognitive stability, while others have not found such reciprocal associations. This study investigated the possible antagonistic correlation between cognitive flexibility and stability using a novel version of the flexibility-stability paradigm and the classic cued task switching paradigm. In Experiment 1, we showed that cognitive flexibility was inversely correlated with cognitive stability, as increased distractor proportions were associated with decreased cognitive flexibility and greater cognitive stability. Moreover, cognitive flexibility and stability were regulated by a single control system instead of two independent control mechanisms, as the model selection results indicated that the reciprocally regulated model with one integration parameter outperformed all other models, and the model parameter was inversely linked to cognitive flexibility and stability. We found similar results using the classic cued task switching paradigm in Experiment 2. Therefore, a trade-off between cognitive flexibility and stability was observed from the paradigms used in this study.

Reliability of gamified reinforcement learning in densely sampled longitudinal assessments

Reinforcement learning is a core facet of motivation and alterations have been associated with various mental disorders. To build better models of individual learning, repeated measurement of value-based decision-making is crucial. However, the focus on lab-based assessment of reward learning has limited the number of measurements and the test-retest reliability of many decision-related parameters is therefore unknown. In this paper, we present an open-source cross-platform application Influenca that provides a novel reward learning task complemented by ecological momentary assessment (EMA) of current mental and physiological states for repeated assessment over weeks. In this task, players have to identify the most effective medication by integrating reward values with changing probabilities to win (according to random Gaussian walks). Participants can complete up to 31 runs with 150 trials each. To encourage replay, in-game screens provide feedback on the progress. Using an initial validation sample of 384 players (9729 runs), we found that reinforcement learning parameters such as the learning rate and reward sensitivity show poor to fair intra-class correlations (ICC: 0.22-0.53), indicating substantial within- and between-subject variance. Notably, items assessing the psychological state showed comparable ICCs as reinforcement learning parameters. To conclude, our innovative and openly customizable app framework provides a gamified task that optimizes repeated assessments of reward learning to better quantify intra- and inter-individual differences in value-based decision-making over time.

Why mismatch negativity continues to hold potential in probing altered brain function in schizophrenia

The brain potential known as mismatch negativity (MMN) is one of the most studied indices of altered brain function in schizophrenia. This review looks at what has been learned about MMN in schizophrenia over the last three decades and why the level of interest and activity in this field of research remains strong. A diligent consideration of available evidence suggests that MMN can serve as a biomarker in schizophrenia, but perhaps not the kind of biomarker that early research supposed. This review concludes that MMN measurement is likely to be most useful as a monitoring and response biomarker enabling tracking of an underlying pathology and efficacy of interventions, respectively. The role of, and challenges presented by, pre-clinical models is discussed as well as the merits of different methodologies that can be brought to bear in pursuing a deeper understanding of pathophysiology that might explain smaller MMN in schizophrenia.

The impact of spatial variance on precision estimates in an auditory oddball paradigm

Predictive processing theories suggest that a principal function of the brain is to reduce the surprise of incoming sensory information by creating accurate and precise models of the environment. These models are commonly explored by looking at the prediction errors elicited when experience departs from predictions. One such prediction error is the mismatch negativity (MMN). Using this component, it is possible to examine the effect of external noise on the precision of the developed model. Recent studies have shown that the brain may not update its model every time there is a change in the environment, rather it will only update it when doing so will increase precision and or accuracy of the model. The current study examined this process using oddball sound sequences with high and low spatial variability and examining how this affected the elicited MMN to a duration deviant sound. The results showed a strong null effect of spatial variance both at a local and sequence levels. These results indicate that variability in the sound sequence will not invariably affect model precision estimates and thus the amplitude of the MMN component.

Learning of irrelevant stimulus-response associations modulates cognitive control

It has been shown that manipulating the proportion of congruent to incongruent trials in conflict tasks (e.g., Stroop, Simon, and flanker tasks) can vary the size of conflict effects, however, by two different mechanisms. One theory is the control learning account (the brain learns the probability of conflict and uses it to proactively adjust the control demand for future trials). The other is the irrelevant stimulus-response learning account (the brain learns the probability of irrelevant stimulus-response associations and uses it to prepare responses). Previous fMRI studies have detected the brain regions that contribute to the control-learning-modulated conflict effects, but it is less known what neural substrates underlie the conflict effects modulated by irrelevant S-R learning. We here investigated this question with a model-based fMRI study, in which the proportion of congruent to incongruent trials changed dynamically in the Simon task and the models learned the probability of irrelevant S-R associations quantitatively. Behavioral analyses showed that the unsigned prediction errors (PEs) of responses generated by the learning models correlated with reaction times irrespective of congruent and incongruent trials, indicating that large unsigned PEs associated with slow responses. The fMRI results showed that the regions of fronto-parietal and cingulo-opercular network involved in cognitive control were significantly modulated by the unsigned PEs, also irrespective of congruent and incongruent trials, indicating that large unsigned PEs associated with transiently increased activity in these regions. These results together suggest that learning of irrelevant S-R associations modulates reactive control, which demonstrates a new way to modulate cognitive control compared to the control learning account.

Liraglutide restores impaired associative learning in individuals with obesity

Survival under selective pressure is driven by the ability of our brain to use sensory information to our advantage to control physiological needs. To that end, neural circuits receive and integrate external environmental cues and internal metabolic signals to form learned sensory associations, consequently motivating and adapting our behaviour. The dopaminergic midbrain plays a crucial role in learning adaptive behaviour and is particularly sensitive to peripheral metabolic signals, including intestinal peptides, such as glucagon-like peptide 1 (GLP-1). In a single-blinded, randomized, controlled, crossover basic human functional magnetic resonance imaging study relying on a computational model of the adaptive learning process underlying behavioural responses, we show that adaptive learning is reduced when metabolic sensing is impaired in obesity, as indexed by reduced insulin sensitivity (participants: N = 30 with normal insulin sensitivity; N = 24 with impaired insulin sensitivity). Treatment with the GLP-1 receptor agonist liraglutide normalizes impaired learning of sensory associations in men and women with obesity. Collectively, our findings reveal that GLP-1 receptor activation modulates associative learning in people with obesity via its central effects within the mesoaccumbens pathway. These findings provide evidence for how metabolic signals can act as neuromodulators to adapt our behaviour to our body's internal state and how GLP-1 receptor agonists work in clinics.

Blocking D2/D3 dopamine receptors in male participants increases volatility of beliefs when learning to trust others

The ability to learn about other people is crucial for human social functioning. Dopamine has been proposed to regulate the precision of beliefs, but direct behavioural evidence of this is lacking. In this study, we investigate how a high dose of the D2/D3 dopamine receptor antagonist sulpiride impacts learning about other people's prosocial attitudes in a repeated Trust game. Using a Bayesian model of belief updating, we show that in a sample of 76 male participants sulpiride increases the volatility of beliefs, which leads to higher precision weights on prediction errors. This effect is driven by participants with genetically conferred higher dopamine availability (Taq1a polymorphism) and remains even after controlling for working memory performance. Higher precision weights are reflected in higher reciprocal behaviour in the repeated Trust game but not in single-round Trust games. Our data provide evidence that the D2 receptors are pivotal in regulating prediction error-driven belief updating in a social context.

P3-like signatures of temporal predictions: a computational EEG study

Many cognitive processes, ranging from perception to action, depend on the ability to predict the timing of forthcoming events. Yet, how the brain uses predictive models in the temporal domain is still an unsolved question. In previous work, we began to explore the neural correlates of temporal predictions by using a computational approach in which an ideal Bayesian observer learned the temporal probabilities of target onsets in a simple reaction time task. Because the task was specifically designed to disambiguate updating of predictive models and surprise, changes in temporal probabilities were explicitly cued. However, in the real world, we are usually incidentally exposed to changes in the statistics of the environment. Here, we thus aimed to further investigate the electroencephalographic (EEG) correlates of Bayesian belief updating and surprise associated with incidental learning of temporal probabilities. In line with our previous EEG study, results showed distinct P3-like modulations for updating and surprise. While surprise was indexed by an early fronto-central P3-like modulation, updating was associated with a later and more posterior P3 modulation. Moreover, updating was associated with a P2-like potential at centro-parietal electrodes, likely capturing integration processes between prior beliefs and likelihood of the observed event. These findings support previous evidence of trial-by-trial variability of P3 amplitudes as an index of dissociable inferential processes. Coupled with our previous findings, the present study strongly bolsters the view of the P3 as a key brain signature of temporal Bayesian inference. Data and scripts are shared on OSF: osf.io/sdy8j/.

Neural correlates of hierarchical predictive processes in autistic adults

Bayesian theories of autism spectrum disorders (ASD) suggest that atypical predictive mechanisms could underlie the autistic symptomatology, but little is known about their neural correlates. Twenty-six neurotypical (NT) and 26 autistic adults participated in an fMRI study where they performed an associative learning task in a volatile environment. By inverting a model of perceptual inference, we characterized the neural correlates of hierarchically structured predictions and prediction errors in ASD. Behaviorally, the predictive abilities of autistic adults were intact. Neurally, predictions were encoded hierarchically in both NT and ASD participants and biased their percepts. High-level predictions were following activity levels in a set of regions more closely in ASD than NT. Prediction errors yielded activation in shared regions in NT and ASD, but group differences were found in the anterior cingulate cortex and putamen. This study sheds light on the neural specificities of ASD that might underlie atypical predictive processing.

Human orbitofrontal cortex signals decision outcomes to sensory cortex during behavioral adaptations

The ability to respond flexibly to an ever-changing environment relies on the orbitofrontal cortex (OFC). However, how the OFC associates sensory information with predicted outcomes to enable flexible sensory learning in humans remains elusive. Here, we combine a probabilistic tactile reversal learning task with functional magnetic resonance imaging (fMRI) to investigate how lateral OFC (lOFC) interacts with the primary somatosensory cortex (S1) to guide flexible tactile learning in humans. fMRI results reveal that lOFC and S1 exhibit distinct task-dependent engagement: while the lOFC responds transiently to unexpected outcomes immediately following reversals, S1 is persistently engaged during re-learning. Unlike the contralateral stimulus-selective S1, activity in ipsilateral S1 mirrors the outcomes of behavior during re-learning, closely related to top-down signals from lOFC. These findings suggest that lOFC contributes to teaching signals to dynamically update representations in sensory areas, which implement computations critical for adaptive behavior.

Relationships between cognitive biases, decision-making, and delusions

Multiple measures of decision-making under uncertainty (e.g. jumping to conclusions (JTC), bias against disconfirmatory evidence (BADE), win-switch behavior, random exploration) have been associated with delusional thinking in independent studies. Yet, it is unknown whether these variables explain shared or unique variance in delusional thinking, and whether these relationships are specific to paranoia or delusional ideation more broadly. Additionally, the underlying computational mechanisms require further investigation. To investigate these questions, task and self-report data were collected in 88 individuals (46 healthy controls, 42 schizophrenia-spectrum) and included measures of cognitive biases and behavior on probabilistic reversal learning and explore/exploit tasks. Of those, only win-switch rate significantly differed between groups. In regression, reversal learning performance, random exploration, and poor evidence integration during BADE showed significant, independent associations with paranoia. Only self-reported JTC was associated with delusional ideation, controlling for paranoia. Computational parameters increased the proportion of variance explained in paranoia. Overall, decision-making influenced by strong volatility and variability is specifically associated with paranoia, whereas self-reported hasty decision-making is specifically associated with other themes of delusional ideation. These aspects of decision-making under uncertainty may therefore represent distinct cognitive processes that, together, have the potential to worsen delusional thinking across the psychosis spectrum.

Transdiagnostic computations of uncertainty: towards a new lens on intolerance of uncertainty

People radically differ in how they cope with uncertainty. Clinical researchers describe a dispositional characteristic known as "intolerance of uncertainty", a tendency to find uncertainty aversive, reported to be elevated across psychiatric and neurodevelopmental conditions. Concurrently, recent research in computational psychiatry has leveraged theoretical work to characterise individual differences in uncertainty processing. Under this framework, differences in how people estimate different forms of uncertainty can contribute to mental health difficulties. In this review, we briefly outline the concept of intolerance of uncertainty within its clinical context, and we argue that the mechanisms underlying this construct may be further elucidated through modelling how individuals make inferences about uncertainty. We will review the evidence linking psychopathology to different computationally specified forms of uncertainty and consider how these findings might suggest distinct mechanistic routes towards intolerance of uncertainty. We also discuss the implications of this computational approach for behavioural and pharmacological interventions, as well as the importance of different cognitive domains and subjective experiences in studying uncertainty processing.

Assumed shared belief about conspiracy theories in social networks protects paranoid individuals against distress

Paranoia is the belief that others intend you harm. It is related to conspiracy theories, wherein those others represent an organized faction, coordinating the harm against self and others, and violating societal norms. Current psychological studies of paranoid conspiracy theorizing focus either on the individual or their broader social network. Likewise, theories of belief formation and updating often contain individual level processes as well as broader interpersonal and organizational factors. Here we examine paranoia and conspiracy theorizing in terms of individual behavioral predictors (performance on a probabilistic reversal learning task which assays belief updating) as well as social sensing: we ask participants to report the features of their social network, including whether their friends and acquaintances share their paranoid conspiratorial beliefs. We find that people who believe paranoid conspiracy theories expect more volatility during the task. They also assume that members of their social network share their paranoid beliefs. Critically, those participants with larger social networks and greater assumed shared belief tend to harbor their conspiratorial beliefs with less emotional distress and expect less volatility in the task. This is evidence that, like political and religious beliefs, conspiracy theories may flourish under a sacred canopy of belief consensus. These data suggest that friends and acquaintances may serve as sources of credulity and moving between them may sustain conspiracy beliefs when there is detraction. This hybrid individual/social account may shed light on clinical paranoia and persecutory delusion, wherein disability is defined normatively, and social supports are fewer.

Individual learning as a driver of changes in community vulnerability under repeated hurricanes and changing climate

The risks from singular natural hazards such as a hurricane have been extensively investigated in the literature. However, little is understood about how individual and collective responses to repeated hazards change communities and impact their preparation for future events. Individual mitigation actions may drive how a community's resilience evolves under repeated hazards. In this paper, we investigate the effect that learning by homeowners can have on household mitigation decisions and on how this influences a region's vulnerability to natural hazards over time, using hurricanes along the east coast of the United States as our case study. To do this, we build an agent-based model (ABM) to simulate homeowners' adaptation to repeated hurricanes and how this affects the vulnerability of the regional housing stock. Through a case study, we explore how different initial beliefs about the hurricane hazard and how the memory of recent hurricanes could change a community's vulnerability both under current and potential future hurricane scenarios under climate change. In some future hurricane environments, different initial beliefs can result in large differences in the region's long-term vulnerability to hurricanes. We find that when some homeowners mitigate soon after a hurricane-when their memory of the event is the strongest-it can help to substantially decrease the vulnerability of a community.

Aberrant uncertainty processing is linked to psychotic-like experiences, autistic traits, and is reflected in pupil dilation during probabilistic learning

Aberrant belief updating due to misestimation of uncertainty and an increased perception of the world as volatile (i.e., unstable) has been found in autism and psychotic disorders. Pupil dilation tracks events that warrant belief updating, likely reflecting the adjustment of neural gain. However, whether subclinical autistic or psychotic symptoms affect this adjustment and how they relate to learning in volatile environments remains to be unraveled. We investigated the relationship between behavioral and pupillometric markers of subjective volatility (i.e., experience of the world as unstable), autistic traits, and psychotic-like experiences in 52 neurotypical adults with a probabilistic reversal learning task. Computational modeling revealed that participants with higher psychotic-like experience scores overestimated volatility in low-volatile task periods. This was not the case for participants scoring high on autistic-like traits, who instead showed a diminished adaptation of choice-switching behavior in response to risk. Pupillometric data indicated that individuals with higher autistic- or psychotic-like trait and experience scores differentiated less between events that warrant belief updating and those that do not when volatility was high. These findings are in line with misestimation of uncertainty accounts of psychosis and autism spectrum disorders and indicate that aberrancies are already present at the subclinical level.

Anterior cingulate and medial prefrontal cortex oscillations underlie learning alterations in trait anxiety in humans

Anxiety has been linked to altered belief formation and uncertainty estimation, impacting learning. Identifying the neural processes underlying these changes is important for understanding brain pathology. Here, we show that oscillatory activity in the medial prefrontal, anterior cingulate and orbitofrontal cortex (mPFC, ACC, OFC) explains anxiety-related learning alterations. In a magnetoencephalography experiment, two groups of human participants pre-screened with high and low trait anxiety (HTA, LTA: 39) performed a probabilistic reward-based learning task. HTA undermined learning through an overestimation of volatility, leading to faster belief updating, more stochastic decisions and pronounced lose-shift tendencies. On a neural level, we observed increased gamma activity in the ACC, dmPFC, and OFC during encoding of precision-weighted prediction errors in HTA, accompanied by suppressed ACC alpha/beta activity. Our findings support the association between altered learning and belief updating in anxiety and changes in gamma and alpha/beta activity in the ACC, dmPFC, and OFC.

Habitual daily intake of a sweet and fatty snack modulates reward processing in humans

Western diets rich in fat and sugar promote excess calorie intake and weight gain; however, the underlying mechanisms are unclear. Despite a well-documented association between obesity and altered brain dopamine function, it remains elusive whether these alterations are (1) pre-existing, increasing the individual susceptibility to weight gain, (2) secondary to obesity, or (3) directly attributable to repeated exposure to western diet. To close this gap, we performed a randomized, controlled study (NCT05574660) with normal-weight participants exposed to a high-fat/high-sugar snack or a low-fat/low-sugar snack for 8 weeks in addition to their regular diet. The high-fat/high-sugar intervention decreased the preference for low-fat food while increasing brain response to food and associative learning independent of food cues or reward. These alterations were independent of changes in body weight and metabolic parameters, indicating a direct effect of high-fat, high-sugar foods on neurobehavioral adaptations that may increase the risk for overeating and weight gain.

Computational Modeling of Oddball Sequence Processing Exposes Common and Differential Auditory Network Changes in First-Episode Schizophrenia-Spectrum Disorders and Schizophrenia

BACKGROUND AND HYPOTHESIS

Differences in sound relevance filtering in schizophrenia are proposed to represent a key index of biological changes in brain function in the illness. This study featured a computational modeling approach to test the hypothesis that processing differences might already be evident in first-episode, becoming more pronounced in the established illness.

STUDY DESIGN

Auditory event-related potentials to a typical oddball sequence (rare pitch deviations amongst regular sounds) were recorded from 90 persons with schizophrenia-spectrum disorders (40 first-episode schizophrenia-spectrum, 50 established illness) and age-matched healthy controls. The data were analyzed using dynamic causal modeling to identify the changes in effective connectivity that best explained group differences.

STUDY RESULTS

Group differences were linked to intrinsic (within brain region) connectivity changes. In activity-dependent measures these were restricted to the left auditory cortex in first-episode schizophrenia-spectrum but were more widespread in the established illness. Modeling suggested that both established illness and first-episode schizophrenia-spectrum groups expressed significantly lower inhibition of inhibitory interneuron activity and altered gain on superficial pyramidal cells with the data indicative of differences in both putative N-methyl-d-aspartate glutamate receptor activity-dependent plasticity and classic neuromodulation.

CONCLUSIONS

The study provides further support for the notion that examining the ability to alter responsiveness to structured sound sequences in schizophrenia and first-episode schizophrenia-spectrum could be informative to uncovering the nature and progression of changes in brain function during the illness. Furthermore, modeling suggested that limited differences present at first-episode schizophrenia-spectrum may become more expansive with illness progression.

Modulation of Motor Vigor by Expectation of Reward Probability Trial-by-Trial Is Preserved in Healthy Ageing and Parkinson's Disease Patients

Motor improvements, such as faster movement times or increased velocity, have been associated with reward magnitude in deterministic contexts. Yet whether individual inferences on reward probability influence motor vigor dynamically remains undetermined. We investigated how dynamically inferring volatile action-reward contingencies modulated motor performance trial-by-trial. We conducted three studies that coupled a reversal learning paradigm with a motor sequence task and used a validated hierarchical Bayesian model to fit trial-by-trial data. In Study 1, we tested healthy younger [HYA; 37 (24 females)] and older adults [HOA; 37 (17 females)], and medicated Parkinson's disease (PD) patients [20 (7 females)]. We showed that stronger predictions about the tendency of the action-reward contingency led to faster performance tempo, commensurate with movement time, on a trial-by-trial basis without robustly modulating reaction time (RT). Using Bayesian linear mixed models, we demonstrated a similar invigoration effect on performance tempo in HYA, HOA, and PD, despite HOA and PD being slower than HYA. In Study 2 [HYA, 39 (29 females)], we additionally showed that retrospective subjective inference about credit assignment did not contribute to differences in motor vigor effects. Last, Study 3 [HYA, 33 (27 females)] revealed that explicit beliefs about the reward tendency (confidence ratings) modulated performance tempo trial-by-trial. Our study is the first to reveal that the dynamic updating of beliefs about volatile action-reward contingencies positively biases motor performance through faster tempo. We also provide robust evidence for a preserved sensitivity of motor vigor to inferences about the action-reward mapping in aging and medicated PD.SIGNIFICANCE STATEMENT Navigating a world rich in uncertainty relies on updating beliefs about the probability that our actions lead to reward. Here, we investigated how inferring the action-reward contingencies in a volatile environment modulated motor vigor trial-by-trial in healthy younger and older adults, and in Parkinson's disease (PD) patients on medication. We found an association between trial-by-trial predictions about the tendency of the action-reward contingency and performance tempo, with stronger expectations speeding the movement. We additionally provided evidence for a similar sensitivity of performance tempo to the strength of these predictions in all groups. Thus, dynamic beliefs about the changing relationship between actions and their outcome enhanced motor vigor. This positive bias was not compromised by age or Parkinson's disease.

A characterization of the neural representation of confidence during probabilistic learning

Learning in a stochastic and changing environment is a difficult task. Models of learning typically postulate that observations that deviate from the learned predictions are surprising and used to update those predictions. Bayesian accounts further posit the existence of a confidence-weighting mechanism: learning should be modulated by the confidence level that accompanies those predictions. However, the neural bases of this confidence are much less known than the ones of surprise. Here, we used a dynamic probability learning task and high-field MRI to identify putative cortical regions involved in the representation of confidence about predictions during human learning. We devised a stringent test based on the conjunction of four criteria. We localized several regions in parietal and frontal cortices whose activity is sensitive to the confidence of an ideal observer, specifically so with respect to potential confounds (surprise and predictability), and in a way that is invariant to which item is predicted. We also tested for functionality in two ways. First, we localized regions whose activity patterns at the subject level showed an effect of both confidence and surprise in qualitative agreement with the confidence-weighting principle. Second, we found neural representations of ideal confidence that also accounted for subjective confidence. Taken together, those results identify a set of cortical regions potentially implicated in the confidence-weighting of learning.

A novel computational approach to pain perception modelling within a Bayesian framework using quantitative sensory testing

Pain perception can be studied as an inferential process in which prior information influences the perception of nociceptive input. To date, there are no suitable psychophysical paradigms to measure this at an individual level. We developed a quantitative sensory testing paradigm allowing for quantification of the influence of prior expectations versus current nociceptive input during perception. Using a Pavlovian-learning task, we investigated the influence of prior expectations on the belief about the varying strength of association between a painful electrical cutaneous stimulus and a visual cue in healthy subjects (N = 70). The belief in cue-pain associations was examined with computational modelling using a Hierarchical Gaussian Filter (HGF). Prior weighting estimates in the HGF model were compared with the established measures of conditioned pain modulation (CPM) and temporal summation of pain (TSP) assessed by cuff algometry. Subsequent HGF-modelling and estimation of the influence of prior beliefs on perception showed that 70% of subjects had a higher reliance on nociceptive input during perception of acute pain stimuli, whereas 30% showed a stronger weighting of prior expectations over sensory evidence. There was no association between prior weighting estimates and CPM or TSP. The data demonstrates relevant individual differences in prior weighting and suggests an importance of top-down cognitive processes on pain perception. Our new psychophysical testing paradigm provides a method to identify individuals with traits suggesting greater reliance on prior expectations in pain perception, which may be a risk factor for developing chronic pain and may be differentially responsive to learning-based interventions.

Brain connectivity modulation by Bayesian surprise in relation to control demand drives cognitive flexibility via control engagement

Human control is characterized by its flexibility and adaptability in response to the conditional probability in the environment. Previous studies have revealed that efficient conflict control could be attained by predicting and adapting to the changing control demand. However, it is unclear whether cognitive flexibility could also be gained by predicting and adapting to the changing control demand. The present study aimed to explore this issue by combining the model-based analyses of behavioral and neuroimaging data with a probabilistic cued task switching paradigm. We demonstrated that the Bayesian surprise (i.e. unsigned precision-weighted prediction error [PE]) negatively modulated the connections among stimulus processing brain regions and control regions/networks. The effect of Bayesian surprise modulation on these connections guided control engagement as reflected by the control PE effect on behavior, which in turn facilitated cognitive flexibility. These results bridge a gap in the literature by illustrating the neural and behavioral effect of control demand prediction (or PE) on cognitive flexibility and offer novel insights into the source of switch cost and the mechanism of cognitive flexibility.

Functional connectivity abnormalities of brain networks in obsessive–compulsive disorder: a systematic review

Obsessive-compulsive disorder (OCD) is characterized by cognitive abnormalities encompassing several executive processes. Neuroimaging studies highlight functional abnormalities of executive fronto-parietal network (FPN) and default-mode network (DMN) in OCD patients, as well as of the prefrontal cortex (PFC) more specifically. We aim at assessing the presence of functional connectivity (FC) abnormalities of intrinsic brain networks and PFC in OCD, possibly underlying specific computational impairments and clinical manifestations. A systematic review of resting-state fMRI studies investigating FC was conducted in unmedicated OCD patients by querying three scientific databases (PubMed, Scopus, PsycInfo) up to July 2022 (search terms: “obsessive–compulsive disorder” AND “resting state” AND “fMRI” AND “function* *connect*” AND “task-positive” OR “executive” OR “central executive” OR “executive control” OR “executive-control” OR “cognitive control” OR “attenti*” OR “dorsal attention” OR “ventral attention” OR “frontoparietal” OR “fronto-parietal” OR “default mode” AND “network*” OR “system*”). Collectively, 20 studies were included. A predominantly reduced FC of DMN – often related to increased symptom severity – emerged. Additionally, intra-network FC of FPN was predominantly increased and often positively related to clinical scores. Concerning PFC, a predominant hyper-connectivity of right-sided prefrontal links emerged. Finally, FC of lateral prefrontal areas correlated with specific symptom dimensions. Several sources of heterogeneity in methodology might have affected results in unpredictable ways and were discussed. Such findings might represent endophenotypes of OCD manifestations, possibly reflecting computational impairments and difficulties in engaging in self-referential processes or in disengaging from cognitive control and monitoring processes.

Understanding mental health through computers: An introduction to computational psychiatry

Computational psychiatry recently established itself as a new tool in the study of mental disorders and problems. Integration of different levels of analysis is creating computational phenotypes with clinical and research values, and constructing a way to arrive at precision psychiatry are part of this new branch. It conceptualizes the brain as a computational organ that receives from the environment parameters to respond to challenges through calculations and algorithms in continuous feedback and feedforward loops with a permanent degree of uncertainty. Through this conception, one can seize an understanding of the cerebral and mental processes in the form of theories or hypotheses based on data. Using these approximations, a better understanding of the disorder and its different determinant factors facilitates the diagnostics and treatment by having an individual, ecologic, and holistic approach. It is a tool that can be used to homologate and integrate multiple sources of information given by several theoretical models. In conclusion, it helps psychiatry achieve precision and reproducibility, which can help the mental health field achieve significant advancement. This article is a narrative review of the basis of the functioning of computational psychiatry with a critical analysis of its concepts.

Two-stage reinforcement learning task predicts psychological traits

External sources of information influence human actions. However, psychological traits (PTs), considered internal variables, also play a crucial role in decision making. PTs are stable across time and contexts and define the set of behavioral repertoires that individuals express. Here, we explored how multiple metrics of adaptive behavior under uncertainty related to several PTs. Participants solved a reversal-learning task with volatile contingencies, from which we characterized a detailed behavioral profile based on their response sequences. We then tested the relationship between this multimetric behavioral profile and scores obtained from self-report psychological questionnaires. The PT measurements were based on the Hierarchical Taxonomy Of Psychopathology (HiTOP) model. By using multiple linear regression models (MLRMs), we found that the learning curves predicted important differences in the PTs and task response times. We confirmed the significance of these relationships by using random permutations of the predictors of the MLRM. Therefore, the behavioral profile configurations predicted the PTs and served as a "fingerprint" to identify participants with a high certainty level. We discuss briefly how this characterization and approach could contribute to better nosological classifications.

Reaction Time "Mismatch Costs" Change with the Likelihood of Stimulus-Response Compatibility

Dyadic interactions require dynamic correspondence between one's own movements and those of the other agent. This mapping is largely viewed as imitative, with the behavioural hallmark being a reaction-time cost for mismatched actions. Yet the complex motor patterns humans enact together extend beyond direct-matching, varying adaptively between imitation, complementary movements, and counter-imitation. Optimal behaviour requires an agent to predict not only what is likely to be observed but also how that observed action will relate to their own motor planning. In 28 healthy adults, we examined imitation and counter-imitation in a task that varied the likelihood of stimulus-response congruence from highly predictable, to moderately predictable, to unpredictable. To gain mechanistic insights into the statistical learning of stimulus-response compatibility, we compared two computational models of behaviour: (1) a classic fixed learning-rate model (Rescorla-Wagner reinforcement [RW]) and (2) a hierarchical model of perceptual-behavioural processes in which the learning rate adapts to the inferred environmental volatility (hierarchical Gaussian filter [HGF]). Though more complex and hence penalized by model selection, the HGF provided a more likely model of the participants' behaviour. Matching motor responses were only primed (faster) in the most experimentally volatile context. This bias was reversed so that mismatched actions were primed when beliefs about volatility were lower. Inferential statistics indicated that matching responses were only primed in unpredictable contexts when stimuli-response congruence was at 50:50 chance. Outside of these unpredictable blocks the classic stimulus-response compatibility effect was reversed: Incongruent responses were faster than congruent ones. We show that hierarchical Bayesian learning of environmental statistics may underlie response priming during dyadic interactions.

Emotions as computations

Emotions ubiquitously impact action, learning, and perception, yet their essence and role remain widely debated. Computational accounts of emotion aspire to answer these questions with greater conceptual precision informed by normative principles and neurobiological data. We examine recent progress in this regard and find that emotions may implement three classes of computations, which serve to evaluate states, actions, and uncertain prospects. For each of these, we use the formalism of reinforcement learning to offer a new formulation that better accounts for existing evidence. We then consider how these distinct computations may map onto distinct emotions and moods. Integrating extensive research on the causes and consequences of different emotions suggests a parsimonious one-to-one mapping, according to which emotions are integral to how we evaluate outcomes (pleasure & pain), learn to predict them (happiness & sadness), use them to inform our (frustration & content) and others' (anger & gratitude) actions, and plan in order to realize (desire & hope) or avoid (fear & anxiety) uncertain outcomes.