Enhancing the Psychometric Properties of the Iowa Gambling Task Using Full Generative Modeling

Hot and Cool Executive Function in Children with Autism Spectrum Disorder and Schizotypal Traits

Background: Schizotypal traits are notably prevalent among children diagnosed with Autism spectrum disorder (ASD). Both conditions commonly exhibit impairments in executive functions (EF), which encompass cool and hot processes. The observed deficits in these EF domains across ASD and schizotypy underscore a compelling need to investigate how their co-occurrence impacts EF. Methods: This study investigated the impact of co-occurring autistic and schizotypal traits on EF in 63 children diagnosed with ASD, aged 7 to 12 years. Participants were assessed using the Autism Diagnostic Observation Schedule-2 (ADOS-2), the Melbourne Assessment of Schizotypy in Kids (MASK), and a battery of hot and cool EF tests. Results: Correlational analyses revealed a significant association between MASK score and working memory, as well as between ADOS scores and various cool EF components (i.e., working memory, inhibition and planning). Hierarchical regression analyses showed that the interaction between ADOS and MASK scores significantly predicted performance on hot EF (i.e., affective decision-making), but not on cool EF tasks. Conclusions: These findings suggest that the co-occurrence of ASD and schizotypal traits may have differential effects on cool and hot EF domains. Understanding how the combination of autistic and schizotypal traits affects cognitive processes may inform tailored interventions and support strategies for individuals presenting with these traits.

Does the reliability of computational models truly improve with hierarchical modeling? Some recommendations and considerations for the assessment of model parameter reliability : Reliability of computational model parameters

Computational modeling of behavior is increasingly being adopted as a standard methodology in psychology, cognitive neuroscience, and computational psychiatry. This approach involves estimating parameters in a computational (or cognitive) model that represents the computational processes of the underlying behavior. In this approach, the reliability of the parameter estimates is an important issue. The use of hierarchical (Bayesian) approaches, which place a prior on each model parameter of the individual participants, is thought to improve the reliability of the parameters. However, the characteristics of reliability in parameter estimates, especially when individual-level priors are assumed, as in hierarchical models, have not yet been fully discussed. Furthermore, the suitability of different reliability measures for assessing parameter reliability is not thoroughly understood. In this study, we conduct a systematic examination of these issues through theoretical analysis and numerical simulations, focusing specifically on reinforcement learning models. We note that the heterogeneity in the estimation precision of individual parameters, particularly with priors, can skew reliability measures toward individuals with higher precision. We further note that there are two factors that reduce reliability, namely estimation error and intersession variation in the true parameters, and we discuss how to evaluate these factors separately. Based on the considerations of this study, we present several recommendations and cautions for assessing the reliability of the model parameters.

Test-retest reliability of behavioral and computational measures of advice taking under volatility

The development of computational models for studying mental disorders is on the rise. However, their psychometric properties remain understudied, posing a risk of undermining their use in empirical research and clinical translation. Here we investigated test-retest reliability (with a 2-week interval) of a computational assay probing advice-taking under volatility with a Hierarchical Gaussian Filter (HGF) model. In a sample of 39 healthy participants, we found the computational measures to have largely poor reliability (intra-class correlation coefficient or ICC < 0.5), on par with the behavioral measures of task performance. Further analysis revealed that reliability was substantially impacted by intrinsic measurement noise (indicated by parameter recovery analysis) and to a smaller extent by practice effects. However, a large portion of within-subject variance remained unexplained and may be attributable to state-like fluctuations. Despite the poor test-retest reliability, we found the assay to have face validity at the group level. Overall, our work highlights that the different sources of variance affecting test-retest reliability need to be studied in greater detail. A better understanding of these sources would facilitate the design of more psychometrically sound assays, which would improve the quality of future research and increase the probability of clinical translation.

The effects of threat on complex decision-making: evidence from a virtual environment

Individuals living and working in dangerous settings (e.g., first responders and military personnel) make complex decisions amidst serious threats. However, controlled studies on decision-making under threat are limited given obvious ethical concerns. Here, we embed a complex decision-making task within a threatening, immersive virtual environment. Based on the Iowa Gambling Task (IGT), a paradigm widely used to study complex decision-making, the task requires participants to make a series of choices to escape a collapsing building. In Study 1 we demonstrate that, as with the traditional IGT, participants learn to make advantageous decisions over time and that their behavioural data can be described by reinforcement-learning based computational models. In Study 2 we created threatening and neutral versions of the environment. In the threat condition, participants performed worse, taking longer to improve from baseline and scoring lower through the final trials. Computational modelling further revealed that participants in the threat condition were more responsive to short term rewards and less likely to perseverate on a given choice. These findings suggest that when threat is integral to decision-making, individuals make more erratic choices and focus on short term gains. They furthermore demonstrate the utility of virtual environments for making threat integral to cognitive tasks.

Temporal stability of Bayesian belief updating in perceptual decision-making

Bayesian inference suggests that perception is inferred from a weighted integration of prior contextual beliefs with current sensory evidence (likelihood) about the world around us. The perceived precision or uncertainty associated with prior and likelihood information is used to guide perceptual decision-making, such that more weight is placed on the source of information with greater precision. This provides a framework for understanding a spectrum of clinical transdiagnostic symptoms associated with aberrant perception, as well as individual differences in the general population. While behavioral paradigms are commonly used to characterize individual differences in perception as a stable characteristic, measurement reliability in these behavioral tasks is rarely assessed. To remedy this gap, we empirically evaluate the reliability of a perceptual decision-making task that quantifies individual differences in Bayesian belief updating in terms of the relative precision weighting afforded to prior and likelihood information (i.e., sensory weight). We analyzed data from participants (n = 37) who performed this task twice. We found that the precision afforded to prior and likelihood information showed high internal consistency and good test-retest reliability (ICC = 0.73, 95% CI [0.53, 0.85]) when averaged across participants, as well as at the individual level using hierarchical modeling. Our results provide support for the assumption that Bayesian belief updating operates as a stable characteristic in perceptual decision-making. We discuss the utility and applicability of reliable perceptual decision-making paradigms as a measure of individual differences in the general population, as well as a diagnostic tool in psychiatric research.

Hot and cool executive function and theory of mind in children with and without specific learning disorders

Executive function (EF) in specific learning disorders (SLD) has been investigated using mainly cool EF tasks, whilst less is known about hot EF and theory of mind (ToM) in this population. The aim of this study was to examine group differences in hot and cool EF and ToM in school-aged children with SLD relative to typically developing peers. It also attempted to investigate whether EF measures are significant predictors of ToM in SLD and typical development. Cross-sectional data were collected from 135 school-aged children with and without SLD (8-10 years old), tested on measures of cool & hot EF and ToM. Significant group differences were observed in EFs inhibition (p= .04), working memory (p= .04) and delay of gratification (p < .001), as well as ToM mental state/emotion recognition (p = .019). Inhibition and planning contributed to 22% of the explained variance of ToM mental state/emotion recognition, but not false belief overall. Results suggest that cool EF may be a crucial predictor of ToM in children with and without SLD. Finally, stepwise logistic regression analysis identified specific hot EF and ToM measures contributing to group differentiation, specifically delay of gratification (odds ratio=.995, 95% CI [.993-.998]) and mental state/emotion recognition (odds ratio= .89, 95% CI [.796-.995]). This study contributes to our understanding of cognitive deficits and socio-cognitive impairment in children with SLD, which hold promise for informing interventions aimed at addressing these cognitive challenges.

Dynamic computational phenotyping of human cognition

Computational phenotyping has emerged as a powerful tool for characterizing individual variability across a variety of cognitive domains. An individual's computational phenotype is defined as a set of mechanistically interpretable parameters obtained from fitting computational models to behavioural data. However, the interpretation of these parameters hinges critically on their psychometric properties, which are rarely studied. To identify the sources governing the temporal variability of the computational phenotype, we carried out a 12-week longitudinal study using a battery of seven tasks that measure aspects of human learning, memory, perception and decision making. To examine the influence of state effects, each week, participants provided reports tracking their mood, habits and daily activities. We developed a dynamic computational phenotyping framework, which allowed us to tease apart the time-varying effects of practice and internal states such as affective valence and arousal. Our results show that many phenotype dimensions covary with practice and affective factors, indicating that what appears to be unreliability may reflect previously unmeasured structure. These results support a fundamentally dynamic understanding of cognitive variability within an individual.

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Individual differences in computational psychiatry: a review of current challenges

Bringing precision to the understanding and treatment of mental disorders requires instruments for studying clinically relevant individual differences. One promising approach is the development of computational assays: integrating computational models with cognitive tasks to infer latent patient-specific disease processes in brain computations. While recent years have seen many methodological advancements in computational modelling and many cross-sectional patient studies, much less attention has been paid to basic psychometric properties (reliability and construct validity) of the computational measures provided by the assays. In this review, we assess the extent of this issue by examining emerging empirical evidence. We find that many computational measures suffer from poor psychometric properties, which poses a risk of invalidating previous findings and undermining ongoing research efforts using computational assays to study individual (and even group) differences. We provide recommendations for how to address these problems and, crucially, embed them within a broader perspective on key developments that are needed for translating computational assays to clinical practice.