Modeling the Covariance Structure of Complex Datasets Using Cognitive Models: An Application to Individual Differences and the Heritability of Cognitive Ability

Winner Takes All! What Are Race Models, and Why and How Should Psychologists Use Them?

Interest in the processes that mediate between stimuli and responses is at the heart of most modern psychology and neuroscience. These processes cannot be directly measured but instead must be inferred from observed responses. Race models, through their ability to account for both response choices and response times, have been a key enabler of such inferences. Examples of such models appeared contemporaneously with the cognitive revolution, and since then have become increasingly prominent and elaborated, so that psychologists now have a powerful array of race models at their disposal. We showcase the state of the art for race models and describe why and how they are used.

Behavioural and neural indices of perceptual decision-making in autistic children during visual motion tasks

Many studies report atypical responses to sensory information in autistic individuals, yet it is not clear which stages of processing are affected, with little consideration given to decision-making processes. We combined diffusion modelling with high-density EEG to identify which processing stages differ between 50 autistic and 50 typically developing children aged 6-14 years during two visual motion tasks. Our pre-registered hypotheses were that autistic children would show task-dependent differences in sensory evidence accumulation, alongside a more cautious decision-making style and longer non-decision time across tasks. We tested these hypotheses using hierarchical Bayesian diffusion models with a rigorous blind modelling approach, finding no conclusive evidence for our hypotheses. Using a data-driven method, we identified a response-locked centro-parietal component previously linked to the decision-making process. The build-up in this component did not consistently relate to evidence accumulation in autistic children. This suggests that the relationship between the EEG measure and diffusion-modelling is not straightforward in autistic children. Compared to a related study of children with dyslexia, motion processing differences appear less pronounced in autistic children. Exploratory analyses also suggest weak evidence that ADHD symptoms moderate perceptual decision-making in autistic children.

Task-general efficiency of evidence accumulation as a computationally-defined neurocognitive trait: Implications for clinical neuroscience

Quantifying individual differences in higher-order cognitive functions is a foundational area of cognitive science that also has profound implications for research on psychopathology. For the last two decades, the dominant approach in these fields has been to attempt to fractionate higher-order functions into hypothesized components (e.g., "inhibition", "updating") through a combination of experimental manipulation and factor analysis. However, the putative constructs obtained through this paradigm have recently been met with substantial criticism on both theoretical and empirical grounds. Concurrently, an alternative approach has emerged focusing on parameters of formal computational models of cognition that have been developed in mathematical psychology. These models posit biologically plausible and experimentally validated explanations of the data-generating process for cognitive tasks, allowing them to be used to measure the latent mechanisms that underlie performance. One of the primary insights provided by recent applications of such models is that individual and clinical differences in performance on a wide variety of cognitive tasks, ranging from simple choice tasks to complex executive paradigms, are largely driven by efficiency of evidence accumulation (EEA), a computational mechanism defined by sequential sampling models. This review assembles evidence for the hypothesis that EEA is a central individual difference dimension that explains neurocognitive deficits in multiple clinical disorders and identifies ways in which in this insight can advance clinical neuroscience research. We propose that recognition of EEA as a major driver of neurocognitive differences will allow the field to make clearer inferences about cognitive abnormalities in psychopathology and their links to neurobiology.

Visual Motion and Decision-Making in Dyslexia: Reduced Accumulation of Sensory Evidence and Related Neural Dynamics

Children with and without dyslexia differ in their behavioral responses to visual information, particularly when required to pool dynamic signals over space and time. Importantly, multiple processes contribute to behavioral responses. Here we investigated which processing stages are affected in children with dyslexia when performing visual motion processing tasks, by combining two methods that are sensitive to the dynamic processes leading to responses. We used a diffusion model which decomposes response time and accuracy into distinct cognitive constructs, and high-density EEG. Fifty children with dyslexia (24 male) and 50 typically developing children (28 male) 6-14 years of age judged the direction of motion as quickly and accurately as possible in two global motion tasks (motion coherence and direction integration), which varied in their requirements for noise exclusion. Following our preregistered analyses, we fitted hierarchical Bayesian diffusion models to the data, blinded to group membership. Unblinding revealed reduced evidence accumulation in children with dyslexia compared with typical children for both tasks. Additionally, we identified a response-locked EEG component which was maximal over centro-parietal electrodes which indicated a neural correlate of reduced drift rate in dyslexia in the motion coherence task, thereby linking brain and behavior. We suggest that children with dyslexia tend to be slower to extract sensory evidence from global motion displays, regardless of whether noise exclusion is required, thus furthering our understanding of atypical perceptual decision-making processes in dyslexia.SIGNIFICANCE STATEMENT Reduced sensitivity to visual information has been reported in dyslexia, with a lively debate about whether these differences causally contribute to reading difficulties. In this large preregistered study with a blind modeling approach, we combine state-of-the art methods in both computational modeling and EEG analysis to pinpoint the stages of processing that are atypical in children with dyslexia in two visual motion tasks that vary in their requirement for noise exclusion. We find reduced evidence accumulation in children with dyslexia across both tasks, and identify a neural marker, allowing us to link brain and behavior. We show that children with dyslexia exhibit general difficulties with extracting sensory evidence from global motion displays, not just in tasks that require noise exclusion.

A cognitive model of response omissions in distraction paradigms

The effects of distraction on responses manifest in three ways: prolonged reaction times, and increased error and response omission rates. However, the latter effect is often ignored or assumed to be due to a separate cognitive process. We investigated omissions occurring in two paradigms that manipulated distraction. One required simple stimulus detection of younger participants, the second required choice responses and was completed by both younger and older participants. We fit data from these paradigms with a model that identifies three causes of omissions: two are related to the process of accumulating the evidence on which a response is based: intrinsic omissions (due to between-trial variation in accumulation rates making it impossible to ever reach the evidence threshold) and design omissions (due to response windows that cause slow responses not to be recorded; a third, contaminant omissions, allows for a cause unrelated to the response process. In both data sets systematic differences in omission rates across conditions were accounted for by task-related omissions. Intrinsic omissions played a lesser role than design omissions, even though the presence of design omissions was not evident in descriptive analyses of the data. The model provided an accurate account of all aspects of the detection data and the choice-response data, but slightly underestimated overall omissions in the choice paradigm, particularly in older participants, suggesting that further investigation of contaminant omission effects is needed.

Preregistration in diverse contexts: a preregistration template for the application of cognitive models

In recent years, open science practices have become increasingly popular in psychology and related sciences. These practices aim to increase rigour and transparency in science as a potential response to the challenges posed by the replication crisis. Many of these reforms-including the increasingly used preregistration-have been designed for purely experimental work that tests straightforward hypotheses with standard inferential statistical analyses, such as assessing whether an experimental manipulation has an effect on a variable of interest. But psychology is a diverse field of research. The somewhat narrow focus of the prevalent discussions surrounding and templates for preregistration has led to debates on how appropriate these reforms are for areas of research with more diverse hypotheses and more intricate methods of analysis, such as cognitive modelling research within mathematical psychology. Our article attempts to bridge the gap between open science and mathematical psychology, focusing on the type of cognitive modelling that Crüwell et al. (Crüwell S, Stefan AM, Evans NJ. 2019 Robust standards in cognitive science. Comput. Brain Behav. 2, 255-265) labelled model application, where researchers apply a cognitive model as a measurement tool to test hypotheses about parameters of the cognitive model. Specifically, we (i) discuss several potential researcher degrees of freedom within model application, (ii) provide the first preregistration template for model application and (iii) provide an example of a preregistered model application using our preregistration template. More broadly, we hope that our discussions and concrete proposals constructively advance the mostly abstract current debate surrounding preregistration in cognitive modelling, and provide a guide for how preregistration templates may be developed in other diverse or intricate research contexts.

Evidence accumulation and associated error-related brain activity as computationally-informed prospective predictors of substance use in emerging adulthood

RATIONALE

Substance use peaks during the developmental period known as emerging adulthood (ages 18-25), but not every individual who uses substances during this period engages in frequent or problematic use. Although individual differences in neurocognition appear to predict use severity, mechanistic neurocognitive risk factors with clear links to both behavior and neural circuitry have yet to be identified. Here, we aim to do so with an approach rooted in computational psychiatry, an emerging field in which formal models are used to identify candidate biobehavioral dimensions that confer risk for psychopathology.

OBJECTIVES

We test whether lower efficiency of evidence accumulation (EEA), a computationally characterized individual difference variable that drives performance on the go/no-go and other neurocognitive tasks, is a risk factor for substance use in emerging adults.

METHODS AND RESULTS

In an fMRI substudy within a sociobehavioral longitudinal study (n = 106), we find that lower EEA and reductions in a robust neural-level correlate of EEA (error-related activations in salience network structures) measured at ages 18-21 are both prospectively related to greater substance use during ages 22-26, even after adjusting for other well-known risk factors. Results from Bayesian model comparisons corroborated inferences from conventional hypothesis testing and provided evidence that both EEA and its neuroimaging correlates contain unique predictive information about substance use involvement.

CONCLUSIONS

These findings highlight EEA as a computationally characterized neurocognitive risk factor for substance use during a critical developmental period, with clear links to both neuroimaging measures and well-established formal theories of brain function.

Think fast! The implications of emphasizing urgency in decision-making

Evidence accumulation models (EAMs) have become the dominant explanation of how the decision-making process operates, proposing that decisions are the result of a process of evidence accumulation. The primary use of EAMs has been as "measurement tools" of the underlying decision-making process, where researchers apply EAMs to empirical data to estimate participants' task ability (i.e., the "drift rate"), response caution (i.e., the "decision threshold"), and the time taken for other processes (i.e., the "non-decision time"), making EAMs a powerful tool for discriminating between competing psychological theories. Recent studies have brought into question the mapping between the latent parameters of EAMs and the theoretical constructs that they are thought to represent, showing that emphasizing urgent responding - which intuitively should selectively influence decision threshold - may also influence drift rate and/or non-decision time. However, these findings have been mixed, leading to differences in opinion between experts in the field. The current study aims to provide a more conclusive answer to the implications of emphasizing urgent responding, providing a re-analysis of 6 data sets from previous studies using two different EAMs - the diffusion model and the linear ballistic accumulator (LBA) - with state-of-the-art methods for model selection based inference. The findings display clear evidence for a difference in conclusions between the two models, with the diffusion model suggesting that decision threshold and non-decision time decrease when urgency is emphasized, and the LBA suggesting that decision threshold and drift rate decrease when urgency is emphasized. Furthermore, although these models disagree regarding whether non-decision time or drift rate decrease under urgency emphasis, both show clear evidence that emphasizing urgency does not selectively influence decision threshold. These findings suggest that researchers should revise their assumptions about certain experimental manipulations, the specification of certain EAMs, or perhaps both.

A method, framework, and tutorial for efficiently simulating models of decision-making

Evidence accumulation models (EAMs) have become the dominant models of rapid decision-making. Several variants of these models have been proposed, ranging from the simple linear ballistic accumulator (LBA) to the more complex leaky-competing accumulator (LCA), and further extensions that include time-varying rates of evidence accumulation or decision thresholds. Although applications of the simpler variants have been widespread, applications of the more complex models have been fewer, largely due to their intractable likelihood function and the computational cost of mass simulation. Here, I present a framework for efficiently fitting complex EAMs, which uses a new, efficient method of simulating these models. I find that the majority of simulation time is taken up by random number generation (RNG) from the normal distribution, needed for the stochastic noise of the differential equation. To reduce this inefficiency, I propose using the well-known concept within computer science of “look-up tables” (LUTs) as an approximation to the inverse cumulative density function (iCDF) method of RNG, which I call “LUT-iCDF”. I show that when using an appropriately sized LUT, simulations using LUT-iCDF closely match those from the standard RNG method in R. My framework, which I provide a detailed tutorial on how to implement, includes C code for 12 different variants of EAMs using the LUT-iCDF method, and should make the implementation of complex EAMs easier and faster.

An Integrated Approach to Testing Dynamic, Multilevel Theory: Using Computational Models to Connect Theory, Model, and Data

Some of the most influential theories in organizational sciences explicitly describe a dynamic, multilevel process. Yet the inherent complexity of such theories makes them difficult to test. These theories often describe multiple subprocesses that interact reciprocally over time at different levels of analysis and over different time scales. Computational (i.e., mathematical) modeling is increasingly advocated as a method for developing and testing theories of this type. In organizational sciences, however, efforts that have been made to test models empirically are often indirect. We argue that the full potential of computational modeling as a tool for testing dynamic, multilevel theory is yet to be realized. In this article, we demonstrate an approach to testing dynamic, multilevel theory using computational modeling. The approach uses simulations to generate model predictions and Bayesian parameter estimation to fit models to empirical data and facilitate model comparisons. This approach enables a direct integration between theory, model, and data that we believe enables a more rigorous test of theory.

Assessing the practical differences between model selection methods in inferences about choice response time tasks

Evidence accumulations models (EAMs) have become the dominant modeling framework within rapid decision-making, using choice response time distributions to make inferences about the underlying decision process. These models are often applied to empirical data as "measurement tools", with different theoretical accounts being contrasted within the framework of the model. Some method is then needed to decide between these competing theoretical accounts, as only assessing the models on their ability to fit trends in the empirical data ignores model flexibility, and therefore, creates a bias towards more flexible models. However, there is no objectively optimal method to select between models, with methods varying in both their computational tractability and theoretical basis. I provide a systematic comparison between nine different model selection methods using a popular EAM-the linear ballistic accumulator (LBA; Brown & Heathcote, Cognitive Psychology 57(3), 153-178 2008)-in a large-scale simulation study and the empirical data of Dutilh et al. (Psychonomic Bulletin and Review, 1-19 2018). I find that the "predictive accuracy" class of methods (i.e., the Akaike Information Criterion [AIC], the Deviance Information Criterion [DIC], and the Widely Applicable Information Criterion [WAIC]) make different inferences to the "Bayes factor" class of methods (i.e., the Bayesian Information Criterion [BIC], and Bayes factors) in many, but not all, instances, and that the simpler methods (i.e., AIC and BIC) make inferences that are highly consistent with their more complex counterparts. These findings suggest that researchers should be able to use simpler "parameter counting" methods when applying the LBA and be confident in their inferences, but that researchers need to carefully consider and justify the general class of model selection method that they use, as different classes of methods often result in different inferences.

Thermodynamic integration via differential evolution: A method for estimating marginal likelihoods

A typical goal in cognitive psychology is to select the model that provides the best explanation of the observed behavioral data. The Bayes factor provides a principled approach for making these selections, though the integral required to calculate the marginal likelihood for each model is intractable for most cognitive models. In these cases, Monte Carlo techniques must be used to approximate the marginal likelihood, such as thermodynamic integration (TI; Friel & Pettitt, Journal of the Royal Statistical Society: Series B (Statistical Methodology), 70(3), 589–607 2008; Lartillot & Philippe, Systematic Biology, 55(2), 195–207 2006), which relies on sampling from the posterior at different powers (called power posteriors). TI can become computationally expensive when using population Markov chain Monte Carlo (MCMC) approaches such as differential evolution MCMC (DE-MCMC; Turner et al., Psychological Methods, 18(3), 368 2013) that require several interacting chains per power posterior. Here, we propose a method called thermodynamic integration via differential evolution (TIDE), which aims to reduce the computational burden associated with TI by using a single chain per power posterior (R code available at https://osf.io/ntmgw/). We show that when applied to non-hierarchical models, TIDE produces an approximation of the marginal likelihood that closely matches TI. When extended to hierarchical models, we find that certain assumptions about the dependence between the individual- and group-level parameters samples (i.e., dependent/independent) have sizable effects on the TI approximated marginal likelihood. We propose two possible extensions of TIDE to hierarchical models, which closely match the marginal likelihoods obtained through TI with dependent/independent sampling in many, but not all, situations. Based on these findings, we believe that TIDE provides a promising method for estimating marginal likelihoods, though future research should focus on a detailed comparison between the methods of estimating marginal likelihoods for cognitive models.

Some Evidence for an Association Between Early Life Adversity and Decision Urgency

The relationship between early life adversity and adult outcomes is traditionally investigated relative to risk and protective factors (e.g., resilience, cognitive appraisal), and poor self-control or decision-making. However, life history theory suggests this relationship may be adaptive-underpinned by mechanisms that use early environmental cues to alter the developmental trajectory toward more short-term strategies. These short-term strategies have some theoretical overlap with the most common process models of decision-making-evidence accumulation models-which model decision urgency as a decision threshold. The current study examined the relationship between decision urgency (through the linear ballistic accumulator) and early life adversity. A mixture of analysis methods, including a joint model analysis designed to explicitly account for uncertainty in estimated decision urgency values, revealed weak-to-strong evidence in favor of a relationship between decision urgency and early life adversity, suggesting a possible effect of life history strategy on even the most basic decisions.