Approximating Bayesian Inference through Model Simulation

PyBEAM: A Bayesian approach to parameter inference for a wide class of binary evidence accumulation models

Many decision-making theories are encoded in a class of processes known as evidence accumulation models (EAM). These assume that noisy evidence stochastically accumulates until a set threshold is reached, triggering a decision. One of the most successful and widely used of this class is the Diffusion Decision Model (DDM). The DDM however is limited in scope and does not account for processes such as evidence leakage, changes of evidence, or time varying caution. More complex EAMs can encode a wider array of hypotheses, but are currently limited by computational challenges. In this work, we develop the Python package PyBEAM (Bayesian Evidence Accumulation Models) to fill this gap. Toward this end, we develop a general probabilistic framework for predicting the choice and response time distributions for a general class of binary decision models. In addition, we have heavily computationally optimized this modeling process and integrated it with PyMC, a widely used Python package for Bayesian parameter estimation. This 1) substantially expands the class of EAM models to which Bayesian methods can be applied, 2) reduces the computational time to do so, and 3) lowers the entry fee for working with these models. Here we demonstrate the concepts behind this methodology, its application to parameter recovery for a variety of models, and apply it to a recently published data set to demonstrate its practical use.

Explaining the flaws in human random generation as local sampling with momentum

In many tasks, human behavior is far noisier than is optimal. Yet when asked to behave randomly, people are typically too predictable. We argue that these apparently contrasting observations have the same origin: the operation of a general-purpose local sampling algorithm for probabilistic inference. This account makes distinctive predictions regarding random sequence generation, not predicted by previous accounts-which suggests that randomness is produced by inhibition of habitual behavior, striving for unpredictability. We verify these predictions in two experiments: people show the same deviations from randomness when randomly generating from non-uniform or recently-learned distributions. In addition, our data show a novel signature behavior, that people's sequences have too few changes of trajectory, which argues against the specific local sampling algorithms that have been proposed in past work with other tasks. Using computational modeling, we show that local sampling where direction is maintained across trials best explains our data, which suggests it may be used in other tasks too. While local sampling has previously explained why people are unpredictable in standard cognitive tasks, here it also explains why human random sequences are not unpredictable enough.

"Reliable organisms from unreliable components" revisited: the linear drift, linear infinitesimal variance model of decision making

Diffusion models of decision making, in which successive samples of noisy evidence are accumulated to decision criteria, provide a theoretical solution to von Neumann's (1956) problem of how to increase the reliability of neural computation in the presence of noise. I introduce and evaluate a new neurally-inspired dual diffusion model, the linear drift, linear infinitesimal variance (LDLIV) model, which embodies three features often thought to characterize neural mechanisms of decision making. The accumulating evidence is intrinsically positively-valued, saturates at high intensities, and is accumulated for each alternative separately. I present explicit integral-equation predictions for the response time distribution and choice probabilities for the LDLIV model and compare its performance on two benchmark sets of data to three other models: the standard diffusion model and two dual diffusion model composed of racing Wiener processes, one between absorbing and reflecting boundaries and one with absorbing boundaries only. The LDLIV model and the standard diffusion model performed similarly to one another, although the standard diffusion model is more parsimonious, and both performed appreciably better than the other two dual diffusion models. I argue that accumulation of noisy evidence by a diffusion process and drift rate variability are both expressions of how the cognitive system solves von Neumann's problem, by aggregating noisy representations over time and over elements of a neural population. I also argue that models that do not solve von Neumann's problem do not address the main theoretical question that historically motivated research in this area.

The quest for simplicity in human learning: Identifying the constraints on attention

For better or worse, humans live a resource-constrained existence; only a fraction of physical sensations ever reach conscious awareness, and we store a shockingly small subset of these experiences in memory for later use. Here, we examined the effects of attention constraints on learning. Among models that frame selective attention as an optimization problem, attention orients toward information that will reduce errors. Using this framing as a basis, we developed a suite of models with a range of constraints on the attention available during each learning event. We fit these models to both choice and eye-fixation data from four benchmark category-learning data sets, and choice data from another dynamic categorization data set. We found consistent evidence for computations we refer to as "simplicity", where attention is deployed to as few dimensions of information as possible during learning, and "competition", where dimensions compete for selective attention via lateral inhibition.

Understanding the structure of cognitive noise

Human cognition is fundamentally noisy. While routinely regarded as a nuisance in experimental investigation, the few studies investigating properties of cognitive noise have found surprising structure. A first line of research has shown that inter-response-time distributions are heavy-tailed. That is, response times between subsequent trials usually change only a small amount, but with occasional large changes. A second, separate, line of research has found that participants’ estimates and response times both exhibit long-range autocorrelations (i.e., 1/f noise). Thus, each judgment and response time not only depends on its immediate predecessor but also on many previous responses. These two lines of research use different tasks and have distinct theoretical explanations: models that account for heavy-tailed response times do not predict 1/f autocorrelations and vice versa. Here, we find that 1/f noise and heavy-tailed response distributions co-occur in both types of tasks. We also show that a statistical sampling algorithm, developed to deal with patchy environments, generates both heavy-tailed distributions and 1/f noise, suggesting that cognitive noise may be a functional adaptation to dealing with a complex world.

Beyond Drift Diffusion Models: Fitting a Broad Class of Decision and Reinforcement Learning Models with HDDM

Computational modeling has become a central aspect of research in the cognitive neurosciences. As the field matures, it is increasingly important to move beyond standard models to quantitatively assess models with richer dynamics that may better reflect underlying cognitive and neural processes. For example, sequential sampling models (SSMs) are a general class of models of decision-making intended to capture processes jointly giving rise to RT distributions and choice data in n-alternative choice paradigms. A number of model variations are of theoretical interest, but empirical data analysis has historically been tied to a small subset for which likelihood functions are analytically tractable. Advances in methods designed for likelihood-free inference have recently made it computationally feasible to consider a much larger spectrum of SSMs. In addition, recent work has motivated the combination of SSMs with reinforcement learning models, which had historically been considered in separate literatures. Here, we provide a significant addition to the widely used HDDM Python toolbox and include a tutorial for how users can easily fit and assess a (user-extensible) wide variety of SSMs and how they can be combined with reinforcement learning models. The extension comes batteries included, including model visualization tools, posterior predictive checks, and ability to link trial-wise neural signals with model parameters via hierarchical Bayesian regression.

Flexible and efficient simulation-based inference for models of decision-making

Inferring parameters of computational models that capture experimental data is a central task in cognitive neuroscience. Bayesian statistical inference methods usually require the ability to evaluate the likelihood of the model—however, for many models of interest in cognitive neuroscience, the associated likelihoods cannot be computed efficiently. Simulation-based inference (SBI) offers a solution to this problem by only requiring access to simulations produced by the model. Previously, Fengler et al. introduced likelihood approximation networks (LANs, Fengler et al., 2021) which make it possible to apply SBI to models of decision-making but require billions of simulations for training. Here, we provide a new SBI method that is substantially more simulation efficient. Our approach, mixed neural likelihood estimation (MNLE), trains neural density estimators on model simulations to emulate the simulator and is designed to capture both the continuous (e.g., reaction times) and discrete (choices) data of decision-making models. The likelihoods of the emulator can then be used to perform Bayesian parameter inference on experimental data using standard approximate inference methods like Markov Chain Monte Carlo sampling. We demonstrate MNLE on two variants of the drift-diffusion model and show that it is substantially more efficient than LANs: MNLE achieves similar likelihood accuracy with six orders of magnitude fewer training simulations and is significantly more accurate than LANs when both are trained with the same budget. Our approach enables researchers to perform SBI on custom-tailored models of decision-making, leading to fast iteration of model design for scientific discovery.

Likelihood approximation networks (LANs) for fast inference of simulation models in cognitive neuroscience

In cognitive neuroscience, computational modeling can formally adjudicate between theories and affords quantitative fits to behavioral/brain data. Pragmatically, however, the space of plausible generative models considered is dramatically limited by the set of models with known likelihood functions. For many models, the lack of a closed-form likelihood typically impedes Bayesian inference methods. As a result, standard models are evaluated for convenience, even when other models might be superior. Likelihood-free methods exist but are limited by their computational cost or their restriction to particular inference scenarios. Here, we propose neural networks that learn approximate likelihoods for arbitrary generative models, allowing fast posterior sampling with only a one-off cost for model simulations that is amortized for future inference. We show that these methods can accurately recover posterior parameter distributions for a variety of neurocognitive process models. We provide code allowing users to deploy these methods for arbitrary hierarchical model instantiations without further training.

Irritability as a Transdiagnostic Vulnerability Trait:Current Issues and Future Directions

In recent years, irritability has received increasing attention among mental health professionals given its transdiagnostic associations with diverse forms of psychopathology. In contrast to other emotional states and traits, however, literature addressing associations between irritability and related temperament and personality constructs is limited. In addition, those who study irritability have diverse perspectives on its neurobiological substrates. In this comment, we situate irritability in the literatures on child temperament and adult personality, and describe a model in which irritability derives from low tonic dopamine (DA) levels and low phasic DA reactivity in subcortical neural structures implicated in appetitive responding. We note that different findings often emerge in neuroimaging studies when irritability is assessed in circumscribed diagnostic groups versus representative samples. We conclude with directions for future research, and propose that more authors use hierarchical Bayesian modeling, which captures functional dependencies between irritability and other dispositional traits (e.g., trait anxiety) that standard regression models are insensitive too. Treatment implications are also considered.

Parameter Inference for Computational Cognitive Models with Approximate Bayesian Computation

This paper addresses a common challenge with computational cognitive models: identifying parameter values that are both theoretically plausible and generate predictions that match well with empirical data. While computational models can offer deep explanations of cognition, they are computationally complex and often out of reach of traditional parameter fitting methods. Weak methodology may lead to premature rejection of valid models or to acceptance of models that might otherwise be falsified. Mathematically robust fitting methods are, therefore, essential to the progress of computational modeling in cognitive science. In this article, we investigate the capability and role of modern fitting methods—including Bayesian optimization and approximate Bayesian computation—and contrast them to some more commonly used methods: grid search and Nelder–Mead optimization. Our investigation consists of a reanalysis of the fitting of two previous computational models: an Adaptive Control of Thought—Rational model of skill acquisition and a computational rationality model of visual search. The results contrast the efficiency and informativeness of the methods. A key advantage of the Bayesian methods is the ability to estimate the uncertainty of fitted parameter values. We conclude that approximate Bayesian computation is (a) efficient, (b) informative, and (c) offers a path to reproducible results.

Multimodal Simon Effect: A Multimodal Extension of the Diffusion Model for Conflict Tasks

In conflict tasks, like the Simon task, it is usually investigated how task-irrelevant information affects the processing of task-relevant information. In the present experiments, we extended the Simon task to a multimodal setup, in which task-irrelevant information emerged from two sensory modalities. Specifically, in Experiment 1, participants responded to the identity of letters presented at a left, right, or central position with a left- or right-hand response. Additional tactile stimulation occurred on a left, right, or central position on the horizontal body plane. Response congruency of the visual and tactile stimulation was orthogonally varied. In Experiment 2, the tactile stimulation was replaced by auditory stimulation. In both experiments, the visual task-irrelevant information produced congruency effects such that responses were slower and less accurate in incongruent than incongruent conditions. Furthermore, in Experiment 1, such congruency effects, albeit smaller, were also observed for the tactile task-irrelevant stimulation. In Experiment 2, the auditory task-irrelevant stimulation produced the smallest effects. Specifically, the longest reaction times emerged in the neutral condition, while incongruent and congruent conditions differed only numerically. This suggests that in the co-presence of multiple task-irrelevant information sources, location processing is more strongly determined by visual and tactile spatial information than by auditory spatial information. An extended version of the Diffusion Model for Conflict Tasks (DMC) was fitted to the results of both experiments. This Multimodal Diffusion Model for Conflict Tasks (MDMC), and a model variant involving faster processing in the neutral visual condition (FN-MDMC), provided reasonable fits for the observed data. These model fits support the notion that multimodal task-irrelevant information superimposes across sensory modalities and automatically affects the controlled processing of task-relevant information.