Trial-history biases in evidence accumulation can give rise to apparent lapses in decision-making

Disentangling sources of variability in decision-making

Even the most highly-trained observers presented with identical choice-relevant stimuli will reliably exhibit substantial trial-to-trial variability in the timing and accuracy of their choices. Despite being a pervasive feature of choice behaviour and a prominent phenotype for numerous clinical disorders, the capability to disentangle the sources of such intra-individual variability (IIV) remains limited. In principle, computational models of decision-making offer a means of parsing and estimating these sources, but methodological limitations have prevented this potential from being fully realized in practice. In this Review, we first discuss current limitations of algorithmic models for understanding variability in decision-making behaviour. We then highlight recent advances in behavioural paradigm design, novel analyses of cross-trial behavioural and neural dynamics, and the development of neurally grounded computational models that are now making it possible to link distinct components of IIV to well-defined neural processes. Taken together, we demonstrate how these methods are opening up new avenues for systematically analysing the neural origins of IIV, paving the way for a more refined, holistic understanding of decision-making in health and disease.

Optimal allocation of time in risky choices under opportunity costs

In economic decision-making there is a trade-off between deliberation time to make a good decision and opportunity costs of other rewarding activities. Recent theories describe how the optimal strategy of evidence accumulation for this problem depends on the environment. If the utility difference between two options is known a priori, but not the identity of the better option, decision-makers should accumulate evidence according to a drift diffusion model with constant decision boundaries. If this difference is unknown beforehand, collapsing boundaries should be used. The exact position of the boundaries depends on the opportunity costs. In two experiments, we examined whether people can adaptively adjust their decision bounds. Participants rated and chose between risky lotteries, while we varied prior information about the utility difference. We also varied opportunity costs, by imposing time limits on task blocks. We found that participants used collapsing boundaries in all examined conditions, even in those where constant boundaries would have been optimal. This means they reduced their target strength of evidence during the choice process, even when they should not. In contrast, participants were sensitive to opportunity costs, deciding faster when choice time was more costly. In sum, people adapted to opportunity costs but not to prior information about utility differences.

Location- and feature-based selection histories make independent, qualitatively distinct contributions to urgent visuomotor performance

Attention mechanisms guide visuomotor behavior by weighing physical salience and internal goals to prioritize stimuli as choices for action. Although less well studied, selection history, which reflects multiple facets of experience with recent events, is increasingly recognized as a distinct source of attentional bias. To examine how selection history impacts saccadic choices, we trained two macaque monkeys to perform an urgent version of an oddball search task in which a red target appeared among three green distracters, or vice versa. By imposing urgency, performance could be tracked continuously as it transitioned from uninformed guesses to informed choices as a function of processing time. This, in turn, permitted assessment of attentional control as manifest in motor biases, processing speed, and asymptotic accuracy. Here, we found that the probability of making a correct choice was strongly modulated by the histories of preceding target locations and target colors. Crucially, although both effects were gated by success (or reward), their dynamics were clearly distinct: whereas location history promoted a motor bias, color history modulated perceptual sensitivity, and these influences acted independently. Thus, combined selection histories can give rise to enormous swings in visuomotor performance even in simple tasks with highly discriminable stimuli.

Toward a Free-Response Paradigm of Decision Making in Spiking Neural Networks

Spiking neural networks (SNNs) have attracted significant interest in the development of brain-inspired computing systems due to their energy efficiency and similarities to biological information processing. In contrast to continuous-valued artificial neural networks, which produce results in a single step, SNNs require multiple steps during inference to achieve a desired accuracy level, resulting in a burden in real-time response and energy efficiency. Inspired by the tradeoff between speed and accuracy in human and animal decision-making processes, which exhibit correlations among reaction times, task complexity, and decision confidence, an inquiry emerges regarding how an SNN model can benefit by implementing these attributes. Here, we introduce a theory of decision making in SNNs by untangling the interplay between signal and noise. Under this theory, we introduce a new learning objective that trains an SNN not only to make the correct decisions but also to shape its confidence. Numerical experiments demonstrate that SNNs trained in this way exhibit improved confidence expression, reduced trial-to-trial variability, and shorter latency to reach the desired accuracy. We then introduce a stopping policy that can stop inference in a way that further enhances the time efficiency of SNNs. The stopping time can serve as an indicator to whether a decision is correct, akin to the reaction time in animal behavior experiments. By integrating stochasticity into decision making, this study opens up new possibilities to explore the capabilities of SNNs and advance SNNs and their applications in complex decision-making scenarios where model performance is limited.

Contributions of distinct attention mechanisms to saccadic choices in a gamified, dynamic environment

Visuospatial attention is key for parsing visual information and selecting targets to look at. In turn, three types of mechanism determine when and where attention is deployed: stimulus-driven (exogenous), goal-driven (endogenous), and history-driven (reflecting recent experience). It is unclear, however, how these distinct attentional signals interact and contribute during natural visual scanning, when stimuli may change rapidly and no fixation requirements are imposed. Here, we investigate this via a gamified task in which participants make continuous saccadic choices at a rapid pace - and yet, perceptual performance can be accurately tracked over time as the choice process unfolds. The results reveal unequivocal markers of exogenous capture toward salient stimuli; endogenous guidance toward valuable targets and relevant locations; and history-driven effects, which produce large, involuntary modulations in processing capacity. Under dynamic conditions, success probability is dictated by temporally precise interplay between different forms of spatial attention, with recent history making a particularly prominent contribution.

Significance Statement

Visuospatial attention comprises a collection of mental mechanisms that allow us to focus on (or look at) specific objects or parts of space and ignore others. The next target to be inspected is generally selected based on how much it stands out (salience), its relevance to current goals, and recent experience. We designed a gamified visual scanning task in which all such forms of attentional control interact rapidly, more akin to real life situations (e.g., driving through traffic). Each mechanism affected in characteristic ways the probability that participants would look to the correct target at each moment in time. Most notably, we found that the history of recently seen stimuli determines visual processing capacity much more strongly than previously thought.

Value construction through sequential sampling explains serial dependencies in decision making

Deciding between a pair of familiar items is thought to rely on a comparison of their subjective values. When the values are similar, decisions take longer, and the choice may be inconsistent with stated value. These regularities are thought to be explained by the same mechanism of noisy evidence accumulation that leads to perceptual errors under conditions of low signal to noise. However, unlike perceptual decisions, subjective values may vary with internal states (e.g. desires, priorities) that change over time. This raises the possibility that the apparent stochasticity of choice reflects changes in value rather than mere noise. We hypothesized that these changes would manifest in serial dependencies across decision sequences. We analyzed data from a task in which participants chose between snack items. We developed an algorithm, Reval, that revealed significant fluctuations of the subjective values of items within an experimental session. The dynamic values predicted choices and response times more accurately than stated values. The dynamic values also furnished a superior account of the BOLD signal in ventromedial prefrontal cortex. A novel bounded-evidence accumulation model with temporally correlated evidence samples supports the idea that revaluation reflects the dynamic construction of subjective value during deliberation, which in turn influences subsequent decisions.

Rapid, systematic updating of movement by accumulated decision evidence

Acting in the natural world requires not only deciding among multiple options but also converting decisions into motor commands. How the dynamics of decision formation influence the fine kinematics of response movement remains, however, poorly understood. Here we investigate how the accumulation of decision evidence shapes the response orienting trajectories in a task where freely-moving rats combine prior expectations and auditory information to select between two possible options. Response trajectories and their motor vigor are initially determined by the prior. Rats movements then incorporate sensory information in less than 100 ms after stimulus onset by accelerating or slowing depending on how much the stimulus supports their initial choice. When the stimulus evidence is in strong contradiction, rats change their mind and reverse their initial trajectory. Human subjects performing an equivalent task display a remarkably similar behavior. We encapsulate these results in a computational model that maps the decision variable onto the movement kinematics at discrete time points, capturing subjects' choices, trajectories and changes of mind. Our results show that motor responses are not ballistic. Instead, they are systematically and rapidly updated, as they smoothly unfold over time, by the parallel dynamics of the underlying decision process.

A cross-species framework for investigating perceptual evidence accumulation

Cross-species studies are important for a comprehensive understanding of brain functions. However, direct quantitative comparison of behaviors across species presents a significant challenge. To enable such comparisons in perceptual decision-making, we developed a synchronized evidence accumulation task for human and non-human animals, by aligning mechanics, stimuli, and training. The task was readily learned by rats, mice and humans, with each species exhibiting qualitatively similar performance. Quantitative model comparison revealed that all three species employed an evidence accumulation strategy, but differed in speed, accuracy, and key decision parameters. Human performance prioritized accuracy, whereas rodent performance was limited by internal time-pressure. Rats optimized reward rate, while mice appeared to switch between evidence accumulation and other strategies trial-to-trial. Together, these results reveal striking similarities and species-specific priorities in decision-making. Furthermore, the synchronized behavioral framework we present may facilitate future studies involving cross-species comparisons, such as evaluating the face validity of animal models of neuropsychiatric disorders.

Highlights

Development of an evidence accumulation task for rats and miceSynchronized video game allows direct comparisons with humansRat, mouse and human behavior are well fit by the same decision modelsModel parameters reveal species-specific priorities in accumulation strategy.

Sensory History Drives Adaptive Neural Geometry in LP/Pulvinar-Prefrontal Cortex Circuits

Prior expectations guide attention and support perceptual filtering for efficient processing during decision-making. Here we show that during a visual discrimination task, mice adaptively use prior stimulus history to guide ongoing choices by estimating differences in evidence between consecutive trials (| Δ Dir |). The thalamic lateral posterior (LP)/pulvinar nucleus provides robust inputs to the Anterior Cingulate Cortex (ACC), which has been implicated in selective attention and predictive processing, but the function of the LP-ACC projection is unknown. We found that optogenetic manipulations of LP-ACC axons disrupted animals' ability to effectively estimate and use information across stimulus history, leading to | Δ Dir |-dependent ipsilateral biases. Two-photon calcium imaging of LP-ACC axons revealed an engagement-dependent low-dimensional organization of stimuli along a curved manifold. This representation was scaled by | Δ Dir | in a manner that emphasized greater deviations from prior evidence. Thus, our work identifies the LP-ACC pathway as essential for selecting and evaluating stimuli relative to prior evidence to guide decisions.

Rat movements reflect internal decision dynamics in an evidence accumulation task

Perceptual decision-making involves multiple cognitive processes, including accumulation of sensory evidence, planning, and executing a motor action. How these processes are intertwined is unclear; some models assume that decision-related processes precede motor execution, whereas others propose that movements reflecting ongoing decision processes occur before commitment to a choice. Here we combine two complementary methods to study the relationship between decision processes and the movements leading up to a choice. The first is a free-response pulse-based evidence accumulation task, in which stimuli continue until choice is reported, and the second is a motion-based drift diffusion model (mDDM), in which movement variables from video pose estimation constrain decision parameters on a trial-by-trial basis. We find that the mDDM provides a better fit to rats' decisions in the free-response accumulation task than traditional drift diffusion models. Interestingly, on each trial we observed a period, before choice, that was characterized by head immobility. The length of this period was positively correlated with the rats' decision bounds, and stimuli presented during this period had the greatest impact on choice. Together these results support a model in which internal decision dynamics are reflected in movements and demonstrate that inclusion of movement parameters improves the performance of diffusion-to-bound decision models.NEW & NOTEWORTHY In this study we combine a novel pulse-based evidence accumulation task with a newly developed motion-based drift diffusion model (mDDM). In this model, we incorporate movement parameters derived from high-resolution video data to estimate parameters of the model on a trial-by-trial basis. We find that this new model is an improved description of animal choice behavior.

Value construction through sequential sampling explains serial dependencies in decision making

Many decisions are expressed as a preference for one item over another. When these items are familiar, it is often assumed that the decision maker assigns a value to each of the items and chooses the item with the highest value. These values may be imperfectly recalled, but are assumed to be stable over the course of an interview or psychological experiment. Choices that are inconsistent with a stated valuation are thought to occur because of unspecified noise that corrupts the neural representation of value. Assuming that the noise is uncorrelated over time, the pattern of choices and response times in value-based decisions are modeled within the framework of Bounded Evidence Accumulation (BEA), similar to that used in perceptual decision-making. In BEA, noisy evidence samples accumulate over time until the accumulated evidence for one of the options reaches a threshold. Here, we argue that the assumption of temporally uncorrelated noise, while reasonable for perceptual decisions, is not reasonable for value-based decisions. Subjective values depend on the internal state of the decision maker, including their desires, needs, priorities, attentional state, and goals. These internal states may change over time, or undergo revaluation, as will the subjective values. We reasoned that these hypothetical value changes should be detectable in the pattern of choices made over a sequence of decisions. We reanalyzed data from a well-studied task in which participants were presented with pairs of snacks and asked to choose the one they preferred. Using a novel algorithm (Reval), we show that the subjective value of the items changes significantly during a short experimental session (about 1 hour). Values derived with Reval explain choice and response time better than explicitly stated values. They also better explain the BOLD signal in the ventromedial prefrontal cortex, known to represent the value of decision alternatives. Revaluation is also observed in a BEA model in which successive evidence samples are not assumed to be independent. We argue that revaluation is a consequence of the process by which values are constructed during deliberation to resolve preference choices.

Contribution of amygdala to dynamic model arbitration under uncertainty

Intrinsic uncertainty in the reward environment requires the brain to run multiple models simultaneously to predict outcomes based on preceding cues or actions, commonly referred to as stimulus- and action-based learning. Ultimately, the brain also must adopt appropriate choice behavior using reliability of these models. Here, we combined multiple experimental and computational approaches to quantify concurrent learning in monkeys performing tasks with different levels of uncertainty about the model of the environment. By comparing behavior in control monkeys and monkeys with bilateral lesions to the amygdala or ventral striatum, we found evidence for dynamic, competitive interaction between stimulus-based and action-based learning, and for a distinct role of the amygdala. Specifically, we demonstrate that the amygdala adjusts the initial balance between the two learning systems, thereby altering the interaction between arbitration and learning that shapes the time course of both learning and choice behaviors. This novel role of the amygdala can account for existing contradictory observations and provides testable predictions for future studies into circuit-level mechanisms of flexible learning and choice under uncertainty.

A cognitive process model captures near-optimal confidence-guided waiting in rats

Rational decision-makers invest more time pursuing rewards they are more confident they will eventually receive. A series of studies have therefore used willingness to wait for delayed rewards as a proxy for decision confidence. However, interpretation of waiting behavior is limited because it is unclear how environmental statistics influence optimal waiting, and how sources of internal variability influence subjects' behavior. We trained rats to perform a confidence-guided waiting task, and derived expressions for optimal waiting that make relevant environmental statistics explicit, including travel time incurred traveling from one reward opportunity to another. We found that rats waited longer than fully optimal agents, but that their behavior was closely matched by optimal agents with travel times constrained to match their own. We developed a process model describing the decision to stop waiting as an accumulation to bound process, which allowed us to compare the effects of multiple sources of internal variability on waiting. Surprisingly, although mean wait times grew with confidence, variability did not, inconsistent with scalar invariant timing, and best explained by variability in the stopping bound. Our results describe a tractable process model that can capture the influence of environmental statistics and internal sources of variability on subjects' decision process during confidence-guided waiting.

From innate to instructed: A new look at perceptual decision-making

Understanding how subjects perceive sensory stimuli in their environment and use this information to guide appropriate actions is a major challenge in neuroscience. To study perceptual decision-making in animals, researchers use tasks that either probe spontaneous responses to stimuli (often described as "naturalistic") or train animals to associate stimuli with experimenter-defined responses. Spontaneous decisions rely on animals' pre-existing knowledge, while trained tasks offer greater versatility, albeit often at the cost of extensive training. Here, we review emerging approaches to investigate perceptual decision-making using both spontaneous and trained behaviors, highlighting their strengths and limitations. Additionally, we propose how trained decision-making tasks could be improved to achieve faster learning and a more generalizable understanding of task rules.

Performance errors during rodent learning reflect a dynamic choice strategy

Humans, even as infants, use cognitive strategies, such as exploration and hypothesis testing, to learn about causal interactions in the environment. In animal learning studies, however, it is challenging to disentangle higher-order behavioral strategies from errors arising from imperfect task knowledge or inherent biases. Here, we trained head-fixed mice on a wheel-based auditory two-choice task and exploited the intra- and inter-animal variability to understand the drivers of errors during learning. During learning, performance errors are dominated by a choice bias, which, despite appearing maladaptive, reflects a dynamic strategy. Early in learning, mice develop an internal model of the task contingencies such that violating their expectation of reward on correct trials (by using short blocks of non-rewarded "probe" trials) leads to an abrupt shift in strategy. During the probe block, mice behave more accurately with less bias, thereby using their learned stimulus-action knowledge to test whether the outcome contingencies have changed. Despite having this knowledge, mice continued to exhibit a strong choice bias during reinforced trials. This choice bias operates on a timescale of tens to hundreds of trials with a dynamic structure, shifting between left, right, and unbiased epochs. Biased epochs also coincided with faster motor kinematics. Although bias decreased across learning, expert mice continued to exhibit short bouts of biased choices interspersed with longer bouts of unbiased choices and higher performance. These findings collectively suggest that during learning, rodents actively probe their environment in a structured manner to refine their decision-making and maintain long-term flexibility.

Rapid, systematic updating of movement by accumulated decision evidence

Acting in the natural world requires not only deciding among multiple options but also converting decisions into motor commands. How the dynamics of decision formation influence the fine kinematics of response movement remains, however, poorly understood. Here we investigate how the accumulation of decision evidence shapes the response orienting trajectories in a task where freely-moving rats combine prior expectations and auditory information to select between two possible options. Response trajectories and their motor vigor are initially determined by the prior. Rats movements then incorporate sensory information as early as 60 ms after stimulus onset by accelerating or slowing depending on how much the stimulus supports their initial choice. When the stimulus evidence is in strong contradiction, rats change their mind and reverse their initial trajectory. Human subjects performing an equivalent task display a remarkably similar behavior. We encapsulate these results in a computational model that, by mapping the decision variable onto the movement kinematics at discrete time points, captures subjects' choices, trajectories and changes of mind. Our results show that motor responses are not ballistic. Instead, they are systematically and rapidly updated, as they smoothly unfold over time, by the parallel dynamics of the underlying decision process.