Optimal policy for attention-modulated decisions explains human fixation behavior

The role of visual attention in opportunity cost neglect and consideration

Choices necessitate opportunity costs: choosing one option means foregoing another. Despite their critical role in decision making, people often neglect opportunity costs and are less likely to make purchases when reminded of them. Here, we seek to understand whether and how opportunity-cost neglect can be explained by attention, a relationship that has been proposed but not explicitly tested. Participants made eye-tracked, incentivized purchase decisions in two conditions: one with implicit opportunity costs (e.g., "Buy" vs. "Do Not Buy") and one with explicit opportunity costs (e.g., "Buy" vs. "Keep Money"). Across two studies (approximately 30,000 choices), we find lower purchase rates when opportunity costs are explicit. More importantly, we show that the relationship between attention and opportunity cost considerations is two-fold. First, the amount of attention to the outside option is greater when opportunity costs are explicit, which partly accounts for the effect of opportunity cost salience on choice. Second, for some framings, the predictive power of attention to opportunity costs is greater when opportunity costs are explicit. Using the attentional drift-diffusion model, we model the effect of opportunity cost salience on choice via attention. These findings help explain why people are more likely to purchase when explicit opportunity cost reminders are absent.

A timeline of cognitive costs in decision-making

Recent research from economics, psychology, cognitive science, computer science, and marketing is increasingly interested in the idea that people face cognitive costs when making decisions. Reviewing and synthesizing this research, we develop a framework of cognitive costs that organizes concepts along a temporal dimension and maps out when costs occur in the decision-making process and how they impact decisions. Our unifying framework broadens the scope of research on cognitive costs to a wider timeline of cognitive processing. We identify implications and recommendations emerging from our framework for intervening on behavior to tackle some of the most pressing issues of our day, from improving health and saving decisions to mitigating the consequences of climate change.

Early versus late noise differentially enhances or degrades context-dependent choice

Noise is a fundamental problem for information processing in neural systems. In decision-making, noise is thought to cause stochastic errors in choice. However, little is known about how noise arising from different sources may contribute differently to value coding and choice behaviors. Here, we examine how noise arising early versus late in the decision process differentially impacts context-dependent choice behavior. We find in model simulations that under early noise, contextual information enhances choice accuracy, while under late noise, context degrades choice accuracy. Furthermore, we verify these opposing predictions in experimental human choice behavior. Manipulating early and late noise - by inducing uncertainty in option values and controlling time pressure - produces dissociable positive and negative context effects. These findings reconcile controversial experimental findings in the literature, suggesting a unified mechanism for context-dependent choice. More broadly, these findings highlight how different sources of noise can interact with neural computations to differentially modulate behavior.

Decision makers consider all options in choice triplets

Most contemporary decision-making research focuses on choices between only two alternative options, in spite of the fact that most real-world decisions involve more than two options. Beyond this practical point, multi-option decisions are also important from a theoretical perspective. Experimental and computational studies have demonstrated that the composition of a set of choice options has predictable effects on choice outcomes. Specifically, with more options available to choose from, responses are slower and more stochastic. This effect is amplified when the values of the options (including the worst option in the set) are more similar to each other. In this study, we provide further evidence of these known effects. We also provide evidence that metacognitive factors such as feelings of confidence in the response or mental effort exertion during deliberation show similar effects as the cognitive factors (consistency between choices and value estimates, response speed). Finally, we provide novel evidence that value estimates are refined during deliberation for all options in choice triplets, similar to what has previously been show for choice pairs.

Origins of noise in both improving and degrading decision making

Noise is a fundamental problem for information processing in neural systems. In decision-making, noise is thought to cause stochastic errors in choice. However, little is known about how noise arising from different sources may contribute differently to value coding and choice behaviors. Here, we examine how noise arising early versus late in the decision process differentially impacts context-dependent choice behavior. We find in model simulations that under early noise, contextual information enhances choice accuracy, while under late noise, context degrades choice accuracy. Furthermore, we verify these opposing predictions in experimental human choice behavior. Manipulating early and late noise - by inducing uncertainty in option values and controlling time pressure - produces dissociable positive and negative context effects. These findings reconcile controversial experimental findings in the literature, suggesting a unified mechanism for context-dependent choice. More broadly, these findings highlight how different sources of noise can interact with neural computations to differentially modulate behavior.

Significance

The current study addresses the role of noise origin in decision-making, reconciling controversies around how decision-making is impacted by context. We demonstrate that different types of noise - either arising early during evaluation or late during option comparison - leads to distinct results: with early noise, context enhances choice accuracy, while with late noise, context impairs it. Understanding these dynamics offers potential strategies for improving decision-making in noisy environments and refining existing neural computation models. Overall, our findings advance our understanding of how neural systems handle noise in essential cognitive tasks, suggest a beneficial role for contextual modulation under certain conditions, and highlight the profound implications of noise structure in decision-making.

Common neural choice signals can emerge artefactually amid multiple distinct value signals

Previous work has identified characteristic neural signatures of value-based decision-making, including neural dynamics that closely resemble the ramping evidence accumulation process believed to underpin choice. Here we test whether these signatures of the choice process can be temporally dissociated from additional, choice-'independent' value signals. Indeed, EEG activity during value-based choice revealed distinct spatiotemporal clusters, with a stimulus-locked cluster reflecting affective reactions to choice sets and a response-locked cluster reflecting choice difficulty. Surprisingly, 'neither' of these clusters met the criteria for an evidence accumulation signal. Instead, we found that stimulus-locked activity can 'mimic' an evidence accumulation process when aligned to the response. Re-analysing four previous studies, including three perceptual decision-making studies, we show that response-locked signatures of evidence accumulation disappear when stimulus-locked and response-locked activity are modelled jointly. Collectively, our findings show that neural signatures of value can reflect choice-independent processes and look deceptively like evidence accumulation.

Peripheral Visual Information Halves Attentional Choice Biases

A growing body of research has shown that simple choices involve the construction and comparison of values at the time of decision. These processes are modulated by attention in a way that leaves decision makers susceptible to attentional biases. Here, we studied the role of peripheral visual information on the choice process and on attentional choice biases. We used an eye-tracking experiment in which participants (N = 50 adults) made binary choices between food items that were displayed in marked screen "shelves" in two conditions: (a) where both items were displayed, and (b) where items were displayed only when participants fixated within their shelves. We found that removing the nonfixated option approximately doubled the size of the attentional biases. The results show that peripheral visual information is crucial in facilitating good decisions and suggest that individuals might be influenceable by settings in which only one item is shown at a time, such as e-commerce.

Establishing gaze markers of perceptual load during multi-target visual search

Highly-automated technologies are increasingly incorporated into existing systems, for instance in advanced car models. Although highly automated modes permit non-driving activities (e.g. internet browsing), drivers are expected to reassume control upon a 'take over' signal from the automation. To assess a person's readiness for takeover, non-invasive eye tracking can indicate their attentive state based on properties of their gaze. Perceptual load is a well-established determinant of attention and perception, however, the effects of perceptual load on a person's ability to respond to a takeover signal and the related gaze indicators are not yet known. Here we examined how load-induced attentional state affects detection of a takeover-signal proxy, as well as the gaze properties that change with attentional state, in an ongoing task with no overt behaviour beyond eye movements (responding by lingering the gaze). Participants performed a multi-target visual search of either low perceptual load (shape targets) or high perceptual load (targets were two separate conjunctions of colour and shape), while also detecting occasional auditory tones (the proxy takeover signal). Across two experiments, we found that high perceptual load was associated with poorer search performance, slower detection of cross-modal stimuli, and longer fixation durations, while saccade amplitude did not consistently change with load. Using machine learning, we were able to predict the load condition from fixation duration alone. These results suggest monitoring fixation duration may be useful in the design of systems to track users' attentional states and predict impaired user responses to stimuli outside of the focus of attention.

Learning how to reason and deciding when to decide

Research on human reasoning has both popularized and struggled with the idea that intuitive and deliberate thoughts stem from two different systems, raising the question how people switch between them. Inspired by research on cognitive control and conflict monitoring, we argue that detecting the need for further thought relies on an intuitive, context-sensitive process that is learned in itself.

Complex economic decisions from simple neurocognitive processes: the role of interactive attention

Neurocognitive theories of value-based choice propose that people additively accumulate choice attributes when making decisions. These theories cannot explain the emergence of complex multiplicative preferences such as those assumed by prospect theory and other economic models. We investigate an interactive attention mechanism, according to which attention to attributes (like payoffs) depends on other attributes (like probabilities) attended to previously. We formalize this mechanism using a Markov attention model combined with an accumulator decision process, and test our model on eye-tracking and mouse-tracking data in risky choice. Our tests show that interactive attention is necessary to make good choices, that most participants display interactive attention and that allowing for interactive attention in accumulation-based decision models improves their predictions. By equipping established decision models with sophisticated attentional dynamics, we extend these models to describe complex economic choice, and in the process, we unify two prominent theoretical approaches to studying value-based decision making.

Phantom controllers: Misspecified models create the false appearance of adaptive control during value-based choice

Decision scientists have grown increasingly interested in how people adaptively control their decision making. Researchers have demonstrated that parameters governing the accumulation of evidence towards a choice, such as the decision threshold, are shaped by information available prior to or in parallel with one's evaluation of an option set (e.g., recent outcomes or choice conflict). A recent account has taken a bold leap forward in this approach, suggesting that adjustments in decision parameters can be motivated by the value of the options under consideration. This motivated control account predicts that when faced with difficult choices (similarly valued options) under time pressure, people will adaptively lower their decision threshold to ensure that they make a choice in time. This account was supported by drift diffusion modeling of a deadlined choice task, demonstrating that decision thresholds decrease for difficult relative to easy choices. Here, we reanalyze the data from this experiment, and show that evidence for this novel account does not hold up to further scrutiny. Using a more systematic and comprehensive modeling approach, we show that this previously observed threshold adjustment disappears (or even reverses) under a more complete model of the data. Importantly, we further show how this and other apparent evidence for motivated control arises as an artifact of model (mis)specification, where one model's putatively controlled decision process (e.g., value-driven threshold adjustments) can mimic another model's stimulus-driven decision processes (e.g., accumulator competition or collapsing bounds). Collectively, this work reveals crucial insights and constraints in the pursuit of understanding how control guides decision-making, and when it doesn't.

Value-directed information search in partner choice

It is a widely held view that people rely on incomplete information to find a relationship partner, resulting in non-compensatory choice heuristics. However, recent experimental work typically finds that partner choice follows compensatory choice strategies. To bridge this gap between theory and experimental evidence, we characterize the mate choice problem by distinguishing the information search process from the evaluation process. In an eye-tracking experiment and a MouseLab experiment, we show that people display strong value-directed search heuristics in response to all types of cues and that the magnitude of value-directed searches increases with cue primacy. Cue primacy also explains the interaction effect of cue type and participant sex on the extent of valued-directed search. We further argue that value-directed searching does not necessarily lead to non-compensatory choice rules but may serve compensatory decision-making. Our results demonstrate that people may adopt remarkably smart search heuristics to find an ideal partner efficiently.

Vigilance, arousal, and acetylcholine: Optimal control of attention in a simple detection task

Paying attention to particular aspects of the world or being more vigilant in general can be interpreted as forms of ‘internal’ action. Such arousal-related choices come with the benefit of increasing the quality and situational appropriateness of information acquisition and processing, but incur potentially expensive energetic and opportunity costs. One implementational route for these choices is widespread ascending neuromodulation, including by acetylcholine (ACh). The key computational question that elective attention poses for sensory processing is when it is worthwhile paying these costs, and this includes consideration of whether sufficient information has yet been collected to justify the higher signal-to-noise ratio afforded by greater attention and, particularly if a change in attentional state is more expensive than its maintenance, when states of heightened attention ought to persist. We offer a partially observable Markov decision-process treatment of optional attention in a detection task, and use it to provide a qualitative model of the results of studies using modern techniques to measure and manipulate ACh in rodents performing a similar task.

Paying attention to a stimulus is costly, both in terms of energy and the lost opportunity to pay attention to something else. It is also beneficial, providing more information about its target. Thus, whether and when we pay more or less attention may best be considered as a choice of internal action that responds to this trade-off. Furthermore, measurements and manipulation of the neuromodulator acetylcholine have suggested that it is one of the instruments of attention, providing us with a window onto this choice. Here, we build an abstract model of a task in which an animal must look out for a brief visual stimulus that may or may not occur on each trial. We show that optimal attentional choices in the model depend on many factors, including how likely a signal is to occur across time, the balance between the improvement in information possible by paying greater attention and its increased cost, and whether there are also costs associated with switching between different attentional states. We also show that our model can qualitatively match results from experiments involving acetylcholine.

Toward an attentional turn in research on risky choice

For a long time, the dominant approach to studying decision making under risk has been to use psychoeconomic functions to account for how behavior deviates from the normative prescriptions of expected value maximization. While this neo-Bernoullian tradition has advanced the field in various ways—such as identifying seminal phenomena of risky choice (e.g., Allais paradox, fourfold pattern)—it contains a major shortcoming: Psychoeconomic curves are mute with regard to the cognitive mechanisms underlying risky choice. This neglect of the mechanisms both limits the explanatory value of neo-Bernoullian models and fails to provide guidance for designing effective interventions to improve decision making. Here we showcase a recent “attentional turn” in research on risk choice that elaborates how deviations from normative prescriptions can result from imbalances in attention allocation (rather than distortions in the representation or processing of probability and outcome information) and that thus promises to overcome the challenges of the neo-Bernoullian tradition. We argue that a comprehensive understanding of preference formation in risky choice must provide an account on a mechanistic level, and we delineate directions in which existing theories that rely on attentional processes may be extended to achieve this objective.

Gaze-dependent evidence accumulation predicts multi-alternative risky choice behaviour

Choices are influenced by gaze allocation during deliberation, so that fixating an alternative longer leads to increased probability of choosing it. Gaze-dependent evidence accumulation provides a parsimonious account of choices, response times and gaze-behaviour in many simple decision scenarios. Here, we test whether this framework can also predict more complex context-dependent patterns of choice in a three-alternative risky choice task, where choices and eye movements were subject to attraction and compromise effects. Choices were best described by a gaze-dependent evidence accumulation model, where subjective values of alternatives are discounted while not fixated. Finally, we performed a systematic search over a large model space, allowing us to evaluate the relative contribution of different forms of gaze-dependence and additional mechanisms previously not considered by gaze-dependent accumulation models. Gaze-dependence remained the most important mechanism, but participants with strong attraction effects employed an additional similarity-dependent inhibition mechanism found in other models of multi-alternative multi-attribute choice.

Optimal Allocation of Finite Sampling Capacity in Accumulator Models of Multialternative Decision Making

When facing many options, we narrow down our focus to very few of them. Although behaviors like this can be a sign of heuristics, they can actually be optimal under limited cognitive resources. Here, we study the problem of how to optimally allocate limited sampling time to multiple options, modeled as accumulators of noisy evidence, to determine the most profitable one. We show that the effective sampling capacity of an agent increases with both available time and the discriminability of the options, and optimal policies undergo a sharp transition as a function of it. For small capacity, it is best to allocate time evenly to exactly five options and to ignore all the others, regardless of the prior distribution of rewards. For large capacities, the optimal number of sampled accumulators grows sublinearly, closely following a power law as a function of capacity for a wide variety of priors. We find that allocating equal times to the sampled accumulators is better than using uneven time allocations. Our work highlights that multialternative decisions are endowed with breadth–depth tradeoffs, demonstrates how their optimal solutions depend on the amount of limited resources and the variability of the environment, and shows that narrowing down to a handful of options is always optimal for small capacities.

Rational use of cognitive resources in human planning

Making good decisions requires thinking ahead, but the huge number of actions and outcomes one could consider makes exhaustive planning infeasible for computationally constrained agents, such as humans. How people are nevertheless able to solve novel problems when their actions have long-reaching consequences is thus a long-standing question in cognitive science. To address this question, we propose a model of resource-constrained planning that allows us to derive optimal planning strategies. We find that previously proposed heuristics such as best-first search are near optimal under some circumstances but not others. In a mouse-tracking paradigm, we show that people adapt their planning strategies accordingly, planning in a manner that is broadly consistent with the optimal model but not with any single heuristic model. We also find systematic deviations from the optimal model that might result from additional cognitive constraints that are yet to be uncovered.

Humans actively sample evidence to support prior beliefs

No one likes to be wrong. Previous research has shown that participants may underweight information incompatible with previous choices, a phenomenon called confirmation bias. In this paper, we argue that a similar bias exists in the way information is actively sought. We investigate how choice influences information gathering using a perceptual choice task and find that participants sample more information from a previously chosen alternative. Furthermore, the higher the confidence in the initial choice, the more biased information sampling becomes. As a consequence, when faced with the possibility of revising an earlier decision, participants are more likely to stick with their original choice, even when incorrect. Critically, we show that agency controls this phenomenon. The effect disappears in a fixed sampling condition where presentation of evidence is controlled by the experimenter, suggesting that the way in which confirmatory evidence is acquired critically impacts the decision process. These results suggest active information acquisition plays a critical role in the propagation of strongly held beliefs over time.

Filling the gaps: Cognitive control as a critical lens for understanding mechanisms of value-based decision-making

While often seeming to investigate rather different problems, research into value-based decision making and cognitive control have historically offered parallel insights into how people select thoughts and actions. While the former studies how people weigh costs and benefits to make a decision, the latter studies how they adjust information processing to achieve their goals. Recent work has highlighted ways in which decision-making research can inform our understanding of cognitive control. Here, we provide the complementary perspective: how cognitive control research has informed understanding of decision-making. We highlight three particular areas of research where this critical interchange has occurred: (1) how different types of goals shape the evaluation of choice options, (2) how people use control to adjust the ways they make their decisions, and (3) how people monitor decisions to inform adjustments to control at multiple levels and timescales. We show how adopting this alternate viewpoint offers new insight into the determinants of both decisions and control; provides alternative interpretations for common neuroeconomic findings; and generates fruitful directions for future research.

Advances in modeling learning and decision-making in neuroscience

An organism's survival depends on its ability to learn about its environment and to make adaptive decisions in the service of achieving the best possible outcomes in that environment. To study the neural circuits that support these functions, researchers have increasingly relied on models that formalize the computations required to carry them out. Here, we review the recent history of computational modeling of learning and decision-making, and how these models have been used to advance understanding of prefrontal cortex function. We discuss how such models have advanced from their origins in basic algorithms of updating and action selection to increasingly account for complexities in the cognitive processes required for learning and decision-making, and the representations over which they operate. We further discuss how a deeper understanding of the real-world complexities in these computations has shed light on the fundamental constraints on optimal behavior, and on the complex interactions between corticostriatal pathways to determine such behavior. The continuing and rapid development of these models holds great promise for understanding the mechanisms by which animals adapt to their environments, and what leads to maladaptive forms of learning and decision-making within clinical populations.

Reframing rationality: Exogenous constraints on controlled information search

Bermúdez argues that framing effects are rational because particular frames provide goal-consistent reasons for choice and that people exert some control over the framing of a decision-problem. We propose instead that these observations raise the question of whether frame selection itself is a rational process and highlight how constraints in the choice environment severely limit the rational selection of frames.

The elusiveness of context effects in decision making

Contextual features influence human and non-human decision making, giving rise to preference reversals. Decades of research have documented the species and situations in which these effects are observed. More recently, however, researchers have focused on boundary conditions, that is, settings in which established effects disappear or reverse. This work is scattered across academic disciplines and some results appear to contradict each other. We synthesize recent findings and resolve apparent contradictions by considering them in terms of three core categories of decision context: spatial arrangement, attribute concreteness, and deliberation time. We suggest that these categories could be understood using theories of choice representation, which specify how context shapes the information over which deliberation processes operate.

An uncertainty-based model of the effects of fixation on choice

When people view a consumable item for a longer amount of time, they choose it more frequently; this also seems to be the direction of causality. The leading model of this effect is a drift-diffusion model with a fixation-based attentional bias. Here, we propose an explicitly Bayesian account for the same data. This account is based on the notion that the brain builds a posterior belief over the value of an item in the same way it would over a sensory variable. As the agent gathers evidence about the item from sensory observations and from retrieved memories, the posterior distribution narrows. We further postulate that the utility of an item is a weighted sum of the posterior mean and the negative posterior standard deviation, with the latter accounting for risk aversion. Fixating for longer can increase or decrease the posterior mean, but will inevitably lower the posterior standard deviation. This model fits the data better than the original attentional drift-diffusion model but worse than a variant with a collapsing bound. We discuss the often overlooked technical challenges in fitting models simultaneously to choice and response time data in the absence of an analytical expression. Our results hopefully contribute to emerging accounts of valuation as an inference process.

Trading mental effort for confidence in the metacognitive control of value-based decision-making

Why do we sometimes opt for actions or items that we do not value the most? Under current neurocomputational theories, such preference reversals are typically interpreted in terms of errors that arise from the unreliable signaling of value to brain decision systems. But, an alternative explanation is that people may change their mind because they are reassessing the value of alternative options while pondering the decision. So, why do we carefully ponder some decisions, but not others? In this work, we derive a computational model of the metacognitive control of decisions or MCD. In brief, we assume that fast and automatic processes first provide initial (and largely uncertain) representations of options' values, yielding prior estimates of decision difficulty. These uncertain value representations are then refined by deploying cognitive (e.g., attentional, mnesic) resources, the allocation of which is controlled by an effort-confidence tradeoff. Importantly, the anticipated benefit of allocating resources varies in a decision-by-decision manner according to the prior estimate of decision difficulty. The ensuing MCD model predicts response time, subjective feeling of effort, choice confidence, changes of mind, as well as choice-induced preference change and certainty gain. We test these predictions in a systematic manner, using a dedicated behavioral paradigm. Our results provide a quantitative link between mental effort, choice confidence, and preference reversals, which could inform interpretations of related neuroimaging findings.

Fixation patterns in simple choice reflect optimal information sampling

Simple choices (e.g., eating an apple vs. an orange) are made by integrating noisy evidence that is sampled over time and influenced by visual attention; as a result, fluctuations in visual attention can affect choices. But what determines what is fixated and when? To address this question, we model the decision process for simple choice as an information sampling problem, and approximate the optimal sampling policy. We find that it is optimal to sample from options whose value estimates are both high and uncertain. Furthermore, the optimal policy provides a reasonable account of fixations and choices in binary and trinary simple choice, as well as the differences between the two cases. Overall, the results show that the fixation process during simple choice is influenced dynamically by the value estimates computed during the decision process, in a manner consistent with optimal information sampling.