Dissociation in decision bias mechanism between probabilistic information and previous decision

Active reinforcement learning versus action bias and hysteresis: control with a mixture of experts and nonexperts

Active reinforcement learning enables dynamic prediction and control, where one should not only maximize rewards but also minimize costs such as of inference, decisions, actions, and time. For an embodied agent such as a human, decisions are also shaped by physical aspects of actions. Beyond the effects of reward outcomes on learning processes, to what extent can modeling of behavior in a reinforcement-learning task be complicated by other sources of variance in sequential action choices? What of the effects of action bias (for actions per se) and action hysteresis determined by the history of actions chosen previously? The present study addressed these questions with incremental assembly of models for the sequential choice data from a task with hierarchical structure for additional complexity in learning. With systematic comparison and falsification of computational models, human choices were tested for signatures of parallel modules representing not only an enhanced form of generalized reinforcement learning but also action bias and hysteresis. We found evidence for substantial differences in bias and hysteresis across participants-even comparable in magnitude to the individual differences in learning. Individuals who did not learn well revealed the greatest biases, but those who did learn accurately were also significantly biased. The direction of hysteresis varied among individuals as repetition or, more commonly, alternation biases persisting from multiple previous actions. Considering that these actions were button presses with trivial motor demands, the idiosyncratic forces biasing sequences of action choices were robust enough to suggest ubiquity across individuals and across tasks requiring various actions. In light of how bias and hysteresis function as a heuristic for efficient control that adapts to uncertainty or low motivation by minimizing the cost of effort, these phenomena broaden the consilient theory of a mixture of experts to encompass a mixture of expert and nonexpert controllers of behavior.

Modelling decision-making biases

Biases are a fundamental aspect of everyday life decision-making. A variety of modelling approaches have been suggested to capture decision-making biases. Statistical models are a means to describe the data, but the results are usually interpreted according to a verbal theory. This can lead to an ambiguous interpretation of the data. Mathematical cognitive models of decision-making outline the structure of the decision process with formal assumptions, providing advantages in terms of prediction, simulation, and interpretability compared to statistical models. We compare studies that used both signal detection theory and evidence accumulation models as models of decision-making biases, concluding that the latter provides a more comprehensive account of the decision-making phenomena by including response time behavior. We conclude by reviewing recent studies investigating attention and expectation biases with evidence accumulation models. Previous findings, reporting an exclusive influence of attention on the speed of evidence accumulation and prior probability on starting point, are challenged by novel results suggesting an additional effect of attention on non-decision time and prior probability on drift rate.

The interpretation of behavior-model correlations in unidentified cognitive models

The rise of computational modeling in the past decade has led to a substantial increase in the number of papers that report parameter estimates of computational cognitive models. A common application of computational cognitive models is to quantify individual differences in behavior by estimating how these are expressed in differences in parameters. For these inferences to hold, models need to be identified, meaning that one set of parameters is most likely, given the behavior under consideration. For many models, model identification can be achieved up to a scaling constraint, which means that under the assumption that one parameter has a specific value, all remaining parameters are identified. In the current note, we argue that this scaling constraint implies a strong assumption about the cognitive process that the model is intended to explain, and warn against an overinterpretation of the associative relations found in this way. We will illustrate these points using signal detection theory, reinforcement learning models, and the linear ballistic accumulator model, and provide suggestions for a clearer interpretation of modeling results.

Increased influence of prior choices on perceptual decisions in autism

Autism spectrum disorder (ASD) manifests sensory and perceptual atypicalities. Recent theories suggest that these may reflect a reduced influence of prior information in ASD. Some studies have found reduced adaptation to recent sensory stimuli in ASD. However, the effects of prior stimuli and prior perceptual choices can counteract one-another. Here, we investigated this using two different tasks (in two different cohorts): (i) visual location discrimination and (ii) multisensory (visual-vestibular) heading discrimination. We fit the data using a logistic regression model to dissociate the specific effects of prior stimuli and prior choices. In both tasks, perceptual decisions were biased toward recent choices. Notably, the 'attractive' effect of prior choices was significantly larger in ASD (in both tasks and cohorts), while there was no difference in the influence of prior stimuli. These results challenge theories of reduced priors in ASD, and rather suggest an increased consistency bias for perceptual decisions in ASD.

Propagation and update of auditory perceptual priors through alpha and theta rhythms

To maintain a continuous and coherent percept over time, the brain makes use of past sensory information to anticipate forthcoming stimuli. We recently showed that auditory experience of the immediate past is propagated through ear-specific reverberations, manifested as rhythmic fluctuations of decision bias at alpha frequencies. Here, we apply the same time-resolved behavioural method to investigate how perceptual performance changes over time under conditions of stimulus expectation and to examine the effect of unexpected events on behaviour. As in our previous study, participants were required to discriminate the ear-of-origin of a brief monaural pure tone embedded in uncorrelated dichotic white noise. We manipulated stimulus expectation by increasing the target probability in one ear to 80%. Consistent with our earlier findings, performance did not remain constant across trials, but varied rhythmically with delay from noise onset. Specifically, decision bias showed a similar oscillation at ~9 Hz, which depended on ear congruency between successive targets. This suggests rhythmic communication of auditory perceptual history occurs early and is not readily influenced by top-down expectations. In addition, we report a novel observation specific to infrequent, unexpected stimuli that gave rise to oscillations in accuracy at ~7.6 Hz one trial after the target occurred in the non-anticipated ear. This new behavioural oscillation may reflect a mechanism for updating the sensory representation once a prediction error has been detected.

Perceptual decisions are biased toward relevant prior choices

Perceptual decisions are biased by recent perceptual history-a phenomenon termed 'serial dependence.' Here, we investigated what aspects of perceptual decisions lead to serial dependence, and disambiguated the influences of low-level sensory information, prior choices and motor actions. Participants discriminated whether a brief visual stimulus lay to left/right of the screen center. Following a series of biased 'prior' location discriminations, subsequent 'test' location discriminations were biased toward the prior choices, even when these were reported via different motor actions (using different keys), and when the prior and test stimuli differed in color. By contrast, prior discriminations about an irrelevant stimulus feature (color) did not substantially influence subsequent location discriminations, even though these were reported via the same motor actions. Additionally, when color (not location) was discriminated, a bias in prior stimulus locations no longer influenced subsequent location discriminations. Although low-level stimuli and motor actions did not trigger serial-dependence on their own, similarity of these features across discriminations boosted the effect. These findings suggest that relevance across perceptual decisions is a key factor for serial dependence. Accordingly, serial dependence likely reflects a high-level mechanism by which the brain predicts and interprets new incoming sensory information in accordance with relevant prior choices.

Suboptimality in perceptual decision making

Human perceptual decisions are often described as optimal. Critics of this view have argued that claims of optimality are overly flexible and lack explanatory power. Meanwhile, advocates for optimality have countered that such criticisms single out a few selected papers. To elucidate the issue of optimality in perceptual decision making, we review the extensive literature on suboptimal performance in perceptual tasks. We discuss eight different classes of suboptimal perceptual decisions, including improper placement, maintenance, and adjustment of perceptual criteria; inadequate tradeoff between speed and accuracy; inappropriate confidence ratings; misweightings in cue combination; and findings related to various perceptual illusions and biases. In addition, we discuss conceptual shortcomings of a focus on optimality, such as definitional difficulties and the limited value of optimality claims in and of themselves. We therefore advocate that the field drop its emphasis on whether observed behavior is optimal and instead concentrate on building and testing detailed observer models that explain behavior across a wide range of tasks. To facilitate this transition, we compile the proposed hypotheses regarding the origins of suboptimal perceptual decisions reviewed here. We argue that verifying, rejecting, and expanding these explanations for suboptimal behavior - rather than assessing optimality per se - should be among the major goals of the science of perceptual decision making.