A neurocomputational model of decision and confidence in object recognition task

How does the brain process natural visual stimuli to make a decision? Imagine driving through fog. An object looms ahead. What do you do? This decision requires not only identifying the object but also choosing an action based on your decision confidence. In this circumstance, confidence is making a bridge between seeing and believing. Our study unveils how the brain processes visual information to make such decisions with an assessment of confidence, using a model inspired by the visual cortex. To computationally model the process, this study uses a spiking neural network inspired by the hierarchy of the visual cortex in mammals to investigate the dynamics of feedforward object recognition and decision-making in the brain. The model consists of two modules: a temporal dynamic object representation module and an attractor neural network-based decision-making module. Unlike traditional models, ours captures the evolution of evidence within the visual cortex, mimicking how confidence forms in the brain. This offers a more biologically plausible approach to decision-making when encountering real-world stimuli. We conducted experiments using natural stimuli and measured accuracy, reaction time, and confidence. The model's estimated confidence aligns remarkably well with human-reported confidence. Furthermore, the model can simulate the human change-of-mind phenomenon, reflecting the ongoing evaluation of evidence in the brain. Also, this finding offers decision-making and confidence encoding share the same neural circuit.

Knowledge of Threat Biases Perceptual Decision Making in Anxiety: Evidence From Signal Detection Theory and Drift Diffusion Modeling

Background

Threat biases are considered key factors in the development and maintenance of anxiety. However, these biases are poorly operationalized and remain unquantified. Furthermore, it is unclear whether and how prior knowledge of threat and its uncertainty induce these biases and how they manifest in anxiety.

Method

Participants (n = 55) used prestimulus cues to decide whether the subsequently presented stimuli were threatening or neutral. The cues either provided no information about the probability (high uncertainty) or indicated high probability (low uncertainty) of encountering threatening or neutral targets. We used signal detection theory and hierarchical drift diffusion modeling to quantify bias.

Results

High-uncertainty threat cues improved discrimination of subsequent threatening and neutral stimuli more than neutral cues. However, anxiety was associated with worse discrimination of threatening versus neutral stimuli following high-uncertainty threat cues. Using hierarchical drift diffusion modeling, we found that threat cues biased decision making not only by shifting the starting point of evidence accumulation toward the threat decision but also by increasing the efficiency with which sensory evidence was accumulated for both threat-related and neutral decisions. However, higher anxiety was associated with a greater shift of starting point toward the threat decision but not with the efficiency of evidence accumulation.

Conclusions

Using computational modeling, these results highlight the biases by which knowledge regarding uncertain threat improves perceptual decision making but impairs it in case of anxiety.

The timing of confidence computations in human prefrontal cortex

Knowing when confidence computations take place is critical for building a mechanistic understanding of the neural and computational bases of metacognition. Yet, even though a substantial amount of research has focused on revealing the neural correlates and computations underlying human confidence judgments, very little is known about the timing of confidence computations. To understand when confidence is computed, we delivered single pulses of transcranial magnetic stimulation (TMS) at different times after stimulus presentation while subjects judged the orientation of a briefly presented visual stimulus and provided a confidence rating. TMS was delivered to either the right dorsolateral prefrontal cortex (DLPFC) in the experimental group or to vertex in the control group. We found that TMS to right DLPFC, but not to vertex, led to increased confidence in the absence of changes to accuracy or metacognitive efficiency. Critically, equivalent levels of confidence increase occurred for TMS delivered between 200 and 500 msec after stimulus presentation. These results suggest that confidence computations occur during a broad window that begins before the perceptual decision has been fully made and thus provide important constraints for theories of confidence generation.

The presence of irrelevant alternatives paradoxically increases confidence in perceptual decisions

Confidence in perceptual decisions is thought to reflect the probability of being correct. According to this view, confidence should be unaffected or minimally reduced by the presence of irrelevant alternatives. To test this prediction, we designed five experiments. In Experiment 1, participants had to identify the largest geometrical shape among two or three alternatives. In the three-alternative condition, one of the shapes was much smaller than the other two, being a clearly incorrect option. Counter-intuitively, confidence was higher when the irrelevant alternative was present, evidencing that confidence construction is more complex than previously thought. Four computational models were tested, only one of them accounting for the results. This model predicts that confidence increases monotonically with the number of irrelevant alternatives, a prediction we tested in Experiment 2. In Experiment 3, we evaluated whether this effect replicated in a categorical task, but we did not find supporting evidence. Experiments 4 and 5 allowed us to discard stimuli presentation time as a factor driving the effect. Our findings suggest that confidence models cannot ignore the effect of multiple, possibly irrelevant alternatives to build a thorough understanding of confidence.

Across-subject correlation between confidence and accuracy: A meta-analysis of the Confidence Database

If one friend confidently tells us to buy Product A while another friend thinks that Product B is better but is not confident, we may go with the advice of our confident friend. Should we? The relationship between people's confidence and accuracy has been of great interest in many fields, especially in high-stakes situations like eyewitness testimony. However, there is still little consensus about how much we should trust someone's overall confidence level. Here, we examine the across-subject relationship between average accuracy and average confidence in 213 unique datasets from the Confidence Database. This approach allows us to empirically address this issue with unprecedented statistical power and check for the presence of various moderators. We find an across-subject correlation between average accuracy and average confidence of R = .22. Importantly, this relationship is much stronger for memory than for perception tasks ("domain effect"), as well as for confidence scales with fewer points ("granularity effect"). These results show that we should take one's confidence seriously (and perhaps buy Product A) and suggest several factors that moderate the relative consistency of how people make confidence judgments.

No selective integration required: A race model explains responses to audiovisual motion-in-depth

Looming motion is an ecologically salient signal that often signifies danger. In both audition and vision, humans show behavioral biases in response to perceiving looming motion, which is suggested to indicate an adaptation for survival. However, it is an open question whether such biases occur also in the combined processing of multisensory signals. Towards this aim, Cappe, Thut, Romei, and Murraya (2009) found that responses to audiovisual signals were faster for congruent looming motion compared to receding motion or incongruent combinations. They considered this as evidence for selective integration of multisensory looming signals. To test this proposal, here, we successfully replicate the behavioral results by Cappe et al. (2009). We then show that the redundant signals effect (RSE - a speedup of multisensory compared to unisensory responses) is not distinct for congruent looming motion. Instead, as predicted by a simple probability summation rule, the RSE is primarily modulated by the looming bias in audition, which suggests that multisensory processing inherits a unisensory effect. Finally, we compare a large set of so-called race models that implement probability summation, but that allow for interference between auditory and visual processing. The best-fitting model, selected by the Akaike Information Criterion (AIC), virtually perfectly explained the RSE across conditions with interference parameters that were either constant or varied only with auditory motion. In the absence of effects jointly caused by auditory and visual motion, we conclude that selective integration is not required to explain the behavioral benefits that occur with audiovisual looming motion.

Examining the robustness of the relationship between metacognitive efficiency and metacognitive bias

We recently found a positive relationship between estimates of metacognitive efficiency and metacognitive bias. However, this relationship was only examined on a within-subject level and required binarizing the confidence scale, a technique that introduces methodological difficulties. Here we examined the robustness of the positive relationship between estimates of metacognitive efficiency and metacognitive bias by conducting two different types of analyses. First, we developed a new within-subject analysis technique where the original n-point confidence scale is transformed into two different (n-1)-point scales in a way that mimics a naturalistic change in confidence. Second, we examined the across-subject correlation between metacognitive efficiency and metacognitive bias. Importantly, for both types of analyses, we not only established the direction of the effect but also computed effect sizes. We applied both techniques to the data from three tasks from the Confidence Database (N > 400 in each). We found that both approaches revealed a small to medium positive relationship between metacognitive efficiency and metacognitive bias. These results demonstrate that the positive relationship between metacognitive efficiency and metacognitive bias is robust across several analysis techniques and datasets, and have important implications for future research.

Brain-behavior relationships in the perceptual decision-making process through cognitive processing stages

Perceptual decision making - the process of detecting and categorizing information - has been studied extensively over the last two decades. In this study, we aim to bridge the gap between neural and behavioral representations of the perceptual decision-making process. The neural characterization of decision-making was investigated by evaluating the duration and neural signature of the information processing stages. We further evaluated the processing stages of the decision-making process at the behavioral level by estimating the drift rate and non-decision time parameters. We asked whether the neural and behavioral characterizations of the decision-making process provided consistent results under different stimulus coherency levels and spatial attention. Our statistical analysis revealed that, at both representational levels, decision-making was affected more by the coherency factor. We further found that among different information processing stages, the decision stage had the highest role in the performance of the decision-making process. Such that, the shorter decision stage duration at the neural level and higher drift rate at the behavioral level lead to faster decision-making. Through our consistent neural and behavioral results, we have shown that the decision-making components at these two representational levels were significantly associated. Moreover, the neural signature of the processing stages gave information about the regions that contributed more to the decision-making process. Our overall results demonstrate that uncovering the cognitive processing stages provided more insights into the decision-making process.

Response modality-dependent categorical choice representations for vibrotactile comparisons

Previous electrophysiological studies in monkeys and humans suggest that premotor regions are the primary loci for the encoding of perceptual choices during vibrotactile comparisons. However, these studies employed paradigms wherein choices were inextricably linked with the stimulus order and selection of manual movements. It remains largely unknown how vibrotactile choices are represented when they are decoupled from these sensorimotor components of the task. To address this question, we used fMRI-MVPA and a variant of the vibrotactile frequency discrimination task which enabled the isolation of choice-related signals from those related to stimulus order and selection of the manual decision reports. We identified the left contralateral dorsal premotor cortex (PMd) and intraparietal sulcus (IPS) as carrying information about vibrotactile choices. Our finding provides empirical evidence for an involvement of the PMd and IPS in vibrotactile decisions that goes above and beyond the coding of stimulus order and specific action selection. Considering findings from recent studies in animals, we speculate that the premotor region likely serves as a temporary storage site for information necessary for the specification of concrete manual movements, while the IPS might be more directly involved in the computation of choice. Moreover, this finding replicates results from our previous work using an oculomotor variant of the task, with the important difference that the informative premotor cluster identified in the previous work was centered in the bilateral frontal eye fields rather than in the PMd. Evidence from these two studies indicates that categorical choices in human vibrotactile comparisons are represented in a response modality-dependent manner.

Reward modulates the association between sensory noise and brain activity during perceptual decision-making

Perceptual decisions entail the accumulation of evidence until a decision criterion is reached. The amount of noise in this process is inversely related to the behavioral performance of the decision-maker. Hence, reducing the amount of perceived noise could improve performance in perceptual decisions. In this study, we investigated whether providing monetary reward for correct responses in a perceptual decision-making task would enhance performance based on prior research linking noise reduction to the administration of reward. To this end, thirty-one healthy young adults carried out an incentivized dot tracking task (iDT) during recording of functional magnetic resonance imaging (fMRI). Behavioral responses were fitted to a Bayesian version of the drift-diffusion model that, among other parameters, also includes an estimate of sensory noise. Fifty percent of the trials were incentivized to compare rewarded with unrewarded trials regarding behavior, brain responses and estimates of model parameters. In order to establish a link between the noise parameter and fMRI activity, we correlated percent signal change (PSC) values from nucleus accumbens and caudate nucleus with noise levels in rewarded and unrewarded trials respectively. Although reward did not affect behavioral performance and model parameters, the fMRI analyses showed notable differences in nucleus accumbens, caudate nucleus and rostral anterior cingulate cortex in rewarded relative to unrewarded trials. Furthermore, higher PSC within nucleus accumbens was significantly associated with lower sensory noise levels, which was specific to rewarded trials. This work is consistent with previous findings on reward modulation of brain responses and marks a first step towards elucidating the effects of reward-induced noise suppression during perceptual decision-making.

Trait anxiety is linked to increased usage of priors in a perceptual decision making task

Current predictive processing accounts consider negative affect to result from elevated rates of prediction error, thereby motivating changes in the degree with which prior expectancies and sensory evidence influence our perceptions. Trait anxiety is associated with the amount of negative affect a person is experiencing and has been linked to aberrant strategies in decision making and belief updating. Here, we assessed the degree to which induced prior expectancies influenced motion judgements in a simple perceptual decision making task in 117 healthy participants with varying levels of trait anxiety. High trait anxious individuals showed increased usage of priors, independent from the amount of sensory uncertainty that was perceived. This finding demonstrates aberrant strategies of belief updating in anxiety even in evaluating nonthreatening visual motion stimuli, and thus suggest an influential role of affective traits in processes of perceptual inference.

Bayesian Semiparametric Longitudinal Drift-Diffusion Mixed Models for Tone Learning in Adults

Understanding how adult humans learn nonnative speech categories such as tone information has shed novel insights into the mechanisms underlying experience-dependent brain plasticity. Scientists have traditionally examined these questions using longitudinal learning experiments under a multi-category decision making paradigm. Drift-diffusion processes are popular in such contexts for their ability to mimic underlying neural mechanisms. Motivated by these problems, we develop a novel Bayesian semiparametric inverse Gaussian drift-diffusion mixed model for multi-alternative decision making in longitudinal settings. We design a Markov chain Monte Carlo algorithm for posterior computation. We evaluate the method's empirical performances through synthetic experiments. Applied to our motivating longitudinal tone learning study, the method provides novel insights into how the biologically interpretable model parameters evolve with learning, differ between input-response tone combinations, and differ between well and poorly performing adults. supplementary materials for this article, including a standardized description of the materials available for reproducing the work, are available as an online supplement.

The interplay between multisensory integration and perceptual decision making

Facing perceptual uncertainty, the brain combines information from different senses to make optimal perceptual decisions and to guide behavior. However, decision making has been investigated mostly in unimodal contexts. Thus, how the brain integrates multisensory information during decision making is still unclear. Two opposing, but not mutually exclusive, scenarios are plausible: either the brain thoroughly combines the signals from different modalities before starting to build a supramodal decision, or unimodal signals are integrated during decision formation. To answer this question, we devised a paradigm mimicking naturalistic situations where human participants were exposed to continuous cacophonous audiovisual inputs containing an unpredictable signal cue in one or two modalities and had to perform a signal detection task or a cue categorization task. First, model-based analyses of behavioral data indicated that multisensory integration takes place alongside perceptual decision making. Next, using supervised machine learning on concurrently recorded EEG, we identified neural signatures of two processing stages: sensory encoding and decision formation. Generalization analyses across experimental conditions and time revealed that multisensory cues were processed faster during both stages. We further established that acceleration of neural dynamics during sensory encoding and decision formation was directly linked to multisensory integration. Our results were consistent across both signal detection and categorization tasks. Taken together, the results revealed a continuous dynamic interplay between multisensory integration and decision making processes (mixed scenario), with integration of multimodal information taking place both during sensory encoding as well as decision formation.

Predictive cues reduce but do not eliminate intrinsic response bias

Predictive cues induce large changes in people's choices by biasing responses towards the expected stimulus category. At the same time, even in the absence of predictive cues, humans often exhibit substantial intrinsic response biases. Despite the ubiquity of both of these biasing effects, it remains unclear how predictive cues interact with intrinsic bias. To understand the nature of this interaction, we examined data across three previous experiments that featured a combination of neutral cues (revealing intrinsic biases) and predictive cues. We found that predictive cues decreased the intrinsic bias to about half of its original size. This result held both when bias was quantified as the criterion location estimated using signal detection theory and as the probability of choosing a particular stimulus category. Our findings demonstrate that predictive cues reduce but do not eliminate intrinsic response bias, testifying to both the malleability and rigidity of intrinsic biases.

Cross-decoding supramodal information in the human brain

Perceptual decision making is the cognitive process wherein the brain classifies stimuli into abstract categories for more efficient downstream processing. A system that, during categorization, can process information regardless of the information's original sensory modality (i.e., a supramodal system) would have a substantial advantage over a system with dedicated processes for specific sensory modalities. While many studies have probed decision processes through the lens of one sensory modality, it remains unclear whether there are such supramodal brain areas that can flexibly process task-relevant information regardless of the original "format" of the information. To investigate supramodality, one must ensure that supramodal information exists somewhere within the functional architecture by rendering information from multiple sensory systems necessary but insufficient for categorization. To this aim, we tasked participants with categorizing auditory and tactile frequency-modulated sweeps according to learned, supramodal categories in a delayed match-to-category paradigm while we measured their blood-oxygen-level dependent signal with functional MRI. To detect supramodal information, we implemented a set of cross-modality pattern classification analyses, which demonstrated that the left caudate nucleus encodes category-level information but not stimulus-specific information (such as spatial directions and stimulus modalities), while the right inferior frontal gyrus, showing the opposite pattern, encodes stimulus-specific information but not category-level information. Given our paradigm, these results reveal abstract representations in the brain that are independent of motor, semantic, and sensory-specific processing, instead reflecting supramodal, categorical information, which points to the caudate nucleus as a locus of cognitive processes involved in complex behavior.

The impact of natural aging on computational and neural indices of perceptual decision making: A review

It is well established that natural aging negatively impacts on a wide variety of cognitive functions and research has sought to identify core neural mechanisms that may account for these disparate changes. A central feature of any cognitive task is the requirement to translate sensory information into an appropriate action - a process commonly known as perceptual decision making. While computational, psychophysical, and neurophysiological research has made substantial progress in establishing the key computations and neural mechanisms underpinning decision making, it is only relatively recently that this knowledge has begun to be applied to research on aging. The purpose of this review is to provide an overview of this work which is beginning to offer new insights into the core psychological processes that mediate age-related cognitive decline in adults aged 65 years and over. Mathematical modelling studies have consistently reported that older adults display longer non-decisional processing times and implement more conservative decision policies than their younger counterparts. However, there are limits on what we can learn from behavioural modeling alone and neurophysiological analyses can play an essential role in empirically validating model predictions and in pinpointing the precise neural mechanisms that are impacted by aging. Although few studies to date have explicitly examined correspondences between computational models and neural data with respect to cognitive aging, neurophysiological studies have already highlighted age-related changes at multiple levels of the sensorimotor hierarchy that are likely to be consequential for decision making behaviour. Here, we provide an overview of this literature and suggest some future directions for the field.

The Effects of Methylphenidate on the Neural Signatures of Sustained Attention

BACKGROUND

Although it is well established that methylphenidate (MPH) enhances sustained attention, the neural mechanisms underpinning this improvement remain unclear. We examined how MPH influenced known electrophysiological precursors of lapsing attention over different time scales.

METHODS

We measured the impact of MPH, compared with placebo, on behavioral and electrocortical markers while healthy adults (n = 40) performed a continuous monitoring paradigm designed to elicit attentional lapses.

RESULTS

MPH led to increased rates of target detection, and electrophysiological analyses were conducted to identify the mechanisms underlying these improvements. Lapses of attention were reliably preceded by progressive increases in alpha activity that emerged over periods of several seconds. MPH led to an overall suppression of alpha activity across the entire task but also diminished the frequency of these maladaptive pretarget increases through a reduction of alpha variability. A drug-related linear increase in the amplitude of the frontal P3 event-related component was also observed in the pretarget timeframe (3 or 4 seconds). Furthermore, during immediate target processing, there was a significant increase in the parietal P3 amplitude with MPH, indicative of enhanced perceptual evidence accumulation underpinning target detection. MPH-related enhancements occurred without significant changes to early visual processing (visual P1 and 25-Hz steady-state visual evoked potential).

CONCLUSIONS

MPH serves to reduce maladaptive electrophysiological precursors of lapsing attention by acting selectively on top-down endogenous mechanisms that support sustained attention and target detection with no significant effect on bottom-up sensory excitability. These findings offer candidate markers to monitor the therapeutic efficacy of psychostimulants or to predict therapeutic responses.

Dysfunctions in striatal microstructure can enhance perceptual decision making through deficits in predictive coding

An important brain function is to predict upcoming events on the basis of extracted regularities of previous inputs. These predictive coding processes can disturb performance in concurrent perceptual decision-making and are known to depend on fronto-striatal circuits. However, it is unknown whether, and if so, to what extent striatal microstructural properties modulate these processes. We addressed this question in a human disease model of striosomal dysfunction, i.e. X-linked dystonia-parkinsonism (XDP), using high-density EEG recordings and source localization. The results show faster and more accurate perceptual decision-making performance during distraction in XDP patients compared to healthy controls. The electrophysiological data show that sensory memory and predictive coding processes reflected by the mismatch negativity related to lateral prefrontal brain regions were weakened in XDP patients and thus induced less cognitive conflict than in controls as reflected by the N2 event-related potential (ERP). Consequently, attentional shifting (P3a ERP) and reorientation processes (RON ERP) were less pronounced in the XDP group. Taken together, these results suggests that striosomal dysfunction is related to predictive coding deficits leading to a better performance in concomitant perceptual decision-making, probably because predictive coding does not interfere with perceptual decision-making processes. These effects may reflect striatal imbalances between the striosomes and the matrix compartment.

Vestibular cognition: the effect of prior belief on vestibular perceptual decision making

Vestibular cognition is a growing field of interest and relatively little is known about the underlying mechanisms. We tested the effect of prior beliefs about the relative probability (50:50 vs. 80:20) of motion direction (yaw rotation) using a direction discrimination task. We analyzed choices individually with a logistic regression model and together with response times using a cognitive process model. The results show that self-motion perception is altered by prior belief, leading to a shift of the psychometric function, without a loss of sensitivity. Hierarchical drift diffusion analysis showed that at the group level, prior belief manifests itself as an offset to the drift criterion. However, individual model fits revealed that participants vary in how they use cognitive information in perceptual decision making. At the individual level, the response bias induced by a prior belief resulted either in a change in starting point (prior to evidence accumulation) or drift rate (during evidence accumulation). Participants incorporate prior belief in a self-motion discrimination task, albeit in different ways.

How attention influences perceptual decision making: Single-trial EEG correlates of drift-diffusion model parameters

Perceptual decision making can be accounted for by drift-diffusion models, a class of decision-making models that assume a stochastic accumulation of evidence on each trial. Fitting response time and accuracy to a drift-diffusion model produces evidence accumulation rate and non-decision time parameter estimates that reflect cognitive processes. Our goal is to elucidate the effect of attention on visual decision making. In this study, we show that measures of attention obtained from simultaneous EEG recordings can explain per-trial evidence accumulation rates and perceptual preprocessing times during a visual decision making task. Models assuming linear relationships between diffusion model parameters and EEG measures as external inputs were fit in a single step in a hierarchical Bayesian framework. The EEG measures were features of the evoked potential (EP) to the onset of a masking noise and the onset of a task-relevant signal stimulus. Single-trial evoked EEG responses, P200s to the onsets of visual noise and N200s to the onsets of visual signal, explain single-trial evidence accumulation and preprocessing times. Within-trial evidence accumulation variance was not found to be influenced by attention to the signal or noise. Single-trial measures of attention lead to better out-of-sample predictions of accuracy and correct reaction time distributions for individual subjects.

Numerical discrimination is mediated by neural coding variation

One foundation of numerical cognition is that discrimination accuracy depends on the proportional difference between compared values, closely following the Weber-Fechner discrimination law. Performance in non-symbolic numerical discrimination is used to calculate individual Weber fraction, a measure of relative acuity of the approximate number system (ANS). Individual Weber fraction is linked to symbolic arithmetic skills and long-term educational and economic outcomes. The present findings suggest that numerical discrimination performance depends on both the proportional difference and absolute value, deviating from the Weber-Fechner law. The effect of absolute value is predicted via computational model based on the neural correlates of numerical perception. Specifically, that the neural coding "noise" varies across corresponding numerosities. A computational model using firing rate variation based on neural data demonstrates a significant interaction between ratio difference and absolute value in predicting numerical discriminability. We find that both behavioral and computational data show an interaction between ratio difference and absolute value on numerical discrimination accuracy. These results further suggest a reexamination of the mechanisms involved in non-symbolic numerical discrimination, how researchers may measure individual performance, and what outcomes performance may predict.