Consensus Goals in the Field of Visual Metacognition

Knowing that you know that you know? An extreme-confidence heuristic can lead to above-chance discrimination of metacognitive performance

In daily life, we can not only estimate confidence in our inferences ('I'm sure I failed that exam'), but can also estimate whether those feelings of confidence are good predictors of decision accuracy ('I feel sure I failed, but my feeling is probably wrong; I probably passed'). In the lab, by using simple perceptual tasks and collecting trial-by-trial confidence ratings visual metacognition research has repeatedly shown that participants can successfully predict the accuracy of their perceptual choices. Can participants also successfully evaluate 'confidence in confidence' in these tasks? This is the question addressed in this study. Participants performed a simple, two-interval forced choice numerosity task framed as an exam. Confidence judgements were collected in the form of a 'predicted exam grade'. Finally, we collected 'meta-metacognitive' reports in a two-interval forced-choice design: trials were presented in pairs, and participants had to select that in which they thought their confidence (predicted grade) best matched their accuracy (actual grade), effectively minimizing their quadratic scoring rule (QSR) score. Participants successfully selected trials on which their metacognition was better when metacognitive performance was quantified using area under the type 2 ROC (AUROC2) but not when using the 'gold-standard' measure m-ratio. However, further analyses suggested that participants selected trials on which AUROC2 is lower in part via an extreme-confidence heuristic, rather than through explicit evaluation of metacognitive inferences: when restricting analyses to trials on which participants gave the same confidence rating AUROC2 no longer differed as a function of selection, and likewise when we excluded trials on which extreme confidence ratings were given. Together, our results show that participants are able to make effective metacognitive discriminations on their visual confidence ratings, but that explicit 'meta-metacognitive' processes may not be required.

Reality monitoring and metacognitive judgments in a false-memory paradigm

How well do we distinguish between different memory sources when the information from imagination and perception is similar? And how do metacognitive (confidence) judgments differ across different sources of experiences? To study these questions, we developed a reality monitoring task using semantically related words from the Deese-Roediger-McDermott (DRM) paradigm of false memories. In an orientation phase, participants either perceived word pairs or had to voluntarily imagine the second word of a word pair. In a test phase, participants viewed words and had to judge whether the paired word was previously perceived, imagined, or new. Results revealed an interaction between memory source and judgment type on both response rates and confidence judgments: reality monitoring was better for new and perceived (compared to imagined) sources, and participants often incorrectly reported imagined experiences to be perceived. Individuals exhibited similar confidence between correct imagined source judgments and incorrect imagined sources reported to be perceived. Modeling results indicated that the observed judgments were likely due to an externalizing bias (i.e., a bias to judge the memory source as perceived). Additionally, we found that overall metacognitive ability was best in the perceived source. Together, these results reveal a source-dependent effect on response rates and confidence ratings, and provide evidence that observers are surprisingly prone to externalizing biases when monitoring their own memories.

Subjective Confidence as a Monitor of the Replicability of the Response

Confidence is commonly assumed to monitor the accuracy of responses. However, intriguing results, examined in the light of philosophical discussions of epistemic justification, suggest that confidence actually monitors the reliability of choices rather than (directly) their accuracy. The focus on reliability is consistent with the view that the construction of truth has much in common with the construction of reality: extracting reliable properties that afford prediction. People are assumed to make a binary choice by sampling cues from a "collective wisdomware," and their confidence is based on the consistency of these cues, in line with the self-consistency model. Here, however, I propose that internal consistency is taken to index the reliability of choices themselves-the likelihood that they will be repeated. The results of 10 studies using binary decisions from different domains indicated that confidence in a choice predicts its replicability both within individuals and across individuals. This was so for domains for which choices have a truth value and for those for which they do not. For the former domains, differences in replicability mediated the prediction of accuracy whether confidence was diagnostic or counterdiagnostic of accuracy. Metatheoretical, methodological, and practical implications are discussed.

The future of metacognition research: Balancing construct breadth with measurement rigor

Foundational work in the psychology of metacognition identified a distinction between metacognitive knowledge (stable beliefs about one's capacities) and metacognitive experiences (local evaluations of performance). More recently, the field has focused on developing tasks and metrics that seek to identify metacognitive capacities from momentary estimates of confidence in performance, and providing precise computational accounts of metacognitive failure. However, this notable progress in formalising models of metacognitive judgments may come at a cost of ignoring broader elements of the psychology of metacognition - such as how stable meta-knowledge is formed, how social cognition and metacognition interact, and how we evaluate affective states that do not have an obvious ground truth. We propose that construct breadth in metacognition research can be restored while maintaining rigour in measurement, and highlight promising avenues for expanding the scope of metacognition research. Such a research programme is well placed to recapture qualitative features of metacognitive knowledge and experience while maintaining the psychophysical rigor that characterises modern research on confidence and performance monitoring.

Think twice: Re-assessing confidence improves visual metacognition

Metacognition is a fundamental feature of human behavior that has adaptive functional value. Current understanding of the factors that influence metacognition remains incomplete, and we lack protocols to improve metacognition. Here, we introduce a two-step confidence choice paradigm to test whether metacognitive performance may improve by asking subjects to reassess their initial confidence. Previous work on perceptual and mnemonic decision-making has shown that (type 1) perceptual sensitivity benefits from reassessing the primary choice, however, it is not clear whether such an effect occurs for type 2 confidence choices. To test this hypothesis, we ran two separate online experiments, in which participants completed a type 1 task followed by two consecutive confidence choices. The results of the two experiments indicated that metacognitive sensitivity improved after re-evaluation. Since post-decisional evidence accumulation following the first confidence choice is likely to be minimal, this metacognitive improvement is better accounted for by an attenuation of metacognitive noise during the process of confidence generation. Thus, here we argue that metacognitive noise may be filtered out by additional post-decisional processing, thereby improving metacognitive sensitivity. We discuss the ramifications of these findings for models of metacognition and for developing protocols to train and manipulate metacognitive processes.

Metacognition and Confidence: A Review and Synthesis

Determining the psychological, computational, and neural bases of confidence and uncertainty holds promise for understanding foundational aspects of human metacognition. While a neuroscience of confidence has focused on the mechanisms underpinning subpersonal phenomena such as representations of uncertainty in the visual or motor system, metacognition research has been concerned with personal-level beliefs and knowledge about self-performance. I provide a road map for bridging this divide by focusing on a particular class of confidence computation: propositional confidence in one's own (hypothetical) decisions or actions. Propositional confidence is informed by the observer's models of the world and their cognitive system, which may be more or less accurate-thus explaining why metacognitive judgments are inferential and sometimes diverge from task performance. Disparate findings on the neural basis of uncertainty and performance monitoring are integrated into a common framework, and a new understanding of the locus of action of metacognitive interventions is developed.

Introspective inference counteracts perceptual distortion

Introspective agents can recognize the extent to which their internal perceptual experiences deviate from the actual states of the external world. This ability, also known as insight, is critically required for reality testing and is impaired in psychosis, yet little is known about its cognitive underpinnings. We develop a Bayesian modeling framework and a psychophysics paradigm to quantitatively characterize this type of insight while people experience a motion after-effect illusion. People can incorporate knowledge about the illusion into their decisions when judging the actual direction of a motion stimulus, compensating for the illusion (and often overcompensating). Furthermore, confidence, reaction-time, and pupil-dilation data all show signatures consistent with inferential adjustments in the Bayesian insight model. Our results suggest that people can question the veracity of what they see by making insightful inferences that incorporate introspective knowledge about internal distortions.

Confidence in visual detection, familiarity and recollection judgments is preserved in schizophrenia spectrum disorder

An effective way to quantify metacognitive performance is to ask participants to estimate their confidence in the accuracy of their response during a cognitive task. A recent meta-analysis1 raised the issue that most assessments of metacognitive performance in schizophrenia spectrum disorders may be confounded with cognitive deficits, which are known to be present in this population. Therefore, it remains unclear whether the reported metacognitive deficits are metacognitive in nature or rather inherited from cognitive deficits. Arbitrating between these two possibilities requires equating task performance between experimental groups. Here, we aimed to characterize metacognitive performance among individuals with schizophrenia across three tasks (visual detection, familiarity, recollection) using a within-subject design while controlling experimentally for intra-individual task performance and statistically for between-subject task performance. In line with our hypotheses, we found no metacognitive deficit for visual detection and familiarity judgments. While we expected metacognition for recollection to be specifically impaired among individuals with schizophrenia, we found evidence in favor of an absence of a deficit in that domain also. We found no specific metacognitive deficit in schizophrenia spectrum disorder in the visual or memory domain. The clinical relevance of our findings is discussed in light of a hierarchical framework of metacognition.

Metacognitive bias resulting from trade-off between local and global motion signals

Visual confidence generally depends on performance in targeted perceptual tasks. However, it remains unclear how factors unrelated to performance affect confidence. Given the hierarchical nature of visual processing, both local and global stimulus features can influence confidence, but their strengths of influence remain unknown. To address this question, we independently manipulated the local contrast signals and the global coherence signals in a multiple-aperture motion pattern. The drifting-Gabor elements were individually manipulated to give rise to a coherent global motion percept. In both dichotomous direction-discrimination task (Experiment 1) and analog direction-judgment task (Experiment 2), we found stimulus-dependent biases in metacognition despite matched perceptual performance. Specifically, participants systematically gave higher confidence ratings to an incoherent pattern with clear elements (i.e., strong local but weak global signals) than a coherent pattern with noisy elements (i.e., weak local but strong global signals). We did not find any systematic effects of local/global stimulus features on metacognitive sensitivity. Model comparisons show that variation in local/global signals in the stimulus should be considered a factor influencing confidence, even after controlling for the effects of performance. Our results suggest that the metacognitive system, when generating confidence for a perceptual task, puts more weights on local than global signals.

Confidence response times: Challenging postdecisional models of confidence

Even though the nature of confidence computations has been the topic of intense interest, little attention has been paid to what confidence response times (cRTs) reveal about the underlying confidence computations. Several previous studies found cRTs to be negatively correlated with confidence in the group as a whole and consequently hypothesized the existence of an intrinsic relationship of cRT with confidence for all subjects. This hypothesis was further used to support postdecisional models of confidence that predict that cRT and confidence should always be negatively correlated. Here we test the alternative hypothesis that cRT is driven by the frequency of confidence responses such that the most frequent confidence ratings are inherently made faster regardless of whether they are high or low. We examined cRTs in three large data sets from the Confidence Database and found that the lowest cRTs occurred for the most frequent confidence rating. In other words, subjects who gave high confidence ratings most frequently had negative confidence-cRT relationships, whereas subjects who gave low confidence ratings most frequently had positive confidence-cRT relationships. In addition, we found a strong across-subject correlation between response time and cRT, suggesting that response speed for both the decision and the confidence rating is influenced by a common factor. Our results show that cRT is not intrinsically linked to confidence and strongly challenge several postdecisional models of confidence.

Confidence reflects a noisy decision reliability estimate

Decisions vary in difficulty. Humans know this and typically report more confidence in easy than in difficult decisions. However, confidence reports do not perfectly track decision accuracy, but also reflect response biases and difficulty misjudgements. To isolate the quality of confidence reports, we developed a model of the decision-making process underlying choice-confidence data. In this model, confidence reflects a subject's estimate of the reliability of their decision. The quality of this estimate is limited by the subject's uncertainty about the uncertainty of the variable that informs their decision ('meta-uncertainty'). This model provides an accurate account of choice-confidence data across a broad range of perceptual and cognitive tasks, investigated in six previous studies. We find meta-uncertainty varies across subjects, is stable over time, generalizes across some domains and can be manipulated experimentally. The model offers a parsimonious explanation for the computational processes that underlie and constrain the sense of confidence.