Assessing the validity of inferences from scores on the cognitive reflection test

Formal Versus Self-Identified Neurodivergence: A Comparative Study in Work Environments

This study investigated the added value of using self-identification of neurodivergence next to formal diagnosis in predicting cognitive differences. We collected and analyzed data from neurodivergent and neurotypical employees in a sample of 357 participants in 19 organizations across seven industries. Our results are aligned with previous results that support a systematic information processing tendency in highly gifted and autistic employees and decision impulsiveness in those with ADHD and ADD. Supporting previous findings, our results show different cognitive profiles of dyslexia and dyscalculia. Finally, our results show that self-identified neurodivergence adds to the predictive value of formally diagnosed conditions and that in empirical organizational research, self-identified neurodivergence is sufficient to capture the cognitive differentiation tied to neurodivergence.

Cognitive reflection is a distinct and measurable trait

We report the findings of an adversarial collaboration examining whether the cognitive reflection test (CRT) measures anything beyond mathematical aptitude and, if so, whether its incremental predictive validity can be attributed to reflection, per se. We found that an 8-item CRT has greater predictive validity than an 8-item Mathematical Aptitude Test (MAT) consisting of comparably difficult items which lack dominant intuitive lures. Further, the incremental predictive validity stems from the CRT's measurement of reflection, which we show using both structural equation models and a dual-response paradigm that helps distinguish susceptibility to intuitions from inadequate mathematical aptitude.

Thinking false and slow: Implausible beliefs and the Cognitive Reflection Test

Why do people believe implausible claims like conspiracy theories, pseudoscience, and fake news? Past studies using the Cognitive Reflection Test (CRT) suggest that implausible beliefs may result from an unwillingness to effortfully process information (i.e., cognitive miserliness). Our analysis (N = 664) tests this account by comparing CRT performance (total score, number and proportion of incorrect intuitive responses, and completion time) for endorsers and non-endorsers of implausible claims. Our results show that endorsers performed worse than non-endorsers on the CRT, but they took significantly longer to answer the questions and did not make proportionally more intuitive mistakes. Endorsers therefore appear to process information effortfully but nonetheless score lower on the CRT. Poorer overall CRT performance may not necessarily indicate that those who endorse implausible beliefs have a more reflexive, intuitive, or non-analytical cognitive style than non-endorsers.

Does cognitive reflection predict attentional control in visual tasks?

The cognitive reflection test (CRT) measures the ability to suppress an intuitive, but incorrect, answer that easily comes to mind. The relationship between the CRT and different cognitive biases has been widely studied. However, whether cognitive reflection is related to attentional control is less well studied. The aim of this paper is to investigate whether the inhibitory component of the CRT, measured by the number of non-intuitive answers of the CRT (Inhibitory Control Score), is related to the control of visual attention in visual tasks that involve overriding a bias in what to attend: an anti-saccade task and a visual search task. To test this possibility, we analyzed whether the CRT-Inhibitory Control Score (CRT-ICS) predicted attention allocation in each task. We compared the relationship between the CRT-ICS to two other potential predictors of attentional control: numeracy and visual working memory (VWM). Participants who scored lower on the CRT-ICS made more errors in the "look-away" trials in the anti-saccade task. Participants who scored higher on the CRT-ICS looked more often towards more informative color subsets in the visual search task. However, when controlling for numeracy and visual working memory, CRT-ICS scores were only related to the control of visual attention in the anti-saccade task.

A diffusion model analysis of belief bias: Different cognitive mechanisms explain how cognitive abilities and thinking styles contribute to conflict resolution in reasoning

Recent results have challenged the widespread assumption of dual process models of belief bias that sound reasoning relies on slow, careful reflection, whereas biased reasoning is based on fast intuition. Instead, parallel process models of reasoning suggest that rule- and belief-based problem features are processed in parallel and that reasoning problems that elicit a conflict between rule- and belief-based problem features may also elicit more than one Type 1 response. This has important implications for individual-differences research on reasoning, because rule-based responses by certain individuals may reflect that these individuals were either more likely to give a rule-based default response or that they successfully inhibited and overrode a belief-based default response. In two studies, we used the diffusion model to describe decision making in a transitive reasoning task. In Study 1, 41 participants were asked to evaluate conclusions based on their validity. In Study 2, 133 participants evaluated conclusions based on their validity or believability. We tested which diffusion model parameters reflected conflict resolution and related those model parameters to individual differences in cognitive abilities and thinking styles. Individual differences in need for cognition predicted successful conflict resolution under logic instruction, which suggests that a disposition to engage in reflective thinking facilitates the inhibition and override of Type 1 responses. Intelligence, however, was negatively related to successful conflict resolution under belief instruction, which suggests that individuals with high cognitive abilities quickly generated a higher-level logical response that interfered with their ability to evaluate lower-level intrinsic problem features. Taken together, this double dissociation indicates that cognitive abilities and thinking styles affect the processing of conflict information through different mechanisms and at different stages: Greater cognitive abilities facilitate the efficient creation of decoupled problem representations, whereas a greater disposition to engage in critical thinking facilitates the detection and override of Type 1 responses.

Hunger Bias or Gut Instinct? Responses to Judgments of Harm Depending on Visceral State Versus Intuitive Decision-Making

Empirical investigation into the emotional and physiological processes that shape moral decision-making is vast and growing. Yet, relatively less attention has been paid to measures of interoception in morality research despite its centrality in both emotional and physiological processes. Hunger and thirst represent two everyday interoceptive states, and hunger, in particular, has been shown to be influential for moral decision-making in numerous studies. It is possible that a tendency to focus on internal sensations interoceptive sensibility (IS), as well as the emotional and physiological states associated with visceral states, could be important in the relationships between hunger, thirst, and moral judgments. This cross-sectional online research (n = 154) explored whether IS, hunger, thirst, and emotional state influenced appropriateness and acceptability judgments of harm. The moral dilemma stimuli used allowed the independent calculation of (1) people's tendency to avoid harmful action at all costs and (2) people's tendency to maximize outcomes that benefit the greater good. The Cognitive Reflection Task (CRT) was implemented to determine whether an ability to override intuitive responses to counterintuitive problems predicted harm-based moral judgments, as found previously. Hunger bias, independent of IS and emotional state, was influential for non-profitable acceptability judgments of harmful actions. Contrary to dual-process perspectives, a novel finding was that more intuitive responses on the CRT predicted a reduced aversion to harmful actions that was indirectly associated with IS. We suggest that IS may indicate people's vulnerability to cognitive miserliness on the CRT task and reduced deliberation of moral dilemma stimuli. The framing of moral dilemmatic questions to encourage allocentric (acceptability questions) versus egocentric perspectives (appropriateness questions) could explain the divergence between hunger bias and intuitive decision-making for predicting these judgments, respectively. The findings are discussed in relation to dual-process accounts of harm-based moral judgments and evidence linking visceral experiences to harm aversion and moral decision-making.

A reflection on cognitive reflection – testing convergent/divergent validity of two measures of cognitive reflection

The aim of the study was to test convergent/discriminant validity of two measures of cognitive reflection, cognitive reflection test (CRT) and belief bias syllogisms (BBS) and to investigate whether their distinctive characteristic of luring participants into giving wrong intuitive responses explains their relationships with various abilities and disposition measures. Our results show that the same traits largely account for performance on both non-lure task, the Berlin Numeracy Test (BNT), and CRT and explain their correlations with other variables. These results also imply that the predictive validity of CRT for wide range of outcomes does not stem from lures. Regarding the BBS, we found that its correlations with other measures were substantially diminished once we accounted for the effects of BNT. This also implies that the lures are not the reason for the correlation between BBS and these measure. We conclude that the lures are not the reason why cognitive reflection tasks correlate with different outcomes. Our results call into question an original definition of CRT as a measure of ability or disposition to resist reporting first response that comes to mind, as well as the validity of results of studies showing “incremental validity” of CRT over numeracy.