Matching bias in syllogistic reasoning: Evidence for a dual-process account from response times and confidence ratings

Set-shifting and inhibition interplay affect the rule-matching bias occurrence during conditional reasoning task

The rule-matching bias is a common error during conditional reasoning tasks, which refers to a tendency to match responses with the lexical context in the conditional rule and leads to incorrect responses. Conditional reasoning is one of the higher-level cognitive abilities affected by many cognitive skills. We aimed to determine whether inhibition and set-shifting skills with rule-matching bias occurrence could be related and, if so, to what quantitative, at a statistically significant level. A total of 30 healthy university students aged 18 to 30 participated in this study. We used the Wason's Selection Task (WST) to measure conditional reasoning and investigated their inhibition and set-shifting skills with the Stroop and Wisconsin Card Sorting Test, respectively. Results showed a significant positive correlation between the number of correct responses to the Stroop test and the Wason Selection Card Test (p=0.614). There was a positive correlation between the number of correct responses to the Wisconsin Card Sorting Test and the Wason Selection Card Test (p=0.423). Participants with higher inhibition and set-shifting abilities showed better performance in the conditional reasoning test and lower rule-matching bias errors.

Recognizing biased reasoning: Conflict detection during decision-making and decision-evaluation

Although it is well established that our thinking can often be biased, the precise cognitive mechanisms underlying these biases are still debated. The present study builds on recent research showing that biased reasoners often seem aware that their reasoning is incorrect; they show signs of conflict detection. One important shortcoming in this research is that the conflict detection effect has only been studied with classic problem-solving tasks, requiring people to make a decision themselves. However, in many reasoning situations people are confronted with decisions already made by others. Therefore, the present study (N = 159) investigated whether conflict detection occurs not only during reasoning on problem-solving tasks (i.e., decision-making), but also on vignette tasks, requiring participants to evaluate decisions made by others. We analyzed participants' conflict detection sensitivity on confidence and response time measures. Results showed that conflict detection occurred during both decision-making and decision-evaluation, as indicated by a decreased confidence. The response time index appeared to be a less reliable measure of conflict detection on the novel tasks. These findings are very relevant for studying reasoning in contexts in which recognizing reasoning errors is important; for instance, in education where teachers have to give feedback on students' reasoning.

Comparing fast thinking and slow thinking: The relative benefits of interventions, individual differences, and inferential rules

Research on judgment and decision making has suggested that the System 2 process of slow thinking can help people to improve their decision making by reducing well-established statistical decision biases (including base rate neglect, probability matching, and the conjunction fallacy). In a large pre-registered study with 1,706 participants and 23,292 unique observations, we compare the effects of individual differences and behavioral interventions to test the relative benefits of slow thinking on performance in canonical judgment and decision-making problems, compared to a control condition, a fast thinking condition, an incentive condition, and a condition that combines fast and slow thinking. We also draw on the rule-based reasoning literature to examine the benefits of having access to a simple form of the rule needed to solve a specific focal problem. Overall, we find equivocal evidence of a small benefit from slow thinking, evidence for a small benefit to accuracy incentives, and clear evidence of a larger cost from fast thinking. The difference in performance between fast-thinking and slow-thinking interventions is comparable to a one-scale point difference on the 4-point Cognitive Reflection Test (CRT). Inferential rules contribute unique explanatory power and interact with individual differences to support the idea that System 2 benefits from a combination of slower processes and knowledge appropriate to the problem.

Not all who ponder count costs: Arithmetic reflection predicts utilitarian tendencies, but logical reflection predicts both deontological and utilitarian tendencies

Conventional sacrificial moral dilemmas propose directly causing some harm to prevent greater harm. Theory suggests that accepting such actions (consistent with utilitarian philosophy) involves more reflective reasoning than rejecting such actions (consistent with deontological philosophy). However, past findings do not always replicate, confound different kinds of reflection, and employ conventional sacrificial dilemmas that treat utilitarian and deontological considerations as opposite. In two studies, we examined whether past findings would replicate when employing process dissociation to assess deontological and utilitarian inclinations independently. Findings suggested two categorically different impacts of reflection: measures of arithmetic reflection, such as the Cognitive Reflection Test, predicted only utilitarian, not deontological, response tendencies. However, measures of logical reflection, such as performance on logical syllogisms, positively predicted both utilitarian and deontological tendencies. These studies replicate some findings, clarify others, and reveal opportunity for additional nuance in dual process theorist's claims about the link between reflection and dilemma judgments.

A developmental study of the bat/ball problem of CRT: How to override the bias and its relation to executive functioning

In two experiments, we explored the nature of the bias observed in the bat/ball problem of the cognitive reflection test (Frederick, 2005, J. Econ. Perspect., 19, 25), how to override it, and its relation to executive functioning. Based on the original bat/ball problem, we designed two additional isomorphic items. In Experiment 1, for four age groups, including 7-, 11-, and 15-year-olds and adults, we determined that the bias is related to the System 1 intervention; the performance in this item was not a matter of mathematical ability and it could be facilitated by changing the order in which the problems were presented. In Experiment 2, we determined that for 15-year-olds, good and bad performances in the item were related to executive functioning, particularly response-distractor inhibition, updating information in working memory, and the regulation of attention; however, subtle differences were identified when the problem was performed in a facilitative context compared with a non-facilitative context. The results indicated that cognitive abilities are a necessary but non-sufficient condition to resolve the problem.

Factors underlying suboptimal diagnostic performance in physicians under time pressure

CONTEXT

Time pressure has been implicated in the suboptimal diagnostic performance of doctors and in increases in diagnostic errors. However, the reasons underlying these effects are not clear. The aim of this study was to investigate the influence of time pressure on physicians' diagnostic accuracy and to explore the mediating effects of perceived stress (emotional pathway) and number of plausible diagnostic hypotheses (cognitive pathway) on the proposed relationship.

METHODS

We conducted a randomised controlled experiment. A total of 75 senior internal medicine residents completed eight written clinical cases under conditions with (n = 40) or without (n = 35) time pressure. They were then asked to: (i) rate the overall stress experienced, and (ii) write down any alternative hypotheses they had thought of when diagnosing the cases. In a post hoc analysis, a mediation path analysis was performed to test the causal relationships between time pressure, perceived stress and number of alternative diagnoses.

RESULTS

Participants who were under time pressure spent less time diagnosing the cases (85.54 seconds versus 181.81 seconds; p< 0.001) and had a lower mean diagnostic accuracy score (0.44 versus 0.53; p = 0.01). In addition, they reported more stress (5.80 versus 4.69; p = 0.01) and generated fewer plausible tentative hypotheses (0.37 versus 0.51; p = 0.01). Two path coefficients were found to be statistically significant; the first path coefficient referred to the relationship between time pressure and perceived stress (standardised β = 0.25, p = 0.029), and the second negative path coefficient referred to the relationship between time pressure and number of plausible alternative hypotheses (standardised β = -0.32, p< 0.01).

CONCLUSIONS

Time pressure adversely influences physicians' diagnostic accuracy by increasing their stress response and reducing the number of plausible hypotheses as mediators.

The Reflective Mind: Examining Individual Differences in Susceptibility to Base Rate Neglect with fMRI

Performance on heuristics and bias tasks has been shown to be susceptible to bias. In turn, susceptibility to bias varies as a function of individual differences in cognitive abilities (e.g., intelligence) and thinking styles (e.g., propensity for reflection). Using a classic task (i.e., lawyer–engineer problem), we conducted two experiments to examine the differential contributions of cognitive abilities versus thinking styles to performance. The results of Experiment 1 demonstrated that the Cognitive Reflection Test (CRT)—a well-established measure of reflective thinking—predicted performance on conflict problems (where base rates and intuition point in opposite directions), whereas STM predicted performance on nonconflict problems. Experiment 2 conducted in the fMRI scanner replicated this behavioral dissociation and enabled us to probe their neural correlates. As predicted, conflict problems were associated with greater activation in the ACC—a key region for conflict detection—even in cases when participants responded stereotypically. In participants with higher CRT scores, conflict problems were associated with greater activation in the posterior cingulate cortex (PCC), and activation in PCC covaried in relation to CRT scores during conflict problems. Also, CRT scores predicted activation in PCC in conflict problems (over and above nonconflict problems). Our results suggest that individual differences in reflective thinking as measured by CRT are related to brain activation in PCC—a region involved in regulating attention between external and internal foci. We discuss the implications of our findings in terms of PCC's possible involvement in switching from intuitive to analytic mode of thought.

Individual differences in conflict detection during reasoning

Decades of reasoning and decision-making research have established that human judgment is often biased by intuitive heuristics. Recent “error” or bias detection studies have focused on reasoners’ abilities to detect whether their heuristic answer conflicts with logical or probabilistic principles. A key open question is whether there are individual differences in this bias detection efficiency. Here we present three studies in which co-registration of different error detection measures (confidence, response time and confidence response time) allowed us to assess bias detection sensitivity at the individual participant level in a range of reasoning tasks. The results indicate that although most individuals show robust bias detection, as indexed by increased latencies and decreased confidence, there is a subgroup of reasoners who consistently fail to do so. We discuss theoretical and practical implications for the field.

Slower is not always better: Response-time evidence clarifies the limited role of miserly information processing in the Cognitive Reflection Test

We report a study examining the role of ‘cognitive miserliness’ as a determinant of poor performance on the standard three-item Cognitive Reflection Test (CRT). The cognitive miserliness hypothesis proposes that people often respond incorrectly on CRT items because of an unwillingness to go beyond default, heuristic processing and invest time and effort in analytic, reflective processing. Our analysis (N = 391) focused on people’s response times to CRT items to determine whether predicted associations are evident between miserly thinking and the generation of incorrect, intuitive answers. Evidence indicated only a weak correlation between CRT response times and accuracy. Item-level analyses also failed to demonstrate predicted response-time differences between correct analytic and incorrect intuitive answers for two of the three CRT items. We question whether participants who give incorrect intuitive answers on the CRT can legitimately be termed cognitive misers and whether the three CRT items measure the same general construct.

Dual processes and moral conflict: Evidence for deontological reasoners’ intuitive utilitarian sensitivity

The prominent dual process model of moral cognition suggests that reasoners intuitively detect that harming others is wrong (deontological System-1 morality) but have to engage in demanding deliberation to realize that harm can be acceptable depending on the consequences (utilitarian System-2 morality). But the nature of the interaction between the processes is not clear. To address this key issue we tested whether deontological reasoners also intuitively grasp the utilitarian dimensions of classic moral dilemmas. In three studies subjects solved moral dilemmas in which utilitarian and deontological considerations cued conflicting or non-conflicting decisions while performing a demanding concurrent load task. Results show that reasoners’ sensitivity to conflicting moral perspectives, as reflected in decreased decision confidence and increased experienced processing difficulty, was unaffected by cognitive load. We discuss how these findings argue for a hybrid dual process model interpretation in which System-1 cues both a deontological and utilitarian intuition.

The Doubting System 1: Evidence for automatic substitution sensitivity

A long prevailing view of human reasoning suggests severe limits on our ability to adhere to simple logical or mathematical prescriptions. A key position assumes these failures arise from insufficient monitoring of rapidly produced intuitions. These faulty intuitions are thought to arise from a proposed substitution process, by which reasoners unknowingly interpret more difficult questions as easier ones. Recent work, however, suggests that reasoners are not blind to this substitution process, but in fact detect that their erroneous responses are not warranted. Using the popular bat-and-ball problem, we investigated whether this substitution sensitivity arises out of an automatic System 1 process or whether it depends on the operation of an executive resource demanding System 2 process. Results showed that accuracy on the bat-and-ball problem clearly declined under cognitive load. However, both reduced response confidence and increased response latencies indicated that biased reasoners remained sensitive to their faulty responses under load. Results suggest that a crucial substitution monitoring process is not only successfully engaged, but that it automatically operates as an autonomous System 1 process. By signaling its doubt along with a biased intuition, it appears System 1 is "smarter" than traditionally assumed.

Comprehension and computation in Bayesian problem solving

Humans have long been characterized as poor probabilistic reasoners when presented with explicit numerical information. Bayesian word problems provide a well-known example of this, where even highly educated and cognitively skilled individuals fail to adhere to mathematical norms. It is widely agreed that natural frequencies can facilitate Bayesian inferences relative to normalized formats (e.g., probabilities, percentages), both by clarifying logical set-subset relations and by simplifying numerical calculations. Nevertheless, between-study performance on “transparent” Bayesian problems varies widely, and generally remains rather unimpressive. We suggest there has been an over-focus on this representational facilitator (i.e., transparent problem structures) at the expense of the specific logical and numerical processing requirements and the corresponding individual abilities and skills necessary for providing Bayesian-like output given specific verbal and numerical input. We further suggest that understanding this task-individual pair could benefit from considerations from the literature on mathematical cognition, which emphasizes text comprehension and problem solving, along with contributions of online executive working memory, metacognitive regulation, and relevant stored knowledge and skills. We conclude by offering avenues for future research aimed at identifying the stages in problem solving at which correct vs. incorrect reasoners depart, and how individual differences might influence this time point.

What makes us think? A three-stage dual-process model of analytic engagement

The distinction between intuitive and analytic thinking is common in psychology. However, while often being quite clear on the characteristics of the two processes ('Type 1' processes are fast, autonomous, intuitive, etc. and 'Type 2' processes are slow, deliberative, analytic, etc.), dual-process theorists have been heavily criticized for being unclear on the factors that determine when an individual will think analytically or rely on their intuition. We address this issue by introducing a three-stage model that elucidates the bottom-up factors that cause individuals to engage Type 2 processing. According to the model, multiple Type 1 processes may be cued by a stimulus (Stage 1), leading to the potential for conflict detection (Stage 2). If successful, conflict detection leads to Type 2 processing (Stage 3), which may take the form of rationalization (i.e., the Type 1 output is verified post hoc) or decoupling (i.e., the Type 1 output is falsified). We tested key aspects of the model using a novel base-rate task where stereotypes and base-rate probabilities cued the same (non-conflict problems) or different (conflict problems) responses about group membership. Our results support two key predictions derived from the model: (1) conflict detection and decoupling are dissociable sources of Type 2 processing and (2) conflict detection sometimes fails. We argue that considering the potential stages of reasoning allows us to distinguish early (conflict detection) and late (decoupling) sources of analytic thought. Errors may occur at both stages and, as a consequence, bias arises from both conflict monitoring and decoupling failures.

The intersection between Descriptivism and Meliorism in reasoning research: further proposals in support of 'soft normativism'

The rationality paradox centers on the observation that people are highly intelligent, yet show evidence of errors and biases in their thinking when measured against normative standards. Elqayam and Evans’ (2011) reject normative standards in the psychological study of thinking, reasoning and deciding in favor of a ‘value-free’ descriptive approach to studying high-level cognition. In reviewing Elqayam and Evans’ (2011) position, we defend an alternative to descriptivism in the form of ‘soft normativism,’ which allows for normative evaluations alongside the pursuit of descriptive research goals. We propose that normative theories have considerable value provided that researchers: (1) are alert to the philosophical quagmire of strong relativism; (2) are mindful of the biases that can arise from utilizing normative benchmarks; and (3) engage in a focused analysis of the processing approach adopted by individual reasoners. We address the controversial ‘is–ought’ inference in this context and appeal to a ‘bridging solution’ to this contested inference that is based on the concept of ‘informal reflective equilibrium.’ Furthermore, we draw on Elqayam and Evans’ (2011) recognition of a role for normative benchmarks in research programs that are devised to enhance reasoning performance and we argue that such Meliorist research programs have a valuable reciprocal relationship with descriptivist accounts of reasoning. In sum, we believe that descriptions of reasoning processes are fundamentally enriched by evaluations of reasoning quality, and argue that if such standards are discarded altogether then our explanations and descriptions of reasoning processes are severely undermined.