Education increases decision-rule use: An investigation of education and incentives to improve decision making

Human judgment error in the intensive care unit: a perspective on bias and noise

BACKGROUND

In the Intensive Care Unit (ICU), clinicians frequently make complex, high-stakes judgments, where inaccuracies can profoundly affect patient outcomes. This perspective examines human judgment error in ICU settings, specifically bias (systematic error) and noise (random error). While past research has emphasized bias, we explore the role of noise in clinical decision making and its mitigation.

MAIN BODY

System noise refers to unwanted variability in judgments that should ideally be identical. This variability stems from level noise (variability in clinicians' average judgments), stable pattern noise (variability in clinicians' responses to specific patient characteristics), and occasion noise (random, within-clinician variability). Two strategies to reduce noise are the use of algorithms and the averaging of independent judgments.

CONCLUSION

Recognizing and addressing noise in clinical decision making is essential to enhancing judgment accuracy in critical care. By implementing effective noise reduction strategies, clinicians can reduce errors and improve patient outcomes, ultimately advancing the quality of care delivered in ICU settings.

Reducing cheap talk? How monetary incentives affect the accuracy of metamemory judgments

The accuracy of metacognitive judgments is rarely incentivized in experiments; hence, it depends on the participants' willingness to invest cognitive resources and respond truthfully. According to arguments promoted in economic research that performance cannot reach its full potential without proper motivation, metacognitive abilities might therefore have been underestimated. In two experiments (N = 128 and N = 129), we explored the impact of incentives on the accuracy of judgments of learning (JOLs), memory performance, and cue use in free recall of word lists. We introduced a payoff scheme with 5 cents maximum per judgment to promote the accuracy of predicting recall success while simultaneously discouraging strategic responding in the memory test. Incentivizing JOLs had no effect on memory performance. Metacognitive accuracy in terms of resolution (Kruskal's Gamma) was slightly improved in Experiment 1, but not in Experiment 2. On the more negative side, the incentives boosted JOLs indiscriminately, producing substantial overconfidence. A deeper analysis including cues like word concreteness, imagery, arousal, frequency, subjective relevance, and font size showed the usual and simultaneous cue effects on JOLs. However, cue effects were largely unaffected in size by incentivizing JOLs. In summary, incentives for accuracy do not improve the resolution of JOLs to an extent that outweighs the large inflation of overconfidence. Based on the current results, one cannot recommend the future use of incentivized studies in the field of metamemory.

The simple observational critical care studies: estimations by students, nurses, and physicians of in-hospital and 6-month mortality

Background

Prognostic assessments of the mortality of critically ill patients are frequently performed in daily clinical practice and provide prognostic guidance in treatment decisions. In contrast to several sophisticated tools, prognostic estimations made by healthcare providers are always available and accessible, are performed daily, and might have an additive value to guide clinical decision-making. The aim of this study was to evaluate the accuracy of students’, nurses’, and physicians’ estimations and the association of their combined estimations with in-hospital mortality and 6-month follow-up.

Methods

The Simple Observational Critical Care Studies is a prospective observational single-center study in a tertiary teaching hospital in the Netherlands. All patients acutely admitted to the intensive care unit were included. Within 3 h of admission to the intensive care unit, a medical or nursing student, a nurse, and a physician independently predicted in-hospital and 6-month mortality. Logistic regression was used to assess the associations between predictions and the actual outcome; the area under the receiver operating characteristics (AUROC) was calculated to estimate the discriminative accuracy of the students, nurses, and physicians.

Results

In 827 out of 1,010 patients, in-hospital mortality rates were predicted to be 11%, 15%, and 17% by medical students, nurses, and physicians, respectively. The estimations of students, nurses, and physicians were all associated with in-hospital mortality (OR 5.8, 95% CI [3.7, 9.2], OR 4.7, 95% CI [3.0, 7.3], and OR 7.7 95% CI [4.7, 12.8], respectively). Discriminative accuracy was moderate for all students, nurses, and physicians (between 0.58 and 0.68). When more estimations were of non-survival, the odds of non-survival increased (OR 2.4 95% CI [1.9, 3.1]) per additional estimate, AUROC 0.70 (0.65, 0.76). For 6-month mortality predictions, similar results were observed.

Conclusions

Based on the initial examination, students, nurses, and physicians can only moderately predict in-hospital and 6-month mortality in critically ill patients. Combined estimations led to more accurate predictions and may serve as an example of the benefit of multidisciplinary clinical care and future research efforts.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13054-021-03809-w.