Editorial: Helping health care organizations to define diagnostic errors as missed opportunities in diagnosis

Epidemiology of diagnostic errors in pediatric emergency departments using electronic triggers

OBJECTIVES

We applied three electronic triggers to study frequency and contributory factors of missed opportunities for improving diagnosis (MOIDs) in pediatric emergency departments (EDs): return visits within 10 days resulting in admission (Trigger 1), care escalation within 24 h of ED presentation (Trigger 2), and death within 24 h of ED visit (Trigger 3).

METHODS

We created an electronic query and reporting template for the triggers and applied them to electronic health record systems of five pediatric EDs for visits from 2019. Clinician reviewers manually screened identified charts and initially categorized them as "unlikely for MOIDs" or "unable to rule out MOIDs" without a detailed chart review. For the latter category, reviewers performed a detailed chart review using the Revised Safer Dx Instrument to determine the presence of a MOID.

RESULTS

A total of 2937 ED records met trigger criteria (Trigger 1 1996 [68%], Trigger 2 829 [28%], Trigger 3 112 [4%]), of which 2786 (95%) were categorized as unlikely for MOIDs. The Revised Safer Dx Instrument was applied to 151 (5%) records and 76 (50%) had MOIDs. The overall frequency of MOIDs was 2.6% for the entire cohort, 3.0% for Trigger 1, 1.9% for Trigger 2, and 0% for Trigger 3. Brain lesions, infections, or hemorrhage; pneumonias and lung abscess; and appendicitis were the top three missed diagnoses. The majority (54%) of MOIDs cases resulted in patient harm. Contributory factors were related to patient-provider (52.6%), followed by patient factors (21.1%), system factors (13.2%), and provider factors (10.5%).

CONCLUSIONS

Using electronic triggers with selective record review is an effective process to screen for harmful diagnostic errors in EDs: detailed review of 5% of charts revealed MOIDs in half, of which half were harmful to the patient. With further refining, triggers can be used as effective patient safety tools to monitor diagnostic quality.

Harbingers of sepsis misdiagnosis among pediatric emergency department patients

OBJECTIVES

To identify clinical presentations that acted as harbingers for future sepsis hospitalizations in pediatric patients evaluated in the emergency department (ED) using the Symptom Disease Pair Analysis of Diagnostic Error (SPADE) methodology.

METHODS

We identified patients in the Pediatric Health Information Systems (PHIS) database admitted for sepsis between January 1, 2004 and December 31, 2023 and limited the study cohort to those patients who had an ED treat-and-release visit in the 30 days prior to admission. Using the look-back approach of the SPADE methodology, we identified the most common clinical presentations at the initial ED visit and used an observed to expected (O:E) analysis to determine which presentations were overrepresented. We then employed a graphical, temporal analysis with a comparison group to identify which overrepresented presentations most likely represented harbingers for future sepsis hospitalization.

RESULTS

We identified 184,157 inpatient admissions for sepsis, of which 15,331 hospitalizations (8.3 %) were preceded by a treat-and-release ED visit in the prior 30 days. Based on the O:E and temporal analyses, the presentations of fever and dehydration were both overrepresented in the study cohort and temporally clustered close to sepsis hospitalization. ED treat-and-release visits for fever or dehydration preceded 1.2 % of all sepsis admissions.

CONCLUSIONS

In pediatric patients presenting to the ED, fever and dehydration may represent harbingers for future sepsis hospitalization. The SPADE methodology could be applied to the PHIS database to develop diagnostic performance measures across a wide range of pediatric hospitals.

Root cause analysis of cases involving diagnosis

Diagnostic errors comprise the leading threat to patient safety in healthcare today. Learning how to extract the lessons from cases where diagnosis succeeds or fails is a promising approach to improve diagnostic safety going forward. We present up-to-date and authoritative guidance on how the existing approaches to conducting root cause analyses (RCA's) can be modified to study cases involving diagnosis. There are several diffierences: In cases involving diagnosis, the investigation should begin immediately after the incident, and clinicians involved in the case should be members of the RCA team. The review must include consideration of how the clinical reasoning process went astray (or succeeded), and use a human-factors perspective to consider the system-related contextual factors in the diagnostic process. We present detailed instructions for conducting RCA's of cases involving diagnosis, with advice on how to identify root causes and contributing factors and select appropriate interventions.

Evaluation of a Natural Language Processing Approach to Identify Diagnostic Errors and Analysis of Safety Learning System Case Review Data: Retrospective Cohort Study

BACKGROUND

Diagnostic errors are an underappreciated cause of preventable mortality in hospitals and pose a risk for severe patient harm and increase hospital length of stay.

OBJECTIVE

This study aims to explore the potential of machine learning and natural language processing techniques in improving diagnostic safety surveillance. We conducted a rigorous evaluation of the feasibility and potential to use electronic health records clinical notes and existing case review data.

METHODS

Safety Learning System case review data from 1 large health system composed of 10 hospitals in the mid-Atlantic region of the United States from February 2016 to September 2021 were analyzed. The case review outcome included opportunities for improvement including diagnostic opportunities for improvement. To supplement case review data, electronic health record clinical notes were extracted and analyzed. A simple logistic regression model along with 3 forms of logistic regression models (ie, Least Absolute Shrinkage and Selection Operator, Ridge, and Elastic Net) with regularization functions was trained on this data to compare classification performances in classifying patients who experienced diagnostic errors during hospitalization. Further, statistical tests were conducted to find significant differences between female and male patients who experienced diagnostic errors.

RESULTS

In total, 126 (7.4%) patients (of 1704) had been identified by case reviewers as having experienced at least 1 diagnostic error. Patients who had experienced diagnostic error were grouped by sex: 59 (7.1%) of the 830 women and 67 (7.7%) of the 874 men. Among the patients who experienced a diagnostic error, female patients were older (median 72, IQR 66-80 vs median 67, IQR 57-76; P=.02), had higher rates of being admitted through general or internal medicine (69.5% vs 47.8%; P=.01), lower rates of cardiovascular-related admitted diagnosis (11.9% vs 28.4%; P=.02), and lower rates of being admitted through neurology department (2.3% vs 13.4%; P=.04). The Ridge model achieved the highest area under the receiver operating characteristic curve (0.885), specificity (0.797), positive predictive value (PPV; 0.24), and F1-score (0.369) in classifying patients who were at higher risk of diagnostic errors among hospitalized patients.

CONCLUSIONS

Our findings demonstrate that natural language processing can be a potential solution to more effectively identifying and selecting potential diagnostic error cases for review and therefore reducing the case review burden.

Ten misconceptions regarding decision-making in critical care

Diagnostic errors are prevalent in critical care practice and are associated with patient harm and costs for providers and the healthcare system. Patient complexity, illness severity, and the urgency in initiating proper treatment all contribute to decision-making errors. Clinician-related factors such as fatigue, cognitive overload, and inexperience further interfere with effective decision-making. Cognitive science has provided insight into the clinical decision-making process that can be used to reduce error. This evidence-based review discusses ten common misconceptions regarding critical care decision-making. By understanding how practitioners make clinical decisions and examining how errors occur, strategies may be developed and implemented to decrease errors in Decision-making and improve patient outcomes.

Achieving Equity in EMS Care and Patient Outcomes Through Quality Management Systems: A Position Statement

Improving health and safety in our communities requires deliberate focus and commitment to equity. Inequities are differences in access, treatment, and outcomes between individuals and across populations that are systemic, avoidable, and unjust. Within health care in general, and Emergency Medical Services (EMS) in particular, there are demonstrated inequities in the quality of care provided to patients based on a number of characteristics linked to discrimination, exclusion, or bias. Given the critical role that EMS plays within the health care system, it is imperative that EMS systems reduce inequities by delivering evidence-based, high-quality care for the communities and patients we serve. To achieve equity in EMS care delivery and patient outcomes, the National Association of EMS Physicians recommends that EMS systems and agencies:make health equity a strategic priority and commit to improving equity at all levels.assess and monitor clinical and safety quality measures through the lens of inequities as an integrated part of the quality management process.ensure that data elements are structured to enable equity analysis at every level and routinely evaluate data for limitations hindering equity analysis and improvement.involve patients and community stakeholders in determining data ownership and stewardship to ensure its ongoing evolution and fitness for use for measuring care inequities.address biases as they translate into the quality of care and standards of respect for patients.pursue equity through a framework rooted in the principles of improvement science.

Diagnostic Errors in Hospitalized Adults Who Died or Were Transferred to Intensive Care

Importance

Diagnostic errors contribute to patient harm, though few data exist to describe their prevalence or underlying causes among medical inpatients.

Objective

To determine the prevalence, underlying cause, and harms of diagnostic errors among hospitalized adults transferred to an intensive care unit (ICU) or who died.

Design, Setting, and Participants

Retrospective cohort study conducted at 29 academic medical centers in the US in a random sample of adults hospitalized with general medical conditions and who were transferred to an ICU, died, or both from January 1 to December 31, 2019. Each record was reviewed by 2 trained clinicians to determine whether a diagnostic error occurred (ie, missed or delayed diagnosis), identify diagnostic process faults, and classify harms. Multivariable models estimated association between process faults and diagnostic error. Opportunity for diagnostic error reduction associated with each fault was estimated using the adjusted proportion attributable fraction (aPAF). Data analysis was performed from April through September 2023.

Main Outcomes and Measures

Whether or not a diagnostic error took place, the frequency of underlying causes of errors, and harms associated with those errors.

Results

Of 2428 patient records at 29 hospitals that underwent review (mean [SD] patient age, 63.9 [17.0] years; 1107 [45.6%] female and 1321 male individuals [54.4%]), 550 patients (23.0%; 95% CI, 20.9%-25.3%) had experienced a diagnostic error. Errors were judged to have contributed to temporary harm, permanent harm, or death in 436 patients (17.8%; 95% CI, 15.9%-19.8%); among the 1863 patients who died, diagnostic error was judged to have contributed to death in 121 (6.6%; 95% CI, 5.3%-8.2%). In multivariable models examining process faults associated with any diagnostic error, patient assessment problems (aPAF, 21.4%; 95% CI, 16.4%-26.4%) and problems with test ordering and interpretation (aPAF, 19.9%; 95% CI, 14.7%-25.1%) had the highest opportunity to reduce diagnostic errors; similar ranking was seen in multivariable models examining harmful diagnostic errors.

Conclusions and Relevance

In this cohort study, diagnostic errors in hospitalized adults who died or were transferred to the ICU were common and associated with patient harm. Problems with choosing and interpreting tests and the processes involved with clinician assessment are high-priority areas for improvement efforts.

Diagnostic errors in Dentistry, opinions of egyptian dental teaching staff, a cross-sectional study

BACKGROUND

Diagnostic errors is a known problem in healthcare practice. Data on diagnostic errors in the dental field are extremely lacking. The objective of the study is to explore the perception of dental teaching staff about the prevalence of dental diagnostic errors in Egypt, identify the most commonly misdiagnosed dental conditions and point out the contributing factors and levels of patient harm.

METHODS

A cross-sectional questionnaire-based study was conducted on 151 dental teaching staff of Egyptian governmental and private universities. The questionnaire was distributed electronically via social media and messaging apps to dental staff members with at least five years of clinical experience to assess their opinion regarding the study objectives. Results were collected and statistically analyzed.

RESULTS

94.7% of participants believed that diagnostic errors represent an urgent problem, lecturers believed by 2.703 folds more than professors that diagnostic errors are an urgent problem The percentage of diagnostic errors was estimated to be < 20% and 20-40% by more than 90% of participants. The most commonly misdiagnosed conditions were oral mucosal lesions (83.4%), followed by temporomandibular joint and periodontal conditions (58.9%) for each. More than half of the participants (60.9%) believe that medical education methodology is one of the factors that lead to dental diagnosis errors. For the impact of errors on patients, 53% of participants reported moderate impacts followed by minor impact (37.7%) while 4.6% reported no impact and the same percentage reported major impact.

CONCLUSION

This study with statistically significant results reported that dental diagnostic errors are frequent and need to be approached. Oral mucosal lesions, periodontal and temporomandibular joint diseases represent areas that include the most commonly seen errors. Further, besides the lack of resources, the dental education system and lack of proper training are the main causes of this problem.

A structured approach to EHR surveillance of diagnostic error in acute care: an exploratory analysis of two institutionally-defined case cohorts

OBJECTIVES

To test a structured electronic health record (EHR) case review process to identify diagnostic errors (DE) and diagnostic process failures (DPFs) in acute care.

METHODS

We adapted validated tools (Safer Dx, Diagnostic Error Evaluation Research [DEER] Taxonomy) to assess the diagnostic process during the hospital encounter and categorized 13 postulated e-triggers. We created two test cohorts of all preventable cases (n=28) and an equal number of randomly sampled non-preventable cases (n=28) from 365 adult general medicine patients who expired and underwent our institution's mortality case review process. After excluding patients with a length of stay of more than one month, each case was reviewed by two blinded clinicians trained in our process and by an expert panel. Inter-rater reliability was assessed. We compared the frequency of DE contributing to death in both cohorts, as well as mean DPFs and e-triggers for DE positive and negative cases within each cohort.

RESULTS

Twenty-seven (96.4%) preventable and 24 (85.7%) non-preventable cases underwent our review process. Inter-rater reliability was moderate between individual reviewers (Cohen's kappa 0.41) and substantial with the expert panel (Cohen's kappa 0.74). The frequency of DE contributing to death was significantly higher for the preventable compared to the non-preventable cohort (56% vs. 17%, OR 6.25 [1.68, 23.27], p<0.01). Mean DPFs and e-triggers were significantly and non-significantly higher for DE positive compared to DE negative cases in each cohort, respectively.

CONCLUSIONS

We observed substantial agreement among final consensus and expert panel reviews using our structured EHR case review process. DEs contributing to death associated with DPFs were identified in institutionally designated preventable and non-preventable cases. While e-triggers may be useful for discriminating DE positive from DE negative cases, larger studies are required for validation. Our approach has potential to augment institutional mortality case review processes with respect to DE surveillance.

Diagnostic delays in infectious diseases

OBJECTIVES

Diagnostic delays are a major source of morbidity and mortality. Despite the adverse outcomes associated with diagnostic delays, few studies have examined the incidence and factors that influence diagnostic delays for different infectious diseases. The objective of this study was to understand the relative frequency of diagnostic delays for six infectious diseases commonly seen by infectious diseases (ID) consultants and to examine contributing factors for these delays.

METHODS

A 25-item survey to examine diagnostic delays in six infectious diseases was sent to all infectious diseases physicians in the Emerging Infections Network (EIN) who provide care to adult patients. Diseases included (1) tuberculosis, (2) non-tuberculous mycobacterial infections, (3) syphilis, (4) epidural abscess, (5) infective endocarditis, and (6) endemic fungal infections (e.g., histoplasmosis, blastomycosis).

RESULTS

A total of 533 of 1,323 (40%) EIN members responded to the survey. Respondents perceived the diagnosis not being considered initially and the appropriate test not being ordered as the two most important contributors to diagnostic delays. Unusual clinical presentations and not consulting ID physicians early enough were also reported as a contributing factor to delays. Responses recorded in open-text fields also indicated errors related to testing as a likely cause of delays; specifically, test-related errors included ordering the wrong laboratory test, laboratory delays (specialized labs not available at the facility), and lab processing delays.

CONCLUSIONS

Diagnostic delays commonly occur for the infectious diseases we considered. The contributing factors we identified are potential targets for future interventions to decrease diagnostic delays.

Cerebrovascular disease hospitalizations following emergency department headache visits: A nested case-control study

BACKGROUND

Delayed diagnosis of cerebrovascular disease (CVD) among patients can result in substantial harm. If diagnostic process failures can be identified at emergency department (ED) visits that precede CVD hospitalization, interventions to improve diagnostic accuracy can be developed.

METHODS

We conducted a nested case-control study using a cohort of adult ED patients discharged from a single medical center with a benign headache diagnosis from October 1, 2015 to March 31, 2018. Hospitalizations for CVD within 1 year of index ED visit were identified using a regional health information exchange. Patients with subsequent CVD hospitalization (cases) were individually matched to patients without subsequent hospitalization (controls) using patient age and visit date. Demographic, clinical, and ED process characteristics were assessed via detailed chart review. McNemar's test for categorical and paired t-test for continuous variables were used with statistical significance set at ≤0.05.

RESULTS

Of the 9157 patients with ED headache visits, 57 (0.6%, 95% confidence interval [CI] = 0.5-0.8) had a subsequent CVD hospitalization. Median time from ED visit to hospitalization was 107 days. In 25 patients (43.9%, 25/57) the CVD hospitalization and the index ED visit were at different hospitals. Fifty-three cases and 53 matched controls were included in the final study analysis. Cases and controls had similar baseline demographic and headache characteristics. Cases more often had a history of stroke (32.1% vs. 13.2%, p = 0.02) and neurosurgery (13.2% vs. 1.9%, p = 0.03) prior to the index ED visit. Cases more often had less than two components of the neurologic examination documented (30.2% vs. 11.3%, p = 0.03).

CONCLUSION

We found that 0.6% of patients with an ED headache visit had subsequent CVD hospitalization, often at another medical center. ED visits for headache complaints among patients with prior stroke or neurosurgical procedures may be important opportunities for CVD prevention. Documented neurologic examinations were poorer among cases, which may represent an opportunity for ED process improvement.

Emergency medicine physicians' perspectives on diagnostic accuracy in neurology: a qualitative study

OBJECTIVES

We sought to understand the knowledge, attitudes, and beliefs of emergency medicine (EM) physicians towards non-specific neurological conditions and the use of clinical decision support (CDS) to improve diagnostic accuracy.

METHODS

We conducted semi-structured interviews of EM physicians at four emergency departments (EDs) affiliated with a single US healthcare system. Interviews were conducted until thematic saturation was achieved. Conventional content analysis was used to identify themes related to EM physicians' perspectives on acute diagnostic neurology; directed content analysis was used to explore views regarding CDS. Each interview transcript was independently coded by two researchers using an iteratively refined codebook with consensus-based resolution of coding differences.

RESULTS

We identified two domains regarding diagnostic safety: (1) challenges unique to neurological complaints and (2) challenges in EM more broadly. Themes relevant to neurology included: (1) knowledge gaps and uncertainty, (2) skepticism about neurology, (3) comfort with basic as opposed to detailed neurological examination, and (4) comfort with non-neurological diseases. Themes relevant to diagnostic decision making in the ED included: (1) cognitive biases, (2) ED system/environmental issues, (3) patient barriers, (4) comfort with diagnostic uncertainty, and (5) concerns regarding diagnostic error identification and measurement. Most participating EM physicians were enthusiastic about the potential for well-designed CDS to improve diagnostic accuracy for non-specific neurological complaints.

CONCLUSIONS

Physicians identified diagnostic challenges unique to neurological diseases as well as issues related more generally to diagnostic accuracy in EM. These physician-reported issues should be accounted for when designing interventions to improve ED diagnostic accuracy.

Incidence, origins and avoidable harm of missed opportunities in diagnosis: longitudinal patient record review in 21 English general practices

BACKGROUND

Diagnostic error is a global patient safety priority.

OBJECTIVES

To estimate the incidence, origins and avoidable harm of diagnostic errors in English general practice. Diagnostic errors were defined as missed opportunities to make a correct or timely diagnosis based on the evidence available (missed diagnostic opportunities, MDOs).

METHOD

Retrospective medical record reviews identified MDOs in 21 general practices. In each practice, two trained general practitioner reviewers independently conducted case note reviews on 100 randomly selected adult consultations performed during 2013-2014. Consultations where either reviewer identified an MDO were jointly reviewed.

RESULTS

Across 2057 unique consultations, reviewers agreed that an MDO was possible, likely or certain in 89 cases or 4.3% (95% CI 3.6% to 5.2%) of reviewed consultations. Inter-reviewer agreement was higher than most comparable studies (Fleiss' kappa=0.63). Sixty-four MDOs (72%) had two or more contributing process breakdowns. Breakdowns involved problems in the patient-practitioner encounter such as history taking, examination or ordering tests (main or secondary factor in 61 (68%) cases), performance and interpretation of diagnostic tests (31; 35%) and follow-up and tracking of diagnostic information (43; 48%). 37% of MDOs were rated as resulting in moderate to severe avoidable patient harm.

CONCLUSIONS

Although MDOs occurred in fewer than 5% of the investigated consultations, the high numbers of primary care contacts nationally suggest that several million patients are potentially at risk of avoidable harm from MDOs each year. Causes of MDOs were frequently multifactorial, suggesting the need for development and evaluation of multipronged interventions, along with policy changes to support them.

Automated identification of diagnostic labelling errors in medicine

Objectives

Identification of diagnostic error is complex and mostly relies on expert ratings, a severely limited procedure. We developed a system that allows to automatically identify diagnostic labelling error from diagnoses coded according to the international classification of diseases (ICD), often available as routine health care data.

Methods

The system developed (index test) was validated against rater based classifications taken from three previous studies of diagnostic labeling error (reference standard). The system compares pairs of diagnoses through calculation of their distance within the ICD taxonomy. Calculation is based on four different algorithms. To assess the concordance between index test and reference standard, we calculated the area under the receiver operating characteristics curve (AUROC) and corresponding confidence intervals. Analysis were conducted overall and separately per algorithm and type of available dataset.

Results

Diagnoses of 1,127 cases were analyzed. Raters previously classified 24.58% of cases as diagnostic labelling errors (ranging from 12.3 to 87.2% in the three datasets). AUROC ranged between 0.821 and 0.837 overall, depending on the algorithm used to calculate the index test (95% CIs ranging from 0.8 to 0.86). Analyzed per type of dataset separately, the highest AUROC was 0.924 (95% CI 0.887–0.962).

Conclusions

The trigger system to automatically identify diagnostic labeling error from routine health care data performs excellent, and is unaffected by the reference standards’ limitations. It is however only applicable to cases with pairs of diagnoses, of which one must be more accurate or otherwise superior than the other, reflecting a prevalent definition of a diagnostic labeling error.

A survey of outpatient Internal Medicine clinician perceptions of diagnostic error

Background Little is known about how practicing Internal Medicine (IM) clinicians perceive diagnostic error, and whether perceptions are in agreement with the published literature. Methods A 16-question survey was administered across two IM practices: one a referral practice providing care for patients traveling for a second opinion and the other a traditional community-based primary care practice. Our aim was to identify individual- and system-level factors contributing to diagnostic error (primary outcome) and conditions at greatest risk of diagnostic error (secondary outcome). Results Sixty-five of 125 clinicians surveyed (51%) responded. The most commonly perceived individual factors contributing to diagnostic error included atypical patient presentations (83%), failure to consider other diagnoses (63%) and inadequate follow-up of test results (53%). The most commonly cited system-level factors included cognitive burden created by the volume of data in the electronic health record (EHR) (68%), lack of time to think (64%) and systems that do not support collaboration (40%). Conditions felt to be at greatest risk of diagnostic error included cancer (46%), pulmonary embolism (43%) and infection (37%). Conclusions Inadequate clinician time and sub-optimal patient and test follow-up are perceived by IM clinicians to be persistent contributors to diagnostic error. Clinician perceptions of conditions at greatest risk of diagnostic error may differ from the published literature.

Understanding diagnostic processes in emergency departments: a mixed methods case study protocol

INTRODUCTION

Diagnostic processes in the emergency department (ED) involve multiple interactions among individuals who interface with information systems to access and record information. A better understanding of diagnostic processes is needed to mitigate errors. This paper describes a study protocol to map diagnostic processes in the ED as a foundation for developing future error mitigation strategies.

METHODS AND ANALYSIS

This study of an adult and a paediatric academic ED uses a prospective mixed methods case study design informed by an ED-specific diagnostic decision-making model (the modified ED-National Academies of Sciences, Engineering and Medicine (NASEM) model) and two cognitive theories (dual process theory and distributed cognition). Data sources include audio recordings of patient and care team interactions, electronic health record data, observer field notes and stakeholder interviews. Multiple qualitative analysis methods will be used to explore diagnostic processes in situ, including systems information flow, human-human and human-system interactions and contextual factors influencing cognition. The study has three parts. Part 1 involves prospective field observations of patients with undifferentiated symptoms at high risk for diagnostic error, where each patient is followed throughout the entire care delivery process. Part 2 involves observing individual care team providers over a 4-hour window to capture their diagnostic workflow, team coordination and communication across multiple patients. Part 3 uses interviews with key stakeholders to understand different perspectives on the diagnostic process, as well as perceived strengths and vulnerabilities, in order to enrich the ED-NASEM diagnostic model.

ETHICS AND DISSEMINATION

The University of Michigan Institutional Review Board approved this study, HUM00156261. This foundational work will help identify strengths and vulnerabilities in diagnostic processes. Further, it will inform the future development and testing of patient, provider and systems-level interventions for mitigating error and improving patient safety in these and other EDs. The work will be disseminated through journal publications and presentations at national and international meetings.

Thresholds, rules and defensive strategies: how physicians learn from their prior diagnosis-related experiences

Background Health professionals are encouraged to learn from their errors. Determining how primary care physicians (PCPs) react to a case, in which their original diagnosis differed from the final outcome, could provide new insights on how they learn from experiences. We explored how PCPs altered their diagnostic evaluation of future patients after cases where the originally assumed diagnosis turned out to be wrong. Methods We asked German PCPs to complete an online survey where they described how the patient concerned originally presented, the subsequent course of events and whether they would change their diagnostic work-up of future patients. Qualitative methods were used to analyze narrative text obtained by this survey. Results A total of 29 PCPs submitted cases, most of which were ultimately found to be more severe than originally assumed. PCPs (n = 27) reflected on changes to their subsequent clinical decisions in the form of general maxims (n = 20) or more specific rules (n = 11). Most changes would have resulted in a lower threshold for investigations, referral and/or a more extensive collection of diagnostic information. PCPs decided not only to listen more often to their intuition (gut feelings), but to also practice more analytical reasoning. Participants felt the need for change of practice even if no clinical standards had been violated in the diagnosis of that case. Some decided to resort to defensive strategies in the future. Conclusions We describe mechanisms by which physicians calibrate their decision thresholds, as well as their cognitive mode (intuitive vs. analytical). PCPs reported the need for change in clinical practice despite the absence of error in some cases.

Psychometric Properties of the Greek Version of the Medical Office on Patient Safety Culture in Primary Care Settings

Background: Safety culture is considered one of the most crucial premises for further development of patient care in healthcare. During the eight-year economic crisis (2010–2018), Greece made significant reforms in the way the primary health care system operates, aiming at the more efficient operation of the system without degrading issues of safety and quality of the provided health services. In this context, this study aims to validate a specialized tool—the Medical Office Survey on Patient Safety Culture (MOSPSC)—developed by the Agency for Healthcare Research and Quality (AHRQ) to evaluate primary care settings in terms of safety culture and quality. Methods: Factor analysis determined the correlation of the factor structure in Greek data with the original questionnaire. The relation of the factor analysis with the Cronbach’s coefficient alpha was also determined, including the construct validity. Results: Eight composites with 34 items were extracted by exploratory factor analysis, with acceptable Cronbach’s alpha coefficients and good construct validity. Consequently, the composites jointly explained 62% of the variance in the responses. Five items were removed from the original version of the questionnaire. As a result, three out of the eight composites were a mixture of items from different compounds of the original tool. The composition of the five factors was similar to that in the original questionnaire. Conclusions: The MOSPSC tool in Greek primary healthcare settings can be used to assess patient safety culture in facilities across the country. From the study, the patient safety culture in Greece was positive, although few composites showed a negative correlation and needed improvement.

What Contributes to Diagnostic Error or Delay? A Qualitative Exploration Across Diverse Acute Care Settings in the United States

OBJECTIVES

Diagnostic error and delay is a prevalent and impactful problem. This study was part of a mixed-methods approach to understand the organizational, clinician, and patient factors contributing to diagnostic error and delay among acutely ill patients within a health system, as well as recommendations for the development of tailored, targeted, feasible, and effective interventions.

METHODS

We did a multisite qualitative study using focus group methodology to explore the perspectives of key clinician stakeholders. We used a conceptual framework that characterized diagnostic error and delay as occurring within 1 of 3 stages of the patient's diagnostic journey-critical information gathering, synthesis of key information, and decision making and communication. We developed our moderator guide based on the sociotechnical frameworks previously described by Holden and Singh for understanding noncognitive factors that lead to diagnostic error and delay. Deidentified focus group transcripts were coded in triplicate and to consensus over a series of meetings. A final coded data set was then uploaded into NVivo software. The data were then analyzed to generate overarching themes and categories.

RESULTS

We recruited a total of 64 participants across 4 sites from emergency departments, hospital floor, and intensive care unit settings into 11 focus groups. Clinicians perceive that diverse organizational, communication and coordination, individual clinician, and patient factors interact to impede the process of making timely and accurate diagnoses.

CONCLUSIONS

This study highlights the complex sociotechnical system within which individual clinicians operate and the contributions of systems, processes, and institutional factors to diagnostic error and delay.

Development of a rubric for assessing delayed diagnosis of appendicitis, diabetic ketoacidosis and sepsis

OBJECTIVES

Using case review to determine whether a patient experienced a delayed diagnosis is challenging. Measurement would be more accurate if case reviewers had access to multi-expert consensus on grading the likelihood of delayed diagnosis. Our objective was to use expert consensus to create a guide for objectively grading the likelihood of delayed diagnosis of appendicitis, new-onset diabetic ketoacidosis (DKA), and sepsis.

METHODS

Case vignettes were constructed for each condition. In each vignette, a patient has the condition and had a previous emergency department (ED) visit within 7 days. Condition-specific multi-specialty expert Delphi panels reviewed the case vignettes and graded the likelihood of a delayed diagnosis on a five-point scale. Delayed diagnosis was defined as the condition being present during the previous ED visit. Consensus was defined as ≥75% agreement. In each Delphi round, panelists were given the scores from the previous round and asked to rescore. A case scoring guide was created from the consensus scores.

RESULTS

Eighteen expert panelists participated. Consensus was achieved within three Delphi rounds for all appendicitis and sepsis vignettes. We reached consensus on 23/30 (77%) DKA vignettes. A case review guide was created from the consensus scores.

CONCLUSIONS

Multi-specialty expert reviewers can agree on the likelihood of a delayed diagnosis for cases of appendicitis and sepsis, and for most cases of DKA. We created a guide that can be used by researchers and quality improvement specialists to allow for objective case review to determine when delayed diagnoses have occurred for appendicitis, DKA, and sepsis.

Identifying trigger concepts to screen emergency department visits for diagnostic errors

OBJECTIVES

The diagnostic process is a vital component of safe and effective emergency department (ED) care. There are no standardized methods for identifying or reliably monitoring diagnostic errors in the ED, impeding efforts to enhance diagnostic safety. We sought to identify trigger concepts to screen ED records for diagnostic errors and describe how they can be used as a measurement strategy to identify and reduce preventable diagnostic harm.

METHODS

We conducted a literature review and surveyed ED directors to compile a list of potential electronic health record (EHR) trigger (e-triggers) and non-EHR based concepts. We convened a multidisciplinary expert panel to build consensus on trigger concepts to identify and reduce preventable diagnostic harm in the ED.

RESULTS

Six e-trigger and five non-EHR based concepts were selected by the expert panel. E-trigger concepts included: unscheduled ED return to ED resulting in hospital admission, death following ED visit, care escalation, high-risk conditions based on symptom-disease dyads, return visits with new diagnostic/therapeutic interventions, and change of treating service after admission. Non-EHR based signals included: cases from mortality/morbidity conferences, risk management/safety office referrals, ED medical director case referrals, patient complaints, and radiology/laboratory misreads and callbacks. The panel suggested further refinements to aid future research in defining diagnostic error epidemiology in ED settings.

CONCLUSIONS

We identified a set of e-trigger concepts and non-EHR based signals that could be developed further to screen ED visits for diagnostic safety events. With additional evaluation, trigger-based methods can be used as tools to monitor and improve ED diagnostic performance.

Controversies in diagnosis: contemporary debates in the diagnostic safety literature

Since the 2015 publication of the National Academy of Medicine's (NAM) Improving Diagnosis in Health Care (Improving Diagnosis in Health Care. In: Balogh EP, Miller BT, Ball JR, editors. Improving Diagnosis in Health Care. Washington (DC): National Academies Press, 2015.), literature in diagnostic safety has grown rapidly. This update was presented at the annual international meeting of the Society to Improve Diagnosis in Medicine (SIDM). We focused our literature search on articles published between 2016 and 2018 using keywords in Pubmed and the Agency for Healthcare Research and Quality (AHRQ)'s Patient Safety Network's running bibliography of diagnostic error literature (Diagnostic Errors Patient Safety Network: Agency for Healthcare Research and Quality; Available from: https://psnet.ahrq.gov/search?topic=Diagnostic-Errors&f_topicIDs=407). Three key topics emerged from our review of recent abstracts in diagnostic safety. First, definitions of diagnostic error and related concepts are evolving since the NAM's report. Second, medical educators are grappling with new approaches to teaching clinical reasoning and diagnosis. Finally, the potential of artificial intelligence (AI) to advance diagnostic excellence is coming to fruition. Here we present contemporary debates around these three topics in a pro/con format.

Handshake antimicrobial stewardship as a model to recognize and prevent diagnostic errors

OBJECTIVES

Few studies describe the impact of antimicrobial stewardship programs (ASPs) on recognizing and preventing diagnostic errors. Handshake stewardship (HS-ASP) is a novel ASP model that prospectively reviews hospital-wide antimicrobial usage with recommendations made in person to treatment teams. The purpose of this study was to determine if HS-ASP could identify and intervene on potential diagnostic errors for children hospitalized at a quaternary care children's hospital.

METHODS

Previously self-identified "Great Catch" (GC) interventions by the Children's Hospital Colorado HS-ASP team from 10/2014 through 5/2018 were retrospectively reviewed. Each GC was categorized based on the types of recommendations from HS-ASP, including if any diagnostic recommendations were made to the treatment team. Each GC was independently scored using the "Safer Dx Instrument" to determine presence of diagnostic error based on a previously determined cut-off score of ≤1.50. Interrater reliability for the instrument was measured using a randomized subset of one third of GCs.

RESULTS

During the study period, there were 162 GC interventions. Of these, 65 (40%) included diagnostic recommendations by HS-ASP and 19 (12%) had a Safer Dx Score of ≤1.50, (Κ=0.44; moderate agreement). Of those GCs associated with diagnostic errors, the HS-ASP team made a diagnostic recommendation to the primary treatment team 95% of the time.

CONCLUSIONS

Handshake stewardship has the potential to identify and intervene on diagnostic errors for hospitalized children.

Frequency, Risk Factors, Causes, and Consequences of Diagnostic Errors in Critically Ill Medical Patients: A Retrospective Cohort Study

OBJECTIVE

Diagnostic errors are a source of significant morbidity and mortality but understudied in the critically ill. We sought to characterize the frequency, causes, consequences, and risk factors of diagnostic errors among unplanned ICU admissions.

DESIGN

We conducted a retrospective cohort study of randomly selected nonsurgical ICU admissions between July 2015 and June 2016.

SETTING

Medical ICU at a tertiary academic medical center.

SUBJECTS

Critically ill adults with unplanned admission to the medical ICU.

MEASUREMENTS AND MAIN RESULTS

The primary investigator reviewed patient records using a modified version of the Safer Dx instrument, a validated instrument for detecting diagnostic error. Two intensivists performed secondary reviews of possible errors, and reviewers met periodically to adjudicate errors by consensus. For each confirmed error, we judged harm on a 1-6 rating scale. We also collected detailed demographic and clinical data for each patient. We analyzed 256 unplanned ICU admissions and identified 18 diagnostic errors (7% of admissions). All errors were associated with harm, and only six errors (33%) were recognized by the ICU team within the first 24 hours. More women than men experienced a diagnostic error (11.7% vs 2.7%; p = 0.015, χ test). On multivariable logistic regression analysis, female sex remained independently associated with risk of diagnostic error both at admission (odds ratio, 5.18; 95% CI, 1.34-20.08) and at 24 hours (odds ratio, 11.6; 95% CI, 1.37-98.6). Similarly, Quick Sequential Organ Failure Assessment score greater than or equal to 2 at admission was independently associated with diagnostic error (odds ratio, 5.73; 95% CI, 1.72-19.01).

CONCLUSIONS

Diagnostic errors may be an underappreciated source of ICU-related harm. Women and higher acuity patients appear to be at increased risk for such errors. Further research is merited to define the scope of error-associated harm and to clarify risk factors for diagnostic errors among the critically ill.

Factors Associated With Diagnostic Error on Admission to a PICU: A Pilot Study

OBJECTIVES

Diagnostic errors can harm critically ill children. However, we know little about their prevalence in PICUs and factors associated with error. The objective of this pilot study was to determine feasibility of record review to identify patient, provider, and work system factors associated with diagnostic errors during the first 12 hours after PICU admission.

DESIGN

Pilot retrospective cohort study with structured record review using a structured tool (Safer Dx instrument) to identify diagnostic error.

SETTING

Academic tertiary referral PICU.

PATIENTS

Patients 0-17 years old admitted nonelectively to the PICU.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

Four of 50 patients (8%) had diagnostic errors in the first 12 hours after admission. The Safer Dx instrument helped identify delayed diagnoses of chronic ear infection, increased intracranial pressure (two cases), and Bartonella encephalitis. We calculated that 610 PICU admissions are needed to achieve 80% power (α = 0.05) to detect significant associations with error.

CONCLUSIONS

Our pilot study found four patients with diagnostic error out of 50 children admitted nonelectively to a PICU. Retrospective record review using a structured tool to identify diagnostic errors is feasible in this population. Pilot data are being used to inform a larger and more definitive multicenter study.

Assessing diagnostic error in cerebral venous thrombosis via detailed chart review

Background Diagnostic error in cerebral venous thrombosis (CVT) has been understudied despite the harm associated with misdiagnosis of other cerebrovascular diseases as well as the known challenges of evaluating non-specific neurological symptoms in clinical practice. Methods We conducted a retrospective cohort study of CVT patients hospitalized at a single center. Two independent reviewers used a medical record review tool, the Safer Dx Instrument, to identify diagnostic errors. Demographic and clinical factors were abstracted. We compared subjects with and without a diagnostic error using the t-test for continuous variables and the chi-square (χ2) test or Fisher's exact test for categorical variables; an alpha of 0.05 was the cutoff for significance. Results A total of 72 CVT patients initially met study inclusion criteria; 19 were excluded due to incomplete medical records. Of the 53 patients included in the final analysis, the mean age was 48 years and 32 (60.4%) were women. Diagnostic error occurred in 11 cases [20.8%; 95% confidence interval (CI) 11.8-33.6%]. Subjects with diagnostic errors were younger (42 vs. 49 years, p = 0.13), more often women (81.8% vs. 54.8%, p = 0.17), and were significantly more likely to have a past medical history of a headache disorder prior to the index CVT visit (7.1% vs. 36.4%, p = 0.03). Conclusions Nearly one in five patients with complete medical records experienced a diagnostic error. Prior history of headache was the only evaluated clinical factor that was more common among those with an error in diagnosis. Future work on distinguishing primary from secondary headaches to improve diagnostic accuracy in acute neurological disease is warranted.

Measures to Improve Diagnostic Safety in Clinical Practice

Timely and accurate diagnosis is foundational to good clinical practice and an essential first step to achieving optimal patient outcomes. However, a recent Institute of Medicine report concluded that most of us will experience at least one diagnostic error in our lifetime. The report argues for efforts to improve the reliability of the diagnostic process through better measurement of diagnostic performance. The diagnostic process is a dynamic team-based activity that involves uncertainty, plays out over time, and requires effective communication and collaboration among multiple clinicians, diagnostic services, and the patient. Thus, it poses special challenges for measurement. In this paper, we discuss how the need to develop measures to improve diagnostic performance could move forward at a time when the scientific foundation needed to inform measurement is still evolving. We highlight challenges and opportunities for developing potential measures of "diagnostic safety" related to clinical diagnostic errors and associated preventable diagnostic harm. In doing so, we propose a starter set of measurement concepts for initial consideration that seem reasonably related to diagnostic safety and call for these to be studied and further refined. This would enable safe diagnosis to become an organizational priority and facilitate quality improvement. Health-care systems should consider measurement and evaluation of diagnostic performance as essential to timely and accurate diagnosis and to the reduction of preventable diagnostic harm.

Improving Team-Based Decision Making Using Data Analytics and Informatics: Protocol for a Collaborative Decision Support Design

Background

According to the September 2015 Institute of Medicine report, Improving Diagnosis in Health Care, each of us is likely to experience one diagnostic error in our lifetime, often with devastating consequences. Traditionally, diagnostic decision making has been the sole responsibility of an individual clinician. However, diagnosis involves an interaction among interprofessional team members with different training, skills, cultures, knowledge, and backgrounds. Moreover, diagnostic error is prevalent in the interruption-prone environment, such as the emergency department, where the loss of information may hinder a correct diagnosis.

Objective

The overall purpose of this protocol is to improve team-based diagnostic decision making by focusing on data analytics and informatics tools that improve collective information management.

Methods

To achieve this goal, we will identify the factors contributing to failures in team-based diagnostic decision making (aim 1), understand the barriers of using current health information technology tools for team collaboration (aim 2), and develop and evaluate a collaborative decision-making prototype that can improve team-based diagnostic decision making (aim 3).

Results

Between 2019 to 2020, we are collecting data for this study. The results are anticipated to be published between 2020 and 2021.

Conclusions

The results from this study can shed light on improving diagnostic decision making by incorporating diagnostics rationale from team members. We believe a positive direction to move forward in solving diagnostic errors is by incorporating all team members, and using informatics.

International Registered Report Identifier (IRRID)

DERR1-10.2196/16047

Cognitive bias in clinical medicine

Cognitive bias is increasingly recognised as an important source of medical error, and is both ubiquitous across clinical practice yet incompletely understood. This increasing awareness of bias has resulted in a surge in clinical and psychological research in the area and development of various 'debiasing strategies'. This paper describes the potential origins of bias based on 'dual process thinking', discusses and illustrates a number of the important biases that occur in clinical practice, and considers potential strategies that might be used to mitigate their effect.

Strategies to reduce diagnostic errors: a systematic review

Background

To evaluate the effectiveness of audit and communication strategies to reduce diagnostic errors made by clinicians.

Methods

MEDLINE complete, CINHAL complete, EMBASE, PSNet and Google Advanced. Electronic and manual search of articles on audit systems and communication strategies or interventions, searched for papers published between January 1990 and April 2017. We included studies with interventions implemented by clinicians in a clinical environment with real patients.

Results

A total of 2431 articles were screened of which 26 studies met inclusion criteria. Data extraction was conducted by two groups, each group comprising two independent reviewers. Articles were classified by communication (6) or audit strategies (20) to reduce diagnostic error in clinical settings. The most common interventions were delivered as technology-based systems n = 16 (62%) and within an acute care setting n = 15 (57%). Nine studies reported randomised controlled trials. Three RCT studies on communication interventions and 3 RCTs on audit strategies found the interventions to be effective in reducing diagnostic errors.

Conclusion

Despite numerous studies on interventions targeting diagnostic errors, our analyses revealed limited evidence on interventions being practically used in clinical settings and a bias of studies originating from the US (n = 19, 73% of included studies). There is some evidence that trigger algorithms, including computer based and alert systems, may reduce delayed diagnosis and improve diagnostic accuracy. In trauma settings, strategies such as additional patient review (e.g. trauma teams) reduced missed diagnosis and in radiology departments review strategies such as team meetings and error documentation may reduce diagnostic error rates over time.

Trial registration

The systematic review was registered in the PROSPERO database under registration number CRD42017067056.

Electronic supplementary material

The online version of this article (10.1186/s12911-019-0901-1) contains supplementary material, which is available to authorized users.

Serious misdiagnosis-related harms in malpractice claims: The “Big Three” – vascular events, infections, and cancers

Background

Diagnostic errors cause substantial preventable harm, but national estimates vary widely from 40,000 to 4 million annually. This cross-sectional analysis of a large medical malpractice claims database was the first phase of a three-phase project to estimate the US burden of serious misdiagnosis-related harms.

Methods

We sought to identify diseases accounting for the majority of serious misdiagnosis-related harms (morbidity/mortality). Diagnostic error cases were identified from Controlled Risk Insurance Company (CRICO)’s Comparative Benchmarking System (CBS) database (2006–2015), representing 28.7% of all US malpractice claims. Diseases were grouped according to the Agency for Healthcare Research and Quality (AHRQ) Clinical Classifications Software (CCS) that aggregates the International Classification of Diseases diagnostic codes into clinically sensible groupings. We analyzed vascular events, infections, and cancers (the “Big Three”), including frequency, severity, and settings. High-severity (serious) harms were defined by scores of 6–9 (serious, permanent disability, or death) on the National Association of Insurance Commissioners (NAIC) Severity of Injury Scale.

Results

From 55,377 closed claims, we analyzed 11,592 diagnostic error cases [median age 49, interquartile range (IQR) 36–60; 51.7% female]. These included 7379 with high-severity harms (53.0% death). The Big Three diseases accounted for 74.1% of high-severity cases (vascular events 22.8%, infections 13.5%, and cancers 37.8%). In aggregate, the top five from each category (n = 15 diseases) accounted for 47.1% of high-severity cases. The most frequent disease in each category, respectively, was stroke, sepsis, and lung cancer. Causes were disproportionately clinical judgment factors (85.7%) across categories (range 82.0–88.8%).

Conclusions

The Big Three diseases account for about three-fourths of serious misdiagnosis-related harms. Initial efforts to improve diagnosis should focus on vascular events, infections, and cancers.

Structured case reviews for organizational learning about diagnostic vulnerabilities: initial experiences from two medical centers

Background

An organization’s ability to identify and learn from opportunities for improvement (OFI) is key to increasing diagnostic safety. Many lack effective processes required to capitalize on these learning opportunities. We describe two parallel attempts at creating such a process and identifying generalizable lessons and learn from them.

Methods

Triggered case review programs were created independently at two organizations, Site 1 (Regions Hospital, HealthPartners, Saint Paul, MN, USA) and site 2 (University of California, San Diego). Both used a five-step process to create the review system and provide feedback: (1) identify trigger criteria; (2) establish a review panel; (3) develop a system to conduct reviews; (4) perform reviews; and (5) provide feedback.

Results

Site 1 identified 112 OFI in 184 case reviews (61%), with 66 (59%) provider OFI and 46 (41%) system OFI. Site 2 focused mainly on systems OFI identifying 105 OFI in 346 cases (30%). Opportunities at both sites were variable; common themes included test result management and communication across teams in peri-procedural care and with consultants. Of provider-initiated reviews, 67% of cases had an OFI at site 1 and 87% at site 2.

Conclusions

Lessons learned include the following: (1) peer review of cases provides opportunities to learn and calibrate diagnostic and management decisions at an organizational level; (2) sharing cases in review groups supports a culture of open discussion of OFIs; (3) reviews focused on diagnostic safety identify opportunities that may complement other organization-wide review opportunities.

Recommendations for using the Revised Safer Dx Instrument to help measure and improve diagnostic safety

The medical record continues to be one of the most useful and accessible sources of information to examine the diagnostic process. However, medical record review studies of diagnostic errors have often used subjective judgments and found low inter-rater agreement among reviewers when determining the presence or absence of diagnostic error. In our previous work, we developed a structured data-collection instrument, called the Safer Dx Instrument, consisting of objective criteria to improve the accuracy of assessing diagnostic errors in primary care. This paper proposes recommendations on how clinicians and health care organizations could use the Revised Safer Dx Instrument in identifying and understanding missed opportunities to make correct and timely diagnoses. The instrument revisions addressed both methodological and implementation issues identified during initial use and included refinements to the instrument to allow broader application across all health care settings. In addition to leveraging knowledge from piloting the instrument in several health care settings, we gained insights from multiple researchers who had used the instrument in studies involving emergency care, inpatient care and intensive care unit settings. This allowed us to enhance and extend the scope of this previously validated data collection instrument. In this paper, we describe the refinement process and provide recommendations for application and use of the Revised Safer Dx Instrument across a broad range of health care settings. The instrument can help users identify potential diagnostic errors in a standardized way for further analysis and safety improvement efforts as well as provide data for clinician feedback and reflection. With wider adoption and use by clinicians and health systems, the Revised Safer Dx Instrument could help propel the science of measuring and reducing diagnostic errors forward.

Medical Diagnostic Tests: A Review of Test Anatomy, Phases, and Statistical Treatment of Data

Diagnostic tests are approaches used in clinical practice to identify with high accuracy the disease of a particular patient and thus to provide early and proper treatment. Reporting high-quality results of diagnostic tests, for both basic and advanced methods, is solely the responsibility of the authors. Despite the existence of recommendation and standards regarding the content or format of statistical aspects, the quality of what and how the statistic is reported when a diagnostic test is assessed varied from excellent to very poor. This article briefly reviews the steps in the evaluation of a diagnostic test from the anatomy, to the role in clinical practice, and to the statistical methods used to show their performances. The statistical approaches are linked with the phase, clinical question, and objective and are accompanied by examples. More details are provided for phase I and II studies while the statistical treatment of phase III and IV is just briefly presented. Several free online resources useful in the calculation of some statistics are also given.

A simulation-based approach to training in heuristic clinical decision-making

Background

Cognitive biases may negatively impact clinical decision-making. The dynamic nature of a simulation environment can facilitate heuristic decision-making which can serve as a teaching opportunity.

Methods

Momentum bias, confirmation bias, playing-the-odds bias, and order-effect bias were integrated into four simulation scenarios. Clinical simulation educators and human factors specialists designed a script of events during scenarios to trigger heuristic decision-making. Debriefing included the exploration of frames (mental models) resulting in the observed actions, as well as a discussion of specific bias-prone frames and bias-resistant frames. Simulation sessions and debriefings were coded to measure the occurrence of bias, recovery from biased decision-making, and effectiveness of debriefings.

Results

Twenty medical residents and 18 medical students participated in the study. Twenty pairs (of one medical student and one resident) and two individuals (medical residents alone) completed a simulation session. Evidence of bias was observed in 11 of 20 (55%) sessions. While most participant pairs were able to avoid or recover from the anticipated bias, there were three sessions with no recovery. Evaluation of debriefings showed exploration of frames in all the participant pairs. Establishing new bias-resistant frames occurred more often when the learners experienced the bias.

Conclusions

Instructional design using experiential learning can focus learner attention on the specific elements of diagnostic decision-making. Using scenario design and debriefing enabled trainees to experience and analyze their own cognitive biases.

Teaching Critical Thinking: A Case for Instruction in Cognitive Biases to Reduce Diagnostic Errors and Improve Patient Safety

Diagnostic errors contribute to as many as 70% of medical errors. Prevention of diagnostic errors is more complex than building safety checks into health care systems; it requires an understanding of critical thinking, of clinical reasoning, and of the cognitive processes through which diagnoses are made. When a diagnostic error is recognized, it is imperative to identify where and how the mistake in clinical reasoning occurred. Cognitive biases may contribute to errors in clinical reasoning. By understanding how physicians make clinical decisions, and examining how errors due to cognitive biases occur, cognitive bias awareness training and debiasing strategies may be developed to decrease diagnostic errors and patient harm. Studies of the impact of teaching critical thinking skills have mixed results but are limited by methodological problems.This Perspective explores the role of clinical reasoning and cognitive bias in diagnostic error, as well as the effect of instruction in metacognitive skills on improvement of diagnostic accuracy for both learners and practitioners. Recent literature questioning whether teaching critical thinking skills increases diagnostic accuracy is critically examined, as are studies suggesting that metacognitive practices result in better patient care and outcomes. Instruction in metacognition, reflective practice, and cognitive bias awareness may help learners move toward adaptive expertise and help clinicians improve diagnostic accuracy. The authors argue that explicit instruction in metacognition in medical education, including awareness of cognitive biases, has the potential to reduce diagnostic errors and thus improve patient safety.

A cognitive forcing tool to mitigate cognitive bias - a randomised control trial

BACKGROUND

Cognitive bias is an important source of diagnostic error yet is a challenging area to understand and teach. Our aim was to determine whether a cognitive forcing tool can reduce the rates of error in clinical decision making. A secondary objective was to understand the process by which this effect might occur.

METHODS

We hypothesised that using a cognitive forcing tool would reduce diagnostic error rates. To test this hypothesis, a novel online case-based approach was used to conduct a single blinded randomized clinical trial conducted from January 2017 to September 2018. In addition, a qualitative series of "think aloud" interviews were conducted with 20 doctors from a UK teaching hospital in 2018. The primary outcome was the diagnostic error rate when solving bias inducing clinical vignettes. A volunteer sample of medical professionals from across the UK, Republic of Ireland and North America. They ranged in seniority from medical student to Attending Physician.

RESULTS

Seventy six participants were included in the study. The data showed doctors of all grades routinely made errors related to cognitive bias. There was no difference in error rates between groups (mean 2.8 cases correct in intervention vs 3.1 in control group, 95% CI -0.94 - 0.45 P = 0.49). The qualitative protocol revealed that the cognitive forcing strategy was well received and a produced a subjectively positive impact on doctors' accuracy and thoughtfulness in clinical cases.

CONCLUSIONS

The quantitative data failed to show an improvement in accuracy despite a positive qualitative experience. There is insufficient evidence to recommend this tool in clinical practice, however the qualitative data suggests such an approach has some merit and face validity to users.

Diagnostic Decision-Making in the Emergency Department

Emergency medicine requires diagnosing unfamiliar patients with undifferentiated acute presentations. This requires hypothesis generation and questioning, examination, and testing. Balancing patient load, care across the severity spectrum, and frequent interruptions create time pressures that predispose humans to fast thinking or cognitive shortcuts, including cognitive biases. Diagnostic error is the failure to establish an accurate and timely explanation of the problem or communicate that to the patient, often contributing to physical, emotional, or financial harm. Methods for monitoring diagnostic error in the emergency department are needed to establish frequency and serve as a foundation for future interventions.

The Definition and Scope of Diagnostic Error in the US and How Diagnostic Error is Enabled

Background

The quality of healthcare in the US has been progressively addressed by 3 reports from the National Academy of Medicine, the latest of which, entitled “Improving Diagnosis in Health Care,” was issued in 2015 from a 21-member panel (the author of this report was a member). The report is a review of the longstanding problem of diagnostic error. The infrastructure of healthcare delivery in the US has inadvertently made diagnostic error a major contributor to the high cost of care and preventable poor patient outcomes.

Content

This review describes the failures in US healthcare delivery that have led to the overwhelming number of deaths attributable to diagnostic error. Each failure is associated with recommendations to eliminate it. The review begins with a description of the scope of the diagnostic error problem and then discusses each of the issues that need to be addressed to reduce the number of misdiagnoses.

Summary

The problem of diagnostic error in the US is a large one. Some the contributing factors to this large problem can be resolved at a small expense and with modest change; others require a major overhaul of aspects of medical practice. For the first time, Americans have a “to-do list” to reduce our diagnostic error problem and be on par with other developed countries that are recognized as providing less costly care with better patient outcomes.

Understanding diagnostic safety in emergency medicine: A case-by-case review of closed ED malpractice claims

The report Improving Diagnosis in Health Care calls for collaboration between professional liability insurance carriers and health care providers to identify opportunities to improve diagnostic performance. We used this collaborative approach and involved risk management/patient safety professionals and emergency medicine physician reviewers to analyze diagnosis-related emergency medicine closed claims from a large malpractice insurer. Our aim was to identify opportunities for risk reduction and to develop an approach for improving at-risk processes. Analysis of these cases revealed several missed opportunities in the diagnostic process. A collaborative approach offered greater insight into diagnosis process failures that may not have been evident if cases were reviewed in silos. Focused review findings led to a multidisciplinary improvement collaborative to develop clinical guidelines for improving at-risk practices and informed a simulation-based training initiative.

Diagnostic Errors in Primary Care Pediatrics: Project RedDE

OBJECTIVE

Diagnostic errors (DEs), which encompass failures of accuracy, timeliness, or patient communication, cause appreciable morbidity but are understudied in pediatrics. Pediatricians have expressed interest in reducing high-frequency/subacute DEs, but their epidemiology remains unknown. The objective of this study was to investigate the frequency of two high-frequency/subacute DEs and one missed opportunity for diagnosis (MOD) in primary care pediatrics.

METHODS

As part of a national quality improvement collaborative, 25 primary care pediatric practices were randomized to collect 5 months of retrospective data on one DE or MOD: elevated blood pressure (BP) and abnormal laboratory values (DEs), or adolescent depression evaluation (MOD). Relationships between DE or MOD proportions and patient age, gender, and insurance status were explored with mixed-effects logistic regression models.

RESULTS

DE or MOD rates in pediatric primary care were found to be 54% for patients with elevated BP (n = 389), 11% for patients with abnormal laboratory values (n = 381), and 62% for adolescents with an opportunity to evaluate for depression (n = 400). When examining the number of times a pediatrician may have recognized an abnormal condition but either knowingly or unknowingly did not act according to recommended guidelines, providers did not document recognition of an elevated BP in 51% of patients with elevated BP, and they did not document recognition of an abnormal laboratory value without a delay in 9% of patients with abnormal laboratory values.

CONCLUSIONS

DEs and MODs occur at an appreciable frequency in pediatric primary care. These errors may contribute to care delays and patient harm.

Simulation and the diagnostic process: a pilot study of trauma and rapid response teams

Background:

Simulation is frequently used to recreate many of the crises encountered in patient care settings. Teams learn to manage these crises in an environment that maximizes their learning experiences and eliminates the potential for patient harm. By designing simulation scenarios that include conditions associated with diagnostic errors, teams can experience how their decisions can lead to errors. The purpose of this study was to assess how trauma teams (TrT) and pediatric rapid response teams (RRT) managed scenarios that included a diagnostic error.

Methods:

We developed four scenarios that would require TrT and pediatric RRT to manage an error in diagnosis. The two trauma scenarios (spinal cord injury and tracheobronchial tear) were designed to not respond to the heuristic management approach frequently used in trauma settings. The two pediatric scenarios (foreign body aspiration and coarctation of the aorta) had an incorrect diagnosis on admission. Two raters independently scored the scenarios using a rating system based on how teams managed the diagnostic process (search, establish and confirm a new diagnosis and initiate therapy based on the new diagnosis).

Results:

Twenty-one TrT and 17 pediatric rapid response managed 51 scenarios. All of the teams questioned the initial diagnosis. The teams were able to establish and confirm a new diagnosis in 49% of the scenarios (25 of 51). Only 23 (45%) teams changed their management of the patient based on the new diagnosis.

Conclusions:

Simulation can be used to recreate conditions that engage teams in the diagnostic process. In contrast to most instruction about diagnostic error, teams learn through realistic experiences and receive timely feedback about their decision-making skills. Based on the findings in this pilot study, the majority of teams would benefit from an education intervention designed to improve their diagnostic skills.

Using Fault Trees to Advance Understanding of Diagnostic Errors

PROBLEM DEFINITION

Diagnostic errors annually affect at least 5% of adults in the outpatient setting in the United States. Formal analytic techniques are only infrequently used to understand them, in part because of the complexity of diagnostic processes and clinical work flows involved. In this article, diagnostic errors were modeled using fault tree analysis (FTA), a form of root cause analysis that has been successfully used in other high-complexity, high-risk contexts. How factors contributing to diagnostic errors can be systematically modeled by FTA to inform error understanding and error prevention is demonstrated.

INITIAL APPROACH

A team of three experts reviewed 10 published cases of diagnostic error and constructed fault trees. The fault trees were modeled according to currently available conceptual frameworks characterizing diagnostic error. The 10 trees were then synthesized into a single fault tree to identify common contributing factors and pathways leading to diagnostic error.

KEY INSIGHTS

FTA is a visual, structured, deductive approach that depicts the temporal sequence of events and their interactions in a formal logical hierarchy. The visual FTA enables easier understanding of causative processes and cognitive and system factors, as well as rapid identification of common pathways and interactions in a unified fashion. In addition, it enables calculation of empirical estimates for causative pathways. Thus, fault trees might provide a useful framework for both quantitative and qualitative analysis of diagnostic errors.

NEXT STEPS

Future directions include establishing validity and reliability by modeling a wider range of error cases, conducting quantitative evaluations, and undertaking deeper exploration of other FTA capabilities.

Extent of diagnostic agreement among medical referrals

RATIONALE, AIMS AND OBJECTIVES

Diagnostic uncertainty is often encountered in a medical practice. Patients with ambiguous, uncertain, and undiagnosed problems are frequently referred for second opinions. Comparing referral diagnoses to final diagnoses provides an opportunity to determine how frequently final diagnoses vary and changes the direction of medical care.

METHODS

A retrospective study was done at a single academic medical center using a sample of 286 patients referred by physician assistants, nurse practitioners, and physicians from primary care practices from January 1, 2009 to December 31, 2010. Patients' referral and final diagnoses were compared and classified into 1 of 3 categories: referral diagnosis and final diagnosis the same, referral diagnosis better defined/refined, and referral diagnosis distinctly different from final diagnosis. Episode costs for the respective categories were calculated for the referral visit and services that occurred at our facility within the first 30 days.

RESULTS

In 12% (36/286) of cases, referral diagnoses were the same as final diagnoses. Final diagnoses were better defined/refined in 66% (188/286) of cases; but in 21% of cases (62/286), final diagnoses were distinctly different than referral diagnoses. Total costs for cases in category 3 (different final diagnoses) were significantly higher than costs for cases in category 1 (P = .0001) and category 2 (P = <.0001).

CONCLUSION

Referrals to advanced specialty care for undifferentiated problems are an essential component of patient care. Without adequate resources to handle undifferentiated diagnoses, a potential unintended consequence is misdiagnoses resulting in treatment delays and complications leading to more costly treatments.

The global burden of diagnostic errors in primary care

Diagnosis is one of the most important tasks performed by primary care physicians. The World Health Organization (WHO) recently prioritized patient safety areas in primary care, and included diagnostic errors as a high-priority problem. In addition, a recent report from the Institute of Medicine in the USA, 'Improving Diagnosis in Health Care', concluded that most people will likely experience a diagnostic error in their lifetime. In this narrative review, we discuss the global significance, burden and contributory factors related to diagnostic errors in primary care. We synthesize available literature to discuss the types of presenting symptoms and conditions most commonly affected. We then summarize interventions based on available data and suggest next steps to reduce the global burden of diagnostic errors. Research suggests that we are unlikely to find a 'magic bullet' and confirms the need for a multifaceted approach to understand and address the many systems and cognitive issues involved in diagnostic error. Because errors involve many common conditions and are prevalent across all countries, the WHO's leadership at a global level will be instrumental to address the problem. Based on our review, we recommend that the WHO consider bringing together primary care leaders, practicing frontline clinicians, safety experts, policymakers, the health IT community, medical education and accreditation organizations, researchers from multiple disciplines, patient advocates, and funding bodies among others, to address the many common challenges and opportunities to reduce diagnostic error. This could lead to prioritization of practice changes needed to improve primary care as well as setting research priorities for intervention development to reduce diagnostic error.

Finding Diagnostic Errors in Children Admitted to the PICU

OBJECTIVES

To determine whether the Safer Dx Instrument, a structured tool for finding diagnostic errors in primary care, can be used to reliably detect diagnostic errors in patients admitted to a PICU.

DESIGN AND SETTING

The Safer Dx Instrument consists of 11 questions to evaluate the diagnostic process and a final question to determine if diagnostic error occurred. We used the instrument to analyze four "high-risk" patient cohorts admitted to the PICU between June 2013 and December 2013.

PATIENTS

High-risk cohorts were defined as cohort 1: patients who were autopsied; cohort 2: patients seen as outpatients within 2 weeks prior to PICU admission; cohort 3: patients transferred to PICU unexpectedly from an acute care floor after a rapid response and requiring vasoactive medications and/or endotracheal intubation due to decompensation within 24 hours; and cohort 4: patients transferred to PICU unexpectedly from an acute care floor after a rapid response without subsequent decompensation in 24 hours.

INTERVENTIONS

Two clinicians used the instrument to independently review records in each cohort for diagnostic errors, defined as missed opportunities to make a correct or timely diagnosis. Errors were confirmed by senior expert clinicians.

MEASUREMENTS AND MAIN RESULTS

Diagnostic errors were present in 26 of 214 high-risk patient records (12.1%; 95% CI, 8.2-17.5%) with the following frequency distribution: cohort 1: two of 16 (12.5%); cohort 2: one of 41 (2.4%); cohort 3: 13 of 44 (29.5%); and cohort 4: 10 of 113 (8.8%). Overall initial reviewer agreement was 93.6% (κ, 0.72). Infections and neurologic conditions were the most commonly missed diagnoses across all high-risk cohorts (16/26).

CONCLUSIONS

The Safer Dx Instrument has high reliability and validity for diagnostic error detection when used in high-risk pediatric care settings. With further validation in additional clinical settings, it could be useful to enhance learning and feedback about diagnostic safety in children.

Accuracy of the Safer Dx Instrument to Identify Diagnostic Errors in Primary Care

IMPORTANCE

Diagnostic errors are common and harmful, but difficult to define and measure. Measurement of diagnostic errors often depends on retrospective medical record reviews, frequently resulting in reviewer disagreement.

OBJECTIVES

We aimed to test the accuracy of an instrument to help detect presence or absence of diagnostic error through record reviews.

DESIGN

We gathered questions from several previously used instruments for diagnostic error measurement, then developed and refined our instrument. We tested the accuracy of the instrument against a sample of patient records (n = 389), with and without previously identified diagnostic errors (n = 129 and n = 260, respectively).

RESULTS

The final version of our instrument (titled Safer Dx Instrument) consisted of 11 questions assessing diagnostic processes in the patient-provider encounter and a main outcome question to determine diagnostic error. In comparison with the previous sample, the instrument yielded an overall accuracy of 84 %, sensitivity of 71 %, specificity of 90 %, negative predictive value of 86 %, and positive predictive value of 78 %. All 11 items correlated significantly with the instrument's error outcome question (all p values ≤ 0.01). Using factor analysis, the 11 questions clustered into two domains with high internal consistency (initial diagnostic assessment, and performance and interpretation of diagnostic tests) and a patient factor domain with low internal consistency (Cronbach's alpha coefficients 0.93, 0.92, and 0.38, respectively).

CONCLUSIONS

The Safer Dx Instrument helps quantify the likelihood of diagnostic error in primary care visits, achieving a high degree of accuracy for measuring their presence or absence. This instrument could be useful to identify high-risk cases for further study and quality improvement.

Understanding diagnostic error: looking beyond diagnostic accuracy

Whether a diagnosis is correct or incorrect is often used to determine diagnostic performance despite there being no valid measure of diagnostic accuracy. In this paper we draw on our experience of conducting research on diagnostic error and discuss some of the challenges that a focus on accuracy brings to this field of research. In particular, we discuss whether diagnostic accuracy can be captured and what diagnostic accuracy does and does not tell us about diagnostic judgement. We draw on these points to argue that a focus on diagnostic accuracy may limit progress in this field and suggest that research which tries to understand more about the factors that influence decision making during the diagnostic process may be more useful in helping to improve diagnostic performance.