A unified conceptual model for diagnostic errors: underdiagnosis, overdiagnosis, and misdiagnosis

A Laboratory Medicine Best Practices Systematic Review and Meta-analysis of Nucleic Acid Amplification Tests (NAATs) and Algorithms Including NAATs for the Diagnosis of Clostridioides (Clostridium) difficile in Adults

The evidence base for the optimal laboratory diagnosis of Clostridioides (Clostridium) difficile in adults is currently unresolved due to the uncertain performance characteristics and various combinations of tests. This systematic review evaluates the diagnostic accuracy of laboratory testing algorithms that include nucleic acid amplification tests (NAATs) to detect the presence of C. difficile The systematic review and meta-analysis included eligible studies (those that had PICO [population, intervention, comparison, outcome] elements) that assessed the diagnostic accuracy of NAAT alone or following glutamate dehydrogenase (GDH) enzyme immunoassays (EIAs) or GDH EIAs plus C. difficile toxin EIAs (toxin). The diagnostic yield of NAAT for repeat testing after an initial negative result was also assessed. Two hundred thirty-eight studies met inclusion criteria. Seventy-two of these studies had sufficient data for meta-analysis. The strength of evidence ranged from high to insufficient. The uses of NAAT only, GDH-positive EIA followed by NAAT, and GDH-positive/toxin-negative EIA followed by NAAT are all recommended as American Society for Microbiology (ASM) best practices for the detection of the C. difficile toxin gene or organism. Meta-analysis of published evidence supports the use of testing algorithms that use NAAT alone or in combination with GDH or GDH plus toxin EIA to detect the presence of C. difficile in adults. There is insufficient evidence to recommend against repeat testing of the sample using NAAT after an initial negative result due to a lack of evidence of harm (i.e., financial, length of stay, or delay of treatment) as specified by the Laboratory Medicine Best Practices (LMBP) systematic review method in making such an assessment. Findings from this systematic review provide clarity to diagnostic testing strategies and highlight gaps, such as low numbers of GDH/toxin/PCR studies, in existing evidence on diagnostic performance, which can be used to guide future clinical research studies.

Diagnostic error increases mortality and length of hospital stay in patients presenting through the emergency room

BACKGROUND

Diagnostic errors occur frequently, especially in the emergency room. Estimates about the consequences of diagnostic error vary widely and little is known about the factors predicting error. Our objectives thus was to determine the rate of discrepancy between diagnoses at hospital admission and discharge in patients presenting through the emergency room, the discrepancies' consequences, and factors predicting them.

METHODS

Prospective observational clinical study combined with a survey in a University-affiliated tertiary care hospital. Patients' hospital discharge diagnosis was compared with the diagnosis at hospital admittance through the emergency room and classified as similar or discrepant according to a predefined scheme by two independent expert raters. Generalized linear mixed-effects models were used to estimate the effect of diagnostic discrepancy on mortality and length of hospital stay and to determine whether characteristics of patients, diagnosing physicians, and context predicted diagnostic discrepancy.

RESULTS

755 consecutive patients (322 [42.7%] female; mean age 65.14 years) were included. The discharge diagnosis differed substantially from the admittance diagnosis in 12.3% of cases. Diagnostic discrepancy was associated with a longer hospital stay (mean 10.29 vs. 6.90 days; Cohen's d 0.47; 95% confidence interval 0.26 to 0.70; P = 0.002) and increased patient mortality (8 (8.60%) vs. 25(3.78%); OR 2.40; 95% CI 1.05 to 5.5 P = 0.038). A factor available at admittance that predicted diagnostic discrepancy was the diagnosing physician's assessment that the patient presented atypically for the diagnosis assigned (OR 3.04; 95% CI 1.33-6.96; P = 0.009).

CONCLUSIONS

Diagnostic discrepancies are a relevant healthcare problem in patients admitted through the emergency room because they occur in every ninth patient and are associated with increased in-hospital mortality. Discrepancies are not readily predictable by fixed patient or physician characteristics; attention should focus on context.

TRIAL REGISTRATION

https://bmjopen.bmj.com/content/6/5/e011585.

Determining qualitative effect size ratings using a likelihood ratio scatter matrix in diagnostic test accuracy systematic reviews

Background Diagnostic test accuracy (DTA) systematic reviews (SRs) characterize a test's potential for diagnostic quality and safety. However, interpreting DTA measures in the context of SRs is challenging. Further, some evidence grading methods (e.g. Centers for Disease Control and Prevention, Division of Laboratory Systems Laboratory Medicine Best Practices method) require determination of qualitative effect size ratings as a contributor to practice recommendations. This paper describes a recently developed effect size rating approach for assessing a DTA evidence base. Methods A likelihood ratio scatter matrix will plot positive and negative likelihood ratio pairings for DTA studies. Pairings are graphed as single point estimates with confidence intervals, positioned in one of four quadrants derived from established thresholds for test clinical validity. These quadrants support defensible judgments on "substantial", "moderate", or "minimal" effect size ratings for each plotted study. The approach is flexible in relation to a priori determinations of the relative clinical importance of false positive and false negative test results. Results and conclusions This qualitative effect size rating approach was operationalized in a recent SR that assessed effectiveness of test practices for the diagnosis of Clostridium difficile. Relevance of this approach to other methods of grading evidence, and efforts to measure diagnostic quality and safety are described. Limitations of the approach arise from understanding that a diagnostic test is not an isolated element in the diagnostic process, but provides information in clinical context towards diagnostic quality and safety.

A Bayesian approach for analysis of ordered categorical responses subject to misclassification

Ordinal categorical responses are frequently collected in survey studies, human medicine, and animal and plant improvement programs, just to mention a few. Errors in this type of data are neither rare nor easy to detect. These errors tend to bias the inference, reduce the statistical power and ultimately the efficiency of the decision-making process. Contrarily to the binary situation where misclassification occurs between two response classes, noise in ordinal categorical data is more complex due to the increased number of categories, diversity and asymmetry of errors. Although several approaches have been presented for dealing with misclassification in binary data, only limited practical methods have been proposed to analyze noisy categorical responses. A latent variable model implemented within a Bayesian framework was proposed to analyze ordinal categorical data subject to misclassification using simulated and real datasets. The simulated scenario consisted of a discrete response with three categories and a symmetric error rate of 5% between any two classes. The real data consisted of calving ease records of beef cows. Using real and simulated data, ignoring misclassification resulted in substantial bias in the estimation of genetic parameters and reduction of the accuracy of predicted breeding values. Using our proposed approach, a significant reduction in bias and increase in accuracy ranging from 11% to 17% was observed. Furthermore, most of the misclassified observations (in the simulated data) were identified with a substantially higher probability. Similar results were observed for a scenario with asymmetric misclassification. While the extension to traits with more categories between adjacent classes is straightforward, it could be computationally costly. For traits with high heritability, the performance of the methodology would be expected to improve.

Diagnostic Decision-Making in the Emergency Department

Emergency medicine requires diagnosing unfamiliar patients with undifferentiated acute presentations. This requires hypothesis generation and questioning, examination, and testing. Balancing patient load, care across the severity spectrum, and frequent interruptions create time pressures that predispose humans to fast thinking or cognitive shortcuts, including cognitive biases. Diagnostic error is the failure to establish an accurate and timely explanation of the problem or communicate that to the patient, often contributing to physical, emotional, or financial harm. Methods for monitoring diagnostic error in the emergency department are needed to establish frequency and serve as a foundation for future interventions.

Explaining variations in test ordering in primary care: protocol for a realist review

INTRODUCTION

Studies have demonstrated the existence of significant variation in test-ordering patterns in both primary and secondary care, for a wide variety of tests and across many health systems. Inconsistent practice could be explained by differing degrees of underuse and overuse of tests for diagnosis or monitoring. Underuse of appropriate tests may result in delayed or missed diagnoses; overuse may be an early step that can trigger a cascade of unnecessary intervention, as well as being a source of harm in itself.

METHODS AND ANALYSIS

This realist review will seek to improve our understanding of how and why variation in laboratory test ordering comes about. A realist review is a theory-driven systematic review informed by a realist philosophy of science, seeking to produce useful theory that explains observed outcomes, in terms of relationships between important contexts and generative mechanisms.An initial explanatory theory will be developed in consultation with a stakeholder group and this 'programme theory' will be tested and refined against available secondary evidence, gathered via an iterative and purposive search process. This data will be analysed and synthesised according to realist principles, to produce a refined 'programme theory', explaining the contexts in which primary care doctors fail to order 'necessary' tests and/or order 'unnecessary' tests, and the mechanisms underlying these decisions.

ETHICS AND DISSEMINATION

Ethical approval is not required for this review. A complete and transparent report will be produced in line with the RAMESES standards. The theory developed will be used to inform recommendations for the development of interventions designed to minimise 'inappropriate' testing. Our dissemination strategy will be informed by our stakeholders. A variety of outputs will be tailored to ensure relevance to policy-makers, primary care and pathology practitioners, and patients.

PROSPERO REGISTRATION NUMBER

CRD42018091986.

Diagnostic Errors and the Bedside Clinical Examination

Diagnostic errors are common in clinical practice and lead to adverse patient outcomes. Systematic reviews have shown that inadequate history taking and physical examination lead to a plurality, if not a majority, of diagnostic errors. Recent advances in cognitive science have also shown that unconscious biases likely contribute to many diagnostic errors. Research into diagnostic error has been hampered by methodologic inconsistency and a paucity of studies in real-world clinical settings. The best evidence indicates that educational interventions to reduce diagnostic error should give physicians feedback about clinical outcomes and enhance their ability to recognize signs and symptoms of specific diseases at the bedside.

Diagnostic Errors in Primary Care Pediatrics: Project RedDE

OBJECTIVE

Diagnostic errors (DEs), which encompass failures of accuracy, timeliness, or patient communication, cause appreciable morbidity but are understudied in pediatrics. Pediatricians have expressed interest in reducing high-frequency/subacute DEs, but their epidemiology remains unknown. The objective of this study was to investigate the frequency of two high-frequency/subacute DEs and one missed opportunity for diagnosis (MOD) in primary care pediatrics.

METHODS

As part of a national quality improvement collaborative, 25 primary care pediatric practices were randomized to collect 5 months of retrospective data on one DE or MOD: elevated blood pressure (BP) and abnormal laboratory values (DEs), or adolescent depression evaluation (MOD). Relationships between DE or MOD proportions and patient age, gender, and insurance status were explored with mixed-effects logistic regression models.

RESULTS

DE or MOD rates in pediatric primary care were found to be 54% for patients with elevated BP (n = 389), 11% for patients with abnormal laboratory values (n = 381), and 62% for adolescents with an opportunity to evaluate for depression (n = 400). When examining the number of times a pediatrician may have recognized an abnormal condition but either knowingly or unknowingly did not act according to recommended guidelines, providers did not document recognition of an elevated BP in 51% of patients with elevated BP, and they did not document recognition of an abnormal laboratory value without a delay in 9% of patients with abnormal laboratory values.

CONCLUSIONS

DEs and MODs occur at an appreciable frequency in pediatric primary care. These errors may contribute to care delays and patient harm.

Paediatric genomics: diagnosing rare disease in children

The majority of rare diseases affect children, most of whom have an underlying genetic cause for their condition. However, making a molecular diagnosis with current technologies and knowledge is often still a challenge. Paediatric genomics is an immature but rapidly evolving field that tackles this issue by incorporating next-generation sequencing technologies, especially whole-exome sequencing and whole-genome sequencing, into research and clinical workflows. This complex multidisciplinary approach, coupled with the increasing availability of population genetic variation data, has already resulted in an increased discovery rate of causative genes and in improved diagnosis of rare paediatric disease. Importantly, for affected families, a better understanding of the genetic basis of rare disease translates to more accurate prognosis, management, surveillance and genetic advice; stimulates research into new therapies; and enables provision of better support.

Symptom-Disease Pair Analysis of Diagnostic Error (SPADE): a conceptual framework and methodological approach for unearthing misdiagnosis-related harms using big data

BACKGROUND

The public health burden associated with diagnostic errors is likely enormous, with some estimates suggesting millions of individuals are harmed each year in the USA, and presumably many more worldwide. According to the US National Academy of Medicine, improving diagnosis in healthcare is now considered 'a moral, professional, and public health imperative.' Unfortunately, well-established, valid and readily available operational measures of diagnostic performance and misdiagnosis-related harms are lacking, hampering progress. Existing methods often rely on judging errors through labour-intensive human reviews of medical records that are constrained by poor clinical documentation, low reliability and hindsight bias.

METHODS

Key gaps in operational measurement might be filled via thoughtful statistical analysis of existing large clinical, billing, administrative claims or similar data sets. In this manuscript, we describe a method to quantify and monitor diagnostic errors using an approach we call 'Symptom-Disease Pair Analysis of Diagnostic Error' (SPADE).

RESULTS

We first offer a conceptual framework for establishing valid symptom-disease pairs illustrated using the well-known diagnostic error dyad of dizziness-stroke. We then describe analytical methods for both look-back (case-control) and look-forward (cohort) measures of diagnostic error and misdiagnosis-related harms using 'big data'. After discussing the strengths and limitations of the SPADE approach by comparing it to other strategies for detecting diagnostic errors, we identify the sources of validity and reliability that undergird our approach.

CONCLUSION

SPADE-derived metrics could eventually be used for operational diagnostic performance dashboards and national benchmarking. This approach has the potential to transform diagnostic quality and safety across a broad range of clinical problems and settings.

Simulation and the diagnostic process: a pilot study of trauma and rapid response teams

Background:

Simulation is frequently used to recreate many of the crises encountered in patient care settings. Teams learn to manage these crises in an environment that maximizes their learning experiences and eliminates the potential for patient harm. By designing simulation scenarios that include conditions associated with diagnostic errors, teams can experience how their decisions can lead to errors. The purpose of this study was to assess how trauma teams (TrT) and pediatric rapid response teams (RRT) managed scenarios that included a diagnostic error.

Methods:

We developed four scenarios that would require TrT and pediatric RRT to manage an error in diagnosis. The two trauma scenarios (spinal cord injury and tracheobronchial tear) were designed to not respond to the heuristic management approach frequently used in trauma settings. The two pediatric scenarios (foreign body aspiration and coarctation of the aorta) had an incorrect diagnosis on admission. Two raters independently scored the scenarios using a rating system based on how teams managed the diagnostic process (search, establish and confirm a new diagnosis and initiate therapy based on the new diagnosis).

Results:

Twenty-one TrT and 17 pediatric rapid response managed 51 scenarios. All of the teams questioned the initial diagnosis. The teams were able to establish and confirm a new diagnosis in 49% of the scenarios (25 of 51). Only 23 (45%) teams changed their management of the patient based on the new diagnosis.

Conclusions:

Simulation can be used to recreate conditions that engage teams in the diagnostic process. In contrast to most instruction about diagnostic error, teams learn through realistic experiences and receive timely feedback about their decision-making skills. Based on the findings in this pilot study, the majority of teams would benefit from an education intervention designed to improve their diagnostic skills.

Challenges and opportunities from the Agency for Healthcare Research and Quality (AHRQ) research summit on improving diagnosis: a proceedings review

Background:

The

Content:

The goals of the summit were to learn from the insights of participants; examine issues associated with definitions of diagnostic error and gaps in the evidence base; explore clinician and patient perspectives; gain a better understanding of data and measurement, health information technology, and organizational factors that impact the diagnostic process; and identify potential future directions for research.

Summary and outlook:

Plenary sessions focused on the state of the new diagnostic safety discipline followed by breakout sessions on the use of data and measurement, health information technology, and the role of organizational factors. The proceedings review captures many of the key challenges and areas deserving further research, revealing stimulating yet complex issues.

Identifying and analyzing diagnostic paths: a new approach for studying diagnostic practices

Diagnostic error is a serious public health problem to which knowledge gaps and associated cognitive error contribute significantly. Identifying diagnostic approaches to common problems in ambulatory care associated with more timely and accurate diagnosis and lower cost and harm associated with diagnostic evaluation is an important priority for health care systems, clinicians, and of course patients. Unfortunately, guidance on how best to approach diagnosis in patients with common presenting complaints such as abdominal pain, dizziness, and fatigue is lacking. Exploring diagnostic practice variation and patterns of diagnostic evaluation is a potentially valuable approach to identifying best current diagnostic practices. A “diagnostic path” is the sequence of actions taken to evaluate a new complaint from first presentation until a diagnosis is established, or the evaluation ends for other reasons. A “big data” approach to identifying diagnostic paths from electronic health records can be used to identify practice variation and best practices from a large number of patients. Limitations of this approach include incompleteness and inaccuracy of electronic medical record data, the fact that diagnostic paths may not represent clinician thinking, and the fact that diagnostic paths may be used to identify best current practices, rather than optimal practices.

Misdiagnosis of cerebellar hemorrhage – features of ‘pseudo-gastroenteritis’ clinical presentations to the ED and primary care

Background:

Early-stage cerebellar hemorrhage can present with nausea or vomiting absent other neurological symptoms or signs, potentially leading to an incorrect diagnosis of gastroenteritis. We sought to determine the frequency of gastroenteritis-like presentations and delayed or missed diagnoses among patients with spontaneous cerebellar hemorrhage.

Methods:

This is a retrospective, case-control analysis of atraumatic, primary cerebellar hemorrhages derived from a systematic search of surgical pathology and autopsy databases at two large urban, academic medical centers from 1984 to 2006. Hospital visit and clinical symptom data were abstracted from electronic and paper medical records for included patients. Delayed or missed diagnoses were defined as those at least one previous visit for relevant clinical symptoms in the 7 days prior to the correct diagnosis being confirmed.

Results:

Among 254 records captured by our search filter, we identified 35 cases of pathologically proven primary cerebellar hemorrhage. Four patients (11%) were misdiagnosed initially – three with “gastroenteritis” and one with “hypertension”. In this small sample, misdiagnosed patients presented more often with normal mental state (100% vs. 35%, p=0.07) and nausea/vomiting (100% vs. 58%, p=0.22). Although patients deteriorated clinically after the initial misdiagnosis, and potentially dangerous diagnostic tests and treatment strategies were instituted as a result of misdiagnosis, none of the misdiagnosed patients died or suffered major permanent harms due to diagnostic delay.

Conclusions:

Our study is limited by the small number of identified cases. Nevertheless, it appears that patients with cerebellar hemorrhages can present with relatively unimpressive clinical findings without obvious neurological manifestations. Such individuals are sometimes misdiagnosed with gastroenteritis or other benign disorders initially, possibly when neurologic examination, particularly gait testing, is omitted or abridged. A careful search for subtle cerebellar signs, including dysarthria, limb ataxia, nystagmus or tandem gait instability, absent in true gastroenteritis cases, could potentially reduce misdiagnosis.

Defining excellence: next steps for practicing clinicians seeking to prevent diagnostic error

The Institute of Medicine (IOM) released its report on diagnostic errors in September, 2015. The report highlights the urgency of reducing errors and calls for system-level intervention and changes in our basic clinical interactions. Using the report's controversial definition of diagnostic error as a starting point, we introduce the issues and the potential impact on practicing physicians. We report a case used to illustrate this in an academic conference. Finally, we turn to the challenge of integrating these ideas into the traditional peer-review process. We argue that the medical community must evolve from understanding diagnostic failures to redesigning the diagnostic process. We should see errors as steps toward diagnostic excellence and reliable processes that minimize the risk of mislabeling and harm.

The IOM report on improving diagnosis: new concepts

The IOM report ‘Improving Diagnosis in Health Care’ represents a major advance in summarizing the problem of diagnostic error. Three new concepts in the report will be helpful in future efforts to understand and improve the diagnostic process: a new definition of diagnostic error, a new framework for understanding the diagnostic process, and a new concept of the diagnostic ‘team’. This paper highlights these new concepts and their relevance to improving diagnosis.

The challenges in defining and measuring diagnostic error

Diagnostic errors have emerged as a serious patient safety problem but they are hard to detect and complex to define. At the research summit of the 2013 Diagnostic Error in Medicine 6th International Conference, we convened a multidisciplinary expert panel to discuss challenges in defining and measuring diagnostic errors in real-world settings. In this paper, we synthesize these discussions and outline key research challenges in operationalizing the definition and measurement of diagnostic error. Some of these challenges include 1) difficulties in determining error when the disease or diagnosis is evolving over time and in different care settings, 2) accounting for a balance between underdiagnosis and overaggressive diagnostic pursuits, and 3) determining disease diagnosis likelihood and severity in hindsight. We also build on these discussions to describe how some of these challenges can be addressed while conducting research on measuring diagnostic error.

Missed diagnosis of stroke in the emergency department: a cross-sectional analysis of a large population-based sample

BACKGROUND

Some cerebrovascular events are not diagnosed promptly, potentially resulting in death or disability from missed treatments. We sought to estimate the frequency of missed stroke and examine associations with patient, emergency department (ED), and hospital characteristics.

METHODS

Cross-sectional analysis using linked inpatient discharge and ED visit records from the 2009 Healthcare Cost and Utilization Project State Inpatient Databases and 2008-2009 State ED Databases across nine US states. We identified adult patients admitted for stroke with a treat-and-release ED visit in the prior 30 days, considering those given a non-cerebrovascular diagnosis as probable (benign headache or dizziness diagnosis) or potential (any other diagnosis) missed strokes.

RESULTS

There were 23,809 potential and 2243 probable missed strokes representing 12.7% and 1.2% of stroke admissions, respectively. Missed hemorrhages (n = 406) were linked to headache while missed ischemic strokes (n = 1435) and transient ischemic attacks (n = 402) were linked to headache or dizziness. Odds of a probable misdiagnosis were lower among men (OR 0.75), older individuals (18-44 years [base]; 45-64:OR 0.43; 65-74:OR 0.28; ≥ 75:OR 0.19), and Medicare (OR 0.66) or Medicaid (OR 0.70) recipients compared to privately insured patients. Odds were higher among Blacks (OR 1.18), Asian/Pacific Islanders (OR 1.29), and Hispanics (OR 1.30). Odds were higher in non-teaching hospitals (OR 1.45) and low-volume hospitals (OR 1.57).

CONCLUSIONS

We estimate 15,000-165,000 misdiagnosed cerebrovascular events annually in US EDs, disproportionately presenting with headache or dizziness. Physicians evaluating these symptoms should be particularly attuned to the possibility of stroke in younger, female, and non-White patients.