Advancing the science of measurement of diagnostic errors in healthcare: the Safer Dx framework

DIAGNOSTIC ERROR IN THE UNITED STATES: A SUMMARY OF THE REPORT OF A NATIONAL ACADEMY OF MEDICINE COMMITTEE

This review article summarizes the conclusions of the National Academy of Medicine committee on diagnostic error. The committee deliberated during five in-person meetings and during numerous conference calls between April 2014 and April 2015. At three of the meetings, the committee invited multiple speakers to inform its deliberations. The 21 members of the committee represented a broad range of expertise related in some way to diagnostic errors, their potential causes, or their consequences. The members' specialized knowledge included patient safety, health care quality and measurement, patient engagement, health policy, health care professional education, cognitive psychology, health disparities, human factors and ergonomics, health information technology, decision analysis, nursing, radiology, anatomic pathology, laboratory medicine (clinical pathology), law, and health economics.

Incidence, origins and avoidable harm of missed opportunities in diagnosis: longitudinal patient record review in 21 English general practices

BACKGROUND

Diagnostic error is a global patient safety priority.

OBJECTIVES

To estimate the incidence, origins and avoidable harm of diagnostic errors in English general practice. Diagnostic errors were defined as missed opportunities to make a correct or timely diagnosis based on the evidence available (missed diagnostic opportunities, MDOs).

METHOD

Retrospective medical record reviews identified MDOs in 21 general practices. In each practice, two trained general practitioner reviewers independently conducted case note reviews on 100 randomly selected adult consultations performed during 2013-2014. Consultations where either reviewer identified an MDO were jointly reviewed.

RESULTS

Across 2057 unique consultations, reviewers agreed that an MDO was possible, likely or certain in 89 cases or 4.3% (95% CI 3.6% to 5.2%) of reviewed consultations. Inter-reviewer agreement was higher than most comparable studies (Fleiss' kappa=0.63). Sixty-four MDOs (72%) had two or more contributing process breakdowns. Breakdowns involved problems in the patient-practitioner encounter such as history taking, examination or ordering tests (main or secondary factor in 61 (68%) cases), performance and interpretation of diagnostic tests (31; 35%) and follow-up and tracking of diagnostic information (43; 48%). 37% of MDOs were rated as resulting in moderate to severe avoidable patient harm.

CONCLUSIONS

Although MDOs occurred in fewer than 5% of the investigated consultations, the high numbers of primary care contacts nationally suggest that several million patients are potentially at risk of avoidable harm from MDOs each year. Causes of MDOs were frequently multifactorial, suggesting the need for development and evaluation of multipronged interventions, along with policy changes to support them.

Diagnostic error in the pediatric hospital: a narrative review

INTRODUCTION

Diagnostic error is a prevalent type of medical error that is associated with considerable patient harm and increased medical costs. The majority of literature guiding the current understanding of diagnostic error in the hospital setting is from adult studies. However, there is research to suggest this type of error is also prevalent in the pediatric specialty.

OBJECTIVES

The primary objective of this study was to define the current understanding of diagnostic error in the pediatric hospital through a structured literature review.

METHODS

We searched PubMed and identified studies focusing on three aspects of diagnostic error in pediatric hospitals: the incidence or prevalence, contributing factors, and related interventions. We used a tiered review, and a standardized electronic form to extract data from included articles.

RESULTS

Fifty-nine abstracts were screened and 23 full-text studies were included in the final review. Seventeen of the 23 studies focused on the incidence or prevalence, with only 3 studies investigating the utility of interventions. Most studies took place in an intensive care unit or emergency department with very few studies including only patients on the general wards. Overall, the prevalence of diagnostic error in pediatric hospitals varied greatly and depended on the measurement technique and specific hospital setting. Both healthcare system factors and individual cognitive factors were found to contribute to diagnostic error, with there being limited evidence to guide how best to mitigate the influence of these factors on the diagnostic process.

CONCLUSION

The general knowledge of diagnostic error in pediatric hospital settings is limited. Future work should incorporate structured frameworks to measure diagnostic errors and examine clinicians' diagnostic processes in real-time to help guide effective hospital-wide interventions.

Diagnostic Error: Why Now?

Diagnostic errors remain relatively understudied and underappreciated. They are particularly concerning in the intensive care unit, where they are more likely to result in harm to patients. There is a lack of consensus on the definition of diagnostic error, and current methods to quantify diagnostic error have numerous limitations as noted in the sentinel report by the National Academy of Medicine. Although definitive definition and measurement remain elusive goals, increasing our understanding of diagnostic error is crucial if we are to make progress in reducing the incidence and harm caused by errors in diagnosis.

Diagnostic Error in the Critically Ill: A Hidden Epidemic?

Epidemiologic studies of diagnostic error in the intensive care unit (ICU) consist mostly of descriptive autopsy series. In these studies, rates of diagnostic errors are approximately 5% to 10%. Recently validated methods for retrospectively measuring error have expanded our understanding of the scope of the problem. These alternative measurement strategies have yielded similar estimates for the frequency of diagnostic error in the ICU. Although there is a fair understanding of the frequency of errors, further research is needed to better define the risk factors for diagnostic error in the ICU.

Errors in Diagnosing Infectious Diseases: A Physician Survey

Background: Infectious diseases are commonly missed or misdiagnosed. Errors in diagnosing infectious diseases not only affect the patient but also the community health. Objectives: To describe our investigation on the most common errors in diagnosing infectious diseases and their causes according to the physicians' reports. Methods: Between August 2018 and February 2019, specialist physicians and residents across Mashhad, Iran were invited to participate in a survey to report errors they had made or witnessed regarding the diagnosis of infectious diseases. Results: Overall, 465 cases were reported by 315 participants. The most common infectious diseases affected by diagnostic errors were upper respiratory tract infections (URTIs) (n = 69, 14.8%), tuberculosis (TB) (n = 66, 14.1%), pleuro-pulmonary infections (n = 54, 11.6%), central nervous system (CNS) infections (n = 51, 10.9%), and urinary tract infections (n = 45, 9.6%). Errors occurred most frequently in generating a diagnostic hypothesis (n = 259, 55/7%), followed by history taking (n = 200, 43%), and physical examination (n = 191, 41/1%). Errors related to the diagnosis of TB (odds ratio [OR]: 2.4, 95% confidence interval [CI]:0.9-5.7; P value: 0.047) and intra-abdominal infections (OR: 7.2, 95% CI: 0.9-53.8; P value: 0.02) were associated with more-serious outcomes. Conclusion: A substantial proportion of errors in diagnosing infectious diseases moderately or seriously affect patients' outcomes. URTIs, TB, and pleuropulmonary infections were the most frequently reported infectious diseases involved in diagnostic error while errors related to the diagnosis of TB and intraabdominal infections were more frequently associated with poor outcomes. Therefore, contagious and potentially life-threatening infectious diseases should always be considered in the differential diagnosis of patients who present with compatible clinical syndromes.

Factors affecting incorrect interpretation of abdominal computed tomography in non-traumatic patients by novice emergency physicians

Objective

Accurate interpretation of computed tomography (CT) scans is critical for patient care in the emergency department. We aimed to identify factors associated with an incorrect interpretation of abdominal CT by novice emergency residents and to analyze the characteristics of incorrectly interpreted scans.

Methods

This retrospective analysis of a prospective observational cohort was conducted at three urban emergency departments. Discrepancies between the interpretations by postgraduate year-1 (PGY-1) emergency residents and the final radiologists’ reports were assessed by independent adjudicators. Potential factors associated with incorrect interpretation included patient age, sex, time of interpretation, and organ category. Adjusted odds ratios (aORs) for incorrect interpretation were calculated using multivariable logistic regression analysis.

Results

Among 1,628 eligible cases, 270 (16.6%) were incorrect. The urinary system was the most correctly interpreted organ system (95.8%, 365/381), while the biliary tract was the most incorrectly interpreted (28.4%, 48/169). Normal CT images showed high false-positive rates of incorrect interpretation (28.2%, 96/340). Organ category was found to be a major determinant of incorrect interpretation. Using the urinary system as a reference, the aOR for incorrect interpretation of biliary tract disease was 9.20 (95% confidence interval, 5.0–16.90) and the aOR for incorrectly interpreting normal CT images was 8.47 (95% confidence interval, 4.85–14.78).

Conclusion

Biliary tract disease is a major factor associated with incorrect preliminary interpretations of abdominal CT scans by PGY-1 emergency residents. PGY-1 residents also showed high false-positive interpretation rates for normal CT images. Emergency residents’ training should focus on these two areas to improve abdominal CT interpretation accuracy.

Defining Antibiotic Inertia: Application of a Focused Clinical Scenario Survey to Illuminate A New Target for Antimicrobial Stewardship During Transitions of Care

In clinical scenario surveys, inpatient providers were more likely to continue inappropriate antibiotic therapy (OR 2.02; 95% CI 1.35-3.03, p<0.001) or broad therapy (OR 1.8; 95%CI 1.27-2.56, p=0.001) when initiated by ED providers, as compared to appropriate or narrow antibiotics, respectively. Antibiotic inertia could represent a significant antibiotic stewardship target.

Understanding diagnostic processes in emergency departments: a mixed methods case study protocol

INTRODUCTION

Diagnostic processes in the emergency department (ED) involve multiple interactions among individuals who interface with information systems to access and record information. A better understanding of diagnostic processes is needed to mitigate errors. This paper describes a study protocol to map diagnostic processes in the ED as a foundation for developing future error mitigation strategies.

METHODS AND ANALYSIS

This study of an adult and a paediatric academic ED uses a prospective mixed methods case study design informed by an ED-specific diagnostic decision-making model (the modified ED-National Academies of Sciences, Engineering and Medicine (NASEM) model) and two cognitive theories (dual process theory and distributed cognition). Data sources include audio recordings of patient and care team interactions, electronic health record data, observer field notes and stakeholder interviews. Multiple qualitative analysis methods will be used to explore diagnostic processes in situ, including systems information flow, human-human and human-system interactions and contextual factors influencing cognition. The study has three parts. Part 1 involves prospective field observations of patients with undifferentiated symptoms at high risk for diagnostic error, where each patient is followed throughout the entire care delivery process. Part 2 involves observing individual care team providers over a 4-hour window to capture their diagnostic workflow, team coordination and communication across multiple patients. Part 3 uses interviews with key stakeholders to understand different perspectives on the diagnostic process, as well as perceived strengths and vulnerabilities, in order to enrich the ED-NASEM diagnostic model.

ETHICS AND DISSEMINATION

The University of Michigan Institutional Review Board approved this study, HUM00156261. This foundational work will help identify strengths and vulnerabilities in diagnostic processes. Further, it will inform the future development and testing of patient, provider and systems-level interventions for mitigating error and improving patient safety in these and other EDs. The work will be disseminated through journal publications and presentations at national and international meetings.

Risk Assessment of the Door-In-Door-Out Process at Primary Stroke Centers for Patients With Acute Stroke Requiring Transfer to Comprehensive Stroke Centers

Background Patients with acute stroke at non- or primary stroke centers (PSCs) are transferred to comprehensive stroke centers for advanced treatments that reduce disability but experience significant delays in treatment and increased adjusted mortality. This study reports the results of a proactive, systematic, risk assessment of the door-in-door-out process and its application to solution design. Methods and Results A learning collaborative (clinicians, patients, and caregivers) at 2 PSCs and 3 comprehensive stroke centers in Chicago, Illinois participated in a failure modes, effects, and criticality analysis to identify steps in the process; failures of each step, underlying causes; and to characterize each failure's frequency, impact, and safeguards using standardized scores to calculate risk priority and criticality numbers for ranking. Targets for solution design were selected among the highest-ranked failures. The failure modes, effects, and criticality analysis process map and risk table were completed during in-person and virtual sessions. Failure to detect severe stroke/large-vessel occlusion on arrival at the PSC is the highest-ranked failure and can lead to a 45-minute door-in-door-out delay caused by failure to obtain a head computed tomography and computed tomography angiogram together. Lower risk failures include communication problems and delays within the PSC team and across the PSC comprehensive stroke center and paramedic teams. Seven solution prototypes were iteratively designed and address 4 of the 10 highest-ranked failures. Conclusions The failure modes, effects, and criticality analysis identified and characterized previously unrecognized failures of the door-in-door-out process. Use of a risk-informed approach for solution design is novel for stroke and should mitigate or eliminate the failures.

Factors Related to Serious Safety Events in a Children's Hospital Patient Safety Collaborative

BACKGROUND AND OBJECTIVES

Serious safety events (SSEs) occur infrequently at individual hospitals, making it difficult to establish trends to improve patient care. Patient safety organizations, such as the Child Health Patient Safety Organization (CHILDPSO), can identify trends and support learning across children's hospitals. We aim to describe longitudinal trends in SSE rates among CHILDPSO member hospitals and describe their sources of harm.

METHODS

SSEs from 44 children's hospitals were assigned severity and reported to CHILDPSO from January 1, 2015, to December 31, 2018. SSEs were classified into groups and subgroups based on analysis. Events were then tagged with up to 3 contributing factors. Subgroups with <5 events were excluded.

RESULTS

There were 22.5 million adjusted patient days included. The 12-month rolling average SSE rate per 10 000 adjusted patient days decreased from 0.71 to 0.41 (P < .001). There were 830 SSEs reported to CHILDPSO. The median hospital volume of SSEs was 12 events (interquartile range: 6-23), or ∼3 SSEs per year. Of the 830 events, 21.0% were high severity (SSE 1-3) and approximately two-thirds (67.0%, n = 610) were patient care management events, including subgroups of missed, delayed, or wrong diagnosis or treatment; medication errors; and suboptimal care coordination. The most common contributing factor was lack of situational awareness (17.9%, n = 382), which contributed to 1 in 5 (20%) high-severity SSEs.

CONCLUSIONS

Hospitals sharing SSE data through CHILDPSO have seen a decrease in SSEs. Patient care management was the most frequently seen. Future work should focus on investigation of contributing factors and risk mitigation strategies.

Bringing the clinical laboratory into the strategy to advance diagnostic excellence

OBJECTIVES

Clinical laboratory testing provides essential data for making medical diagnoses. Generating accurate and timely test results clearly communicated to the treating clinician, and ultimately the patient, is a critical component that supports diagnostic excellence. On the other hand, failure to achieve this can lead to diagnostic errors that manifest in missed, delayed and wrong diagnoses.

CONTENT

Innovations that support diagnostic excellence address: 1) test utilization, 2) leveraging clinical and laboratory data, 3) promoting the use of credible information resources, 4) enhancing communication among laboratory professionals, health care providers and the patient, and 5) advancing the use of diagnostic management teams. Integrating evidence-based laboratory and patient-care quality management approaches may provide a strategy to support diagnostic excellence. Professional societies, government agencies, and healthcare systems are actively engaged in efforts to advance diagnostic excellence. Leveraging clinical laboratory capabilities within a healthcare system can measurably improve the diagnostic process and reduce diagnostic errors.

SUMMARY

An expanded quality management approach that builds on existing processes and measures can promote diagnostic excellence and provide a pathway to transition innovative concepts to practice.

OUTLOOK

There are increasing opportunities for clinical laboratory professionals and organizations to be part of a strategy to improve diagnoses.

nalyzing diagnostic errors in the acute setting: a process-driven approach

OBJECTIVES

We describe an approach for analyzing failures in diagnostic processes in a small, enriched cohort of general medicine patients who expired during hospitalization and experienced medical error. Our objective was to delineate a systematic strategy for identifying frequent and significant failures in the diagnostic process to inform strategies for preventing adverse events due to diagnostic error.

METHODS

Two clinicians independently reviewed detailed records of purposively sampled cases identified from established institutional case review forums and assessed the likelihood of diagnostic error using the Safer Dx instrument. Each reviewer used the modified Diagnostic Error Evaluation and Research (DEER) taxonomy, revised for acute care (41 possible failure points across six process dimensions), to characterize the frequency of failure points (FPs) and significant FPs in the diagnostic process.

RESULTS

Of 166 cases with medical error, 16 were sampled: 13 (81.3%) had one or more diagnostic error(s), and a total of 113 FPs and 30 significant FPs were identified. A majority of significant FPs (63.3%) occurred in "Diagnostic Information and Patient Follow-up" and "Patient and Provider Encounter and Initial Assessment" process dimensions. Fourteen (87.5%) cases had a significant FP in at least one of these dimensions.

CONCLUSIONS

Failures in the diagnostic process occurred across multiple dimensions in our purposively sampled cohort. A systematic analytic approach incorporating the modified DEER taxonomy, revised for acute care, offered critical insights into key failures in the diagnostic process that could serve as potential targets for preventative interventions.

Improving diagnostic performance through feedback: the Diagnosis Learning Cycle

Background

Errors in reasoning are a common cause of diagnostic error. However, it is difficult to improve performance partly because providers receive little feedback on diagnostic performance. Examining means of providing consistent feedback and enabling continuous improvement may provide novel insights for diagnostic performance.

Methods

We developed a model for improving diagnostic performance through feedback using a six-step qualitative research process, including a review of existing models from within and outside of medicine, a survey, semistructured interviews with individuals working in and outside of medicine, the development of the new model, an interdisciplinary consensus meeting, and a refinement of the model.

Results

We applied theory and knowledge from other fields to help us conceptualise learning and comparison and translate that knowledge into an applied diagnostic context. This helped us develop a model, the Diagnosis Learning Cycle, which illustrates the need for clinicians to be given feedback about both their confidence and reasoning in a diagnosis and to be able to seamlessly compare diagnostic hypotheses and outcomes. This information would be stored in a repository to allow accessibility. Such a process would standardise diagnostic feedback and help providers learn from their practice and improve diagnostic performance. This model adds to existing models in diagnosis by including a detailed picture of diagnostic reasoning and the elements required to improve outcomes and calibration.

Conclusion

A consistent, standard programme of feedback that includes representations of clinicians’ confidence and reasoning is a common element in non-medical fields that could be applied to medicine. Adapting this approach to diagnosis in healthcare is a promising next step. This information must be stored reliably and accessed consistently. The next steps include testing the Diagnosis Learning Cycle in clinical settings.

Comparison of Diagnostic Recommendations from Individual Physicians versus the Collective Intelligence of Multiple Physicians in Ambulatory Cases Referred for Specialist Consultation

BACKGROUND

Studies report higher diagnostic accuracy using the collective intelligence (CI) of multiple clinicians compared with individual clinicians. However, the diagnostic process is iterative, and unexplored is the value of CI in improving clinical recommendations leading to a final diagnosis.

METHODS

To compare the appropriateness of diagnostic recommendations advised by individual physicians versus the CI of physicians, we entered actual consultation requests sent by primary care physicians to specialists onto a web-based CI platform capable of collecting diagnostic recommendations (next steps for care) from multiple physicians. We solicited responses to 35 cases (12 endocrinology, 13 gynecology, 10 neurology) from ≥3 physicians of any specialty through the CI platform, which aggregated responses into a CI output. The primary outcome was the appropriateness of individual physician recommendations versus the CI output recommendations, using recommendations agreed upon by 2 specialists in the same specialty as a gold standard. The secondary outcome was the recommendations' potential for harm.

RESULTS

A total of 177 physicians responded. Cases had a median of 7 respondents (interquartile range: 5-10). Diagnostic recommendations in the CI output achieved higher levels of appropriateness (69%) than recommendations from individual physicians (45%; χ² = 5.95, P = 0.015). Of the CI recommendations, 54% were potentially harmful, as compared with 41% of individuals' recommendations (χ² = 2.49, P = 0.11).

LIMITATIONS

Cases were from a single institution. CI was solicited using a single algorithm/platform.

CONCLUSIONS

When seeking specialist guidance, diagnostic recommendations from the CI of multiple physicians are more appropriate than recommendations from most individual physicians, measured against specialist recommendations. Although CI provides useful recommendations, some have potential for harm. Future research should explore how to use CI to improve diagnosis while limiting harm from inappropriate tests/therapies.

Clinical Features and Preventability of Delayed Diagnosis of Pediatric Appendicitis

IMPORTANCE

Delayed diagnosis of appendicitis is associated with worse outcomes than timely diagnosis, but clinical features associated with diagnostic delay are uncertain, and the extent to which delays are preventable is unclear.

OBJECTIVE

To determine clinical features associated with delayed diagnosis of pediatric appendicitis, assess the frequency of preventable delay, and compare delay outcomes.

DESIGN, SETTING, AND PARTICIPANTS

This case-control study included 748 children treated at 5 pediatric emergency departments in the US between January 1, 2010, and December 31, 2019. Participants were younger than 21 years and had a diagnosis of appendicitis.

EXPOSURES

Individual features of appendicitis and pretest likelihood of appendicitis were measured by the Pediatric Appendicitis Risk Calculator (pARC).

MAIN OUTCOMES AND MEASURES

Case patients had a delayed diagnosis of appendicitis, defined as 2 emergency department visits leading to diagnosis and a case review showing the patient likely had appendicitis at the first visit. Control patients had a single emergency department visit yielding a diagnosis. Clinical features and pARC scores were compared by case-control status. Preventability of delay was assessed as unlikely, possible, or likely. The proportion of children with indicated imaging based on an evidence-based cost-effectiveness threshold was determined. Outcomes of delayed diagnosis were compared by case-control status, including hospital length of stay, perforation, and multiple surgical procedures.

RESULTS

A total of 748 children (mean [SD] age, 10.2 [4.3] years; 392 boys [52.4%]; 427 White children [57.1%]) were included in the study; 471 (63.0%) had a delayed diagnosis of appendicitis, and 277 (37.0%) had no delay in diagnosis. Children with a delayed diagnosis were less likely to have pain with walking (adjusted odds ratio [aOR], 0.16; 95% CI, 0.10-0.25), maximal pain in the right lower quadrant (aOR, 0.12; 95% CI, 0.07-0.19), and abdominal guarding (aOR, 0.33; 95% CI, 0.21-0.51), and were more likely to have a complex chronic condition (aOR, 2.34; 95% CI, 1.05-5.23). The pretest likelihood of appendicitis was 39% to 52% lower in children with a delayed vs timely diagnosis. Among children with a delayed diagnosis, 109 cases (23.1%) were likely to be preventable, and 247 (52.4%) were possibly preventable. Indicated imaging was performed in 104 (22.0%) to 289 (61.3%) children with delayed diagnosis, depending on the imputation method for missing data on white blood cell count. Patients with delayed diagnosis had longer hospital length of stay (mean difference between the groups, 2.8 days; 95% CI, 2.3-3.4 days) and higher perforation rates (OR, 7.8; 95% CI, 5.5-11.3) and were more likely to undergo 2 or more surgical procedures (OR, 8.0; 95% CI, 2.0-70.4).

CONCLUSIONS AND RELEVANCE

In this case-control study, delayed appendicitis was associated with initially milder symptoms but worse outcomes. These findings suggest that a majority of delayed diagnoses were at least possibly preventable and that many of these patients did not undergo indicated imaging, suggesting an opportunity to prevent delayed diagnosis of appendicitis in some children.

Diagnostic Errors in Pediatric Critical Care: A Systematic Review

OBJECTIVES

To summarize the literature on prevalence, impact, and contributing factors related to diagnostic error in the PICU.

DATA SOURCES

Search of PubMed, EMBASE, and the Cochrane Library up to December 2019.

STUDY SELECTION

Studies on diagnostic error and the diagnostic process in pediatric critical care were included. Non-English studies with no translation, case reports/series, studies providing no information on diagnostic error, studies focused on non-PICU populations, and studies focused on a single condition/disease or a single diagnostic test/tool were excluded.

DATA EXTRACTION

Data on research design, objectives, study sample, and results pertaining to the prevalence, impact, and factors associated with diagnostic error were abstracted from each study.

DATA SYNTHESIS

Using independent tiered review, 396 abstracts were screened, and 17 studies (14 full-text, 3 abstracts) were ultimately included. Fifteen of 17 studies (88%) had an observational research design. Autopsy studies (autopsy rates were 20-47%) showed a 10-23% rate of missed major diagnoses; 5-16% of autopsy-discovered diagnostic errors had a potential adverse impact on survival and would have changed management. Retrospective record reviews reported varying rates of diagnostic error from 8% in a general PICU population to 12% among unexpected critical admissions and 21-25% of patients discussed at PICU morbidity and mortality conferences. Cardiovascular, infectious, congenital, and neurologic conditions were most commonly misdiagnosed. Systems factors (40-67%), cognitive factors (20-3%), and both systems and cognitive factors (40%) were associated with diagnostic error. Limited information was available on the impact of misdiagnosis.

CONCLUSIONS

Knowledge of diagnostic errors in the PICU is limited. Future work to understand diagnostic errors should involve a balanced focus between studying the diagnosis of individual diseases and uncovering common system- and process-related determinants of diagnostic error.

Safer Stroke-Dx Instrument: Identifying Stroke Misdiagnosis in the Emergency Department

BACKGROUND

Missed or delayed diagnosis of acute stroke, or false-negative stroke (FNS), at initial emergency department (ED) presentation occurs in ≈9% of confirmed stroke patients. Failure to rapidly diagnose stroke can preclude time-sensitive treatments, resulting in higher risks of severe sequelae and disability. In this study, we developed and tested a modified version of a structured medical record review tool, the Safer Dx Instrument, to identify FNS in a subgroup of hospitalized patients with stroke to gain insight into sources of ED stroke misdiagnosis.

METHODS

We conducted a retrospective cohort study at 2 unaffiliated comprehensive stroke centers. In the development and confirmatory cohorts, we applied the Safer Stroke-Dx Instrument to report the prevalence and documented sources of ED diagnostic error in FNS cases among confirmed stroke patients upon whom an acute stroke was suspected by the inpatient team, as evidenced by stroke code activation or urgent neurological consultation, but not by the ED team. Inter-rater reliability and agreement were assessed using interclass coefficient and kappa values (κ).

RESULTS

Among 183 cases in the development cohort, the prevalence of FNS was 20.2% (95% CI, 15.0-26.7). Too narrow a differential diagnosis and limited neurological examination were common potential sources of error. The interclass coefficient for the Safer Stroke-Dx Instrument items ranged from 0.42 to 0.91, and items were highly correlated with each other. The κ for diagnostic error identification was 0.90 (95% CI, 0.821-0.978) using the Safer Stroke-Dx Instrument. In the confirmatory cohort of 99 cases, the prevalence of FNS was 21.2% (95% CI, 14.2-30.3) with similar sources of diagnostic error identified.

CONCLUSIONS

Hospitalized patients identified by stroke codes and requests for urgent neurological consultation represent an enriched population for the study of diagnostic error in the ED. The Safer Stroke-Dx Instrument is a reliable tool for identifying FNS and sources of diagnostic error.

What Contributes to Diagnostic Error or Delay? A Qualitative Exploration Across Diverse Acute Care Settings in the United States

OBJECTIVES

Diagnostic error and delay is a prevalent and impactful problem. This study was part of a mixed-methods approach to understand the organizational, clinician, and patient factors contributing to diagnostic error and delay among acutely ill patients within a health system, as well as recommendations for the development of tailored, targeted, feasible, and effective interventions.

METHODS

We did a multisite qualitative study using focus group methodology to explore the perspectives of key clinician stakeholders. We used a conceptual framework that characterized diagnostic error and delay as occurring within 1 of 3 stages of the patient's diagnostic journey-critical information gathering, synthesis of key information, and decision making and communication. We developed our moderator guide based on the sociotechnical frameworks previously described by Holden and Singh for understanding noncognitive factors that lead to diagnostic error and delay. Deidentified focus group transcripts were coded in triplicate and to consensus over a series of meetings. A final coded data set was then uploaded into NVivo software. The data were then analyzed to generate overarching themes and categories.

RESULTS

We recruited a total of 64 participants across 4 sites from emergency departments, hospital floor, and intensive care unit settings into 11 focus groups. Clinicians perceive that diverse organizational, communication and coordination, individual clinician, and patient factors interact to impede the process of making timely and accurate diagnoses.

CONCLUSIONS

This study highlights the complex sociotechnical system within which individual clinicians operate and the contributions of systems, processes, and institutional factors to diagnostic error and delay.

Development of a rubric for assessing delayed diagnosis of appendicitis, diabetic ketoacidosis and sepsis

OBJECTIVES

Using case review to determine whether a patient experienced a delayed diagnosis is challenging. Measurement would be more accurate if case reviewers had access to multi-expert consensus on grading the likelihood of delayed diagnosis. Our objective was to use expert consensus to create a guide for objectively grading the likelihood of delayed diagnosis of appendicitis, new-onset diabetic ketoacidosis (DKA), and sepsis.

METHODS

Case vignettes were constructed for each condition. In each vignette, a patient has the condition and had a previous emergency department (ED) visit within 7 days. Condition-specific multi-specialty expert Delphi panels reviewed the case vignettes and graded the likelihood of a delayed diagnosis on a five-point scale. Delayed diagnosis was defined as the condition being present during the previous ED visit. Consensus was defined as ≥75% agreement. In each Delphi round, panelists were given the scores from the previous round and asked to rescore. A case scoring guide was created from the consensus scores.

RESULTS

Eighteen expert panelists participated. Consensus was achieved within three Delphi rounds for all appendicitis and sepsis vignettes. We reached consensus on 23/30 (77%) DKA vignettes. A case review guide was created from the consensus scores.

CONCLUSIONS

Multi-specialty expert reviewers can agree on the likelihood of a delayed diagnosis for cases of appendicitis and sepsis, and for most cases of DKA. We created a guide that can be used by researchers and quality improvement specialists to allow for objective case review to determine when delayed diagnoses have occurred for appendicitis, DKA, and sepsis.

Evaluation of Consecutive Guided Training to Improve Interrater Agreement in Identifying Elements of Situation Awareness in Objective Structured Clinical Examination Assessments

This article was migrated. The article was marked as recommended. Introduction: Little is known about the medical student's cognitive ability in diagnostic and therapeutic accuracy. Literature does not suggest a methodology to quantify students' cognitive processing. Situation Awareness (SA) is described as having the proficiency to obtain awareness of the surrounding and to integrate this consciousness into the situational context and potential forthcoming development. OSCEs might be a suitable instrument to evaluate students' awareness of the situation. Methods: Consecutive guided training was provided to obtain a consistent comprehension of the model of SA. 4 independent researchers consecutively examined 6 randomised OSCE forms in a qualitative and quantitative method. Final interrater agreement was expressed as Cohens kappa. Generalisability theory determined the impact of the main facets on the variation in disagreement. Results: Evaluation of identifying and categorising elements of SA within OSCE forms demonstrated a moderate to very good interrater agreement. The G-Theory revealed key facets for variance: OSCE forms, Levels of SA, Items embedded in the Levels, Interaction between Forms and Levels and Forms and Items embedded within Levels. Conclusion: Consecutive guided training improved the identification of elements of SA within OSCE assessments. Further research is necessary to improve the assessment of SA in undergraduate medical curricula.

Informing the Design of a New Pragmatic Registry to Stimulate Near Miss Reporting in Ambulatory Care

OBJECTIVE

Ambulatory care safety is of emerging concern, especially in light of recent studies related to diagnostic errors and health information technology-related safety. Safety reporting systems in outpatient care must address the top safety concerns and be practical and simple to use. A registry that can identify common near misses in ambulatory care can be useful to facilitate safety improvements. We reviewed the literature on medical errors in the ambulatory setting to inform the design of a registry for collecting near miss incidents.

METHODS

This narrative review included articles from PubMed that were: 1) original research; 2) discussed near misses or adverse events in the ambulatory setting; 3) relevant to US health care; and 4) published between 2002 and 2013. After full text review, 38 studies were searched for information on near misses and associated factors. Additionally, we used expert opinion and current inpatient near miss registries to inform registry development.

RESULTS

Studies included a variety of safety issues including diagnostic errors, treatment or management-related errors, communication errors, environmental/structural hazards, and health information technology (health IT)-related concerns. The registry, based on the results of the review, updates previous work by including specific sections for errors associated with diagnosis, communication, and environment structure and incorporates specific questions about the role of health information technology.

CONCLUSIONS

Through use of this registry or future registries that incorporate newly identified categories, near misses in the ambulatory setting can be accurately captured, and that information can be used to improve patient safety.

Diagnostic Errors Induced by a Wrong a Priori Diagnosis: A Prospective Randomized Simulator-Based Trial

Preventive strategies against diagnostic errors require the knowledge of underlying mechanisms. We examined the effects of a wrong a priori diagnosis on diagnostic accuracy of a focussed assessment in an acute myocardial infarction scenario. One-hundred-and-fifty-six medical students (cohort 1) were randomized to three study arms differing in the a priori diagnosis revealed: no diagnosis (control group), myocardial infarction (correct diagnosis group), and pulmonary embolism (wrong diagnosis group). Forty-four physicians (cohort 2) were randomized to the control group and the wrong diagnosis group. Primary endpoint was the participants’ final presumptive diagnosis. Among students, the correct diagnosis of an acute myocardial infarction was made by 48/52 (92%) in the control group, 49/52 (94%) in the correct diagnosis group, and 14/52 (27%) in the wrong diagnosis group (p < 0.001 vs. both other groups). Among physicians, the correct diagnosis was made by 20/21 (95%) in the control group and 15/23 (65%) in the wrong diagnosis group (p = 0.023). In the wrong diagnosis group, 31/52 (60%) students and 6/23 (19%) physicians indicated their initially given wrong a priori diagnosis pulmonary embolism as final diagnosis. A wrong a priori diagnosis significantly increases the likelihood of a diagnostic error during a subsequent patient encounter.

Risk of misdiagnosis and delayed diagnosis with COVID-19: A Syndemic Approach

ABSTRACT

Misdiagnosis and delayed diagnosis are common problems in healthcare and are typically related to patient, provider, and socioeconomic factors. A syndemics model of COVID-19 is used to analyze the synergistic relationship between diseases and influences that impact patients' living conditions and health. NPs can use this approach to promote patient safety and equitable healthcare.

Increasing Physician Reporting of Diagnostic Learning Opportunities

BACKGROUND

An estimated 10% of Americans experience a diagnostic error annually, yet little is known about pediatric diagnostic errors. Physician reporting is a promising method for identifying diagnostic errors. However, our pediatric hospital medicine (PHM) division had only 1 diagnostic-related safety report in the preceding 4 years. We aimed to improve attending physician reporting of suspected diagnostic errors from 0 to 2 per 100 PHM patient admissions within 6 months.

METHODS

Our improvement team used the Model for Improvement, targeting the PHM service. To promote a safe reporting culture, we used the term diagnostic learning opportunity (DLO) rather than diagnostic error, defined as a "potential opportunity to make a better or more timely diagnosis." We developed an electronic reporting form and encouraged its use through reminders, scheduled reflection time, and monthly progress reports. The outcome measure, the number of DLO reports per 100 patient admissions, was tracked on an annotated control chart to assess the effect of our interventions over time. We evaluated DLOs using a formal 2-reviewer process.

RESULTS

Over the course of 13 weeks, there was an increase in the number of reports filed from 0 to 1.6 per 100 patient admissions, which met special cause variation, and was subsequently sustained. Most events (66%) were true diagnostic errors and were found to be multifactorial after formal review.

CONCLUSIONS

We used quality improvement methodology, focusing on psychological safety, to increase physician reporting of DLOs. This growing data set has generated nuanced learnings that will guide future improvement work.

Developing Health Care Organizations That Pursue Learning and Exploration of Diagnostic Excellence: An Action Plan

Reducing errors in diagnosis is the next big challenge for patient safety. Diagnostic safety improvement efforts should become a priority for health care organizations, payers, and accrediting bodies; however, external incentives, policies, and practical guidance to develop these efforts are largely absent. In this Perspective, the authors highlight ways in which health care organizations can pursue learning and exploration of diagnostic excellence (LEDE). Building on current evidence and their recent experiences in developing such a learning organization at Geisinger, the authors propose a 5-point action plan and corresponding policy levers to support development of LEDE organizations. These recommendations, which are applicable to many health care organizations, include (1) implementing a virtual hub to coordinate organizational activities for improving diagnosis, such as identifying risks and prioritizing interventions that cross intra-institutional silos while promoting a culture of learning and safety; (2) participating in novel scientific initiatives to generate and translate evidence, given the rapidly evolving "basic science" of diagnostic excellence; (3) avoiding the "tyranny of metrics" by focusing on measurement for improvement rather than using measures to reward or punish; (4) engaging clinicians in activities for improving diagnosis and framing missed opportunities positively as learning opportunities rather than negatively as errors; and (5) developing an accountable culture of engaging and learning from patients, who are often underexplored sources of information. The authors also outline specific policy actions to support organizations in implementing these recommendations. They suggest this action plan can stimulate scientific, practice, and policy progress needed for achieving diagnostic excellence and reducing preventable patient harm.

Operational measurement of diagnostic safety: state of the science

Reducing the incidence of diagnostic errors is increasingly a priority for government, professional, and philanthropic organizations. Several obstacles to measurement of diagnostic safety have hampered progress toward this goal. Although a coordinated national strategy to measure diagnostic safety remains an aspirational goal, recent research has yielded practical guidance for healthcare organizations to start using measurement to enhance diagnostic safety. This paper, concurrently published as an Issue Brief by the Agency for Healthcare Research and Quality, issues a "call to action" for healthcare organizations to begin measurement efforts using data sources currently available to them. Our aims are to outline the state of the science and provide practical recommendations for organizations to start identifying and learning from diagnostic errors. Whether by strategically leveraging current resources or building additional capacity for data gathering, nearly all organizations can begin their journeys to measure and reduce preventable diagnostic harm.

Identifying psychiatric diagnostic errors with the Safer Dx Instrument

OBJECTIVE

Diagnostic errors in psychiatry are understudied partly because they are difficult to measure. The current study aimed to adapt and test the Safer Dx Instrument, a structured tool to review electronic health records (EHR) for errors in medical diagnoses, to evaluate errors in anxiety diagnoses to improve measurement of psychiatric diagnostic errors.

DESIGN

The iterative adaptation process included a review of the revised Safer Dx-Mental Health Instrument by mental health providers to ensure content and face validity and review by a psychometrician to ensure methodologic validity and pilot testing of the revised instrument.

SETTINGS

None.

PARTICIPANTS

Pilot testing was conducted on 128 records of patients diagnosed with anxiety in integrated primary care mental health clinics. Cases with anxiety diagnoses documented in progress notes but not included as a diagnosis for the encounter (n = 25) were excluded.

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

None.

RESULTS

Of 103 records meeting the inclusion criteria, 62 likely involved a diagnostic error (42 from use of unspecified anxiety diagnosis when a specific anxiety diagnosis was warranted; 20 from use of unspecified anxiety diagnosis when anxiety symptoms were either undocumented or documented but not severe enough to warrant diagnosis). Reviewer agreement on presence/absence of errors was 88% (κ = 0.71).

CONCLUSION

The revised Safer Dx-Mental Health Instrument has a high reliability for detecting anxiety-related diagnostic errors and deserves testing in additional psychiatric populations and clinical settings.

Frequency, Risk Factors, Causes, and Consequences of Diagnostic Errors in Critically Ill Medical Patients: A Retrospective Cohort Study

OBJECTIVE

Diagnostic errors are a source of significant morbidity and mortality but understudied in the critically ill. We sought to characterize the frequency, causes, consequences, and risk factors of diagnostic errors among unplanned ICU admissions.

DESIGN

We conducted a retrospective cohort study of randomly selected nonsurgical ICU admissions between July 2015 and June 2016.

SETTING

Medical ICU at a tertiary academic medical center.

SUBJECTS

Critically ill adults with unplanned admission to the medical ICU.

MEASUREMENTS AND MAIN RESULTS

The primary investigator reviewed patient records using a modified version of the Safer Dx instrument, a validated instrument for detecting diagnostic error. Two intensivists performed secondary reviews of possible errors, and reviewers met periodically to adjudicate errors by consensus. For each confirmed error, we judged harm on a 1-6 rating scale. We also collected detailed demographic and clinical data for each patient. We analyzed 256 unplanned ICU admissions and identified 18 diagnostic errors (7% of admissions). All errors were associated with harm, and only six errors (33%) were recognized by the ICU team within the first 24 hours. More women than men experienced a diagnostic error (11.7% vs 2.7%; p = 0.015, χ test). On multivariable logistic regression analysis, female sex remained independently associated with risk of diagnostic error both at admission (odds ratio, 5.18; 95% CI, 1.34-20.08) and at 24 hours (odds ratio, 11.6; 95% CI, 1.37-98.6). Similarly, Quick Sequential Organ Failure Assessment score greater than or equal to 2 at admission was independently associated with diagnostic error (odds ratio, 5.73; 95% CI, 1.72-19.01).

CONCLUSIONS

Diagnostic errors may be an underappreciated source of ICU-related harm. Women and higher acuity patients appear to be at increased risk for such errors. Further research is merited to define the scope of error-associated harm and to clarify risk factors for diagnostic errors among the critically ill.

Assessing diagnostic error in cerebral venous thrombosis via detailed chart review

Background Diagnostic error in cerebral venous thrombosis (CVT) has been understudied despite the harm associated with misdiagnosis of other cerebrovascular diseases as well as the known challenges of evaluating non-specific neurological symptoms in clinical practice. Methods We conducted a retrospective cohort study of CVT patients hospitalized at a single center. Two independent reviewers used a medical record review tool, the Safer Dx Instrument, to identify diagnostic errors. Demographic and clinical factors were abstracted. We compared subjects with and without a diagnostic error using the t-test for continuous variables and the chi-square (χ2) test or Fisher's exact test for categorical variables; an alpha of 0.05 was the cutoff for significance. Results A total of 72 CVT patients initially met study inclusion criteria; 19 were excluded due to incomplete medical records. Of the 53 patients included in the final analysis, the mean age was 48 years and 32 (60.4%) were women. Diagnostic error occurred in 11 cases [20.8%; 95% confidence interval (CI) 11.8-33.6%]. Subjects with diagnostic errors were younger (42 vs. 49 years, p = 0.13), more often women (81.8% vs. 54.8%, p = 0.17), and were significantly more likely to have a past medical history of a headache disorder prior to the index CVT visit (7.1% vs. 36.4%, p = 0.03). Conclusions Nearly one in five patients with complete medical records experienced a diagnostic error. Prior history of headache was the only evaluated clinical factor that was more common among those with an error in diagnosis. Future work on distinguishing primary from secondary headaches to improve diagnostic accuracy in acute neurological disease is warranted.

Competencies for improving diagnosis: an interprofessional framework for education and training in health care

Background Given an unacceptably high incidence of diagnostic errors, we sought to identify the key competencies that should be considered for inclusion in health professions education programs to improve the quality and safety of diagnosis in clinical practice. Methods An interprofessional group reviewed existing competency expectations for multiple health professions, and conducted a search that explored quality, safety, and competency in diagnosis. An iterative series of group discussions and concept prioritization was used to derive a final set of competencies. Results Twelve competencies were identified: Six of these are individual competencies: The first four (#1-#4) focus on acquiring the key information needed for diagnosis and formulating an appropriate, prioritized differential diagnosis; individual competency #5 is taking advantage of second opinions, decision support, and checklists; and #6 is using reflection and critical thinking to improve diagnostic performance. Three competencies focus on teamwork: Involving the patient and family (#1) and all relevant health professionals (#2) in the diagnostic process; and (#3) ensuring safe transitions of care and handoffs, and "closing the loop" on test result communication. The final three competencies emphasize system-related aspects of care: (#1) Understanding how human-factor elements influence the diagnostic process; (#2) developing a supportive culture; and (#3) reporting and disclosing diagnostic errors that are recognized, and learning from both successful diagnosis and from diagnostic errors. Conclusions These newly defined competencies are relevant to all health professions education programs and should be incorporated into educational programs.

Influence of doctor-patient conversations on behaviours of patients presenting to primary care with new or persistent symptoms: a video observation study

BACKGROUND

Most cancers are diagnosed following contact with primary care. Patients diagnosed with cancer often see their doctor multiple times with potentially relevant symptoms before being referred to see a specialist, suggesting missed opportunities during doctor-patient conversations.

OBJECTIVE

To understand doctor-patient communication around the significance of persistent or new presenting problems and its potential impact on timely cancer diagnosis.

RESEARCH DESIGN

Qualitative thematic analysis based on video recordings of doctor-patient consultations in primary care and follow-up interviews with patients and doctors. 80 video observations, 20 patient interviews and 7 doctor interviews across 7 general practices in England.

RESULTS

We found that timeliness of diagnosis may be adversely affected if doctors and patients do not come to an agreement about the presenting problem's significance. 'Disagreements' may involve misaligned cognitive factors such as differences in medical knowledge between doctor and patient or misaligned emotional factors such as patients' unexpressed fear of diagnostic procedures. Interviews suggested that conversations where the difference in views is either not recognised or stays unresolved may lead to unhelpful patient behaviour after the consultation (eg, non-attendance at specialist appointments), creating potential for diagnostic delay and patient harm.

CONCLUSIONS

Our findings highlight how doctor-patient consultations can impact timely diagnosis when patients present with persistent or new problems. Misalignments were common and could go unnoticed, leaving gaps for potential to cause patient harm. These findings have implications for timely diagnosis of cancer and other serious disease because they highlight the complexity and fluidity of the consultation and the subsequent impact on the diagnostic process.

The frequency, nature and impact of GP-assessed avoidable delays in a population-based cohort of cancer patients

BACKGROUND

There is a growing emphasis on the speed of diagnosis as an aspect of cancer prognosis. While epidemiological data in the last decade have quantified diagnostic timeliness and its variation, whether and how often prolonged diagnostic intervals can be considered avoidable is unknown.

METHODS

We used data from the English National Cancer Diagnosis Audit (NCDA) on 17,042 patients diagnosed with cancer in 2014. Participating primary care physicians were asked to identify delays in diagnosis that they deemed avoidable, together with the 'setting' of the avoidable delay and key attributable factors. We used descriptive analysis and regression frameworks to assess validity and examine variation in the frequency and nature of avoidable delays.

RESULTS

Among 14,259 patients, 24% were deemed to have had an avoidable delay to their diagnosis. Patients with a reported avoidable delay had a longer median diagnostic interval (92 days) than those without (30 days). Of all avoidable delays, 13% were deemed to have occurred pre-consultation, 49% within primary care, and 38% within secondary care. Avoidable delays were mostly attributed to the test request/performance phase (25%). Multimorbidity was associated with greater odds of avoidable delay (OR for 3+ vs no comorbidity: 1.43 (95% CI 1.25-1.63)), with heterogeneous associations with cancer site.

CONCLUSION

We have shown that GP-identified instances of avoidable delay have construct validity. Whilst the causes of avoidable diagnostic delays are multi-factorial and occur in different settings and phases of the diagnostic process, their analysis can guide improvement initiatives and enable the examination of any prognostic implications.

Understanding the roles of three academic communities in a prospective learning health ecosystem for diagnostic excellence

Inaccurate, untimely, and miscommunicated medical diagnoses represent a wicked problem requiring comprehensive and coordinated approaches, such as those demonstrated in the characteristics of learning health systems (LHSs). To appreciate a vision for how LHS methods can optimize processes and outcomes in medical diagnosis (diagnostic excellence), we interviewed 32 individuals with relevant expertise: 18 who have studied diagnostic processes using traditional behavioral science and health services research methods, six focused on machine learning (ML) and artificial intelligence (AI) approaches, and eight multidisciplinary researchers experienced in advocating for and incorporating LHS methods, ie, scalable continuous learning in health care. We report on barriers and facilitators, identified by these subjects, to applying their methods toward optimizing medical diagnosis. We then employ their insights to envision the emergence of a learning ecosystem that leverages the tools of each of the three research groups to advance diagnostic excellence. We found that these communities represent a natural fit forward, in which together, they can better measure diagnostic processes and close the loop of putting insights into practice. Members of the three academic communities will need to network and bring in additional stakeholders before they can design and implement the necessary infrastructure that would support ongoing learning of diagnostic processes at an economy of scale and scope.

Comorbid chronic diseases and cancer diagnosis: disease-specific effects and underlying mechanisms

An earlier diagnosis is a key strategy for improving the outcomes of patients with cancer. However, achieving this goal can be challenging, particularly for the growing number of people with one or more chronic conditions (comorbidity/multimorbidity) at the time of diagnosis. Pre-existing chronic diseases might affect patient participation in cancer screening, help-seeking for new and/or changing symptoms and clinicians' decision-making on the use of diagnostic investigations. Evidence suggests, for example, that pre-existing pulmonary, cardiovascular, neurological and psychiatric conditions are all associated with a more advanced stage of cancer at diagnosis. By contrast, hypertension and certain gastrointestinal and musculoskeletal conditions might be associated with a more timely diagnosis. In this Review, we propose a comprehensive framework that encompasses the effects of disease-specific, patient-related and health-care-related factors on the diagnosis of cancer in individuals with pre-existing chronic illnesses. Several previously postulated aetiological mechanisms (including alternative explanations, competing demands and surveillance effects) are integrated with newly identified mechanisms, such as false reassurances, or patient concerns about appearing to be a hypochondriac. By considering specific effects of chronic diseases on diagnostic processes and outcomes, tailored early diagnosis initiatives can be developed to improve the outcomes of the large proportion of patients with cancer who have pre-existing chronic conditions.

Diagnostic errors in the neonatal intensive care unit: State of the science and new directions

Diagnostic errors remain understudied in neonatology. The limited available evidence, however, suggests that diagnostic errors in the neonatal intensive care unit (NICU) result in significant and long-term consequences. In this narrative review, we discuss how the concept of diagnostic errors framed as missed opportunities can be applied to the non-linear nature of diagnosis in a critical care environment such as the NICU. We then explore how the etiology of an error in diagnosis can be related to both individual cognitive factors as well as organizational and systemic factors - all of which often contribute to the error. This multifactorial causation has limited the development of methodology to measure diagnostic errors as well as strategies to mitigate and prevent their adverse effects. We recommend research focused on the frequency and etiology of diagnostic error in the NICU as well as potential mitigation strategies to advance this important field in neonatal intensive care.

Mortality and morbidity rounds (MMR) in pathology: relative contribution of cognitive bias vs. systems failures to diagnostic error

Background Heuristics and cognitive biases are thought to play an important role in diagnostic medical error. How to systematically determine and capture these kinds of errors remains unclear. Morbidity and mortality rounds (MMRs) are generally focused on reducing medical error by identifying and correcting systems failures. However, they may also provide an educational platform for recognizing and raising awareness on cognitive errors. Methods A total of 49 MMR cases spanning the period 2008-2015 in our pathology department were examined for the presence of cognitive errors and/or systems failures by eight study participant raters who were trained on a subset of 16 of these MMR cases (excluded from the main study analysis) to identify such errors. The Delphi method was used to obtain group consensus on error classification on the remaining 33 study cases. Cases with <75% inter-rater agreement were subjected to subsequent rounds of Delphi analysis. Inter-rater agreement at each round was determined by Fleiss' kappa values. Results Thirty-six percent of the cases presented at our pathology MMRs over an 8-year period were found to contain errors likely due to cognitive bias. Conclusions These data suggest that the errors identified in our pathology MMRs represent not only systems failures but may also be composed of a significant proportion of cognitive errors. Teaching trainees and health professionals to correctly identify different types of cognitive errors may present an opportunity for quality improvement interventions in the interests of patient safety.

Recommendations for using the Revised Safer Dx Instrument to help measure and improve diagnostic safety

The medical record continues to be one of the most useful and accessible sources of information to examine the diagnostic process. However, medical record review studies of diagnostic errors have often used subjective judgments and found low inter-rater agreement among reviewers when determining the presence or absence of diagnostic error. In our previous work, we developed a structured data-collection instrument, called the Safer Dx Instrument, consisting of objective criteria to improve the accuracy of assessing diagnostic errors in primary care. This paper proposes recommendations on how clinicians and health care organizations could use the Revised Safer Dx Instrument in identifying and understanding missed opportunities to make correct and timely diagnoses. The instrument revisions addressed both methodological and implementation issues identified during initial use and included refinements to the instrument to allow broader application across all health care settings. In addition to leveraging knowledge from piloting the instrument in several health care settings, we gained insights from multiple researchers who had used the instrument in studies involving emergency care, inpatient care and intensive care unit settings. This allowed us to enhance and extend the scope of this previously validated data collection instrument. In this paper, we describe the refinement process and provide recommendations for application and use of the Revised Safer Dx Instrument across a broad range of health care settings. The instrument can help users identify potential diagnostic errors in a standardized way for further analysis and safety improvement efforts as well as provide data for clinician feedback and reflection. With wider adoption and use by clinicians and health systems, the Revised Safer Dx Instrument could help propel the science of measuring and reducing diagnostic errors forward.

Payment Innovations To Improve Diagnostic Accuracy And Reduce Diagnostic Error

Diagnostic accuracy is essential for treatment decisions but is largely unaccounted for by payers, including in fee-for-service Medicare and proposed Alternative Payment Models (APMs). We discuss three payment-related approaches to reducing diagnostic error. First, coding changes in the Medicare Physician Fee Schedule could facilitate the more effective use of teamwork and information technology in the diagnostic process and better support the cognitive work and time commitment that physicians make in the quest for diagnostic accuracy, especially in difficult or uncertain cases. Second, new APMs could be developed to focus on improving diagnostic accuracy in challenging cases and make available support resources for diagnosis, including condition-specific centers of diagnostic expertise or general diagnostic centers of excellence that provide second (or even third) opinions. Performing quality improvement activities that promote safer diagnosis should be a part of the accountability of APM recipients. Third, the accuracy of diagnoses that trigger APM payments and establish payment amounts should be confirmed by APM recipients. Implementation of these multipronged approaches can make current payment models more accountable for addressing diagnostic error and position diagnostic performance as a critical component of quality-based payment.

Evaluating diagnostic strategies for early detection of cancer: the CanTest framework

BACKGROUND

Novel diagnostic triage and testing strategies to support early detection of cancer could improve clinical outcomes. Most apparently promising diagnostic tests ultimately fail because of inadequate performance in real-world, low prevalence populations such as primary care or general community populations. They should therefore be systematically evaluated before implementation to determine whether they lead to earlier detection, are cost-effective, and improve patient safety and quality of care, while minimising over-investigation and over-diagnosis.

METHODS

We performed a systematic scoping review of frameworks for the evaluation of tests and diagnostic approaches.

RESULTS

We identified 16 frameworks: none addressed the entire continuum from test development to impact on diagnosis and patient outcomes in the intended population, nor the way in which tests may be used for triage purposes as part of a wider diagnostic strategy. Informed by these findings, we developed a new framework, the 'CanTest Framework', which proposes five iterative research phases forming a clear translational pathway from new test development to health system implementation and evaluation.

CONCLUSION

This framework is suitable for testing in low prevalence populations, where tests are often applied for triage testing and incorporated into a wider diagnostic strategy. It has relevance for a wide range of stakeholders including patients, policymakers, purchasers, healthcare providers and industry.

Missed Diagnosis of New-Onset Systolic Heart Failure at First Presentation in Children with No Known Heart Disease

OBJECTIVE

To determine frequency of missed heart failure diagnosis at first presentation among children with no known heart disease admitted with new-onset heart failure.

STUDY DESIGN

Using a retrospective design, we reviewed electronic medical records of all patients aged <21 years with no known heart disease, hospitalized with new-onset heart failure during 2003-2015 at a tertiary-quaternary care institution. We assessed records for missed diagnosis of heart failure (primary outcome), associated process breakdowns, and clinical outcomes using a structured data collection instrument.

RESULTS

Of 191 patients meeting inclusion criteria, 49% (94/191) were missed on first presentation. Most common incorrect diagnostic labels given to "missed" patients were bacterial infection (29%; 27/94), followed by viral illness (22%; 21/94) and gastroenteritis/hepatitis (21%; 20/94). On multivariable analysis, presentation to primary care provider (PCP), longer duration of symptoms (median 7 days), more than 2 symptoms of heart failure, and nausea/emesis were associated with missed diagnosis. On examining process breakdowns, 49% had errors in history-taking and 50% had no documentation of differential diagnoses. There was no difference in hospital mortality, length of stay, or mechanical circulatory support in missed vs not-missed cohorts. Unnecessary noninvasive and invasive tests were performed in 18% and 4% of patients, respectively.

CONCLUSIONS

Nearly one-half of children with no known heart disease hospitalized with systolic heart failure were missed at first presentation and underwent significant nonrelevant treatment and testing. Initial presentation to the PCP, longer duration of symptoms before presentation, and nausea/emesis were associated with missed diagnosis.

A Step Toward Identifying Sources of Medical Errors: Modeling Standards of Care Deviations for Different Disease States

Objective

To examine the feasibility of utilizing electronic health records (EHR) to determine a metric for identifying physician diagnostic and treatment deviations in standards of care for different disease states.

Methods

A Boolean-rule-based model compared deviations in standards of care across four disease states: diabetes, cardiovascular disease, asthma, and rheumatoid arthritis. This metric was used to identify the relationship between physician deviations in standards of care procedures, before and after diagnosis, for 76 internal medicine physicians.

Results

The Boolean-rule-based model identified patterns of standards of care deviation for the physicians before diagnosis and during treatment. The deviations identified for each of the four disease states were then related to Continuing Medical Education courses that could support further training. The rule-based model was extended and improved by including system and process aspects of medical care that are not specifically related to the physician, yet potentially have an impact on the physician's decision to deviate from the standards of care.

Conclusion

The Boolean-rule-based approach provided a means to systematically mine EHRs and use these data to assess deviations in standards of care that could identify quality of care issues stemming from system processes or the need for specific CME for a physician.

Application of electronic trigger tools to identify targets for improving diagnostic safety

Progress in reducing diagnostic errors remains slow partly due to poorly defined methods to identify errors, high-risk situations, and adverse events. Electronic trigger (e-trigger) tools, which mine vast amounts of patient data to identify signals indicative of a likely error or adverse event, offer a promising method to efficiently identify errors. The increasing amounts of longitudinal electronic data and maturing data warehousing techniques and infrastructure offer an unprecedented opportunity to implement new types of e-trigger tools that use algorithms to identify risks and events related to the diagnostic process. We present a knowledge discovery framework, the Safer Dx Trigger Tools Framework, that enables health systems to develop and implement e-trigger tools to identify and measure diagnostic errors using comprehensive electronic health record (EHR) data. Safer Dx e-trigger tools detect potential diagnostic events, allowing health systems to monitor event rates, study contributory factors and identify targets for improving diagnostic safety. In addition to promoting organisational learning, some e-triggers can monitor data prospectively and help identify patients at high-risk for a future adverse event, enabling clinicians, patients or safety personnel to take preventive actions proactively. Successful application of electronic algorithms requires health systems to invest in clinical informaticists, information technology professionals, patient safety professionals and clinicians, all of who work closely together to overcome development and implementation challenges. We outline key future research, including advances in natural language processing and machine learning, needed to improve effectiveness of e-triggers. Integrating diagnostic safety e-triggers in institutional patient safety strategies can accelerate progress in reducing preventable harm from diagnostic errors.

Diagnostic Decision-Making in the Emergency Department

Emergency medicine requires diagnosing unfamiliar patients with undifferentiated acute presentations. This requires hypothesis generation and questioning, examination, and testing. Balancing patient load, care across the severity spectrum, and frequent interruptions create time pressures that predispose humans to fast thinking or cognitive shortcuts, including cognitive biases. Diagnostic error is the failure to establish an accurate and timely explanation of the problem or communicate that to the patient, often contributing to physical, emotional, or financial harm. Methods for monitoring diagnostic error in the emergency department are needed to establish frequency and serve as a foundation for future interventions.

Blinding or information control in diagnosis: could it reduce errors in clinical decision-making?

Background Clinical medicine has long recognized the potential for cognitive bias in the development of new treatments, and in response developed a tradition of blinding both clinicians and patients to address this specific concern. Although cognitive biases have been shown to exist which impact the accuracy of clinical diagnosis, blinding the diagnostician to potentially misleading information has received little attention as a possible solution. Recently, within the forensic sciences, the control of contextual information (i.e. information apart from the objective test results) has been studied as a technique to reduce errors. We consider the applicability of this technique to clinical medicine. Content This article briefly describes the empirical research examining cognitive biases arising from context which impact clinical diagnosis. We then review the recent awakening of forensic sciences to the serious effects of misleading information. Comparing the approaches, we discuss whether blinding to contextual information might (and in what circumstances) reduce clinical errors. Summary and outlook Substantial research indicates contextual information plays a significant role in diagnostic error and conclusions across several medical specialties. The forensic sciences may provide a useful model for the control of potentially misleading information in diagnosis. A conceptual analog of the forensic blinding process (the "agnostic" first reading) may be applicable to diagnostic investigations such as imaging, microscopic tissue examinations and waveform recognition. An "agnostic" approach, where the first reading occurs with minimal clinical referral information, but is followed by incorporation of the clinical history and reinterpretation, has the potential to reduce errors.