Advancing the science of measurement of diagnostic errors in healthcare: the Safer Dx framework

Diagnostic safety and quality optimization in sepsis study protocol

BACKGROUND

Sepsis ranks among the "Big Three" conditions most prone to harmful diagnostic errors. Despite its high prevalence and severity, health systems lack effective and contextually tailored strategies to optimize diagnostic accuracy for sepsis.

OBJECTIVES

The purpose of this study is to understand factors related to high sepsis diagnostic accuracy using principles and tools of safety and implementation science.

METHODS

This is a multi-site study involving 20 hospitals across four states in the United States. The primary objectives are to (1) describe hospital-level variability and understand barriers and facilitators to sepsis diagnostic accuracy and (2) apply cross-case and coincidence analysis to determine minimally sufficient and necessary conditions for optimal sepsis diagnosis that minimizes under- and overtreatment. To identify barriers and facilitators of acute sepsis diagnosis, we will conduct electronic surveys and in-depth interviews with key informants from each hospital. We will use data from electronic health records (EHR) and data warehouses to operationalize sepsis diagnostic accuracy.

RESULTS

We have enrolled 20 hospitals and begum data collection. The findings of this study will be used to develop a context-specific toolkit that guides the selection of feasible and important strategies to promote optimal sepsis diagnosis in diverse hospitals settings.

CONCLUSIONS

The study uses tools and principles from safety and implementation science to generate first-of-its-kind evidence to improve diagnostic excellence in sepsis.

Communication failures and racial disparities in inpatient maternity care: a qualitative content analysis of incident reports

BACKGROUND

Severe maternal morbidity (SMM) and mortality disproportionality affect Black women in the USA. Communication failures are a leading cause of poor maternal outcomes. We examined incident reports to identify communication failures within inpatient maternity care and racial disparities therein.

METHODS

We analysed de-identified incident reports submitted by hospital staff working on antepartum, labour and birth, and postpartum in an urban, academic hospital between 2019 and 2022. Reports were linked to electronic health records to capture race and SMM outcome. We conducted qualitative content analyses using a constant comparative method and an inductive and deductive approach. We explored communication failures by race/ethnicity and SMM outcome. In vivo themes included equity and positive communication.

RESULTS

We identified 541 communication failures within a random sample (n=1006) of incident reports across the study period. Black women represented 28% of births during this time, but 38% of the incident reports. Most of the communication failures occurred within the healthcare team rather than with patients. Communication failures were, broadly, contextual (eg, audience, who was present), conceptual (eg, lack of shared understanding) or sociotechnical (eg, computer-human interface). Of the incident reports coded as contextual failures, errors of omission were the most common. Most conceptual failures were a lack of shared understanding. Sociotechnical failures were predominantly workflow and communication and internal organisational features.

CONCLUSIONS

Our findings suggest that if we want to address communication failures as a root cause of maternal morbidity and mortality, we need to focus on the quality of communication within the healthcare team. These efforts should concentrate on decreasing omission and building shared understanding of responsibilities and processes, especially when teams are caring for Black women.

Let us talk about mistakes

Unfortunately, errors and mistakes are part of life. Errors and mistakes can harm patients and incur unplanned costs. Errors may arise from various sources, which may be classified as systematic, latent, or active. Intrinsic and extrinsic factors also contribute to incorrect decisions. In addition to cognitive biases, our personality, socialization, personal chronobiology, and way of thinking (heuristic versus analytical) are influencing factors. Factors such as overload from private situations, long commuting times, and the complex environment of information technology must also be considered. The objective of this paper is to define and classify errors and mistakes in radiology, to discuss the influencing factors, and to present strategies for prevention. Hierarchical responsibilities and team "well-being" are also discussed.

Implementation of Electronic Triggers to Identify Diagnostic Errors in Emergency Departments

Importance

Missed diagnosis can lead to preventable patient harm.

Objective

To develop and implement a portfolio of electronic triggers (e-triggers) and examine their performance for identifying missed opportunities in diagnosis (MODs) in emergency departments (EDs).

Design, Setting, and Participants

In this retrospective medical record review study of ED visits at 1321 Veterans Affairs health care sites, rules-based e-triggers were developed and implemented using a national electronic health record repository. These e-triggers targeted 6 high-risk presentations for MODs in treat-and-release ED visits. A high-risk stroke e-trigger was applied to treat-and-release ED visits from January 1, 2016, to December 31, 2020. A symptom-disease dyad e-trigger was applied to visits from January 1, 2018, to December 31, 2019. High-risk abdominal pain, unexpected ED return, unexpected hospital return, and test result e-triggers were applied to visits from January 1, 2019, to December 31, 2019. At least 100 randomly selected flagged records were reviewed by physician reviewers for each e-trigger. Data were analyzed between January 2024 and April 2024.

Exposures

Treat-and-release ED visits involving high-risk stroke, symptom-disease dyads, high-risk abdominal pain, unexpected ED return, unexpected hospital return, and abnormal test results not followed up after initial ED visit.

Main Outcomes and Measures

Trained physician reviewers evaluated the presence/absence of MODs at ED visits and recorded data on patient and clinician characteristics, types of diagnostic process breakdowns, and potential harm from MODs.

Results

The high-risk stroke e-trigger was applied to 8 792 672 treat-and-release ED visits (4 967 283 unique patients); the symptom-disease dyad e-trigger was applied to 3 692 454 visits (2 070 979 patients); and high-risk abdominal pain, unexpected ED return, unexpected hospital return, and test result e-triggers were applied to 1 845 905 visits (1 032 969 patients), overall identifying 203, 1981, 170, 116 785, 14 879, and 2090 trigger-positive records, respectively. Review of 625 randomly selected patient records (mean [SD] age, 62.5 [15.2] years; 553 [88.5%] male) showed the following MOD counts and positive predictive values (PPVs) within each category: 47 MODs (PPV, 47.0%) for stroke, 31 MODs (PPV, 25.8%) for abdominal pain, 11 MODs (PPV, 11.0%) for ED returns, 23 MODs (PPV, 23.0%) for hospital returns, 18 MODs (PPV, 18.0%) for symptom-disease dyads, and 55 MODs (PPV, 52.4%) for test results. Patients with MODs were slightly older than those without (mean [SD] age, 65.6 [14.5] vs 61.2 [15.3] years; P < .001). Reviewer agreement was favorable (range, 72%-100%). In 108 of 130 MODs (83.1%; excluding MODs related to the test result e-trigger), the most common diagnostic process breakdown involved the patient-clinician encounter. In 185 total MODs, 20 patients experienced severe harm (10.8%), and 54 patients experienced moderate harm (29.2%).

Conclusions and Relevance

In this retrospective medical record review study, rules-based e-triggers were useful for post hoc detection of MODs in ED visits. Interventions to target ED work system factors are urgently needed to support patient-clinician encounters and minimize harm from diagnostic errors.

Diagnostic inequalities relating to physical healthcare among people with mental health conditions: a systematic review

Background

Inaccurate diagnosis of physical health problems in people with mental health conditions may contribute to poorer health outcomes. We review the evidence on whether individuals with mental health conditions are at risk of diagnostic inequalities related to their physical health.

Methods

We searched MEDLINE, PsycINFO, Embase, and CINAHL, 1 September 2002-18 Septemebr 2024 (PROSPERO 2022: CRD42022375892). Seventy-nine studies were eligible for inclusion. Risk of Bias (RoB) was assessed using the Newcastle-Ottawa or RoB2 tools and results were presented as a narrative synthesis.

Findings

Findings from the included studies suggests that people with mental health conditions face diagnostic inequalities for their physical health. A minority of studies adopted a design that specifically measured professional- and service-related factors associated with diagnostic inequalities. Most studies, however, measured diagnostic endpoints only, meaning that no inference could be made regarding the relative impact of patients' and clinicians' behaviour in producing inequalities.

Interpretation

Further investigations should consider the stage of the diagnostic process at which inequalities occur, to improve knowledge of the mechanisms underpinning diagnostic inequalities, and support the development of targeted improvement interventions.

Funding

This study is funded by The Health Foundation's grant to the University of Cambridge for The Healthcare Improvement Studies (THIS) Institute. Grant number not applicable.

Application of a digital quality measure for cancer diagnosis in Epic Cosmos

OBJECTIVES

Missed and delayed cancer diagnoses are common, harmful, and often preventable. We previously validated a digital quality measure (dQM) of emergency presentation (EP) of lung cancer in 2 US health systems. This study aimed to apply the dQM to a new national electronic health record (EHR) database and examine demographic associations.

MATERIALS AND METHODS

We applied the dQM (emergency encounter followed by new lung cancer diagnosis within 30 days) to Epic Cosmos, a deidentified database covering 184 million US patients. We examined dQM associations with sociodemographic factors.

RESULTS

The overall EP rate was 19.6%. EP rate was higher in Black vs White patients (24% vs 19%, P < .001) and patients with younger age, higher social vulnerability, lower-income ZIP code, and self-reported transport difficulties.

DISCUSSION

We successfully applied a dQM based on cancer EP to the largest US EHR database.

CONCLUSION

This dQM could be a marker for sociodemographic vulnerabilities in cancer diagnosis.

Adverse diagnostic events in hospitalised patients: a single-centre, retrospective cohort study

BACKGROUND

Adverse event surveillance approaches underestimate the prevalence of harmful diagnostic errors (DEs) related to hospital care.

METHODS

We conducted a single-centre, retrospective cohort study of a stratified sample of patients hospitalised on general medicine using four criteria: transfer to intensive care unit (ICU), death within 90 days, complex clinical events, and none of the aforementioned high-risk criteria. Cases in higher-risk subgroups were over-sampled in predefined percentages. Each case was reviewed by two adjudicators trained to judge the likelihood of DE using the Safer Dx instrument; characterise harm, preventability and severity; and identify associated process failures using the Diagnostic Error Evaluation and Research Taxonomy modified for acute care. Cases with discrepancies or uncertainty about DE or impact were reviewed by an expert panel. We used descriptive statistics to report population estimates of harmful, preventable and severely harmful DEs by demographic variables based on the weighted sample, and characteristics of harmful DEs. Multivariable models were used to adjust association of process failures with harmful DEs.

RESULTS

Of 9147 eligible cases, 675 were randomly sampled within each subgroup: 100% of ICU transfers, 38.5% of deaths within 90 days, 7% of cases with complex clinical events and 2.4% of cases without high-risk criteria. Based on the weighted sample, the population estimates of harmful, preventable and severely harmful DEs were 7.2% (95% CI 4.66 to 9.80), 6.1% (95% CI 3.79 to 8.50) and 1.1% (95% CI 0.55 to 1.68), respectively. Harmful DEs were frequently characterised as delays (61.9%). Severely harmful DEs were frequent in high-risk cases (55.1%). In multivariable models, process failures in assessment, diagnostic testing, subspecialty consultation, patient experience, and history were significantly associated with harmful DEs.

CONCLUSIONS

We estimate that a harmful DE occurred in 1 of every 14 patients hospitalised on general medicine, the majority of which were preventable. Our findings underscore the need for novel approaches for adverse DE surveillance.

Harbingers of sepsis misdiagnosis among pediatric emergency department patients

OBJECTIVES

To identify clinical presentations that acted as harbingers for future sepsis hospitalizations in pediatric patients evaluated in the emergency department (ED) using the Symptom Disease Pair Analysis of Diagnostic Error (SPADE) methodology.

METHODS

We identified patients in the Pediatric Health Information Systems (PHIS) database admitted for sepsis between January 1, 2004 and December 31, 2023 and limited the study cohort to those patients who had an ED treat-and-release visit in the 30 days prior to admission. Using the look-back approach of the SPADE methodology, we identified the most common clinical presentations at the initial ED visit and used an observed to expected (O:E) analysis to determine which presentations were overrepresented. We then employed a graphical, temporal analysis with a comparison group to identify which overrepresented presentations most likely represented harbingers for future sepsis hospitalization.

RESULTS

We identified 184,157 inpatient admissions for sepsis, of which 15,331 hospitalizations (8.3 %) were preceded by a treat-and-release ED visit in the prior 30 days. Based on the O:E and temporal analyses, the presentations of fever and dehydration were both overrepresented in the study cohort and temporally clustered close to sepsis hospitalization. ED treat-and-release visits for fever or dehydration preceded 1.2 % of all sepsis admissions.

CONCLUSIONS

In pediatric patients presenting to the ED, fever and dehydration may represent harbingers for future sepsis hospitalization. The SPADE methodology could be applied to the PHIS database to develop diagnostic performance measures across a wide range of pediatric hospitals.

Partnership as a Pathway to Diagnostic Excellence: The Challenges and Successes of Implementing the Safer Dx Learning Lab

BACKGROUND

Learning health system (LHS) approaches could potentially help health care organizations (HCOs) identify and address diagnostic errors. However, few such programs exist, and their implementation is poorly understood.

METHODS

The authors conducted a qualitative evaluation of the Safer Dx Learning Lab, a partnership between a health system and a research team, to identify and learn from diagnostic errors and improve diagnostic safety at an organizational level. The research team conducted virtual interviews to solicit participant feedback regarding experiences with the lab, focusing specifically on implementation and sustainment issues.

RESULTS

Interviews of 25 members associated with the lab identified the following successes: learning and professional growth, improved workflow related to streamlining the process of reporting error cases, and a psychologically safe culture for identifying and reporting diagnostic errors. However, multiple barriers also emerged: competing priorities between clinical responsibilities and research, time-management issues related to a lack of protected time, and inadequate guidance to disseminate findings. Lessons learned included understanding the importance of obtaining buy-in from leadership and interested stakeholders, creating a psychologically safe environment for reporting cases, and the need for more protected time for clinicians to review and learn from cases.

CONCLUSION

Findings suggest that a learning health systems approach using partnerships between researchers and a health system affected organizational culture by prioritizing learning from diagnostic errors and encouraging clinicians to be more open to reporting. The study findings can help organizations overcome barriers to engage clinicians and inform future implementation and sustainment of similar initiatives.

Interventions to improve timely cancer diagnosis: an integrative review

Cancer will affect more than one in three U.S. residents in their lifetime, and although the diagnosis will be made efficiently in most of these cases, roughly one in five patients will experience a delayed or missed diagnosis. In this integrative review, we focus on missed opportunities in the diagnosis of breast, lung, and colorectal cancer in the ambulatory care environment. From a review of 493 publications, we summarize the current evidence regarding the contributing factors to missed or delayed cancer diagnosis in ambulatory care, as well as evidence to support possible strategies for intervention. Cancer diagnoses are made after follow-up of a positive screening test or an incidental finding, or most commonly, by following up and clarifying non-specific initial presentations to primary care. Breakdowns and delays are unacceptably common in each of these pathways, representing failures to follow-up on abnormal test results, incidental findings, non-specific symptoms, or consults. Interventions aimed at 'closing the loop' represent an opportunity to improve the timeliness of cancer diagnosis and reduce the harm from diagnostic errors. Improving patient engagement, using 'safety netting,' and taking advantage of the functionality offered through health information technology are all viable options to address these problems.

Diagnostic error in mental health: a review

Diagnostic errors are associated with patient harm and suboptimal outcomes. Despite national scientific efforts to advance definition, measurement and interventions for diagnostic error, diagnosis in mental health is not well represented in this ongoing work. We aimed to summarise the current state of research on diagnostic errors in mental health and identify opportunities to align future research with the emerging science of diagnostic safety. We review conceptual considerations for defining and measuring diagnostic error, the application of these concepts to mental health settings, and the methods and subject matter focus of recent studies of diagnostic error in mental health. We found that diagnostic error is well understood to be a problem in mental healthcare. Although few studies used clear definitions or frameworks for understanding diagnostic error in mental health, several studies of missed, wrong, delayed and disparate diagnosis of common mental disorders have identified various avenues for future research and development. Nevertheless, a lack of clear consensus on how to conceptualise, define and measure errors in diagnosis will pose a barrier to advancement. Further research should focus on identifying preventable missed opportunities in the diagnosis of mental disorders, which may uncover generalisable opportunities for improvement.

Assessing the Revised Safer Dx Instrument® in the understanding of ambulatory system design changes for type 1 diabetes and autism spectrum disorder in pediatrics

OBJECTIVES

We sought within an ambulatory safety study to understand if the Revised Safer Dx instrument may be helpful in identification of diagnostic missed opportunities in care of children with type 1 diabetes (T1D) and autism spectrum disorder (ASD).

METHODS

We reviewed two months of emergency department (ED) encounters for all patients at our tertiary care site with T1D and a sample of such encounters for patients with ASD over a 15-month period, and their pre-visit communication methods to better understand opportunities to improve diagnosis. We applied the Revised Safer Dx instrument to each diagnostic journey. We chose potentially preventable ED visits for hyperglycemia, diabetic ketoacidosis, and behavioral crises, and reviewed electronic health record data over the prior three months related to the illness that resulted in the ED visit.

RESULTS

We identified 63 T1D and 27 ASD ED visits. Using the Revised Safer Dx instrument, we did not identify any potentially missed opportunities to improve diagnosis in T1D. We found two potential missed opportunities (Safer Dx overall score of 5) in ASD, related to potential for ambulatory medical management to be improved. Over this period, 40 % of T1D and 52 % of ASD patients used communication prior to the ED visit.

CONCLUSIONS

Using the Revised Safer Dx instrument, we uncommonly identified missed opportunities to improve diagnosis in patients who presented to the ED with potentially preventable complications of their chronic diseases. Future researchers should consider prospectively collected data as well as development or adaptation of tools like the Safer Dx.

Development and Implementation of a Digital Quality Measure of Emergency Cancer Diagnosis

PURPOSE

Missed and delayed cancer diagnoses are common, harmful, and often preventable. Automated measures of quality of cancer diagnosis are lacking but could identify gaps and guide interventions. We developed and implemented a digital quality measure (dQM) of cancer emergency presentation (EP) using electronic health record databases of two health systems and characterized the measure's association with missed opportunities for diagnosis (MODs) and mortality.

METHODS

On the basis of literature and expert input, we defined EP as a new cancer diagnosis within 30 days after emergency department or inpatient visit. We identified EPs for lung cancer and colorectal cancer (CRC) in the Department of Veterans Affairs (VA) and Geisinger from 2016 to 2020. We validated measure accuracy and identified preceding MODs through standardized chart review of 100 records per cancer per health system. Using VA's longitudinal encounter and mortality data, we applied logistic regression to assess EP's association with 1-year mortality, adjusting for cancer stage and demographics.

RESULTS

Among 38,565 and 2,914 patients with lung cancer and 14,674 and 1,649 patients with CRCs at VA and Geisinger, respectively, our dQM identified EPs in 20.9% and 9.4% of lung cancers, and 22.4% and 7.5% of CRCs. Chart reviews revealed high positive predictive values for EPs across sites and cancer types (72%-90%), and a substantial percent represented MODs (48.8%-84.9%). EP was associated with significantly higher odds of 1-year mortality for lung cancer and CRC (adjusted odds ratio, 1.78 and 1.83, respectively, 95% CI, 1.63 to 1.86 and 1.61 to 2.07).

CONCLUSION

A dQM for cancer EP was strongly associated with both mortality and MODs. The findings suggest a promising automated approach to measuring quality of cancer diagnosis in US health systems.

Diagnostic Errors in Obstetric Morbidity and Mortality: Methods for and Challenges in Seeking Diagnostic Excellence

Pregnancy-related morbidity and mortality remain high across the United States, with the majority of deaths being deemed preventable. Misdiagnosis and delay in diagnosis are thought to be significant contributors to preventable harm. These diagnostic errors in obstetrics are understudied. Presented here are five selected research methods to ascertain the rates of and harm associated with diagnostic errors and the pros and cons of each. These methodologies include clinicopathologic autopsy studies, retrospective chart reviews based on clinical criteria, obstetric simulations, pregnancy-related harm case reviews, and malpractice and administrative claim database research. We then present a framework for a future study of diagnostic errors and the pursuit of diagnostic excellence in obstetrics: (1) defining and capturing diagnostic errors, (2) targeting bias in diagnostic processes, (3) implementing and monitoring safety bundles, (4) leveraging electronic health record triggers for case reviews, (5) improving diagnostic skills via simulation training, and (6) publishing error rates and reduction strategies. Evaluation of the effectiveness of this framework to ascertain diagnostic error rates, as well as its impact on patient outcomes, is required.

Evaluation of a Structured Review Process for Emergency Department Return Visits with Admission

BACKGROUND

Review of emergency department (ED) revisits with admission allows the identification of improvement opportunities. Applying a health equity lens to revisits may highlight potential disparities in care transitions. Universal definitions or practicable frameworks for these assessments are lacking. The authors aimed to develop a structured methodology for this quality assurance (QA) process, with a layered equity analysis.

METHODS

The authors developed a classification instrument to identify potentially preventable 72-hour returns with admission (PPRA-72), accounting for directed, unrelated, unanticipated, or disease progression returns. A second review team assessed the instrument reliability. A self-reported race/ethnicity (R/E) and language algorithm was developed to minimize uncategorizable data. Disposition distribution, return rates, and PPRA-72 classifications were analyzed for disparities using Pearson chi-square and Fisher's exact tests.

RESULTS

The PPRA-72 rate was 4.8% for 2022 ED return visits requiring admission. Review teams achieved 93% agreement (κ = 0.51) for the binary determination of PPRA-72 vs. nonpreventable returns. There were significant differences between R/E and language in ED dispositions (p < 0.001), with more frequent admissions for the R/E White at the index visit and Other at the 72-hour return visit. Rates of return visits within 72 hours differed significantly by R/E (p < 0.001) but not by language (p = 0.156), with the R/E Black most frequent to have a 72-hour return. There were no differences between R/E (p = 0.446) or language (p = 0.248) in PPRA-72 rates. The initiative led to system improvements through informatics optimizations, triage protocols, provider feedback, and education.

CONCLUSION

The authors developed a review methodology for identifying improvement opportunities across ED 72-hour returns. This QA process enabled the identification of areas of disparity, with the continuous aim to develop next steps in ensuring health equity in care transitions.

Automating detection of diagnostic error of infectious diseases using machine learning

Diagnostic error, a cause of substantial morbidity and mortality, is largely discovered and evaluated through self-report and manual review, which is costly and not suitable to real-time intervention. Opportunities exist to leverage electronic health record data for automated detection of potential misdiagnosis, executed at scale and generalized across diseases. We propose a novel automated approach to identifying diagnostic divergence considering both diagnosis and risk of mortality. Our objective was to identify cases of emergency department infectious disease misdiagnoses by measuring the deviation between predicted diagnosis and documented diagnosis, weighted by mortality. Two machine learning models were trained for prediction of infectious disease and mortality using the first 24h of data. Charts were manually reviewed by clinicians to determine whether there could have been a more correct or timely diagnosis. The proposed approach was validated against manual reviews and compared using the Spearman rank correlation. We analyzed 6.5 million ED visits and over 700 million associated clinical features from over one hundred emergency departments. The testing set performances of the infectious disease (Macro F1 = 86.7, AUROC 90.6 to 94.7) and mortality model (Macro F1 = 97.6, AUROC 89.1 to 89.1) were in expected ranges. Human reviews and the proposed automated metric demonstrated positive correlations ranging from 0.231 to 0.358. The proposed approach for diagnostic deviation shows promise as a potential tool for clinicians to find diagnostic errors. Given the vast number of clinical features used in this analysis, further improvements likely need to either take greater account of data structure (what occurs before when) or involve natural language processing. Further work is needed to explain the potential reasons for divergence and to refine and validate the approach for implementation in real-world settings.

Effect of contextual factors on the prevalence of diagnostic errors among patients managed by physicians of the same specialty: a single-centre retrospective observational study

BACKGROUND

There has been growing recognition that contextual factors influence the physician's cognitive processes. However, given that cognitive processes may depend on the physicians' specialties, the effects of contextual factors on diagnostic errors reported in previous studies could be confounded by difference in physicians.

OBJECTIVE

This study aimed to clarify whether contextual factors such as location and consultation type affect diagnostic accuracy.

METHODS

We reviewed the medical records of 1992 consecutive outpatients consulted by physicians from the Department of Diagnostic and Generalist Medicine in a university hospital between 1 January and 31 December 2019. Diagnostic processes were assessed using the Revised Safer Dx Instrument. Patients were categorised into three groups according to contextual factors (location and consultation type): (1) referred patients with scheduled visit to the outpatient department; (2) patients with urgent visit to the outpatient department; and (3) patients with emergency visit to the emergency room. The effect of the contextual factors on the prevalence of diagnostic errors was investigated using logistic regression analysis.

RESULTS

Diagnostic errors were observed in 12 of 534 referred patients with scheduled visit to the outpatient department (2.2%), 3 of 599 patients with urgent visit to the outpatient department (0.5%) and 13 of 859 patients with emergency visit to the emergency room (1.5%). Multivariable logistic regression analysis showed a significantly higher prevalence of diagnostic errors in referred patients with scheduled visit to the outpatient department than in patients with urgent visit to the outpatient department (OR 4.08, p=0.03), but no difference between patients with emergency and urgent visit to the emergency room and outpatient department, respectively.

CONCLUSION

Contextual factors such as consultation type may affect diagnostic errors; however, since the differences in the prevalence of diagnostic errors were small, the effect of contextual factors on diagnostic accuracy may be small in physicians working in different care settings.

Guideline concordance for timely chest imaging after new presentations of dyspnoea or haemoptysis in primary care: a retrospective cohort study

BACKGROUND

Guidelines recommend urgent chest X-ray for newly presenting dyspnoea or haemoptysis but there is little evidence about their implementation.

METHODS

We analysed linked primary care and hospital imaging data for patients aged 30+ years newly presenting with dyspnoea or haemoptysis in primary care during April 2012 to March 2017. We examined guideline-concordant management, defined as General Practitioner-ordered chest X-ray/CT carried out within 2 weeks of symptomatic presentation, and variation by sociodemographic characteristic and relevant medical history using logistic regression. Additionally, among patients diagnosed with cancer we described time to diagnosis, diagnostic route and stage at diagnosis by guideline-concordant status.

RESULTS

In total, 22 560/162 161 (13.9%) patients with dyspnoea and 4022/8120 (49.5%) patients with haemoptysis received guideline-concordant imaging within the recommended 2-week period. Patients with recent chest imaging pre-presentation were much less likely to receive imaging (adjusted OR 0.16, 95% CI 0.14-0.18 for dyspnoea, and adjusted OR 0.09, 95% CI 0.06-0.11 for haemoptysis). History of chronic obstructive pulmonary disease/asthma was also associated with lower odds of guideline concordance (dyspnoea: OR 0.234, 95% CI 0.225-0.242 and haemoptysis: 0.88, 0.79-0.97). Guideline-concordant imaging was lower among dyspnoea presenters with prior heart failure; current or ex-smokers; and those in more socioeconomically disadvantaged groups.The likelihood of lung cancer diagnosis within 12 months was greater among the guideline-concordant imaging group (dyspnoea: 1.1% vs 0.6%; haemoptysis: 3.5% vs 2.7%).

CONCLUSION

The likelihood of receiving urgent imaging concords with the risk of subsequent cancer diagnosis. Nevertheless, large proportions of dyspnoea and haemoptysis presenters do not receive prompt chest imaging despite being eligible, indicating opportunities for earlier lung cancer diagnosis.

Diagnostic Errors in Hospitalized Adults Who Died or Were Transferred to Intensive Care

Importance

Diagnostic errors contribute to patient harm, though few data exist to describe their prevalence or underlying causes among medical inpatients.

Objective

To determine the prevalence, underlying cause, and harms of diagnostic errors among hospitalized adults transferred to an intensive care unit (ICU) or who died.

Design, Setting, and Participants

Retrospective cohort study conducted at 29 academic medical centers in the US in a random sample of adults hospitalized with general medical conditions and who were transferred to an ICU, died, or both from January 1 to December 31, 2019. Each record was reviewed by 2 trained clinicians to determine whether a diagnostic error occurred (ie, missed or delayed diagnosis), identify diagnostic process faults, and classify harms. Multivariable models estimated association between process faults and diagnostic error. Opportunity for diagnostic error reduction associated with each fault was estimated using the adjusted proportion attributable fraction (aPAF). Data analysis was performed from April through September 2023.

Main Outcomes and Measures

Whether or not a diagnostic error took place, the frequency of underlying causes of errors, and harms associated with those errors.

Results

Of 2428 patient records at 29 hospitals that underwent review (mean [SD] patient age, 63.9 [17.0] years; 1107 [45.6%] female and 1321 male individuals [54.4%]), 550 patients (23.0%; 95% CI, 20.9%-25.3%) had experienced a diagnostic error. Errors were judged to have contributed to temporary harm, permanent harm, or death in 436 patients (17.8%; 95% CI, 15.9%-19.8%); among the 1863 patients who died, diagnostic error was judged to have contributed to death in 121 (6.6%; 95% CI, 5.3%-8.2%). In multivariable models examining process faults associated with any diagnostic error, patient assessment problems (aPAF, 21.4%; 95% CI, 16.4%-26.4%) and problems with test ordering and interpretation (aPAF, 19.9%; 95% CI, 14.7%-25.1%) had the highest opportunity to reduce diagnostic errors; similar ranking was seen in multivariable models examining harmful diagnostic errors.

Conclusions and Relevance

In this cohort study, diagnostic errors in hospitalized adults who died or were transferred to the ICU were common and associated with patient harm. Problems with choosing and interpreting tests and the processes involved with clinician assessment are high-priority areas for improvement efforts.

The PRIDx framework to engage payers in reducing diagnostic errors in healthcare

OBJECTIVES

No framework currently exists to guide how payers and providers can collaboratively develop and implement incentives to improve diagnostic safety. We conducted a literature review and interviews with subject matter experts to develop a multi-component 'Payer Relationships for Improving Diagnoses (PRIDx)' framework, that could be used to engage payers in diagnostic safety efforts.

CONTENT

The PRIDx framework, 1) conceptualizes diagnostic safety links to care provision, 2) illustrates ways to promote payer and provider engagement in the design and adoption of accountability mechanisms, and 3) explicates the use of data analytics. Certain approaches suggested by PRIDx were refined by subject matter expert interviewee perspectives.

SUMMARY

The PRIDx framework can catalyze public and private payers to take specific actions to improve diagnostic safety.

OUTLOOK

Implementation of the PRIDx framework requires new types of partnerships, including external support from public and private payer organizations, and requires creation of strong provider incentives without undermining providers' sense of professionalism and autonomy. PRIDx could help facilitate collaborative payer-provider approaches to improve diagnostic safety and generate research concepts, policy ideas, and potential innovations for engaging payers in diagnostic safety improvement activities.

Characteristics and contributing factors of diagnostic error in surgery: analysis of closed medico-legal cases and complaints in Canada

BACKGROUND

Diagnostic errors lead to patient harm; however, most research has been conducted in nonsurgical disciplines. We sought to characterize diagnostic error in the pre-, intra-, and postoperative surgical phases, describe their contributing factors, and quantify their impact related to patient harm.

METHODS

We performed a retrospective analysis of closed medico-legal cases and complaints using a database representing more than 95% of all Canadian physicians. We included cases if they involved a legal action or complaint that closed between 2014 and 2018 and involved a diagnostic error assigned by peer expert review to a surgeon.

RESULTS

We identified 387 surgical cases that involved a diagnostic error. The surgical specialties most often associated with diagnostic error were general surgery (n = 151, 39.0%), gynecology (n = 71, 18.3%), and orthopedic surgery (n = 48, 12.4%), but most surgical specialties were represented. Errors occurred more often in the postoperative phase (n = 171, 44.2%) than in the pre- (n = 127, 32.8%) or intra-operative (n = 120, 31.0%) phases of surgical care. More than 80% of the contributing factors for diagnostic errors were related to providers, with clinical decision-making being the principal contributing factor. Half of the contributing factors were related to the health care team (n = 194, 50.1%), the most common of which was communication breakdown. More than half of patients involved in a surgical diagnostic error experienced at least moderate harm, with 1 in 7 cases resulting in death.

CONCLUSION

In our cohort, diagnostic errors occurred in most surgical disciplines and across all surgical phases of care; contributing factors were commonly attributed to provider clinical decision-making and communication breakdown. Surgical patient safety efforts should include diagnostic errors with a focus on understanding and reducing errors in surgical clinical decision-making and improving communication.

Emergency Department Volume and Delayed Diagnosis of Pediatric Appendicitis: A Retrospective Cohort Study

OBJECTIVE

To determine the association of emergency department (ED) volume of children and delayed diagnosis of appendicitis.

BACKGROUND

Delayed diagnosis of appendicitis is common in children. The association between ED volume and delayed diagnosis is uncertain, but diagnosis-specific experience might improve diagnostic timeliness.

METHODS

Using Healthcare Cost and Utilization Project 8-state data from 2014 to 2019, we studied all children with appendicitis <18 years old in all EDs. The main outcome was probable delayed diagnosis: >75% likelihood that a delay occurred based on a previously validated measure. Hierarchical models tested associations between ED volumes and delay, adjusting for age, sex, and chronic conditions. We compared complication rates by delayed diagnosis occurrence.

RESULTS

Among 93,136 children with appendicitis, 3,293 (3.5%) had delayed diagnosis. Each 2-fold increase in ED volume was associated with a 6.9% (95% CI: 2.2, 11.3) decreased odds of delayed diagnosis. Each 2-fold increase in appendicitis volume was associated with a 24.1% (95% CI: 21.0, 27.0) decreased odds of delay. Those with delayed diagnosis were more likely to receive intensive care [odds ratio (OR): 1.81, 95% CI: 1.48, 2.21], have perforated appendicitis (OR: 2.81, 95% CI: 2.62, 3.02), undergo abdominal abscess drainage (OR: 2.49, 95% CI: 2.16, 2.88), have multiple abdominal surgeries (OR: 2.56, 95% CI: 2.13, 3.07), or develop sepsis (OR: 2.02, 95% CI: 1.61, 2.54).

CONCLUSIONS

Higher ED volumes were associated with a lower risk of delayed diagnosis of pediatric appendicitis. Delay was associated with complications.

Multicenter evaluation of a method to identify delayed diagnosis of diabetic ketoacidosis and sepsis in administrative data

OBJECTIVES

To derive a method of automated identification of delayed diagnosis of two serious pediatric conditions seen in the emergency department (ED): new-onset diabetic ketoacidosis (DKA) and sepsis.

METHODS

Patients under 21 years old from five pediatric EDs were included if they had two encounters within 7 days, the second resulting in a diagnosis of DKA or sepsis. The main outcome was delayed diagnosis based on detailed health record review using a validated rubric. Using logistic regression, we derived a decision rule evaluating the likelihood of delayed diagnosis using only characteristics available in administrative data. Test characteristics at a maximal accuracy threshold were determined.

RESULTS

Delayed diagnosis was present in 41/46 (89 %) of DKA patients seen twice within 7 days. Because of the high rate of delayed diagnosis, no characteristic we tested added predictive power beyond the presence of a revisit. For sepsis, 109/646 (17 %) of patients were deemed to have a delay in diagnosis. Fewer days between ED encounters was the most important characteristic associated with delayed diagnosis. In sepsis, our final model had a sensitivity for delayed diagnosis of 83.5 % (95 % confidence interval 75.2-89.9) and specificity of 61.3 % (95 % confidence interval 56.0-65.4).

CONCLUSIONS

Children with delayed diagnosis of DKA can be identified by having a revisit within 7 days. Many children with delayed diagnosis of sepsis may be identified using this approach with low specificity, indicating the need for manual case review.

Developing electronic clinical quality measures to assess the cancer diagnostic process

OBJECTIVE

Measures of diagnostic performance in cancer are underdeveloped. Electronic clinical quality measures (eCQMs) to assess quality of cancer diagnosis could help quantify and improve diagnostic performance.

MATERIALS AND METHODS

We developed 2 eCQMs to assess diagnostic evaluation of red-flag clinical findings for colorectal (CRC; based on abnormal stool-based cancer screening tests or labs suggestive of iron deficiency anemia) and lung (abnormal chest imaging) cancer. The 2 eCQMs quantified rates of red-flag follow-up in CRC and lung cancer using electronic health record data repositories at 2 large healthcare systems. Each measure used clinical data to identify abnormal results, evidence of appropriate follow-up, and exclusions that signified follow-up was unnecessary. Clinicians reviewed 100 positive and 20 negative randomly selected records for each eCQM at each site to validate accuracy and categorized missed opportunities related to system, provider, or patient factors.

RESULTS

We implemented the CRC eCQM at both sites, while the lung cancer eCQM was only implemented at the VA due to lack of structured data indicating level of cancer suspicion on most chest imaging results at Geisinger. For the CRC eCQM, the rate of appropriate follow-up was 36.0% (26 746/74 314 patients) in the VA after removing clinical exclusions and 41.1% at Geisinger (1009/2461 patients; P < .001). Similarly, the rate of appropriate evaluation for lung cancer in the VA was 61.5% (25 166/40 924 patients). Reviewers most frequently attributed missed opportunities at both sites to provider factors (84 of 157).

CONCLUSIONS

We implemented 2 eCQMs to evaluate the diagnostic process in cancer at 2 large health systems. Health care organizations can use these eCQMs to monitor diagnostic performance related to cancer.

Lessons in clinical reasoning - pitfalls, myths, and pearls: a case of crushing, substernal chest pain

OBJECTIVES

Diagnostic error is not uncommon and diagnostic accuracy can be improved with the use of problem representation, pre-test probability, and Bayesian analysis for improved clinical reasoning.

CASE PRESENTATION

A 48-year-old female presented as a transfer from another Emergency Department (ED) to our ED with crushing, substernal pain associated with dyspnea, diaphoresis, nausea, and a tingling sensation down both arms with radiation to the back and neck. Troponins were elevated along with an abnormal electrocardiogram. A negative myocardial perfusion scan led to the patient's discharge. The patient presented to the ED 10 days later with an anterior ST-elevation myocardial infarction.

CONCLUSIONS

An overemphasis on a single testing modality led to diagnostic error and a severe event. The use of pre-test probabilities guided by history-taking can lead to improved interpretation of test results, ultimately improving diagnostic accuracy and preventing serious medical errors.

Improving the diagnostic process for patients with possible bladder and kidney cancer: a mixed-methods study to identify potential missed diagnostic opportunities

BACKGROUND

Patients with bladder and kidney cancer may experience diagnostic delays.

AIM

To identify patterns of suboptimal care and contributors of potential missed diagnostic opportunities (MDOs).

DESIGN AND SETTING

Prospective, mixed-methods study recruiting participants from nine general practices in Eastern England between June 2018 and October 2019.

METHOD

Patients with possible bladder and kidney cancer were identified using eligibility criteria based on National Institute for Health and Care Excellence (NICE) guidelines for suspected cancer. Primary care records were reviewed at recruitment and at 1 year for data on symptoms, tests, referrals, and diagnosis. Referral predictors were examined using logistic regression. Semi-structured interviews were undertaken with 15 patients to explore their experiences of the diagnostic process, and these were analysed thematically.

RESULTS

Participants (n = 940) were mostly female (n = 657, 69.9%), with a median age of 71 years (interquartile range 64-77 years). In total, 268 (28.5%) received a referral and 465 (48.5%) had a final diagnosis of urinary tract infection (UTI). There were 33 (3.5%) patients who were diagnosed with cancer, including prostate (n = 17), bladder (n = 7), and upper urothelial tract (n = 1) cancers. Among referred patients, those who had a final diagnosis of UTI had the longest time to referral (median 81.5 days). Only one-third of patients with recurrent UTIs were referred despite meeting NICE referral guidelines. Qualitative findings revealed barriers during the diagnostic process, including inadequate clinical examination, female patients given repeated antibiotics without clinical reviews, and suboptimal communication of test results to patients.

CONCLUSION

Older females with UTIs might be at increased risk of MDOs for cancer. Targeting barriers during the initial diagnostic assessment and follow-up might improve quality of diagnosis.

Opportunities for Diagnostic Improvement Among Pediatric Hospital Readmissions

OBJECTIVES

Diagnostic errors, termed "missed opportunities for improving diagnosis" (MOIDs), are known sources of harm in children but have not been well characterized in pediatric hospital medicine. Our objectives were to systematically identify and describe MOIDs among general pediatric patients who experienced hospital readmission, outline improvement opportunities, and explore factors associated with increased risk of MOID.

PATIENTS AND METHODS

Our retrospective cohort study included unplanned readmissions within 15 days of discharge from a freestanding children's hospital (October 2018-September 2020). Health records from index admissions and readmissions were independently reviewed and discussed by practicing inpatient physicians to identify MOIDs using an established instrument, SaferDx. MOIDs were evaluated using a diagnostic-specific tool to identify improvement opportunities within the diagnostic process.

RESULTS

MOIDs were identified in 22 (6.3%) of 348 readmissions. Opportunities for improvement included: delay in considering the correct diagnosis (n = 11, 50%) and failure to order needed test(s) (n = 10, 45%). Patients with MOIDs were older (median age: 3.8 [interquartile range 1.5-11.2] vs 1.0 [0.3-4.9] years) than patients without MOIDs but similar in sex, primary language, race, ethnicity, and insurance type. We did not identify conditions associated with higher risk of MOID. Lower respiratory tract infections accounted for 26% of admission diagnoses but only 1 (4.5%) case of MOID.

CONCLUSIONS

Standardized review of pediatric readmissions identified MOIDs and opportunities for improvement within the diagnostic process, particularly in clinician decision-making. We identified conditions with low incidence of MOID. Further work is needed to better understand pediatric populations at highest risk for MOID.

Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach

Objective

To describe a user-centered approach to develop, pilot test, and refine requirements for 3 electronic health record (EHR)-integrated interventions that target key diagnostic process failures in hospitalized patients.

Materials and Methods

Three interventions were prioritized for development: a Diagnostic Safety Column (DSC) within an EHR-integrated dashboard to identify at-risk patients; a Diagnostic Time-Out (DTO) for clinicians to reassess the working diagnosis; and a Patient Diagnosis Questionnaire (PDQ) to gather patient concerns about the diagnostic process. Initial requirements were refined from analysis of test cases with elevated risk predicted by DSC logic compared to risk perceived by a clinician working group; DTO testing sessions with clinicians; PDQ responses from patients; and focus groups with clinicians and patient advisors using storyboarding to model the integrated interventions. Mixed methods analysis of participant responses was used to identify final requirements and potential implementation barriers.

Results

Final requirements from analysis of 10 test cases predicted by the DSC, 18 clinician DTO participants, and 39 PDQ responses included the following: DSC configurable parameters (variables, weights) to adjust baseline risk estimates in real-time based on new clinical data collected during hospitalization; more concise DTO wording and flexibility for clinicians to conduct the DTO with or without the patient present; and integration of PDQ responses into the DSC to ensure closed-looped communication with clinicians. Analysis of focus groups confirmed that tight integration of the interventions with the EHR would be necessary to prompt clinicians to reconsider the working diagnosis in cases with elevated diagnostic error (DE) risk or uncertainty. Potential implementation barriers included alert fatigue and distrust of the risk algorithm (DSC); time constraints, redundancies, and concerns about disclosing uncertainty to patients (DTO); and patient disagreement with the care team's diagnosis (PDQ).

Discussion

A user-centered approach led to evolution of requirements for 3 interventions targeting key diagnostic process failures in hospitalized patients at risk for DE.

Conclusions

We identify challenges and offer lessons from our user-centered design process.

Do malpractice claim clinical case vignettes enhance diagnostic accuracy and acceptance in clinical reasoning education during GP training?

BACKGROUND

Using malpractice claims cases as vignettes is a promising approach for improving clinical reasoning education (CRE), as malpractice claims can provide a variety of content- and context-rich examples. However, the effect on learning of adding information about a malpractice claim, which may evoke a deeper emotional response, is not yet clear. This study examined whether knowing that a diagnostic error resulted in a malpractice claim affects diagnostic accuracy and self-reported confidence in the diagnosis of future cases. Moreover, suitability of using erroneous cases with and without a malpractice claim for CRE, as judged by participants, was evaluated.

METHODS

In the first session of this two-phased, within-subjects experiment, 81 first-year residents of general practice (GP) were exposed to both erroneous cases with (M) and erroneous cases without (NM) malpractice claim information, derived from a malpractice claims database. Participants rated suitability of the cases for CRE on a five-point Likert scale. In the second session, one week later, participants solved four different cases with the same diagnoses. Diagnostic accuracy was measured with three questions, scored on a 0-1 scale: (1) What is your next step? (2) What is your differential diagnosis? (3) What is your most probable diagnosis and what is your level of certainty on this? Both subjective suitability and diagnostic accuracy scores were compared between the versions (M and NM) using repeated measures ANOVA.

RESULTS

There were no differences in diagnostic accuracy parameters (M vs. NM next step: 0.79 vs. 0.77, p = 0.505; differential diagnosis 0.68 vs. 0.75, p = 0.072; most probable diagnosis 0.52 vs. 0.57, p = 0.216) and self-reported confidence (53.7% vs. 55.8% p = 0.390) of diagnoses previously seen with or without malpractice claim information. Subjective suitability- and complexity scores for the two versions were similar (suitability: 3.68 vs. 3.84, p = 0.568; complexity 3.71 vs. 3.88, p = 0.218) and significantly increased for higher education levels for both versions.

CONCLUSION

The similar diagnostic accuracy rates between cases studied with or without malpractice claim information suggests both versions are equally effective for CRE in GP training. Residents judged both case versions to be similarly suitable for CRE; both were considered more suitable for advanced than for novice learners.

Physicians' Perspectives on a Multi-Dimensional Model for the Roles of Electronic Health Records in Approaching a Proper Differential Diagnosis

Many healthcare organizations have adopted Electronic Health Records (EHRs) to improve the quality of care and help physicians make proper clinical decisions. The vital roles of EHRs can support the accuracy of diagnosis, suggest, and rationalize the provided care to patients. This study aims to understand the roles of EHRs in approaching proper differential diagnosis and optimizing patient safety. This study utilized a cross-sectional survey-based descriptive research design to assess physicians' perceptions of the roles of EHRs on diagnosis quality and safety. Physicians working in tertiary hospitals in Saudi Arabia were surveyed. Three hundred and fifty-one participants were included in the study, of which 61% were male. The main participants were family/general practice (22%), medicine, general (14%), and OB/GYN (12%). Overall, 66% of the participants ranked themselves as IT competent, most of the participants underwent IT self-guided learning, and 65% of the participants always used the system. The results generally reveal positive physicians' perceptions toward the roles of the EHR system on diagnosis quality and safety. There was a statistically significant relationship between user characteristics and the roles of the EHR by enhancing access to care, patient-physician encounter, clinical reasoning, diagnostic testing and consultation, follow-up, and diagnostic safety functionality. The study participants demonstrate positive perceptions of physicians toward the roles of the EHR system in approaching differential diagnosis. Yet, areas of improvement in the design and using EHRs are emphasized.

Types of diagnostic errors reported by paediatric emergency providers in a global paediatric emergency care research network

Background

Diagnostic errors, reframed as missed opportunities for improving diagnosis (MOIDs), are poorly understood in the paediatric emergency department (ED) setting. We investigated the clinical experience, harm and contributing factors related to MOIDs reported by physicians working in paediatric EDs.

Methods

We developed a web-based survey in which physicians participating in the international Paediatric Emergency Research Network representing five out of six WHO regions, described examples of MOIDs involving their own or a colleague’s patients. Respondents provided case summaries and answered questions regarding harm and factors contributing to the event.

Results

Of 1594 physicians surveyed, 412 (25.8%) responded (mean age=43 years (SD=9.2), 42.0% female, mean years in practice=12 (SD=9.0)). Patient presentations involving MOIDs had common undifferentiated symptoms at initial presentation, including abdominal pain (21.1%), fever (17.2%) and vomiting (16.5%). Patients were discharged from the ED with commonly reported diagnoses, including acute gastroenteritis (16.7%), viral syndrome (10.2%) and constipation (7.0%). Most reported MOIDs (65%) were detected on ED return visits (46% within 24 hours and 76% within 72 hours). The most common reported MOID was appendicitis (11.4%), followed by brain tumour (4.4%), meningitis (4.4%) and non-accidental trauma (4.1%). More than half (59.1%) of the reported MOIDs involved the patient/parent–provider encounter (eg, misinterpreted/ignored history or an incomplete/inadequate physical examination). Types of MOIDs and contributing factors did not differ significantly between countries. More than half of patients had either moderate (48.7%) or major (10%) harm due to the MOID.

Conclusions

An international cohort of paediatric ED physicians reported several MOIDs, often in children who presented to the ED with common undifferentiated symptoms. Many of these were related to patient/parent–provider interaction factors such as suboptimal history and physical examination. Physicians’ personal experiences offer an underexplored source for investigating and mitigating diagnostic errors in the paediatric ED.

Clinical value of second opinions in oncology: A retrospective review of changes in diagnosis and treatment recommendations

BACKGROUND

Data on the clinical value of second opinions in oncology are limited. We examined diagnostic and treatment changes resulting from second opinions and the expected impact on morbidity and prognosis.

METHODS

This retrospective cohort study included patients presenting in 2018 to a high-volume cancer center for second opinions about newly diagnosed colorectal, head and neck, lung, and myeloma cancers or abnormal results. Two sub-specialty physicians from each cancer type reviewed 30 medical records (120 total) using a process and detailed data collection guide meant to mitigate institutional bias. The primary outcome measure was the rate of treatment changes that were "clinically meaningful", i.e., expected to impact morbidity and/or prognosis. Among those with treatment changes, another outcome measure was the rate of clinically meaningful diagnostic changes that led to treatment change.

RESULTS

Of 120 cases, forty-two had clinically meaningful changes in treatment with positive expected outcomes (7 colorectal, 17 head and neck, 11 lung, 7 myeloma; 23-57%). Two patients had negative expected outcomes from having sought a second opinion, with worse short-term morbidity and unchanged long-term morbidity and prognosis. All those with positive expected outcomes had improved expected morbidity (short- and/or long-term); 11 (0-23%) also had improved expected prognosis. Nine involved a shift from treatment to observation; 21 involved eliminating or reducing the extent of surgery, compared to 6 adding surgery or increasing its extent. Of the 42 with treatment changes, 13 were due to clinically meaningful diagnostic changes (1 colorectal, 5 head and neck, 3 lung, 4 myeloma; 3%-17%) .

CONCLUSIONS

Second-opinion consultations sometimes add clinical value by improving expected prognoses; more often, they offer treatment de-escalations, with corresponding reductions in expected short- and/or long-term morbidity. Future research could identify subgroups of patients most likely to benefit from second opinions.

A clinically-guided unsupervised clustering approach to recommend symptoms of disease associated with diagnostic opportunities

OBJECTIVES

A first step in studying diagnostic delays is to select the signs, symptoms and alternative diseases that represent missed diagnostic opportunities. Because this step is labor intensive requiring exhaustive literature reviews, we developed machine learning approaches to mine administrative data sources and recommend conditions for consideration. We propose a methodological approach to find diagnostic codes that exhibit known patterns of diagnostic delays and apply this to the diseases of tuberculosis and appendicitis.

METHODS

We used the IBM MarketScan Research Databases, and consider the initial symptoms of cough before tuberculosis and abdominal pain before appendicitis. We analyze diagnosis codes during healthcare visits before the index diagnosis, and use k-means clustering to recommend conditions that exhibit similar trends to the initial symptoms provided. We evaluate the clinical plausibility of the recommended conditions and the corresponding number of possible diagnostic delays based on these diseases.

RESULTS

For both diseases of interest, the clustering approach suggested a large number of clinically-plausible conditions to consider (e.g., fever, hemoptysis, and pneumonia before tuberculosis). The recommended conditions had a high degree of precision in terms of clinical plausibility: >70% for tuberculosis and >90% for appendicitis. Including these additional clinically-plausible conditions resulted in more than twice the number of possible diagnostic delays identified.

CONCLUSIONS

Our approach can mine administrative datasets to detect patterns of diagnostic delay and help investigators avoid under-identifying potential missed diagnostic opportunities. In addition, the methods we describe can be used to discover less-common presentations of diseases that are frequently misdiagnosed.

Providers' and Patients' Perspectives on Diagnostic Errors in the Acute Care Setting

BACKGROUND

Diagnostic errors (DEs) have been studied extensively in ambulatory care, but less work has been done in the acute care setting. In this study, the authors examined health care providers' and patients' perspectives about the classification of DEs, the main causes and scope of DEs in acute care, the main gaps in current systems, and the need for innovative solutions.

METHODS

A qualitative mixed methods study was conducted, including semistructured interviews with health care providers and focus groups with patient advisors. Using grounded theory approach, thematic categories were derived from the interviews and focus groups.

RESULTS

The research team conducted interviews with 17 providers and two focus groups with seven patient advisors. Both providers and patient advisors struggled to define and describe DEs in acute care settings. Although participants agreed that DEs pose a significant risk to patient safety, their perception of the frequency of DEs was mixed. Most participants identified communication failures, lack of comfort with diagnostic uncertainty, incorrect clinical evaluation, and cognitive load as key causes of DEs. Most respondents believed that non-information technology (IT) tools and processes (for example, communication improvement strategies) could significantly reduce DEs.

CONCLUSION

The study findings represent an important supplement to our understanding of DEs in acute care settings and the advancement of a culture of patient safety in the context of patient-centered care and patient engagement. Health care organizations should consider the key factors identified in this study when trying to create a culture that engages clinicians and patients in reducing DEs.

Developing the Safer Dx Checklist of Ten Safety Recommendations for Health Care Organizations to Address Diagnostic Errors

BACKGROUND

Most health care organizations (HCOs) find diagnostic errors hard to address. The research team developed a checklist (the Safer Dx Checklist) of 10 high-priority safety practices HCOs can use to conduct a proactive risk assessment to address diagnostic error.

METHODS

First, the team identified potential practices based on reviews of recent literature, reports by national and international organizations, and interviews with quality/safety leaders. Then a Delphi panel was conducted, followed by an online expert panel, to prioritize 10 practices. The prioritization process considered impact on safety and feasibility of practice implementation within a one- to three-year time frame. Finally, cognitive walkthroughs were conducted for a face-validity check with end users. The team also conducted content analysis in each step to look for themes that influenced prioritization or checklist implementation.

RESULTS

A total of 71 practices for prioritization were identified through the Delphi panel of 28 experts; 65% of participants reached consensus on 28 practices. A multidisciplinary panel of 10 experts helped prioritize and refine the top 10 practices, which were then developed into a checklist paired with implementation guidance. Practices included themes related to creating organizational and leadership accountability for improving diagnosis, including patients in diagnostic safety work, and developing and implementing organizational infrastructure for measurement and improvement activities. Qualitative analysis revealed insights for implementation. End users at three different HCOs helped refine implementation guidance for the checklist.

CONCLUSION

The researchers identified 10 safety practices to help organizations conduct a proactive, systematic assessment of risks to timely and accurate diagnosis. The Safer Dx Checklist can enable HCOs to begin implementing strategies to address diagnostic error.

A structured approach to EHR surveillance of diagnostic error in acute care: an exploratory analysis of two institutionally-defined case cohorts

OBJECTIVES

To test a structured electronic health record (EHR) case review process to identify diagnostic errors (DE) and diagnostic process failures (DPFs) in acute care.

METHODS

We adapted validated tools (Safer Dx, Diagnostic Error Evaluation Research [DEER] Taxonomy) to assess the diagnostic process during the hospital encounter and categorized 13 postulated e-triggers. We created two test cohorts of all preventable cases (n=28) and an equal number of randomly sampled non-preventable cases (n=28) from 365 adult general medicine patients who expired and underwent our institution's mortality case review process. After excluding patients with a length of stay of more than one month, each case was reviewed by two blinded clinicians trained in our process and by an expert panel. Inter-rater reliability was assessed. We compared the frequency of DE contributing to death in both cohorts, as well as mean DPFs and e-triggers for DE positive and negative cases within each cohort.

RESULTS

Twenty-seven (96.4%) preventable and 24 (85.7%) non-preventable cases underwent our review process. Inter-rater reliability was moderate between individual reviewers (Cohen's kappa 0.41) and substantial with the expert panel (Cohen's kappa 0.74). The frequency of DE contributing to death was significantly higher for the preventable compared to the non-preventable cohort (56% vs. 17%, OR 6.25 [1.68, 23.27], p<0.01). Mean DPFs and e-triggers were significantly and non-significantly higher for DE positive compared to DE negative cases in each cohort, respectively.

CONCLUSIONS

We observed substantial agreement among final consensus and expert panel reviews using our structured EHR case review process. DEs contributing to death associated with DPFs were identified in institutionally designated preventable and non-preventable cases. While e-triggers may be useful for discriminating DE positive from DE negative cases, larger studies are required for validation. Our approach has potential to augment institutional mortality case review processes with respect to DE surveillance.

Creating a Learning Health System for Improving Diagnostic Safety: Pragmatic Insights from US Health Care Organizations

OBJECTIVE

To identify challenges and pragmatic strategies for improving diagnostic safety at an organizational level using concepts from learning health systems METHODS: We interviewed 32 safety leaders across the USA on how their organizations approach diagnostic safety. Participants were recruited through email and represented geographically diverse academic and non-academic settings. The interview included questions on culture of reporting and learning from diagnostic errors; data gathering and analysis activities; diagnostic training and educational activities; and engagement of clinical leadership, staff, patients, and families in diagnostic safety activities. We conducted an inductive content analysis of interview transcripts and two reviewers coded all data.

RESULTS

Of 32 participants, 12 reported having a specific program to address diagnostic errors. Multiple barriers to implement diagnostic safety activities emerged: serious concerns about psychological safety associated with diagnostic error; lack of infrastructure for measurement, monitoring, and improvement activities related to diagnosis; lack of leadership investment, which was often diverted to competing priorities related to publicly reported measures or other incentives; and lack of dedicated teams to work on diagnostic safety. Participants provided several strategies to overcome barriers including adapting trigger tools to identify safety events, engaging patients in diagnostic safety, and appointing dedicated diagnostic safety champions.

CONCLUSIONS

Several foundational building blocks related to learning health systems could inform organizational efforts to reduce diagnostic error. Promoting an organizational culture specific to diagnostic safety, using science and informatics to improve measurement and analysis, leadership incentives to build institutional capacity to address diagnostic errors, and patient engagement in diagnostic safety activities can enable progress.

Contributors to Diagnostic Error or Delay in the Acute Care Setting: A Survey of Clinical Stakeholders

Diagnostic error or delay (DEOD) is common in the acute care setting and results in poor patient outcomes. Many factors contribute to DEOD, but little is known about how contributors may differ across acute care areas and professional roles. As part of a sequential exploratory mixed methods research study, we surveyed acute care clinical stakeholders about the frequency with which different factors contribute to DEOD. Survey respondents could also propose solutions in open text fields. N = 220 clinical stakeholders completed the survey. Care Team Interactions, Systems and Process, Patient, Provider, and Cognitive factors were perceived to contribute to DEOD with similar frequency. Organization and Infrastructure factors were perceived to contribute to DEOD significantly less often. Responses did not vary across acute care setting. Physicians perceived Cognitive factors to contribute to DEOD more frequently compared to those in other roles. Commonly proposed solutions included: technological solutions, organization level fixes, ensuring staff know and are encouraged to work to the full scope of their role, and cultivating a culture of collaboration and respect. Multiple factors contribute to DEOD with similar frequency across acute care areas, suggesting the need for a multi-pronged approach that can be applied across acute care areas.

Development and Usability Testing of the Agency for Healthcare Research and Quality Common Formats to Capture Diagnostic Safety Events

OBJECTIVES

A lack of consensus around definitions and reporting standards for diagnostic errors limits the extent to which healthcare organizations can aggregate, analyze, share, and learn from these events. In response to this problem, the Agency for Healthcare Research and Quality (AHRQ) began the development of the Common Formats for Event Reporting for Diagnostic Safety Events (CFER-DS). We conducted a usability assessment of the draft CFER-DS to inform future revision and implementation.

METHODS

We recruited a purposive sample of quality and safety personnel working in 8 U.S. healthcare organizations. Participants were invited to use the CFER-DS to simulate reporting for a minimum of 5 cases of diagnostic safety events and then provide written and verbal qualitative feedback. Analysis focused on participants' perceptions of content validity, ease of use, and potential for implementation.

RESULTS

Estimated completion time was 30 to 90 minutes per event. Participants shared generally positive feedback about content coverage and item clarity but identified reporter burden as a potential concern. Participants also identified opportunities to clarify several conceptual definitions, ensure applicability across different care settings, and develop guidance to operationalize use of CFER-DS. Findings led to refinement of content and supplementary materials to facilitate implementation.

CONCLUSIONS

Standardized definitions of diagnostic safety events and reporting standards for contextual information and contributing factors can help capture and analyze diagnostic safety events. In addition to usability testing, additional feedback from the field will ensure that AHRQ's CFER-DS is useful to a broad range of users for learning and safety improvement.

Inviting patients to identify diagnostic concerns through structured evaluation of their online visit notes

BACKGROUND

The 21st Century Cures Act mandates patients' access to their electronic health record (EHR) notes. To our knowledge, no previous work has systematically invited patients to proactively report diagnostic concerns while documenting and tracking their diagnostic experiences through EHR-based clinician note review.

OBJECTIVE

To test if patients can identify concerns about their diagnosis through structured evaluation of their online visit notes.

METHODS

In a large integrated health system, patients aged 18-85 years actively using the patient portal and seen between October 2019 and February 2020 were invited to respond to an online questionnaire if an EHR algorithm detected any recent unexpected return visit following an initial primary care consultation ("at-risk" visit). We developed and tested an instrument (Safer Dx Patient Instrument) to help patients identify concerns related to several dimensions of the diagnostic process based on notes review and recall of recent "at-risk" visits. Additional questions assessed patients' trust in their providers and their general feelings about the visit. The primary outcome was a self-reported diagnostic concern. Multivariate logistic regression tested whether the primary outcome was predicted by instrument variables.

RESULTS

Of 293 566 visits, the algorithm identified 1282 eligible patients, of whom 486 responded. After applying exclusion criteria, 418 patients were included in the analysis. Fifty-one patients (12.2%) identified a diagnostic concern. Patients were more likely to report a concern if they disagreed with statements "the care plan the provider developed for me addressed all my medical concerns" [odds ratio (OR), 2.65; 95% confidence interval [CI], 1.45-4.87) and "I trust the provider that I saw during my visit" (OR, 2.10; 95% CI, 1.19-3.71) and agreed with the statement "I did not have a good feeling about my visit" (OR, 1.48; 95% CI, 1.09-2.01).

CONCLUSION

Patients can identify diagnostic concerns based on a proactive online structured evaluation of visit notes. This surveillance strategy could potentially improve transparency in the diagnostic process.

Diagnostic Error in Pediatrics: A Narrative Review

A priority topic for patient safety research is diagnostic errors. However, despite the significant growth in awareness of their unacceptably high incidence and associated harm, a relative paucity of large, high-quality studies of diagnostic error in pediatrics exists. In this narrative review, we present what is known about the incidence and epidemiology of diagnostic error in pediatrics as well as the established research methods for identifying, evaluating, and reducing diagnostic errors, including their strengths and weaknesses. Additionally, we highlight that pediatric diagnostic error remains an area in need of both innovative research and quality improvement efforts to apply learnings from a rapidly growing evidence base. We propose several key research questions aimed at addressing persistent gaps in the pediatric diagnostic error literature that focus on the foundational knowledge needed to inform effective interventions to reduce the incidence of diagnostic errors and their associated harm. Additional research is needed to better establish the epidemiology of diagnostic error in pediatrics, including identifying high-risk clinical scenarios, patient populations, and groups of diagnoses. A critical need exists for validated measures of both diagnostic errors and diagnostic processes that can be adapted for different clinical settings and standardized for use across varying institutions. Pediatric researchers will need to work collaboratively on large-scale, high-quality studies to accomplish the ultimate goal of reducing diagnostic errors and their associated harm in children by addressing these fundamental gaps in knowledge.