Diagnostic error in medicine: analysis of 583 physician-reported errors

What is diagnostic safety? A review of safety science paradigms and rethinking paths to improving diagnosis

Diagnostic errors in health care are a global threat to patient safety. Researchers have traditionally focused diagnostic safety efforts on identifying errors and their causes with the goal of reducing diagnostic error rates. More recently, complementary approaches to diagnostic errors have focused on improving diagnostic performance drawn from the safety sciences. These approaches have been called Safety-II and Safety-III, which apply resilience engineering and system safety principles, respectively. This review explores the safety science paradigms and their implications for analyzing diagnostic errors, highlighting their distinct yet complementary perspectives. The integration of Safety-I, Safety-II, and Safety-III paradigms presents a promising pathway for improving diagnosis. Diagnostic researchers not yet familiar with the various approaches and potential paradigm shift in diagnostic safety research may use this review as a starting point for considering Safety-I, Safety-II, and Safety-III in their efforts to both reduce diagnostic errors and improve diagnostic performance.

Effect of contextual factors on the prevalence of diagnostic errors among patients managed by physicians of the same specialty: a single-centre retrospective observational study

BACKGROUND

There has been growing recognition that contextual factors influence the physician's cognitive processes. However, given that cognitive processes may depend on the physicians' specialties, the effects of contextual factors on diagnostic errors reported in previous studies could be confounded by difference in physicians.

OBJECTIVE

This study aimed to clarify whether contextual factors such as location and consultation type affect diagnostic accuracy.

METHODS

We reviewed the medical records of 1992 consecutive outpatients consulted by physicians from the Department of Diagnostic and Generalist Medicine in a university hospital between 1 January and 31 December 2019. Diagnostic processes were assessed using the Revised Safer Dx Instrument. Patients were categorised into three groups according to contextual factors (location and consultation type): (1) referred patients with scheduled visit to the outpatient department; (2) patients with urgent visit to the outpatient department; and (3) patients with emergency visit to the emergency room. The effect of the contextual factors on the prevalence of diagnostic errors was investigated using logistic regression analysis.

RESULTS

Diagnostic errors were observed in 12 of 534 referred patients with scheduled visit to the outpatient department (2.2%), 3 of 599 patients with urgent visit to the outpatient department (0.5%) and 13 of 859 patients with emergency visit to the emergency room (1.5%). Multivariable logistic regression analysis showed a significantly higher prevalence of diagnostic errors in referred patients with scheduled visit to the outpatient department than in patients with urgent visit to the outpatient department (OR 4.08, p=0.03), but no difference between patients with emergency and urgent visit to the emergency room and outpatient department, respectively.

CONCLUSION

Contextual factors such as consultation type may affect diagnostic errors; however, since the differences in the prevalence of diagnostic errors were small, the effect of contextual factors on diagnostic accuracy may be small in physicians working in different care settings.

Feedback Loop Failure Modes in Medical Diagnosis: How Biases Can Emerge and Be Reinforced

BACKGROUND

Medical diagnosis in practice connects to research through continuous feedback loops: Studies of diagnosed cases shape our understanding of disease, which shapes future diagnostic practice. Without accounting for an imperfect and complex diagnostic process in which some cases are more likely to be diagnosed correctly (or diagnosed at all), the feedback loop can inadvertently exacerbate future diagnostic errors and biases.

FRAMEWORK

A feedback loop failure occurs if misleading evidence about disease etiology encourages systematic errors that self-perpetuate, compromising future diagnoses and patient care. This article defines scenarios for feedback loop failure in medical diagnosis.

DESIGN

Through simulated cases, we characterize how disease incidence, presentation, and risk factors can be misunderstood when observational data are summarized naive to biases arising from diagnostic error. A fourth simulation extends to a progressive disease.

RESULTS

When severe cases of a disease are diagnosed more readily, less severe cases go undiagnosed, increasingly leading to underestimation of the prevalence and heterogeneity of the disease presentation. Observed differences in incidence and symptoms between demographic groups may be driven by differences in risk, presentation, the diagnostic process itself, or a combination of these. We suggested how perceptions about risk factors and representativeness may drive the likelihood of diagnosis. Differing diagnosis rates between patient groups can feed back to increasingly greater diagnostic errors and disparities in the timing of diagnosis and treatment.

CONCLUSIONS

A feedback loop between past data and future medical practice may seem obviously beneficial. However, under plausible scenarios, poorly implemented feedback loops can degrade care. Direct summaries from observational data based on diagnosed individuals may be misleading, especially concerning those symptoms and risk factors that influence the diagnostic process itself.

HIGHLIGHTS

Current evidence about a disease can (and should) influence the diagnostic process. A feedback loop failure may occur if biased "evidence" encourages diagnostic errors, leading to future errors in the evidence base.When diagnostic accuracy varies for mild versus severe cases or between demographic groups, incorrect conclusions about disease prevalence and presentation will result without specifically accounting for such variability.Use of demographic characteristics in the diagnostic process should be done with careful justification, in particular avoiding potential cognitive biases and overcorrection.

The influence of viewing time on visual diagnostic accuracy: Less is more

BACKGROUND

Understanding the factors that contribute to diagnostic errors is critical if we are to correct or prevent them. Some scholars influenced by the default interventionist dual-process theory of cognition (dual-process theory) emphasise a narrow focus on individual clinician's faulty reasoning as a significant contributor. In this paper, we examine the validity of claims that dual process theory is a key to error reduction.

METHODS

We examined the relationship between a clinical experience (staff and resident physicians) and viewing time on accuracy for categorising chest X-rays (CXRs) and electrocardiograms (ECGs). In two studies, participants categorised images as normal or abnormal, presented at viewing times of 175, 250, 500 and 1000 ms, to encourage System 1 processing. Study 2 extended viewing times to 1, 5, 10 and 20 s to allow time for System 2 processing and a diagnosis. Descriptives and repeated measures analysis of variance were used to analyse the proportion of true and false positive rates (TP and FP) as well as correct diagnoses.

RESULTS

In Study 1, physicians were able to detect abnormal CXRs (0.78) and ECGs (0.67) with relatively high accuracy. The effect of experience was found for ECGs only, as staff physicians (0.71, 95% CI = 0.66-0.75) had higher ECG TP than resident physicians (0.63, 95% CI = 0.58-0.68) in Study 1, and staff had lower ECG FP (0.10, 95% CI = 0.03-0.18) than resident physicians (0.27, 95% CI = 0.20-0.33) in Study 2. In other comparisons, experience was equivocal for ECG FPs and CXR TPs and FPs. In Study 2, overall diagnostic accuracy was similar for both ECGs and CXRs, (0.74). There were small interactions between experience and time for TP in ECGs and FP in CXRs, which are discussed further in the discussion and offer insights into the relationship between processing and experience.

CONCLUSION

Overall, our findings raise concerns about the practical application of models that link processing type to diagnostic error, or to specific diagnostic error reduction strategies.

Diagnostic error in mental health: a review

Diagnostic errors are associated with patient harm and suboptimal outcomes. Despite national scientific efforts to advance definition, measurement and interventions for diagnostic error, diagnosis in mental health is not well represented in this ongoing work. We aimed to summarise the current state of research on diagnostic errors in mental health and identify opportunities to align future research with the emerging science of diagnostic safety. We review conceptual considerations for defining and measuring diagnostic error, the application of these concepts to mental health settings, and the methods and subject matter focus of recent studies of diagnostic error in mental health. We found that diagnostic error is well understood to be a problem in mental healthcare. Although few studies used clear definitions or frameworks for understanding diagnostic error in mental health, several studies of missed, wrong, delayed and disparate diagnosis of common mental disorders have identified various avenues for future research and development. Nevertheless, a lack of clear consensus on how to conceptualise, define and measure errors in diagnosis will pose a barrier to advancement. Further research should focus on identifying preventable missed opportunities in the diagnosis of mental disorders, which may uncover generalisable opportunities for improvement.

Stigmatizing Language, Patient Demographics, and Errors in the Diagnostic Process

This cohort study assesses the association between stigmatizing language, demographic characteristics, and errors in the diagnostic process among hospitalized adults.

Patients' Experiences of Dental Diagnostic Failures: A Qualitative Study Using Social Media

OBJECTIVE

Despite the many advancements made in patient safety over the past decade, combating diagnostic errors (DEs) remains a crucial, yet understudied initiative toward improvement. This study sought to understand the perception of dental patients who have experienced a dental diagnostic failure (DDF) and to identify patient-centered strategies to help reduce future occurrences of DDF.

METHODS

Through social media recruitment, we conducted a screening survey, initial assessment, and 67 individual patient interviews to capture the effects of misdiagnosis, missed diagnosis, or delayed diagnosis on patient lives. Audio recordings of patient interviews were transcribed, and a hybrid thematic analysis approach was used to capture details about 4 main domains of interest: the patient's DDF experience, contributing factors, impact, and strategies to mitigate future occurrences.

RESULTS

Dental patients endured prolonged suffering, disease progression, unnecessary treatments, and the development of new symptoms as a result of experiencing DE. Poor provider communication, inadequate time with provider, and lack of patient self-advocacy and health literacy were among the top attributes patients believed contributed to the development of a DE. Patients suggested that improvements in provider chairside manners, more detailed patient diagnostic workups, and improving personal self-advocacy; along with enhanced reporting systems, could help mitigate future DE.

CONCLUSIONS

This study demonstrates the valuable insight the patient perspective provides in understanding DEs, therefore aiding the development of strategies to help reduce the occurrences of future DDF events. Given the challenges patients expressed, there is a significant need to create an accessible reporting system that fosters constructive clinician learning.

Definitions and Measurements for Atypical Presentations at Risk for Diagnostic Errors in Internal Medicine: Protocol for a Scoping Review

BACKGROUND

Atypical presentations have been increasingly recognized as a significant contributing factor to diagnostic errors in internal medicine. However, research to address associations between atypical presentations and diagnostic errors has not been evaluated due to the lack of widely applicable definitions and criteria for what is considered an atypical presentation.

OBJECTIVE

The aim of the study is to describe how atypical presentations are defined and measured in studies of diagnostic errors in internal medicine and use this new information to develop new criteria to identify atypical presentations at high risk for diagnostic errors.

METHODS

This study will follow an established framework for conducting scoping reviews. Inclusion criteria are developed according to the participants, concept, and context framework. This review will consider studies that fulfill all of the following criteria: include adult patients (participants); explore the association between atypical presentations and diagnostic errors using any definition, criteria, or measurement to identify atypical presentations and diagnostic errors (concept); and focus on internal medicine (context). Regarding the type of sources, this scoping review will consider quantitative, qualitative, and mixed methods study designs; systematic reviews; and opinion papers for inclusion. Case reports, case series, and conference abstracts will be excluded. The data will be extracted through MEDLINE, Web of Science, CINAHL, Embase, Cochrane Library, and Google Scholar searches. No limits will be applied to language, and papers indexed from database inception to December 31, 2023, will be included. Two independent reviewers (YH and RK) will conduct study selection and data extraction. The data extracted will include specific details about the patient characteristics (eg, age, sex, and disease), the definitions and measuring methods for atypical presentations and diagnostic errors, clinical settings (eg, department and outpatient or inpatient), type of evidence source, and the association between atypical presentations and diagnostic errors relevant to the review question. The extracted data will be presented in tabular format with descriptive statistics, allowing us to identify the key components or types of atypical presentations and develop new criteria to identify atypical presentations for future studies of diagnostic errors. Developing the new criteria will follow guidance for a basic qualitative content analysis with an inductive approach.

RESULTS

As of January 2024, a literature search through multiple databases is ongoing. We will complete this study by December 2024.

CONCLUSIONS

This scoping review aims to provide rigorous evidence to develop new criteria to identify atypical presentations at high risk for diagnostic errors in internal medicine. Such criteria could facilitate the development of a comprehensive conceptual model to understand the associations between atypical presentations and diagnostic errors in internal medicine.

TRIAL REGISTRATION

Open Science Framework; www.osf.io/27d5m.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/56933.

Exploring Factors That Prolong the Diagnosis of Myasthenia Gravis

Background and Objectives

Myasthenia gravis (MG) is a condition with significant phenotypic variability, posing a diagnostic challenge to many clinicians worldwide. Prolonged diagnosis can lead to reduced remission rates and morbidity. This study aimed to identify factors leading to a longer time to diagnosis in MG that could be addressed in future to optimize diagnosis time.

Methods

One hundred and ten patients from 3 institutions in Melbourne, Australia, were included in this retrospective cohort study. Demographic and clinical data were collected for these patients over the first 5 years from diagnosis and at 10 years. Nonparametric statistical analysis was used to identify factors contributing to a longer diagnosis time.

Results

The median time for MG diagnosis was 102 (345) days. 90% of patients were diagnosed before 1 year. Female patients took longer than male patients to be diagnosed (p = 0.013). The time taken for first presentation after symptom onset contributed most to diagnosis time (median 17 [141] days), with female patients and not working as contributory factors. Neurology referral took longer if patients had diplopia (p = 0.022), respiratory (p = 0.026) symptoms, or saw an ophthalmologist first (p < 0.001). Outpatient management compared with inpatient was associated with a longer time to be seen by a neurologist from referral (p < 0.001), for the first diagnostic result to return (p = 0.001), and for the result to be reviewed (p < 0.001). Ocular MG had a median greater time to neurologist review than generalized MG (median 5 [25] days vs 1 [13] days, p = 0.035). Electrophysiology tests took longer for outpatients than inpatients (median 21 [35] days vs 2 [8] days, p < 0.001). Outpatients were also started on treatment later than inpatients (p < 0.001). There was no association of MG severity, ethnicity, age, medical and ocular comorbidities, and public or private health service on diagnosis time. There was also no impact of time to diagnosis on Myasthenia Gravis Foundation of America outcomes, number of follow-ups or hospitalizations, or prevalence of treatments used. This study is limited by low patient numbers and its retrospective nature.

Discussion

This study identified several factors that can contribute to a prolonged diagnosis time of MG. Patient and clinician education about MG and outpatient diagnostic efficiency needs emphasis. Further studies are also needed to explore the delayed presentation time of women and nonworking patients in MG.

Assessing clinical reasoning skills following a virtual patient dizziness curriculum

OBJECTIVES

Dizziness is a common medical symptom that is frequently misdiagnosed. While virtual patient (VP) education has been shown to improve diagnostic accuracy for dizziness as assessed by VPs, trainee performance has not been assessed on human subjects. The study aimed to assess whether internal medicine (IM) interns after training on a VP-based dizziness curriculum using a deliberate practice framework would demonstrate improved clinical reasoning when assessed in an objective structured clinical examination (OSCE).

METHODS

All available interns volunteered and were randomized 2:1 to intervention (VP education) vs. control (standard clinical teaching) groups. This quasi-experimental study was conducted at one academic medical center from January to May 2021. Both groups completed pre-posttest VP case assessments (scored as correct diagnosis across six VP cases) and participated in an OSCE done 6 weeks later. The OSCEs were recorded and assessed using a rubric that was systematically developed and validated.

RESULTS

Out of 21 available interns, 20 participated. Between intervention (n=13) and control (n=7), mean pretest VP diagnostic accuracy scores did not differ; the posttest VP scores improved for the intervention group (3.5 [SD 1.3] vs. 1.6 [SD 0.8], p=0.007). On the OSCE, the means scores were higher in the intervention (n=11) compared to control group (n=4) for physical exam (8.4 [SD 4.6] vs. 3.9 [SD 4.0], p=0.003) and total rubric score (43.4 [SD 12.2] vs. 32.6 [SD 11.3], p=0.04).

CONCLUSIONS

The VP-based dizziness curriculum resulted in improved diagnostic accuracy among IM interns with enhanced physical exam skills retained at 6 weeks post-intervention.

Preferred language and diagnostic errors in the pediatric emergency department

OBJECTIVES

To investigate the relationship between language and diagnostic errors (DxE) in the pediatric emergency department (ED).

METHODS

Electronic trigger identified ED encounters resulting in unplanned hospital admission that occurred within 10 days of an index visit from January 2018 through February 2022. Manual screening of each triggered encounter identified cases where the index visit diagnosis and hospitalization discharge diagnosis differed, and these were screened in for review using the Revised Safer Dx instrument to determine if a diagnostic error (DxE) occurred. Non-English primary language (NEPL) and English-proficient (EP) groups were established based on caregiver language. The primary outcome was the proportion of DxE each group. Data were analyzed using univariate analysis and multivariable logistic regression to identify independent predictors of DxE.

RESULTS

Electronic trigger identified 3,551 patients, of which 806 (22.7 %) screened in for Safer Dx review. 172 (21.3 %) experienced DxE. The proportion of DxE was similar between EP and NEPL groups (21.5 vs. 21.7 %; p=0.97). Age≥12 years and fewer prior admissions in the preceding 6 months predicted higher odds of DxE. NEPL did not predict higher odds of DxE.

CONCLUSIONS

NEPL was not associated with increased odds DxE resulting in unplanned admission.

Diagnostic Errors in Hospitalized Adults Who Died or Were Transferred to Intensive Care

Importance

Diagnostic errors contribute to patient harm, though few data exist to describe their prevalence or underlying causes among medical inpatients.

Objective

To determine the prevalence, underlying cause, and harms of diagnostic errors among hospitalized adults transferred to an intensive care unit (ICU) or who died.

Design, Setting, and Participants

Retrospective cohort study conducted at 29 academic medical centers in the US in a random sample of adults hospitalized with general medical conditions and who were transferred to an ICU, died, or both from January 1 to December 31, 2019. Each record was reviewed by 2 trained clinicians to determine whether a diagnostic error occurred (ie, missed or delayed diagnosis), identify diagnostic process faults, and classify harms. Multivariable models estimated association between process faults and diagnostic error. Opportunity for diagnostic error reduction associated with each fault was estimated using the adjusted proportion attributable fraction (aPAF). Data analysis was performed from April through September 2023.

Main Outcomes and Measures

Whether or not a diagnostic error took place, the frequency of underlying causes of errors, and harms associated with those errors.

Results

Of 2428 patient records at 29 hospitals that underwent review (mean [SD] patient age, 63.9 [17.0] years; 1107 [45.6%] female and 1321 male individuals [54.4%]), 550 patients (23.0%; 95% CI, 20.9%-25.3%) had experienced a diagnostic error. Errors were judged to have contributed to temporary harm, permanent harm, or death in 436 patients (17.8%; 95% CI, 15.9%-19.8%); among the 1863 patients who died, diagnostic error was judged to have contributed to death in 121 (6.6%; 95% CI, 5.3%-8.2%). In multivariable models examining process faults associated with any diagnostic error, patient assessment problems (aPAF, 21.4%; 95% CI, 16.4%-26.4%) and problems with test ordering and interpretation (aPAF, 19.9%; 95% CI, 14.7%-25.1%) had the highest opportunity to reduce diagnostic errors; similar ranking was seen in multivariable models examining harmful diagnostic errors.

Conclusions and Relevance

In this cohort study, diagnostic errors in hospitalized adults who died or were transferred to the ICU were common and associated with patient harm. Problems with choosing and interpreting tests and the processes involved with clinician assessment are high-priority areas for improvement efforts.

The power of written word: Reflection reduces errors of omission

BACKGROUND

Medical trainees are expected to perform complex tasks while experiencing interruptions, which increases susceptibility to errors of omission. In our study, we examine whether documentation of clinical encounters increases reflective thinking and reduces errors of omission among novice learners in a simulated setting.

METHODS

In 2021, 56 senior medical students participated in a simulated paging curriculum involving urgent inpatient cross-cover scenarios (sepsis and atrial fibrillation). Students responded to pages from standardized registered nurses (SRNs) via telephone, gathered history, and discussed clinical decision-making. Following the phone encounter, students documented a brief note (documentation encounter). A 'phone' score (number of checklist items completed in the phone encounter) and a 'combined' score (number of checklist items completed in the phone and documentation encounters) were calculated. Data were analyzed for differences between the phone scores (control) and combined scores using T-tests and McNemar test of symmetry.

FINDINGS

Fifty-four students (96%) participated. Combined scores were higher than phone scores for sepsis (72.8 ± 11.3% vs. 67.9 ± 11.9%, p < 0.001) and atrial fibrillation (74.0 ± 10.1% vs. 67.6 ± 10.0%, p < 0.001) cases. Important items, such as ordering blood cultures for sepsis (p = 0.023) and placing the patient on telemetry for atrial fibrillation (p = 0.013), were more likely to be present when a note was documented.

DISCUSSION

This study suggests that documentation provides a mechanism for learners to reflect, which could increase important diagnostic and therapeutic interventions.

CONCLUSION

Documentation by novice medical learners may improve patient care by allowing for reflection and reducing errors of omission.

The PRIDx framework to engage payers in reducing diagnostic errors in healthcare

OBJECTIVES

No framework currently exists to guide how payers and providers can collaboratively develop and implement incentives to improve diagnostic safety. We conducted a literature review and interviews with subject matter experts to develop a multi-component 'Payer Relationships for Improving Diagnoses (PRIDx)' framework, that could be used to engage payers in diagnostic safety efforts.

CONTENT

The PRIDx framework, 1) conceptualizes diagnostic safety links to care provision, 2) illustrates ways to promote payer and provider engagement in the design and adoption of accountability mechanisms, and 3) explicates the use of data analytics. Certain approaches suggested by PRIDx were refined by subject matter expert interviewee perspectives.

SUMMARY

The PRIDx framework can catalyze public and private payers to take specific actions to improve diagnostic safety.

OUTLOOK

Implementation of the PRIDx framework requires new types of partnerships, including external support from public and private payer organizations, and requires creation of strong provider incentives without undermining providers' sense of professionalism and autonomy. PRIDx could help facilitate collaborative payer-provider approaches to improve diagnostic safety and generate research concepts, policy ideas, and potential innovations for engaging payers in diagnostic safety improvement activities.

Deep learning-aided decision support for diagnosis of skin disease across skin tones

Although advances in deep learning systems for image-based medical diagnosis demonstrate their potential to augment clinical decision-making, the effectiveness of physician-machine partnerships remains an open question, in part because physicians and algorithms are both susceptible to systematic errors, especially for diagnosis of underrepresented populations. Here we present results from a large-scale digital experiment involving board-certified dermatologists (n = 389) and primary-care physicians (n = 459) from 39 countries to evaluate the accuracy of diagnoses submitted by physicians in a store-and-forward teledermatology simulation. In this experiment, physicians were presented with 364 images spanning 46 skin diseases and asked to submit up to four differential diagnoses. Specialists and generalists achieved diagnostic accuracies of 38% and 19%, respectively, but both specialists and generalists were four percentage points less accurate for the diagnosis of images of dark skin as compared to light skin. Fair deep learning system decision support improved the diagnostic accuracy of both specialists and generalists by more than 33%, but exacerbated the gap in the diagnostic accuracy of generalists across skin tones. These results demonstrate that well-designed physician-machine partnerships can enhance the diagnostic accuracy of physicians, illustrating that success in improving overall diagnostic accuracy does not necessarily address bias.

Japanese Internists' Most Memorable Diagnostic Error Cases: A Self-reflection Survey

Objective The etiologies of diagnostic errors among internal medicine physicians are unclear. To understand the causes and characteristics of diagnostic errors through reflection by those involved in them. Methods We conducted a cross-sectional study using a web-based questionnaire in Japan in January 2019. Over a 10-day period, a total of 2,220 participants agreed to participate in the study, of whom 687 internists were included in the final analysis. Participants were asked about their most memorable diagnostic error cases, in which the time course, situational factors, and psychosocial context could be most vividly recalled and where the participant provided care. We categorized diagnostic errors and identified contributing factors (i.e., situational factors, data collection/interpretation factors, and cognitive biases). Results Two-thirds of the identified diagnostic errors occurred in the clinic or emergency department. Errors were most frequently categorized as wrong diagnoses, followed by delayed and missed diagnoses. Errors most often involved diagnoses related to malignancy, circulatory system disorders, or infectious diseases. Situational factors were the most cited error cause, followed by data collection factors and cognitive bias. Common situational factors included limited consultation during office hours and weekends and barriers that prevented consultation with a supervisor or another department. Conclusion Internists reported situational factors as a significant cause of diagnostic errors. Other factors, such as cognitive biases, were also evident, although the difference in clinical settings may have influenced the proportions of the etiologies of the errors that were observed. Furthermore, wrong, delayed, and missed diagnoses may have distinctive associated cognitive biases.

Quantifying and Reducing Errors in Vascular Imaging Examination Orders Through a Multistage Quality Improvement Intervention

PURPOSE

The aims of this study were to quantify order error rates for vascular imaging examinations and to assess the effects of a multistage quality improvement intervention on those rates.

METHODS

In this prospective, institutional review board-exempt project at a large academic quaternary care hospital, the authors aimed to quantify and reduce the order error rate by 50%. The authors analyzed 844 orders for all vascular imaging examinations placed before the intervention (July 19 to August 1, 2021, and September 13 to September 26, 2021), after an intervention in the cardiac surgery department consisting of a new customized order option in the electronic health record for routine preoperative patients (postintervention 1, February 28 to March 27, 2022); and after an educational and feedback campaign (postintervention 2, May 23 to June 5, 2022). Incorrect orders were identified by a radiology trainee during protocoling if the reasons for ordered examination and imaging examination were discordant and subsequently confirmed with the ordering provider. The primary outcome, order error rate, was compared across the project periods using the χ2 test and by ordering department using the χ2 and Fisher exact tests.

RESULTS

The preintervention order error rate of 16% (50 of 306) decreased by 83% to 3% (10 of 353) at postintervention 1 (P < .001) and was durable at 3% (6 of 185) by project end. Chest CT with or without contrast constituted the majority of incorrect orders (44%, 22 of 50); "Pre-Op" was the most common examination reason (32% [16 of 50]). Cardiac surgery orderers were responsible for the most incorrect orders (32% [16 of 50]). All four most common ordering departments, including cardiac surgery, reduced their order error rates after the intervention (P < .001).

CONCLUSIONS

Incorrect orders for imaging examinations can be reduced through targeted quality improvement interventions combining tailored electronic health record order options with education and feedback on practice habits.

Financial Impact of a Radiology Safety Net Program for Resolution of Clinically Necessary Follow-up Imaging Recommendations

OBJECTIVE

Health care safety net (SN) programs can potentially improve patient safety and decrease risk associated with missed or delayed follow-up care, although they require financial resources. This study aimed to assess whether the revenue generated from completion of clinically necessary recommendations for additional imaging (RAI) made possible by an IT-enabled SN program could fund the required additional labor resources.

METHODS

Clinically necessary RAI generated October 21, 2019, to September 24, 2021, were tracked to resolution as of April 13, 2023. A new radiology SN team worked with existing schedulers and care coordinators, performing chart review and patient and provider outreach to ensure RAI resolution. We applied relevant Current Procedural Terminology, version 4 codes of the completed imaging examinations to estimate total revenue. Coprimary outcomes included revenue generated by total performed examinations and estimated revenue attributed to SN involvement. We used Student's t test to compare the secondary outcome, RAI time interval, for higher versus lower revenue-generating modalities.

RESULTS

In all, 24% (3,243) of eligible follow-up recommendations (13,670) required SN involvement. Total estimated revenue generated by performed recommended examinations was $6,116,871, with $980,628 attributed to SN. Net SN-generated revenue per 1.0 full-time equivalent was an estimated $349,768. Greatest proportion of performed examinations were cross-sectional modalities (CT, MRI, PET/CT), which were higher revenue-generating than non-cross-sectional modalities (x-ray, ultrasound, mammography), and had shorter recommendation time frames (153 versus 180 days, P < .001).

DISCUSSION

The revenue generated from completion of RAI facilitated by an IT-enabled quality and safety program supplemented by an SN team can fund the required additional labor resources to improve patient safety. Realizing early revenue may require 5 to 6 months postimplementation.

Are the latest point-of-care D-dimer devices ready for use in general practice? A prospective clinical evaluation of five test systems with a capillary blood feature for suspected venous thromboembolism

INTRODUCTION

We evaluated clinical performance of five novel point-of-care (POC) D-dimer devices with a capillary finger stick feature for predicting venous thromboembolism (VTE) in general practice: Exdia TRF Plus (E), AFIAS-1® (A), Standard F200® (S), LumiraDx™ (L) and Hipro AFS/1® (H).

MATERIALS AND METHODS

Primary care patients with a low suspicion of a VTE were asked to consent to (i) draw additional venous blood samples, (ii) perform a capillary POC D-dimer test, (iii) approach their general practitioner afterwards for clinical outcomes. Venous plasma samples were processed on all POC devices and a laboratory-based assay (STA-Liatest®D-Di PLUS assay). Results were compared with clinical outcomes to generate performance characteristics. Capillary and venous blood results were used for a matrix comparison.

RESULTS

Venous plasma samples from 511 participants, of whom 57 had VTE, were used for clinical performance analyses. Areas under Receiving Operating Characteristic Curves ranged from 0.90 (95 % CI: 0.86-0.94) (H) to 0.93 (0.90-0.96) (E). All false-negative rates were below 1.4 % (95 % CI: 0.5 %-3.4 %). Matrix comparison demonstrated correlation coefficients ranging from r = 0.11 (95 % CI: -0.15-0.36) (H) to r = 0.94 (0.90-0.97) (A) with concordance percentages ranging from 71.4 % (applying a D-dimer cutoff of 500 ng/mL) (H) to 100 % (applying an age-dependent D-dimer cutoff) (A).

CONCLUSIONS

Clinical performance of the POC D-dimer devices for predicting a VTE in low-risk patients was comparable to that of a laboratory-based assay. However, our results indicate that the finger stick feature of certain devices should be further improved. (NL71809.028.19.).

Achieving diagnostic excellence through prevention and teamwork (ADEPT) study protocol: A multicenter, prospective quality and safety program to improve diagnostic processes in medical inpatients

BACKGROUND

Few hospitals have built surveillance for diagnostic errors into usual care or used comparative quantitative and qualitative data to understand their diagnostic processes and implement interventions designed to reduce these errors.

OBJECTIVES

To build surveillance for diagnostic errors into usual care, benchmark diagnostic performance across sites, pilot test interventions, and evaluate the program's impact on diagnostic error rates.

METHODS AND ANALYSIS

Achieving diagnostic excellence through prevention and teamwork (ADEPT) is a multicenter, real-world quality and safety program utilizing interrupted time-series techniques to evaluate outcomes. Study subjects will be a randomly sampled population of medical patients hospitalized at 16 US hospitals who died, were transferred to intensive care, or had a rapid response during the hospitalization. Surveillance for diagnostic errors will occur on 10 events per month per site using a previously established two-person adjudication process. Concurrent reviews of patients who had a qualifying event in the previous week will allow for surveys of clinicians to better understand contributors to diagnostic error, or conversely, examples of diagnostic excellence, which cannot be gleaned from medical record review alone. With guidance from national experts in quality and safety, sites will report and benchmark diagnostic error rates, share lessons regarding underlying causes, and design, implement, and pilot test interventions using both Safety I and Safety II approaches aimed at patients, providers, and health systems. Safety II approaches will focus on cases where diagnostic error did not occur, applying theories of how people and systems are able to succeed under varying conditions. The primary outcome will be the number of diagnostic errors per patient, using segmented multivariable regression to evaluate change in y-intercept and change in slope after initiation of the program.

ETHICS AND DISSEMINATION

The study has been approved by the University of California, San Francisco Institutional Review Board (IRB), which is serving as the single IRB. Intervention toolkits and study findings will be disseminated through partners including Vizient, The Joint Commission, and Press-Ganey, and through national meetings, scientific journals, and publications aimed at the general public.

The Nature, Causes, and Clinical Impact of Errors in the Clinical Laboratory Testing Process Leading to Diagnostic Error: A Voluntary Incident Report Analysis

OBJECTIVES

Diagnostic errors, that is, missed, delayed, or wrong diagnoses, are a common type of medical errors and preventable iatrogenic harm. Errors in the laboratory testing process can lead to diagnostic errors. This retrospective analysis of voluntary incident reports aimed to investigate the nature, causes, and clinical impact of errors, including diagnostic errors, in the clinical laboratory testing process.

METHODS

We used a sample of 600 voluntary incident reports concerning diagnostic testing selected from all incident reports filed at the University Medical Center Utrecht in 2017-2018. From these incident reports, we included all reports concerning the clinical laboratory testing process. For these incidents, we determined the following: nature: in which phase of the testing process the error occurred; cause: human, technical, organizational; and clinical impact: the type and severity of the harm to the patient, including diagnostic error.

RESULTS

Three hundred twenty-seven reports were included in the analysis. In 77.1%, the error occurred in the preanalytical phase, 13.5% in the analytical phase and 8.0% in the postanalytical phase (1.5% undetermined). Human factors were the most frequent cause (58.7%). Severe clinical impact occurred relatively more often in the analytical and postanalytical phase, 32% and 28%, respectively, compared with the preanalytical phase (40%). In 195 cases (60%), there was a potential diagnostic error as consequence, mainly a potential delay in the diagnostic process (50.5%).

CONCLUSIONS

Errors in the laboratory testing process often lead to potential diagnostic errors. Although prone to incomplete information on causes and clinical impact, voluntary incident reports are a valuable source for research on diagnostic error related to errors in the clinical laboratory testing process.

Diagnostic errors in uncommon conditions: a systematic review of case reports of diagnostic errors

OBJECTIVES

To assess the usefulness of case reports as sources for research on diagnostic errors in uncommon diseases and atypical presentations.

CONTENT

We reviewed 563 case reports of diagnostic error. The commonality of the final diagnoses was classified based on the description in the articles, Orphanet, or epidemiological data on available references; the typicality of presentation was classified based on the description in the articles and the judgment of the physician researchers. Diagnosis Error Evaluation and Research (DEER), Reliable Diagnosis Challenges (RDC), and Generic Diagnostic Pitfalls (GDP) taxonomies were used to assess the factors contributing to diagnostic errors.

SUMMARY AND OUTLOOK

Excluding three cases in that commonality could not be classified, 560 cases were classified into four categories: typical presentations of common diseases (60, 10.7 %), atypical presentations of common diseases (35, 6.2 %), typical presentations of uncommon diseases (276, 49.3 %), and atypical presentations of uncommon diseases (189, 33.8 %). The most important DEER taxonomy was "Failure/delay in considering the diagnosis" among the four categories, whereas the most important RDC and GDP taxonomies varied with the categories. Case reports can be a useful data source for research on the diagnostic errors of uncommon diseases with or without atypical presentations.

Associations Between Hospitalist Shift Busyness, Diagnostic Confidence, and Resource Utilization: A Pilot Study

OBJECTIVES

Hospitalized patients are at risk for diagnostic errors. Hospitalists caring for these patients are often multitasking when overseeing patient care. We aimed to measure hospitalist workload and understand its influences on diagnostic performance in a real-world clinical setting.

METHODS

We conducted a single-center, prospective, pilot observational study of hospitalists admitting new patients to the hospital. Hospitalists completed an abridged Mindful Attention Awareness Tool and a survey about diagnostic confidence at shift completion. Data on differential diagnoses and resource utilization (e.g., laboratory, imaging tests ordered, and consultations) were collected from the medical record. The number of admissions and paging volume per shift were used as separate proxies for shift busyness. Data were analyzed using linear mixed effects models (continuous outcomes) or mixed effects logistic regression (dichotomous outcomes).

RESULTS

Of the 53 hospitalists approached, 47 (89%) agreed to participate; complete data were available for 37 unique hospitalists who admitted 160 unique patients. Increases in admissions (odds ratio, 1.99; 95% confidence interval [CI], 1.04 to 3.82; P = 0.04) and pages (odds ratio, 1.11; 95% CI, 1.02 to 1.21; P = 0.01) were associated with increased odds of hospitalists finding it "difficult to focus on what is happening in the present." Increased pages was associated with a decrease in the number of listed differential diagnoses (coefficient, -0.02; 95% CI, -0.04 to -0.003; P = 0.02).

CONCLUSIONS

Evaluation of hospitalist busyness and its associations with factors that may influence diagnosis in a real-world environment was feasible and demonstrated important implications on physician focus and differential diagnosis.

Common contributing factors of diagnostic error: A retrospective analysis of 109 serious adverse event reports from Dutch hospitals

INTRODUCTION

Although diagnostic errors have gained renewed focus within the patient safety domain, measuring them remains a challenge. They are often measured using methods that lack information on decision-making processes given by involved physicians (eg, record reviews). The current study analyses serious adverse event (SAE) reports from Dutch hospitals to identify common contributing factors of diagnostic errors in hospital medicine. These reports are the results of thorough investigations by highly trained, independent hospital committees into the causes of SAEs. The reports include information from involved healthcare professionals and patients or family obtained through interviews.

METHODS

All 71 Dutch hospitals were invited to participate in this study. Participating hospitals were asked to send four diagnostic SAE reports of their hospital. Researchers applied the Safer Dx Instrument, a Generic Analysis Framework, the Diagnostic Error Evaluation and Research (DEER) taxonomy and the Eindhoven Classification Model (ECM) to analyse reports.

RESULTS

Thirty-one hospitals submitted 109 eligible reports. Diagnostic errors most often occurred in the diagnostic testing, assessment and follow-up phases according to the DEER taxonomy. The ECM showed human errors as the most common contributing factor, especially relating to communication of results, task planning and execution, and knowledge. Combining the most common DEER subcategories and the most common ECM classes showed that clinical reasoning errors resulted from failures in knowledge, and task planning and execution. Follow-up errors and errors with communication of test results resulted from failures in coordination and monitoring, often accompanied by usability issues in electronic health record design and missing protocols.

DISCUSSION

Diagnostic errors occurred in every hospital type, in different specialties and with different care teams. While clinical reasoning errors remain a common problem, often caused by knowledge and skill gaps, other frequent errors in communication of test results and follow-up require different improvement measures (eg, improving technological systems).

Effects of Combinational Use of Additional Differential Diagnostic Generators on the Diagnostic Accuracy of the Differential Diagnosis List Developed by an Artificial Intelligence-Driven Automated History-Taking System: Pilot Cross-Sectional Study

BACKGROUND

Low diagnostic accuracy is a major concern in automated medical history-taking systems with differential diagnosis (DDx) generators. Extending the concept of collective intelligence to the field of DDx generators such that the accuracy of judgment becomes higher when accepting an integrated diagnosis list from multiple people than when accepting a diagnosis list from a single person may be a possible solution.

OBJECTIVE

The purpose of this study is to assess whether the combined use of several DDx generators improves the diagnostic accuracy of DDx lists.

METHODS

We used medical history data and the top 10 DDx lists (index DDx lists) generated by an artificial intelligence (AI)-driven automated medical history-taking system from 103 patients with confirmed diagnoses. Two research physicians independently created the other top 10 DDx lists (second and third DDx lists) per case by imputing key information into the other 2 DDx generators based on the medical history generated by the automated medical history-taking system without reading the index lists generated by the automated medical history-taking system. We used the McNemar test to assess the improvement in diagnostic accuracy from the index DDx lists to the three types of combined DDx lists: (1) simply combining DDx lists from the index, second, and third lists; (2) creating a new top 10 DDx list using a 1/n weighting rule; and (3) creating new lists with only shared diagnoses among DDx lists from the index, second, and third lists. We treated the data generated by 2 research physicians from the same patient as independent cases. Therefore, the number of cases included in analyses in the case using 2 additional lists was 206 (103 cases × 2 physicians' input).

RESULTS

The diagnostic accuracy of the index lists was 46% (47/103). Diagnostic accuracy was improved by simply combining the other 2 DDx lists (133/206, 65%, P<.001), whereas the other 2 combined DDx lists did not improve the diagnostic accuracy of the DDx lists (106/206, 52%, P=.05 in the collective list with the 1/n weighting rule and 29/206, 14%, P<.001 in the only shared diagnoses among the 3 DDx lists).

CONCLUSIONS

Simply adding each of the top 10 DDx lists from additional DDx generators increased the diagnostic accuracy of the DDx list by approximately 20%, suggesting that the combinational use of DDx generators early in the diagnostic process is beneficial.

Diagnostic Blood Biomarkers for Acute Pulmonary Embolism: A Systematic Review

(1) Background: The current diagnostic algorithm for acute pulmonary embolism (PE) is associated with the overuse of CT pulmonary angiography (CTPA). An additional highly specific blood test could potentially lower the proportion of patients with suspected PE that require CTPA. The aim was to summarize the literature on the diagnostic performance of biomarkers of patients admitted to an emergency department with suspected acute PE. (2) Methods: Medline and Embase databases were searched from 1995 to the present. The study selection process, data extraction, and risk of bias assessment were conducted by two reviewers. Eligibility criteria accepted all blood biomarkers except D-dimer, and CTPA was used as the reference standard. Qualitative data synthesis was performed. (3) Results: Of the 8448 identified records, only 6 were included. Eight blood biomarkers were identified, of which, three were investigated in two separate studies. Red distribution width and mean platelet volume were reported to have a specificity of ≥ 90% in one study, although these findings were not confirmed by other studies. The majority of the studies contained a high risk of selection bias. (4) Conclusions: The modest findings and the uncertain validity of the included studies suggest that none of the biomarkers identified in this systematic review have the potential to improve the current diagnostic algorithm for acute PE by reducing the overuse of CTPA.

Preventing lost-to-follow up diagnostic imaging in ambulatory care: evaluation of an electronic notification tool

OBJECTIVE

Missed or cancelled imaging tests may be invisible to the ordering clinician and result in diagnostic delay. We developed an outpatient results notification tool (ORNT) to alert physicians of patients' missed radiology studies.

DESIGN

Randomised controlled evaluation of a quality improvement intervention.

SETTING

23 primary care and subspecialty ambulatory clinics at an urban academic medical centre.

PARTICIPANTS

276 physicians randomised to intervention or usual care.

MAIN OUTCOME MEASURE

90-day test completion of missed imaging tests.

RESULTS

We included 3675 radiology tests in our analysis: 1769 ordered in the intervention group and 1906 in the usual care group. A higher per cent of studies were completed for intervention compared with usual care groups in CT (20.7% vs 15.3%, p=0.06), general radiology (19.6% vs 12.0%, p=0.02) and, in aggregate, across all modalities (18.1% vs 16.1%, p=0.03). In the multivariable regression model adjusting for sex, age and insurance type and accounting for clustering with random effects at the level of the physician, the intervention group had a 36% greater odds of test completion than the usual care group (OR: 1.36 (1.097-1.682), p=0.005). In the Cox regression model, patients in the intervention group were 1.32 times more likely to complete their test in a timely fashion (HR: 1.32 (1.10-1.58), p=0.003).

CONCLUSIONS

An electronic alert that notified the responsible clinician of a missed imaging test ordered in an ambulatory clinic reduced the number of incomplete tests at 90 days. Further study of the obstacles to completing recommended diagnostic testing may allow for the development of better tools to support busy clinicians and their patients and reduce the risk of diagnostic delays.

Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach

Objective

To describe a user-centered approach to develop, pilot test, and refine requirements for 3 electronic health record (EHR)-integrated interventions that target key diagnostic process failures in hospitalized patients.

Materials and Methods

Three interventions were prioritized for development: a Diagnostic Safety Column (DSC) within an EHR-integrated dashboard to identify at-risk patients; a Diagnostic Time-Out (DTO) for clinicians to reassess the working diagnosis; and a Patient Diagnosis Questionnaire (PDQ) to gather patient concerns about the diagnostic process. Initial requirements were refined from analysis of test cases with elevated risk predicted by DSC logic compared to risk perceived by a clinician working group; DTO testing sessions with clinicians; PDQ responses from patients; and focus groups with clinicians and patient advisors using storyboarding to model the integrated interventions. Mixed methods analysis of participant responses was used to identify final requirements and potential implementation barriers.

Results

Final requirements from analysis of 10 test cases predicted by the DSC, 18 clinician DTO participants, and 39 PDQ responses included the following: DSC configurable parameters (variables, weights) to adjust baseline risk estimates in real-time based on new clinical data collected during hospitalization; more concise DTO wording and flexibility for clinicians to conduct the DTO with or without the patient present; and integration of PDQ responses into the DSC to ensure closed-looped communication with clinicians. Analysis of focus groups confirmed that tight integration of the interventions with the EHR would be necessary to prompt clinicians to reconsider the working diagnosis in cases with elevated diagnostic error (DE) risk or uncertainty. Potential implementation barriers included alert fatigue and distrust of the risk algorithm (DSC); time constraints, redundancies, and concerns about disclosing uncertainty to patients (DTO); and patient disagreement with the care team's diagnosis (PDQ).

Discussion

A user-centered approach led to evolution of requirements for 3 interventions targeting key diagnostic process failures in hospitalized patients at risk for DE.

Conclusions

We identify challenges and offer lessons from our user-centered design process.

Prevalence and Causes of Diagnostic Errors in Hospitalized Patients Under Investigation for COVID-19

BACKGROUND

The COVID-19 pandemic required clinicians to care for a disease with evolving characteristics while also adhering to care changes (e.g., physical distancing practices) that might lead to diagnostic errors (DEs).

OBJECTIVE

To determine the frequency of DEs and their causes among patients hospitalized under investigation (PUI) for COVID-19.

DESIGN

Retrospective cohort.

SETTING

Eight medical centers affiliated with the Hospital Medicine ReEngineering Network (HOMERuN).

TARGET POPULATION

Adults hospitalized under investigation (PUI) for COVID-19 infection between February and July 2020.

MEASUREMENTS

We randomly selected up to 8 cases per site per month for review, with each case reviewed by two clinicians to determine whether a DE (defined as a missed or delayed diagnosis) occurred, and whether any diagnostic process faults took place. We used bivariable statistics to compare patients with and without DE and multivariable models to determine which process faults or patient factors were associated with DEs.

RESULTS

Two hundred and fifty-seven patient charts underwent review, of which 36 (14%) had a diagnostic error. Patients with and without DE were statistically similar in terms of socioeconomic factors, comorbidities, risk factors for COVID-19, and COVID-19 test turnaround time and eventual positivity. Most common diagnostic process faults contributing to DE were problems with clinical assessment, testing choices, history taking, and physical examination (all p < 0.01). Diagnostic process faults associated with policies and procedures related to COVID-19 were not associated with DE risk. Fourteen patients (35.9% of patients with errors and 5.4% overall) suffered harm or death due to diagnostic error.

LIMITATIONS

Results are limited by available documentation and do not capture communication between providers and patients.

CONCLUSION

Among PUI patients, DEs were common and not associated with pandemic-related care changes, suggesting the importance of more general diagnostic process gaps in error propagation.

Improving decision accuracy using a clinical decision support system for medical students during history-taking: a randomized clinical trial

BACKGROUND

A clinical diagnostic support system (CDSS) can support medical students and physicians in providing evidence-based care. In this study, we investigate diagnostic accuracy based on the history of present illness between groups of medical students using a CDSS, Google, and neither (control). Further, the degree of diagnostic accuracy of medical students using a CDSS is compared with that of residents using neither a CDSS nor Google.

METHODS

This study is a randomized educational trial. The participants comprised 64 medical students and 13 residents who rotated in the Department of General Medicine at Chiba University Hospital from May to December 2020. The medical students were randomly divided into the CDSS group (n = 22), Google group (n = 22), and control group (n = 20). Participants were asked to provide the three most likely diagnoses for 20 cases, mainly a history of a present illness (10 common and 10 emergent diseases). Each correct diagnosis was awarded 1 point (maximum 20 points). The mean scores of the three medical student groups were compared using a one-way analysis of variance. Furthermore, the mean scores of the CDSS, Google, and residents' (without CDSS or Google) groups were compared.

RESULTS

The mean scores of the CDSS (12.0 ± 1.3) and Google (11.9 ± 1.1) groups were significantly higher than those of the control group (9.5 ± 1.7; p = 0.02 and p = 0.03, respectively). The residents' group's mean score (14.7 ± 1.4) was higher than the mean scores of the CDSS and Google groups (p = 0.01). Regarding common disease cases, the mean scores were 7.4 ± 0.7, 7.1 ± 0.7, and 8.2 ± 0.7 for the CDSS, Google, and residents' groups, respectively. There were no significant differences in mean scores (p = 0.1).

CONCLUSIONS

Medical students who used the CDSS and Google were able to list differential diagnoses more accurately than those using neither. Furthermore, they could make the same level of differential diagnoses as residents in the context of common diseases.

TRIAL REGISTRATION

This study was retrospectively registered with the University Hospital Medical Information Network Clinical Trials Registry on 24/12/2020 (unique trial number: UMIN000042831).

Physicians' experiences and actions in making complex level-of-care decisions during acute situations within older patients' homes: a critical incident study

BACKGROUND

Complex level-of-care decisions involve uncertainty in which decisions are beneficial for older patients. Knowledge of physicians' decision-making during acute situations in older patients' homes is limited. Therefore, this study aimed to describe physicians' experiences and actions in making complex level-of-care decisions during the assessment of older patients in acute situations within their own homes.

METHODS

Individual interviews and analyses were performed according to the critical incident technique (CIT). In total, 14 physicians from Sweden were included.

RESULTS

In making complex level-of-care decisions, physicians experienced collaborating with and including older patients, significant others and health care professionals to be essential for making individualized decisions regarding the patients' and their significant others' needs. During decision-making, physicians experienced difficulties when doubt or collaborative obstructions occurred. Physicians' actions involved searching for an understanding of older patients' and their significant others' wishes and needs, considering their unique conditions, guiding them, and adjusting care according to their wishes. Actions further involved promoting collaboration and reaching a consensus with all persons involved.

CONCLUSION

Physicians strive to individualize complex level-of-care decisions based on older patients' and their significant others' wishes and needs. Furthermore, individualized decisions depend on successful collaboration and consensus among older patients, their significant others and other health care professionals. Therefore, to facilitate individualized level-of-care decisions, the health care organizations need to support physicians when they are making individualized decisions, provide sufficient resources and promote 24 - 7 collaboration between organizations and health care professionals.

Twelve tips for physicians’ mastering expertise in diagnostic excellence

Diagnostic errors, which account for a large proportion of medical errors, are a global medical challenge. The slogan of reducing diagnostic errors has recently shifted to a new strategy of diagnostic excellence, the core of which is the importance of improving the multidisciplinary diagnostic process. Many of the elements and strategies necessary for diagnostic excellence have been presented. In the context of this diagnostic improvement, some reports have been structured to improve the quality of performance of individual physicians as players. Still, surprisingly, only a few reports have focused on specific day-to-day training strategies for the diagnostic thinking process as expertise. This paper focuses on this point and proposes strategies for refining the diagnostic thinking expertise of frontline physicians in the new era, based on the following four elements: knowledge and experience, diagnostic thinking strategies, information management skills, and calibration and reflection.

Identifying Electronic Health Record Contributions to Diagnostic Error in Ambulatory Settings Through Legal Claims Analysis

This qualitative study analyzes closed legal claims data to determine whether problems with electronic health records are associated with diagnostic errors, in which part of the diagnostic process errors occur, and the specific types of errors that occur.

Artificial Intelligence Tool for Detection and Worklist Prioritization Reduces Time to Diagnosis of Incidental Pulmonary Embolism at CT

Purpose

To evaluate the diagnostic efficacy of artificial intelligence (AI) software in detecting incidental pulmonary embolism (IPE) at CT and shorten the time to diagnosis with use of radiologist reading worklist prioritization.

Materials and Methods

In this study with historical controls and prospective evaluation, regulatory-cleared AI software was evaluated to prioritize IPE on routine chest CT scans with intravenous contrast agent in adult oncology patients. Diagnostic accuracy metrics were calculated, and temporal end points, including detection and notification times (DNTs), were assessed during three time periods (April 2019 to September 2020): routine workflow without AI, human triage without AI, and worklist prioritization with AI.

Results

In total, 11 736 CT scans in 6447 oncology patients (mean age, 63 years ± 12 [SD]; 3367 men) were included. Prevalence of IPE was 1.3% (51 of 3837 scans), 1.4% (54 of 3920 scans), and 1.0% (38 of 3979 scans) for the respective time periods. The AI software detected 131 true-positive, 12 false-negative, 31 false-positive, and 11 559 true-negative results, achieving 91.6% sensitivity, 99.7% specificity, 99.9% negative predictive value, and 80.9% positive predictive value. During prospective evaluation, AI-based worklist prioritization reduced the median DNT for IPE-positive examinations to 87 minutes (vs routine workflow of 7714 minutes and human triage of 4973 minutes). Radiologists' missed rate of IPE was significantly reduced from 44.8% (47 of 105 scans) without AI to 2.6% (one of 38 scans) when assisted by the AI tool (P < .001).

Conclusion

AI-assisted workflow prioritization of IPE on routine CT scans in oncology patients showed high diagnostic accuracy and significantly shortened the time to diagnosis in a setting with a backlog of examinations.Keywords: CT, Computer Applications, Detection, Diagnosis, Embolism, Thorax, ThrombosisSupplemental material is available for this article.© RSNA, 2023See also the commentary by Elicker in this issue.

Factors Associated With Diagnostic Error: An Analysis of Closed Medical Malpractice Claims

INTRODUCTION

Missed and delayed diagnoses have received substantial attention as a quality and patient safety priority. To the extent that electronic health records, team-based care, and other mitigation strategies have been successful in improving diagnosis since the last large-scale study, we would expect that the contributing factors to diagnostic claims may have changed.

METHODS

This study sought to examine paid medical malpractice claims as a proxy to identify contributing factors that reflect a clear diagnostic error. Diagnostic error cases with indemnity payments (2009-2020) were identified using the Candello (formerly known as CRICO) proprietary taxonomy. Factors associated with indemnity payments were analyzed using a multivariable logistic regression model.

RESULTS

Of 5367 included claims, 2161 (40%) had indemnity payments. A majority of claims had multiple contributing factors on the diagnostic pathway. In multivariable analysis, factors independently associated with an indemnity payment included the insurer (odds ratio and 95% confidence interval, 2.8 [2.4-3.3]), high injury severity (1.9 [1.3-2.8]) or death (1.5 [0.99-2.1]), and case setting (inpatient (0.77 [0.65-0.91]) or emergency department (0.67 [0.49-0.92])). Importantly, cases with contributing factors outside of Candello's diagnostic pathway were more likely to lead to indemnity payment.

CONCLUSIONS

The digital transformation and acceleration of team-based care in medicine have not mitigated the malpractice risks of complex cases with severe injuries and multiple missteps.

Types of diagnostic errors reported by paediatric emergency providers in a global paediatric emergency care research network

Background

Diagnostic errors, reframed as missed opportunities for improving diagnosis (MOIDs), are poorly understood in the paediatric emergency department (ED) setting. We investigated the clinical experience, harm and contributing factors related to MOIDs reported by physicians working in paediatric EDs.

Methods

We developed a web-based survey in which physicians participating in the international Paediatric Emergency Research Network representing five out of six WHO regions, described examples of MOIDs involving their own or a colleague’s patients. Respondents provided case summaries and answered questions regarding harm and factors contributing to the event.

Results

Of 1594 physicians surveyed, 412 (25.8%) responded (mean age=43 years (SD=9.2), 42.0% female, mean years in practice=12 (SD=9.0)). Patient presentations involving MOIDs had common undifferentiated symptoms at initial presentation, including abdominal pain (21.1%), fever (17.2%) and vomiting (16.5%). Patients were discharged from the ED with commonly reported diagnoses, including acute gastroenteritis (16.7%), viral syndrome (10.2%) and constipation (7.0%). Most reported MOIDs (65%) were detected on ED return visits (46% within 24 hours and 76% within 72 hours). The most common reported MOID was appendicitis (11.4%), followed by brain tumour (4.4%), meningitis (4.4%) and non-accidental trauma (4.1%). More than half (59.1%) of the reported MOIDs involved the patient/parent–provider encounter (eg, misinterpreted/ignored history or an incomplete/inadequate physical examination). Types of MOIDs and contributing factors did not differ significantly between countries. More than half of patients had either moderate (48.7%) or major (10%) harm due to the MOID.

Conclusions

An international cohort of paediatric ED physicians reported several MOIDs, often in children who presented to the ED with common undifferentiated symptoms. Many of these were related to patient/parent–provider interaction factors such as suboptimal history and physical examination. Physicians’ personal experiences offer an underexplored source for investigating and mitigating diagnostic errors in the paediatric ED.

Assessing the Assessment-Developing and Deploying a Novel Tool for Evaluating Clinical Notes' Diagnostic Assessment Quality

BACKGROUND

Ambulatory diagnostic errors are increasingly being recognized as an important quality and safety issue, and while measures of diagnostic quality have been sought, tools to evaluate diagnostic assessments in the medical record are lacking.

OBJECTIVE

To develop and test a tool to measure diagnostic assessment note quality in primary care urgent encounters and identify common elements and areas for improvement in diagnostic assessment.

DESIGN

Retrospective chart review of urgent care encounters at an urban academic setting.

PARTICIPANTS

Primary care physicians.

MAIN MEASURES

The Assessing the Assessment (ATA) instrument was evaluated for inter-rater reliability, internal consistency, and findings from its application to EHR notes.

KEY RESULTS

ATA had reasonable performance characteristics (kappa 0.63, overall Cronbach's alpha 0.76). Variability in diagnostic assessment was seen in several domains. Two components of situational awareness tended to be well-documented ("Don't miss diagnoses" present in 84% of charts, red flag symptoms in 87%), while Psychosocial context was present only 18% of the time.

CONCLUSIONS

The ATA tool is a promising framework for assessing and identifying areas for improvement in diagnostic assessments documented in clinical encounters.

Identification of delayed diagnosis of paediatric appendicitis in administrative data: a multicentre retrospective validation study

OBJECTIVE

To derive and validate a tool that retrospectively identifies delayed diagnosis of appendicitis in administrative data with high accuracy.

DESIGN

Cross-sectional study.

SETTING

Five paediatric emergency departments (EDs).

PARTICIPANTS

669 patients under 21 years old with possible delayed diagnosis of appendicitis, defined as two ED encounters within 7 days, the second with appendicitis.

OUTCOME

Delayed diagnosis was defined as appendicitis being present but not diagnosed at the first ED encounter based on standardised record review. The cohort was split into derivation (2/3) and validation (1/3) groups. We derived a prediction rule using logistic regression, with covariates including variables obtainable only from administrative data. The resulting trigger tool was applied to the validation group to determine area under the curve (AUC). Test characteristics were determined at two predicted probability thresholds.

RESULTS

Delayed diagnosis occurred in 471 (70.4%) patients. The tool had an AUC of 0.892 (95% CI 0.858 to 0.925) in the derivation group and 0.859 (95% CI 0.806 to 0.912) in the validation group. The positive predictive value (PPV) for delay at a maximal accuracy threshold was 84.7% (95% CI 78.2% to 89.8%) and identified 87.3% of delayed cases. The PPV at a stricter threshold was 94.9% (95% CI 87.4% to 98.6%) and identified 46.8% of delayed cases.

CONCLUSIONS

This tool accurately identified delayed diagnosis of appendicitis. It may be used to screen for potential missed diagnoses or to specifically identify a cohort of children with delayed diagnosis.

Ear, Nose and Throat (ENT) disease diagnostic error in low-resource health care: Observations from a hospital-based cross-sectional study

Although the global burden of ear, nose and throat (ENT) diseases is high, data relating to ENT disease epidemiology and diagnostic error in resource-limited settings remain scarce. We conducted a retrospective cross-sectional review of ENT patients' clinical records at a resource-limited tertiary hospital. We determined the diagnostic accuracy and appropriateness of patient referrals for ENT specialist care using descriptive statistics. Cohens kappa coefficient (κ) was calculated to determine the diagnostic agreement between non-ENT clinicians and the ENT specialist, and logistic regression applied to establish the likelihood of patient misdiagnosis by non-ENT clinicians. Of the 1543 patients studied [age 0-87 years, mean age 25(21) years (mean(SD)], non-ENT clinicians misdiagnosed 67.4% and inappropriately referred 50.4%. Compared to those aged 0-5 years, patients aged 51-87 years were 1.77 (95%CI: 1.03-3.04) fold more likely to have a referral misdiagnosis for specialist care. Patients with ear (aOR: 1.63; 95% CI: 1.14-2.33) and those with sinonasal diseases (aOR: 1.80; 95% CI: 1.14-2.45) had greater likelihood of referral misdiagnosis than those with head and neck diseases. Agreement in diagnosis between the ENT specialist and non-ENT clinicians was poor (κ = 0.0001). More effective, accelerated training of clinicians may improve diagnostic accuracy in low-resource settings.

The diagnostic potential of plasma SCUBE-1 concentration for pulmonary embolism: A pilot study

INTRODUCTION

This study aimed to investigate the potential application of plasma signal peptide-complement C1r/C1s, Uegf and Bmp1-epidermal growth factor domain-containing protein 1 (SCUBE-1) as a biomarker in the diagnosis of pulmonary embolism (PE).

METHODS

This cross-sectional study enrolled 177 patients who underwent PE diagnostic test and 87 healthy controls. The results of CT pulmonary angiogram (CTPA) were used as reference standards for PE diagnosis. The levels of SCUBE-1 and D-dimer in participants' plasma were detected with enzyme-linked immunosorbent assay and compared among patients with confirmed PE, suspicious PE and healthy controls. The diagnostic values were analysed using receiver operating characteristic (ROC) curve analysis. In addition, differences in plasma SCUBE-1 levels were compared among patients with different risk stratifications.

RESULTS

The plasma SCUBE-1 concentration levels in patients with CTPA confirmed PE (14.28 ± 7.74 ng/ml) was significantly higher than those in the suspicious patients (11.11 ± 4.48 ng/ml) and in healthy control (4.40 ± 3.23 ng/ml) (P < 0.01). ROC curve analysis showed that at the cut-off of 7.789 ng/ml, SCUBE-1 has significant diagnostic value in differentiating PE patients from healthy control (AUC = 0.919, sensitivity = 81.25%, specificity = 92.13%), and the performance is more accurate than D-dimer (cut-off 273.4 ng/ml, AUC = 0.648, sensitivity = 65.75%, specificity = 67.42%). The combination of D-dimer with SCUBE-1 did not further improve the diagnostic value. However, SCUBE-1 did not show significant diagnostic value in identifying PE among suspicious patients There was no significant difference in SCUBE-1 level among different risk groups (P > 0.05).

CONCLUSION

We believe that SCUBE-1 could be a potential coagulation-related marker for the diagnosis of PE.

Diagnostic Delays in Sepsis: Lessons Learned From a Retrospective Study of Canadian Medico-Legal Claims

Although rapid treatment improves outcomes for patients presenting with sepsis, early detection can be difficult, especially in otherwise healthy adults.

OBJECTIVES

Using medico-legal data, we aimed to identify areas of focus to assist with early recognition of sepsis.

DESIGN SETTING AND PARTICIPANTS

Retrospective descriptive design. We analyzed closed medico-legal cases involving physicians from a national database repository at the Canadian Medical Protective Association. The study included cases closed between 2011 and 2020 that had documented peer expert criticism of a diagnostic issue related to sepsis or relevant infections.

MAIN OUTCOMES AND MEASURES

We used univariate statistics to describe patients and physicians and applied published frameworks to classify contributing factors (provider, team, system) and diagnostic pitfalls based on peer expert criticisms.

RESULTS

Of 162 involved patients, the median age was 53 years (interquartile range [IQR], 34-66 yr) and mortality was 49%. Of 218 implicated physicians, 169 (78%) were from family medicine, emergency medicine, or surgical specialties. Eighty patients (49%) made multiple visits to outpatient care leading up to sepsis recognition/hospitalization (median = two visits; IQR, 2-4). Almost 40% of patients were admitted to the ICU. Deficient assessments, such as failing to consider sepsis or not reassessing the patient prior to discharge, contributed to the majority of cases (81%).

CONCLUSIONS AND RELEVANCE

Sepsis continues to be a challenging diagnosis for clinicians. Multiple visits to outpatient care may be an early warning sign requiring vigilance in the patient assessment.

Clinical-GAN: Trajectory forecasting of clinical events using transformer and Generative Adversarial Networks

Predicting the trajectory of a disease at an early stage can aid physicians in offering effective treatment, prompt care to patients, and also avoid misdiagnosis. However, forecasting patient trajectories is challenging due to long-range dependencies, irregular intervals between consecutive admissions, and non-stationarity data. To address these challenges, we propose a novel method called Clinical-GAN, a Transformer-based Generative Adversarial Networks (GAN) to forecast the patients' medical codes for subsequent visits. First, we represent the patients' medical codes as a time-ordered sequence of tokens akin to language models. Then, a Transformer mechanism is used as a Generator to learn from existing patients' medical history and is trained adversarially against a Transformer-based Discriminator. We address the above mentioned challenges based on our data modeling and Transformer-based GAN architecture. Additionally, we enable the local interpretation of the model's prediction using a multi-head attention mechanism. We evaluated our method using a publicly available dataset, Medical Information Mart for Intensive Care IV v1.0 (MIMIC-IV), with more than 500,000 visits completed by around 196,000 adult patients over an 11-year period from 2008-2019. Clinical-GAN significantly outperforms baseline methods and existing works, as demonstrated through various experiments. Source code is at https://github.com/vigi30/Clinical-GAN.

A Machine Learning Approach to Support Urgent Stroke Triage Using Administrative Data and Social Determinants of Health at Hospital Presentation: Retrospective Study

BACKGROUND

The key to effective stroke management is timely diagnosis and triage. Machine learning (ML) methods developed to assist in detecting stroke have focused on interpreting detailed clinical data such as clinical notes and diagnostic imaging results. However, such information may not be readily available when patients are initially triaged, particularly in rural and underserved communities.

OBJECTIVE

This study aimed to develop an ML stroke prediction algorithm based on data widely available at the time of patients' hospital presentations and assess the added value of social determinants of health (SDoH) in stroke prediction.

METHODS

We conducted a retrospective study of the emergency department and hospitalization records from 2012 to 2014 from all the acute care hospitals in the state of Florida, merged with the SDoH data from the American Community Survey. A case-control design was adopted to construct stroke and stroke mimic cohorts. We compared the algorithm performance and feature importance measures of the ML models (ie, gradient boosting machine and random forest) with those of the logistic regression model based on 3 sets of predictors. To provide insights into the prediction and ultimately assist care providers in decision-making, we used TreeSHAP for tree-based ML models to explain the stroke prediction.

RESULTS

Our analysis included 143,203 hospital visits of unique patients, and it was confirmed based on the principal diagnosis at discharge that 73% (n=104,662) of these patients had a stroke. The approach proposed in this study has high sensitivity and is particularly effective at reducing the misdiagnosis of dangerous stroke chameleons (false-negative rate <4%). ML classifiers consistently outperformed the benchmark logistic regression in all 3 input combinations. We found significant consistency across the models in the features that explain their performance. The most important features are age, the number of chronic conditions on admission, and primary payer (eg, Medicare or private insurance). Although both the individual- and community-level SDoH features helped improve the predictive performance of the models, the inclusion of the individual-level SDoH features led to a much larger improvement (area under the receiver operating characteristic curve increased from 0.694 to 0.823) than the inclusion of the community-level SDoH features (area under the receiver operating characteristic curve increased from 0.823 to 0.829).

CONCLUSIONS

Using data widely available at the time of patients' hospital presentations, we developed a stroke prediction model with high sensitivity and reasonable specificity. The prediction algorithm uses variables that are routinely collected by providers and payers and might be useful in underresourced hospitals with limited availability of sensitive diagnostic tools or incomplete data-gathering capabilities.

IKAR: An Interdisciplinary Knowledge-Based Automatic Retrieval Method from Chinese Electronic Medical Record

To date, information retrieval methods in the medical field have mainly focused on English medical reports, but little work has studied Chinese electronic medical reports, especially in the field of obstetrics and gynecology. In this paper, a dataset of 180,000 complete Chinese ultrasound reports in obstetrics and gynecology was established and made publicly available. Based on the ultrasound reports in the dataset, a new information retrieval method (IKAR) is proposed to extract key information from the ultrasound reports and automatically generate the corresponding ultrasound diagnostic results. The model can both extract what is already in the report and analyze what is not in the report by inference. After applying the IKAR method to the dataset, it is proved that the method could achieve 89.38% accuracy, 91.09% recall, and 90.23% F-score. Moreover, the method achieves an F-score of over 90% on 50% of the 10 components of the report. This study provides a quality dataset for the field of electronic medical records and offers a reference for information retrieval methods in the field of obstetrics and gynecology or in other fields.