Validation of an electronic trigger to measure missed diagnosis of stroke in emergency departments

Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach

Objective

To describe a user-centered approach to develop, pilot test, and refine requirements for 3 electronic health record (EHR)-integrated interventions that target key diagnostic process failures in hospitalized patients.

Materials and Methods

Three interventions were prioritized for development: a Diagnostic Safety Column (DSC) within an EHR-integrated dashboard to identify at-risk patients; a Diagnostic Time-Out (DTO) for clinicians to reassess the working diagnosis; and a Patient Diagnosis Questionnaire (PDQ) to gather patient concerns about the diagnostic process. Initial requirements were refined from analysis of test cases with elevated risk predicted by DSC logic compared to risk perceived by a clinician working group; DTO testing sessions with clinicians; PDQ responses from patients; and focus groups with clinicians and patient advisors using storyboarding to model the integrated interventions. Mixed methods analysis of participant responses was used to identify final requirements and potential implementation barriers.

Results

Final requirements from analysis of 10 test cases predicted by the DSC, 18 clinician DTO participants, and 39 PDQ responses included the following: DSC configurable parameters (variables, weights) to adjust baseline risk estimates in real-time based on new clinical data collected during hospitalization; more concise DTO wording and flexibility for clinicians to conduct the DTO with or without the patient present; and integration of PDQ responses into the DSC to ensure closed-looped communication with clinicians. Analysis of focus groups confirmed that tight integration of the interventions with the EHR would be necessary to prompt clinicians to reconsider the working diagnosis in cases with elevated diagnostic error (DE) risk or uncertainty. Potential implementation barriers included alert fatigue and distrust of the risk algorithm (DSC); time constraints, redundancies, and concerns about disclosing uncertainty to patients (DTO); and patient disagreement with the care team's diagnosis (PDQ).

Discussion

A user-centered approach led to evolution of requirements for 3 interventions targeting key diagnostic process failures in hospitalized patients at risk for DE.

Conclusions

We identify challenges and offer lessons from our user-centered design process.

Increased Stroke Risk Following Herpes Zoster Infection and Protection With Zoster Vaccine

BACKGROUND

Studies evaluating stroke following varicella zoster virus (VZV) infection are limited, and the utility of zoster vaccination against this phenomenon is unclear. This study aimed to determine the risk of stroke 30 days following zoster infection and to evaluate the impact of zoster vaccinations on the risk of stroke in VZV-infected patients.

METHODS

This retrospective case-control study was conducted from January 2010 to January 2020 utilizing nationwide patient data retrieved from the Veterans Affairs' Corporate Data Warehouse.

RESULTS

A total of 2 165 505 patients ≥18 years of age who received care at a Veterans Affairs facility were included in the study, of whom 71 911 had a history of zoster infection. Zoster patients were found to have 1.9 times increased likelihood of developing a stroke within 30 days following infection (odds ratio [OR], 1.93 [95% confidence interval {CI}, 1.57-2.4]; P < .0001). A decreased risk of stroke was seen in patients who received the recombinant zoster vaccine (OR, 0.57 [95% CI, .46-.72]; P < .0001) or the live zoster vaccine (OR, 0.77 [95% CI, .65-.91]; P = .002).

CONCLUSIONS

Patients had a significantly higher risk of stroke within the first month following recent herpes zoster infection. Receipt of at least 1 zoster vaccination was found to mitigate this increased risk. Vaccination may therefore be viewed as a protective tool against the risk of neurologic postinfection sequelae.

A structured approach to EHR surveillance of diagnostic error in acute care: an exploratory analysis of two institutionally-defined case cohorts

OBJECTIVES

To test a structured electronic health record (EHR) case review process to identify diagnostic errors (DE) and diagnostic process failures (DPFs) in acute care.

METHODS

We adapted validated tools (Safer Dx, Diagnostic Error Evaluation Research [DEER] Taxonomy) to assess the diagnostic process during the hospital encounter and categorized 13 postulated e-triggers. We created two test cohorts of all preventable cases (n=28) and an equal number of randomly sampled non-preventable cases (n=28) from 365 adult general medicine patients who expired and underwent our institution's mortality case review process. After excluding patients with a length of stay of more than one month, each case was reviewed by two blinded clinicians trained in our process and by an expert panel. Inter-rater reliability was assessed. We compared the frequency of DE contributing to death in both cohorts, as well as mean DPFs and e-triggers for DE positive and negative cases within each cohort.

RESULTS

Twenty-seven (96.4%) preventable and 24 (85.7%) non-preventable cases underwent our review process. Inter-rater reliability was moderate between individual reviewers (Cohen's kappa 0.41) and substantial with the expert panel (Cohen's kappa 0.74). The frequency of DE contributing to death was significantly higher for the preventable compared to the non-preventable cohort (56% vs. 17%, OR 6.25 [1.68, 23.27], p<0.01). Mean DPFs and e-triggers were significantly and non-significantly higher for DE positive compared to DE negative cases in each cohort, respectively.

CONCLUSIONS

We observed substantial agreement among final consensus and expert panel reviews using our structured EHR case review process. DEs contributing to death associated with DPFs were identified in institutionally designated preventable and non-preventable cases. While e-triggers may be useful for discriminating DE positive from DE negative cases, larger studies are required for validation. Our approach has potential to augment institutional mortality case review processes with respect to DE surveillance.

Learning from Missed Opportunities Through Reflective Practice

Identification of diagnostic errors is difficult but is not alone sufficient for performance improvement. Instead, cases must be reflected on to identify ways to improve decision-making in the future. There are many tools and modalities to retrospectively reflect on action to study medical decisions and outcomes and improve future performance. Reflection in action-in which diagnostic decisions are considered in real-time-may also improve medical decision-making especially through strategies such as structured reflection. Ongoing regular feedback can normalize the discussion about improving decision-making, enable reflective practice, and improve decision making.