What's going well: a qualitative analysis of positive patient and family feedback in the context of the diagnostic process

OBJECTIVES

Accurate and timely diagnosis relies on close collaboration between patients/families and clinicians. Just as patients have unique insights into diagnostic breakdowns, positive patient feedback may also generate broader perspectives on what constitutes a "good" diagnostic process (DxP).

METHODS

We evaluated patient/family feedback on "what's going well" as part of an online pre-visit survey designed to engage patients/families in the DxP. Patients/families living with chronic conditions with visits in three urban pediatric subspecialty clinics (site 1) and one rural adult primary care clinic (site 2) were invited to complete the survey between December 2020 and March 2022. We adapted the Healthcare Complaints Analysis Tool (HCAT) to conduct a qualitative analysis on a subset of patient/family responses with ≥20 words.

RESULTS

In total, 7,075 surveys were completed before 18,129 visits (39 %) at site 1, and 460 surveys were completed prior to 706 (65 %) visits at site 2. Of all participants, 1,578 volunteered positive feedback, ranging from 1-79 words. Qualitative analysis of 272 comments with ≥20 words described: Relationships (60 %), Clinical Care (36 %), and Environment (4 %). Compared to primary care, subspecialty comments showed the same overall rankings. Within Relationships, patients/families most commonly noted: thorough and competent attention (46 %), clear communication and listening (41 %) and emotional support and human connection (39 %). Within Clinical Care, patients highlighted: timeliness (31 %), effective clinical management (30 %), and coordination of care (25 %).

CONCLUSIONS

Patients/families valued relationships with clinicians above all else in the DxP, emphasizing the importance of supporting clinicians to nurture effective relationships and relationship-centered care in the DxP.

Comparative Perspectives on Diagnostic Error Discussions Between Inpatient and Outpatient Pediatric Providers

Diagnostic error remains understudied and underaddressed despite causing significant morbidity and mortality. One barrier to addressing this issue remains provider discomfort. Survey studies have shown significantly more discomfort among providers in discussing diagnostic error compared with other forms of error. Whether the comfort in discussing diagnostic error differs depending on practice setting has not been previously studied. The objective of this study was to assess differences in provider willingness to discuss diagnostic error in the inpatient versus outpatient setting. A multicenter survey was sent out to 3881 providers between May and June 2018. This survey was designed to assess comfort level of discussing diagnostic error and looking at barriers to discussing diagnostic error. Forty-three percent versus 22% of inpatient versus outpatient providers (P = 0.004) were comfortable discussing short-term diagnostic error publicly. Similarly, 76% versus 60% of inpatient versus outpatient providers (P = 0.010) were comfortable discussing short-term diagnostic error privately. A higher percentage of inpatient (64%) compared with outpatient providers (46%) (P = 0.043) were comfortable discussing long-term diagnostic error privately. Forty percent versus 24% of inpatient versus outpatient providers (P = 0.018) were comfortable discussing long-term error publicly. No difference in barriers cited depending on practice setting. Inpatient providers are more comfortable discussing diagnostic error than their outpatient counterparts. More study is needed to determine the etiology of this discrepancy and to develop strategies to increase outpatient provider comfort.

OUTCOMES OF MISSED DIAGNOSIS OF PEDIATRIC APPENDICITIS, NEW-ONSET DIABETIC KETOACIDOSIS, AND SEPSIS IN FIVE PEDIATRIC HOSPITALS

BACKGROUND

Missed diagnosis can predispose to worse condition-specific outcomes.

OBJECTIVE

To determine 90-day complication rates and hospital utilization after a missed diagnosis of pediatric appendicitis, new-onset diabetic ketoacidosis (DKA), and sepsis.

METHODS

We evaluated patients under 21 years of age visiting five pediatric emergency departments (EDs) with a study condition. Case patients had a preceding ED visit within 7 days of diagnosis and underwent case review to confirm a missed diagnosis. Control patients had no preceding ED visit. We compared complication rates and utilization between case and control patients after adjusting for age, sex, and insurance.

RESULTS

We analyzed 29,398 children with appendicitis, 5366 with DKA, and 3622 with sepsis, of whom 429, 33, and 46, respectively, had a missed diagnosis. Patients with missed diagnosis of appendicitis or DKA had more hospital days and readmissions; there were no significant differences for those with sepsis. Those with missed appendicitis were more likely to have abdominal abscess drainage (adjusted odds ratio [aOR] 3.0, 95% confidence interval [CI] 2.4-3.6) or perforated appendicitis (aOR 3.1, 95% CI 2.5-3.8). Those with missed DKA were more likely to have cerebral edema (aOR 4.6, 95% CI 1.5-11.3), mechanical ventilation (aOR 13.4, 95% CI 3.8-37.1), or death (aOR 28.4, 95% CI 1.4-207.5). Those with missed sepsis were less likely to have mechanical ventilation (aOR 0.5, 95% CI 0.2-0.9). Other illness complications were not significantly different by missed diagnosis.

CONCLUSIONS

Children with delayed diagnosis of appendicitis or new-onset DKA had a higher risk of 90-day complications and hospital utilization than those with a timely diagnosis.

Developing, pilot testing, and refining requirements for 3 EHR-integrated interventions to improve diagnostic safety in acute care: a user-centered approach

Objective

To describe a user-centered approach to develop, pilot test, and refine requirements for 3 electronic health record (EHR)-integrated interventions that target key diagnostic process failures in hospitalized patients.

Materials and Methods

Three interventions were prioritized for development: a Diagnostic Safety Column (DSC) within an EHR-integrated dashboard to identify at-risk patients; a Diagnostic Time-Out (DTO) for clinicians to reassess the working diagnosis; and a Patient Diagnosis Questionnaire (PDQ) to gather patient concerns about the diagnostic process. Initial requirements were refined from analysis of test cases with elevated risk predicted by DSC logic compared to risk perceived by a clinician working group; DTO testing sessions with clinicians; PDQ responses from patients; and focus groups with clinicians and patient advisors using storyboarding to model the integrated interventions. Mixed methods analysis of participant responses was used to identify final requirements and potential implementation barriers.

Results

Final requirements from analysis of 10 test cases predicted by the DSC, 18 clinician DTO participants, and 39 PDQ responses included the following: DSC configurable parameters (variables, weights) to adjust baseline risk estimates in real-time based on new clinical data collected during hospitalization; more concise DTO wording and flexibility for clinicians to conduct the DTO with or without the patient present; and integration of PDQ responses into the DSC to ensure closed-looped communication with clinicians. Analysis of focus groups confirmed that tight integration of the interventions with the EHR would be necessary to prompt clinicians to reconsider the working diagnosis in cases with elevated diagnostic error (DE) risk or uncertainty. Potential implementation barriers included alert fatigue and distrust of the risk algorithm (DSC); time constraints, redundancies, and concerns about disclosing uncertainty to patients (DTO); and patient disagreement with the care team's diagnosis (PDQ).

Discussion

A user-centered approach led to evolution of requirements for 3 interventions targeting key diagnostic process failures in hospitalized patients at risk for DE.

Conclusions

We identify challenges and offer lessons from our user-centered design process.

What Can We Learn From In-Depth Analysis of Human Errors Resulting in Diagnostic Errors in the Emergency Department: An Analysis of Serious Adverse Event Reports

INTRODUCTION

Human error plays a vital role in diagnostic errors in the emergency department. A thorough analysis of these human errors, using information-rich reports of serious adverse events (SAEs), could help to better study and understand the causes of these errors and formulate more specific recommendations.

METHODS

We studied 23 SAE reports of diagnostic events in emergency departments of Dutch general hospitals and identified human errors. Two researchers independently applied the Safer Dx Instrument, Diagnostic Error Evaluation and Research Taxonomy, and the Model of Unsafe acts to analyze reports.

RESULTS

Twenty-one reports contained a diagnostic error, in which we identified 73 human errors, which were mainly based on intended actions (n = 69) and could be classified as mistakes (n = 56) or violations (n = 13). Most human errors occurred during the assessment and testing phase of the diagnostic process.

DISCUSSION

The combination of different instruments and information-rich SAE reports allowed for a deeper understanding of the mechanisms underlying diagnostic error. Results indicated that errors occurred most often during the assessment and the testing phase of the diagnostic process. Most often, the errors could be classified as mistakes and violations, both intended actions. These types of errors are in need of different recommendations for improvement, as mistakes are often knowledge based, whereas violations often happen because of work and time pressure. These analyses provided valuable insights for more overarching recommendations to improve diagnostic safety and would be recommended to use in future research and analysis of (serious) adverse events.

Development and Usability Testing of the Agency for Healthcare Research and Quality Common Formats to Capture Diagnostic Safety Events

OBJECTIVES

A lack of consensus around definitions and reporting standards for diagnostic errors limits the extent to which healthcare organizations can aggregate, analyze, share, and learn from these events. In response to this problem, the Agency for Healthcare Research and Quality (AHRQ) began the development of the Common Formats for Event Reporting for Diagnostic Safety Events (CFER-DS). We conducted a usability assessment of the draft CFER-DS to inform future revision and implementation.

METHODS

We recruited a purposive sample of quality and safety personnel working in 8 U.S. healthcare organizations. Participants were invited to use the CFER-DS to simulate reporting for a minimum of 5 cases of diagnostic safety events and then provide written and verbal qualitative feedback. Analysis focused on participants' perceptions of content validity, ease of use, and potential for implementation.

RESULTS

Estimated completion time was 30 to 90 minutes per event. Participants shared generally positive feedback about content coverage and item clarity but identified reporter burden as a potential concern. Participants also identified opportunities to clarify several conceptual definitions, ensure applicability across different care settings, and develop guidance to operationalize use of CFER-DS. Findings led to refinement of content and supplementary materials to facilitate implementation.

CONCLUSIONS

Standardized definitions of diagnostic safety events and reporting standards for contextual information and contributing factors can help capture and analyze diagnostic safety events. In addition to usability testing, additional feedback from the field will ensure that AHRQ's CFER-DS is useful to a broad range of users for learning and safety improvement.

Advancing Diagnostic Safety Research: Results of a Systematic Research Priority Setting Exercise

BACKGROUND

Diagnostic errors are a major source of preventable harm but the science of reducing them remains underdeveloped.

OBJECTIVE

To identify and prioritize research questions to advance the field of diagnostic safety in the next 5 years.

PARTICIPANTS

Ninety-seven researchers and 42 stakeholders were involved in the identification of the research priorities.

DESIGN

We used systematic prioritization methods based on the Child Health and Nutrition Research Initiative (CHNRI) methodology. We first invited a large international group of expert researchers in various disciplines to submit research questions while considering five prioritization criteria: (1) usefulness, (2) answerability, (3) effectiveness, (4) potential for translation, and (5) maximal potential for effect on diagnostic safety. After consolidation, these questions were prioritized at an in-person expert meeting in April 2019. Top-ranked questions were subsequently reprioritized through scoring on the five prioritization criteria using an online questionnaire. We also invited non-research stakeholders to assign weights to the five criteria and then used these weights to adjust the final prioritization score for each question.

KEY RESULTS

Of the 207 invited researchers, 97 researchers responded and 78 submitted 333 research questions which were then consolidated. Expert meeting participants (n = 21) discussed questions in different breakout sessions and prioritized 50, which were subsequently reduced to the top 20 using the online questionnaire. The top 20 questions addressed mostly system factors (e.g., implementation and evaluation of information technologies), teamwork factors (e.g., role of nurses and other health professionals in the diagnostic process), and strategies to engage patients in the diagnostic process.

CONCLUSIONS

Top research priorities for advancing diagnostic safety in the short-term include strengthening systems and teams and engaging patients to support diagnosis. High-priority areas identified using these systematic methods can inform an actionable research agenda for reducing preventable diagnostic harm.

Development of a rubric for assessing delayed diagnosis of appendicitis, diabetic ketoacidosis and sepsis

OBJECTIVES

Using case review to determine whether a patient experienced a delayed diagnosis is challenging. Measurement would be more accurate if case reviewers had access to multi-expert consensus on grading the likelihood of delayed diagnosis. Our objective was to use expert consensus to create a guide for objectively grading the likelihood of delayed diagnosis of appendicitis, new-onset diabetic ketoacidosis (DKA), and sepsis.

METHODS

Case vignettes were constructed for each condition. In each vignette, a patient has the condition and had a previous emergency department (ED) visit within 7 days. Condition-specific multi-specialty expert Delphi panels reviewed the case vignettes and graded the likelihood of a delayed diagnosis on a five-point scale. Delayed diagnosis was defined as the condition being present during the previous ED visit. Consensus was defined as ≥75% agreement. In each Delphi round, panelists were given the scores from the previous round and asked to rescore. A case scoring guide was created from the consensus scores.

RESULTS

Eighteen expert panelists participated. Consensus was achieved within three Delphi rounds for all appendicitis and sepsis vignettes. We reached consensus on 23/30 (77%) DKA vignettes. A case review guide was created from the consensus scores.

CONCLUSIONS

Multi-specialty expert reviewers can agree on the likelihood of a delayed diagnosis for cases of appendicitis and sepsis, and for most cases of DKA. We created a guide that can be used by researchers and quality improvement specialists to allow for objective case review to determine when delayed diagnoses have occurred for appendicitis, DKA, and sepsis.

Operational measurement of diagnostic safety: state of the science

Reducing the incidence of diagnostic errors is increasingly a priority for government, professional, and philanthropic organizations. Several obstacles to measurement of diagnostic safety have hampered progress toward this goal. Although a coordinated national strategy to measure diagnostic safety remains an aspirational goal, recent research has yielded practical guidance for healthcare organizations to start using measurement to enhance diagnostic safety. This paper, concurrently published as an Issue Brief by the Agency for Healthcare Research and Quality, issues a "call to action" for healthcare organizations to begin measurement efforts using data sources currently available to them. Our aims are to outline the state of the science and provide practical recommendations for organizations to start identifying and learning from diagnostic errors. Whether by strategically leveraging current resources or building additional capacity for data gathering, nearly all organizations can begin their journeys to measure and reduce preventable diagnostic harm.

Responsibility for follow-up during the diagnostic process in primary care: a secondary analysis of International Cancer Benchmarking Partnership data

BACKGROUND

It is unclear to what extent primary care practitioners (PCPs) should retain responsibility for follow-up to ensure that patients are monitored until their symptoms or signs are explained.

AIM

To explore the extent to which PCPs retain responsibility for diagnostic follow-up actions across 11 international jurisdictions.

DESIGN AND SETTING

A secondary analysis of survey data from the International Cancer Benchmarking Partnership.

METHOD

The authors counted the proportion of 2879 PCPs who retained responsibility for each area of follow-up (appointments, test results, and non-attenders). Proportions were weighted by the sample size of each jurisdiction. Pooled estimates were obtained using a random-effects model, and UK estimates were compared with non-UK ones. Free-text responses were analysed to contextualise quantitative findings using a modified grounded theory approach.

RESULTS

PCPs varied in their retention of responsibility for follow-up from 19% to 97% across jurisdictions and area of follow-up. Test reconciliation was inadequate in most jurisdictions. Significantly fewer UK PCPs retained responsibility for test result communication (73% versus 85%, P = 0.04) and non-attender follow-up (78% versus 93%, P<0.01) compared with non-UK PCPs. PCPs have developed bespoke, inconsistent solutions to follow-up. In cases of greatest concern, 'double safety netting' is described, where both patient and PCP retain responsibility.

CONCLUSION

The degree to which PCPs retain responsibility for follow-up is dependent on their level of concern about the patient and their primary care system's properties. Integrated systems to support follow-up are at present underutilised, and research into their development, uptake, and effectiveness seems warranted.

Accuracy of the Safer Dx Instrument to Identify Diagnostic Errors in Primary Care

IMPORTANCE

Diagnostic errors are common and harmful, but difficult to define and measure. Measurement of diagnostic errors often depends on retrospective medical record reviews, frequently resulting in reviewer disagreement.

OBJECTIVES

We aimed to test the accuracy of an instrument to help detect presence or absence of diagnostic error through record reviews.

DESIGN

We gathered questions from several previously used instruments for diagnostic error measurement, then developed and refined our instrument. We tested the accuracy of the instrument against a sample of patient records (n = 389), with and without previously identified diagnostic errors (n = 129 and n = 260, respectively).

RESULTS

The final version of our instrument (titled Safer Dx Instrument) consisted of 11 questions assessing diagnostic processes in the patient-provider encounter and a main outcome question to determine diagnostic error. In comparison with the previous sample, the instrument yielded an overall accuracy of 84 %, sensitivity of 71 %, specificity of 90 %, negative predictive value of 86 %, and positive predictive value of 78 %. All 11 items correlated significantly with the instrument's error outcome question (all p values ≤ 0.01). Using factor analysis, the 11 questions clustered into two domains with high internal consistency (initial diagnostic assessment, and performance and interpretation of diagnostic tests) and a patient factor domain with low internal consistency (Cronbach's alpha coefficients 0.93, 0.92, and 0.38, respectively).

CONCLUSIONS

The Safer Dx Instrument helps quantify the likelihood of diagnostic error in primary care visits, achieving a high degree of accuracy for measuring their presence or absence. This instrument could be useful to identify high-risk cases for further study and quality improvement.