Diagnostic error in mental health: a review

Diagnostic errors are associated with patient harm and suboptimal outcomes. Despite national scientific efforts to advance definition, measurement and interventions for diagnostic error, diagnosis in mental health is not well represented in this ongoing work. We aimed to summarise the current state of research on diagnostic errors in mental health and identify opportunities to align future research with the emerging science of diagnostic safety. We review conceptual considerations for defining and measuring diagnostic error, the application of these concepts to mental health settings, and the methods and subject matter focus of recent studies of diagnostic error in mental health. We found that diagnostic error is well understood to be a problem in mental healthcare. Although few studies used clear definitions or frameworks for understanding diagnostic error in mental health, several studies of missed, wrong, delayed and disparate diagnosis of common mental disorders have identified various avenues for future research and development. Nevertheless, a lack of clear consensus on how to conceptualise, define and measure errors in diagnosis will pose a barrier to advancement. Further research should focus on identifying preventable missed opportunities in the diagnosis of mental disorders, which may uncover generalisable opportunities for improvement.

Patient generated research priorities to improve diagnostic safety: A systematic prioritization exercise

BACKGROUND

Most people experience a diagnostic error at least once in their lifetime. Patients' experiences with their diagnosis could provide important insights when setting research priorities to reduce diagnostic error.

OBJECTIVE

Our objective was to engage patients in research agenda setting for improving diagnosis.

PATIENT INVOLVEMENT

Patients were involved in generating, discussing, prioritizing, and ranking of research questions for diagnostic error reduction.

METHODS

We used the prioritization methodology based on the Child Health and Nutrition Research Initiative (CHNRI). We first solicited research questions important for diagnostic error reduction from a large group of patients. Thirty questions were initially prioritized at an in-person meeting with 8 patients who were supported by 4 researchers. The resulting list was further prioritized by patients who scored questions on five predefined criteria. We then applied previously determined weights to these prioritization criteria to adjust the final prioritization score for each question, resulting in 10 highest priority research questions.

RESULTS

Forty-one patients submitted 171 research questions. After prioritization, the highest priority topics included better care coordination across the diagnostic continuum and improving care transitions, improved identification and measurement of diagnostic errors and attention for implicit bias towards patients who are vulnerable to diagnostic errors.

DISCUSSION

We systematically identified the top-10 patient generated research priorities for diagnostic error reduction using transparent and objective methods. Patients prioritized different research questions than researchers and therefore complemented an agenda previously generated by researchers.

PRACTICAL VALUE

Research priorities identified by patients can be used by funders and researchers to conduct future research focused on reducing diagnostic errors.

FUNDING

This project was funded by the Gordon and Betty Moore Foundation.

Defining Diagnostic Error: A Scoping Review to Assess the Impact of the National Academies' Report Improving Diagnosis in Health Care

BACKGROUND

Standards for accurate and timely diagnosis are ill-defined. In 2015, the National Academies of Science, Engineering, and Medicine (NASEM) committee published a landmark report, Improving Diagnosis in Health Care , and proposed a new definition of diagnostic error, "the failure to ( a ) establish an accurate and timely explanation of the patient's health problem(s) or ( b ) communicate that explanation to the patient."

OBJECTIVE

This study aimed to explore how researchers operationalize the NASEM's definition of diagnostic error with relevance to accuracy, timeliness, and/or communication in peer-reviewed published literature.

METHODS

Using the Arskey and O'Malley's framework framework, we identified published literature from October 2015 to February 2021 using Medline and Google Scholar. We also conducted subject matter expert interviews with researchers.

RESULTS

Of 34 studies identified, 16 were analyzed and abstracted to determine how diagnostic error was operationalized and measured. Studies were grouped by theme: epidemiology, patient focus, measurement/surveillance, and clinician focus. Nine studies indicated using the NASEM definition. Of those, 5 studies also operationalized with existing definitions proposed before the NASEM report. Four studies operationalized the components of the NASEM definition and did not cite existing definitions. Three studies operationalized error using existing definitions only. Subject matter experts indicated that the NASEM definition functions as foundation for researchers to conceptualize diagnostic error.

CONCLUSIONS

The NASEM report produced a common understanding of diagnostic error that includes accuracy, timeliness, and communication. In recent peer-reviewed literature, most researchers continue to use pre-NASEM report definitions to operationalize accuracy and timeliness. The report catalyzed the use of patient-centered concepts in the definition, resulting in emerging studies focused on examining errors related to communicating diagnosis to patients.

Creating a Learning Health System for Improving Diagnostic Safety: Pragmatic Insights from US Health Care Organizations

OBJECTIVE

To identify challenges and pragmatic strategies for improving diagnostic safety at an organizational level using concepts from learning health systems METHODS: We interviewed 32 safety leaders across the USA on how their organizations approach diagnostic safety. Participants were recruited through email and represented geographically diverse academic and non-academic settings. The interview included questions on culture of reporting and learning from diagnostic errors; data gathering and analysis activities; diagnostic training and educational activities; and engagement of clinical leadership, staff, patients, and families in diagnostic safety activities. We conducted an inductive content analysis of interview transcripts and two reviewers coded all data.

RESULTS

Of 32 participants, 12 reported having a specific program to address diagnostic errors. Multiple barriers to implement diagnostic safety activities emerged: serious concerns about psychological safety associated with diagnostic error; lack of infrastructure for measurement, monitoring, and improvement activities related to diagnosis; lack of leadership investment, which was often diverted to competing priorities related to publicly reported measures or other incentives; and lack of dedicated teams to work on diagnostic safety. Participants provided several strategies to overcome barriers including adapting trigger tools to identify safety events, engaging patients in diagnostic safety, and appointing dedicated diagnostic safety champions.

CONCLUSIONS

Several foundational building blocks related to learning health systems could inform organizational efforts to reduce diagnostic error. Promoting an organizational culture specific to diagnostic safety, using science and informatics to improve measurement and analysis, leadership incentives to build institutional capacity to address diagnostic errors, and patient engagement in diagnostic safety activities can enable progress.

Electronic Co-design (ECO-design) Workshop for Increasing Clinician Participation in the Design of Health Services Interventions: Participatory Design Approach

BACKGROUND

Participation from clinician stakeholders can improve the design and implementation of health care interventions. Participatory design methods, especially co-design methods, comprise stakeholder-led design activities that are time-consuming. Competing work demands and increasing workloads make clinicians' commitments to typical participatory methods even harder. The COVID-19 pandemic further exacerbated barriers to clinician participation in such interventions.

OBJECTIVE

The aim of this study was to explore a web-based participatory design approach to conduct economical, electronic co-design (ECO-design) workshops with primary care clinicians.

METHODS

We adapted traditional in-person co-design workshops to web-based delivery and adapted co-design workshop series to fit within a single 1-hour session. We applied the ECO-design workshop approach to codevelop feedback interventions regarding abnormal test result follow-up in primary care. We conducted ECO-design workshops with primary care clinicians at a medical center in Southern Texas, using videoconferencing software. Each workshop focused on one of three types of feedback interventions: conversation guide, email template, and dashboard prototype. We paired electronic materials and software features to facilitate participant interactions, prototyping, and data collection. The workshop protocol included four main activities: problem identification, solution generation, prototyping, and debriefing. Two facilitators were assigned to each workshop and one researcher resolved technical problems. After the workshops, our research team met to debrief and evaluate workshops.

RESULTS

A total of 28 primary care clinicians participated in our ECO-design workshops. We completed 4 parallel workshops, each with 5-10 participants. We conducted traditional analyses and generated a clinician persona (ie, representative description) and user interface prototypes. We also formulated recommendations for future ECO-design workshop recruitment, technology, facilitation, and data collection. Overall, our adapted workshops successfully enabled primary care clinicians to participate without increasing their workload, even during a pandemic.

CONCLUSIONS

ECO-design workshops are viable, economical alternatives to traditional approaches. This approach fills a need for efficient methods to involve busy clinicians in the design of health care interventions.

Inviting patients to identify diagnostic concerns through structured evaluation of their online visit notes

BACKGROUND

The 21st Century Cures Act mandates patients' access to their electronic health record (EHR) notes. To our knowledge, no previous work has systematically invited patients to proactively report diagnostic concerns while documenting and tracking their diagnostic experiences through EHR-based clinician note review.

OBJECTIVE

To test if patients can identify concerns about their diagnosis through structured evaluation of their online visit notes.

METHODS

In a large integrated health system, patients aged 18-85 years actively using the patient portal and seen between October 2019 and February 2020 were invited to respond to an online questionnaire if an EHR algorithm detected any recent unexpected return visit following an initial primary care consultation ("at-risk" visit). We developed and tested an instrument (Safer Dx Patient Instrument) to help patients identify concerns related to several dimensions of the diagnostic process based on notes review and recall of recent "at-risk" visits. Additional questions assessed patients' trust in their providers and their general feelings about the visit. The primary outcome was a self-reported diagnostic concern. Multivariate logistic regression tested whether the primary outcome was predicted by instrument variables.

RESULTS

Of 293 566 visits, the algorithm identified 1282 eligible patients, of whom 486 responded. After applying exclusion criteria, 418 patients were included in the analysis. Fifty-one patients (12.2%) identified a diagnostic concern. Patients were more likely to report a concern if they disagreed with statements "the care plan the provider developed for me addressed all my medical concerns" [odds ratio (OR), 2.65; 95% confidence interval [CI], 1.45-4.87) and "I trust the provider that I saw during my visit" (OR, 2.10; 95% CI, 1.19-3.71) and agreed with the statement "I did not have a good feeling about my visit" (OR, 1.48; 95% CI, 1.09-2.01).

CONCLUSION

Patients can identify diagnostic concerns based on a proactive online structured evaluation of visit notes. This surveillance strategy could potentially improve transparency in the diagnostic process.

Why Test Results Are Still Getting "Lost" to Follow-up: a Qualitative Study of Implementation Gaps

BACKGROUND

Lack of timely follow-up of abnormal test results is common and has been implicated in missed or delayed diagnosis, resulting in potential for patient harm.

OBJECTIVE

As part of a larger project to implement change strategies to improve follow-up of diagnostic test results, this study sought to identify specifically where implementation gaps exist, as well as possible solutions identified by front-line staff.

DESIGN

We used a semi-structured interview guide to collect qualitative data from Veterans Affairs (VA) facility staff who had experience with test results management and patient safety.

SETTING

Twelve VA facilities across the USA.

PARTICIPANTS

Facility staff members (n = 27), including clinicians, lab and imaging professionals, nursing staff, patient safety professionals, and leadership.

APPROACH

We conducted a content analysis of interview transcripts to identify perceived barriers and high-risk areas for effective test result management, as well as recommendations for improvement.

RESULTS

We identified seven themes to guide further development of interventions to improve test result follow-up. Themes related to trainees, incidental findings, tracking systems for electronic health record notifications, outdated contact information, referrals, backup or covering providers, and responsibility for test results pending at discharge. Participants provided recommendations for improvement within each theme.

CONCLUSIONS

Perceived barriers and recommendations for improving test result follow-up often reflected previously known problems and their corresponding solutions, which have not been consistently implemented in practice. Better policy solutions and improvement methods, such as quality improvement collaboratives, may bridge the implementation gaps between knowledge and practice.

Technology-Based Closed-Loop Tracking for Improving Communication and Follow-up of Pathology Results

OBJECTIVE

Failure to follow-up on laboratory test results can lead to missed diagnoses, diagnostic delays, patient harm, and potential malpractice claims against providers. State-of-the-art tracking technologies such as the radio frequency identification (RFID) can potentially improve laboratory order processing and test result communication. We conducted a comparative evaluation of differences in completion rates for 5 testing process milestones and time to reach these process milestones, with and without RFID order tracking for skin biopsy orders.

METHODS

This observational study analyzed 48,515 orders from 20 dermatology providers, sent to 8 pathology laboratories in 2016 to 2017. Descriptive t tests and multiple Cox proportional hazard regressions were used to examine the differences in completion rates and times to the 5 testing process milestones, namely, (1) provider receipt of results, (2) provider review of results, (3) patient notification, (4) follow-up scheduling, and (5) order case closure, for orders processed with and without RFID order tracking.

RESULTS

Descriptive statistics illustrated that all 5 testing process milestone completion rates were statistically higher for RFID tracked orders compared with non-RFID tracked orders, and RFID tracked orders took 3 to 5 days lesser than non-RFID tracked orders to reach the 5 testing process milestones. Multiple cox proportional hazard regressions showed that the process milestones were achieved faster if orders were RFID tracked versus not (hazard ratios ranged from 1.3 to 4.9).

CONCLUSIONS

The RFID tracking technology considerably improved test result communication timeliness and reliability. Such technologies can be beneficial for laboratory order processing, and their effectiveness should be explored in other practice settings.

An Operational Framework to Study Diagnostic Errors in Emergency Departments: Findings From A Consensus Panel

OBJECTIVE

To create an operational definition and framework to study diagnostic error in the emergency department setting.

METHODS

We convened a 17-member multidisciplinary panel with expertise in general and pediatric emergency medicine, nursing, patient safety, informatics, cognitive psychology, social sciences, human factors, and risk management and a patient/caregiver advocate. We used a modified nominal group technique to develop a shared understanding to operationally define diagnostic errors in emergency care and modify the National Academies of Sciences, Engineering, and Medicine's conceptual process framework to this setting.

RESULTS

The expert panel defined diagnostic errors as "a divergence from evidence-based processes that increases the risk of poor outcomes despite the availability of sufficient information to provide a timely and accurate explanation of the patient's health problem(s)." Diagnostic processes include tasks related to (a) acuity recognition, information and synthesis, evaluation coordination, and (b) communication with patients/caregivers and other diagnostic team members. The expert panel also modified the National Academies of Sciences, Engineering, and Medicine's diagnostic process framework to incorporate influence of mode of arrival, triage level, and interventions during emergency care and underscored the importance of outcome feedback to emergency department providers to promote learning and improvement related to diagnosis.

CONCLUSIONS

The proposed operational definition and modified diagnostic process framework can potentially inform the development of measurement tools and strategies to study the epidemiology and interventions to improve emergency care diagnosis.

Patient and clinician experiences of uncertainty in the diagnostic process: Current understanding and future directions

OBJECTIVE

Uncertainty occurs throughout the diagnostic process and must be managed to facilitate accurate and timely diagnoses and treatments. Better characterization of uncertainty can inform strategies to manage it more effectively in clinical practice. We provide a comprehensive overview of current literature on diagnosis-related uncertainty describing (1) where patients and clinicians experience uncertainty within the diagnostic process, (2) how uncertainty affects the diagnostic process, (3) roots of uncertainty related to probability/risk, ambiguity, or complexity, and (4) strategies to manage uncertainty.

DISCUSSION

Each diagnostic process step involves uncertainty, including patient engagement with the healthcare system; information gathering, interpretation, and integration; formulating working diagnoses; and communicating diagnoses to patients. General management strategies include acknowledging uncertainty, obtaining more contextual information from patients (e.g., gathering occupations and family histories), creating diagnostic safety nets (e.g., informing patients what red flags to look for), engaging in worst case/best case scenario planning, and communicating diagnostic uncertainty to patients, families, and colleagues. Potential strategies tailored to various aspects of diagnostic uncertainty are also outlined.

CONCLUSION

Scientific knowledge on diagnostic uncertainty, while previously elusive, is now becoming more clearly defined. Next steps include research to evaluate relationships between management and communication of diagnostic uncertainty and improved patient outcomes.

Use of patient complaints to identify diagnosis-related safety concerns: a mixed-method evaluation

Background

Patient complaints are associated with adverse events and malpractice claims but underused in patient safety improvement.

Objective

To systematically evaluate the use of patient complaint data to identify safety concerns related to diagnosis as an initial step to using this information to facilitate learning and improvement.

Methods

We reviewed patient complaints submitted to Geisinger, a large healthcare organisation in the USA, from August to December 2017 (cohort 1) and January to June 2018 (cohort 2). We selected complaints more likely to be associated with diagnostic concerns in Geisinger’s existing complaint taxonomy. Investigators reviewed all complaint summaries and identified cases as ‘concerning’ for diagnostic error using the National Academy of Medicine’s definition of diagnostic error. For all ‘concerning’ cases, a clinician-reviewer evaluated the associated investigation report and the patient’s medical record to identify any missed opportunities in making a correct or timely diagnosis. In cohort 2, we selected a 10% sample of ‘concerning’ cases to test this smaller pragmatic sample as a proof of concept for future organisational monitoring.

Results

In cohort 1, we reviewed 1865 complaint summaries and identified 177 (9.5%) concerning reports. Review and analysis identified 39 diagnostic errors. Most were categorised as ‘Clinical Care issues’ (27, 69.2%), defined as concerns/questions related to the care that is provided by clinicians in any setting. In cohort 2, we reviewed 2423 patient complaint summaries and identified 310 (12.8%) concerning reports. The 10% sample (n=31 cases) contained five diagnostic errors. Qualitative analysis of cohort 1 cases identified concerns about return visits for persistent and/or worsening symptoms, interpersonal issues and diagnostic testing.

Conclusions

Analysis of patient complaint data and corresponding medical record review identifies patterns of failures in the diagnostic process reported by patients and families. Health systems could systematically analyse available data on patient complaints to monitor diagnostic safety concerns and identify opportunities for learning and improvement.

Identifying trigger concepts to screen emergency department visits for diagnostic errors

OBJECTIVES

The diagnostic process is a vital component of safe and effective emergency department (ED) care. There are no standardized methods for identifying or reliably monitoring diagnostic errors in the ED, impeding efforts to enhance diagnostic safety. We sought to identify trigger concepts to screen ED records for diagnostic errors and describe how they can be used as a measurement strategy to identify and reduce preventable diagnostic harm.

METHODS

We conducted a literature review and surveyed ED directors to compile a list of potential electronic health record (EHR) trigger (e-triggers) and non-EHR based concepts. We convened a multidisciplinary expert panel to build consensus on trigger concepts to identify and reduce preventable diagnostic harm in the ED.

RESULTS

Six e-trigger and five non-EHR based concepts were selected by the expert panel. E-trigger concepts included: unscheduled ED return to ED resulting in hospital admission, death following ED visit, care escalation, high-risk conditions based on symptom-disease dyads, return visits with new diagnostic/therapeutic interventions, and change of treating service after admission. Non-EHR based signals included: cases from mortality/morbidity conferences, risk management/safety office referrals, ED medical director case referrals, patient complaints, and radiology/laboratory misreads and callbacks. The panel suggested further refinements to aid future research in defining diagnostic error epidemiology in ED settings.

CONCLUSIONS

We identified a set of e-trigger concepts and non-EHR based signals that could be developed further to screen ED visits for diagnostic safety events. With additional evaluation, trigger-based methods can be used as tools to monitor and improve ED diagnostic performance.

Health Care Provider Factors Associated with Patient-Reported Adverse Events and Harm

BACKGROUND

Patients can provide valuable information missing from traditional sources of safety data, thus adding new insights about factors that lead to preventable harm. In this study, researchers determined associations between patient-reported contributory factors and patient-reported harms experienced after an adverse event (AE).

METHODS

A secondary analysis was conducted of a national sample of patient-reported AEs (surgical, medication, diagnostic, and hospital-acquired infection) gathered through an online questionnaire between January 2010 and February 2016. Generalized logit multivariable regression was used to assess the association between patient-reported contributory factors and patient-reported harms (grouped as nonphysical harm only, physical harm only, physical harm and emotional or financial harm, and all three harms) and adjusted for patient and AE characteristics.

RESULTS

One third of patients (32.6%) reported experiencing all three harms, 27.3% reported physical harms and one additional harm, 25.5% reported physical harms only, and 14.7% reported nonphysical harms only. Patients reporting all three harms were 2.5 times more likely to have filed a report with a responsible authority (95% confidence interval [CI] = 1.23-5.01) and 3.3 times more likely to have also experienced a surgical complication (95% CI = 1.42-7.51). Odds of reporting problems related to communication between clinician and patients/families or clinician-related behavioral issues was 13% higher in those experiencing all three harm types (95% CI = 1.07-1.19).

CONCLUSION

Patients' experiences are important to identify safety issues and reduce harm and should be included in patient safety measurement and improvement activities. These findings underscore the need for policy and practice changes to identify, address, and support harmed patients.

Patient Perspectives on the Usefulness of an Artificial Intelligence-Assisted Symptom Checker: Cross-Sectional Survey Study

Background

Patients are increasingly seeking Web-based symptom checkers to obtain diagnoses. However, little is known about the characteristics of the patients who use these resources, their rationale for use, and whether they find them accurate and useful.

Objective

The study aimed to examine patients’ experiences using an artificial intelligence (AI)–assisted online symptom checker.

Methods

An online survey was administered between March 2, 2018, through March 15, 2018, to US users of the Isabel Symptom Checker within 6 months of their use. User characteristics, experiences of symptom checker use, experiences discussing results with physicians, and prior personal history of experiencing a diagnostic error were collected.

Results

A total of 329 usable responses was obtained. The mean respondent age was 48.0 (SD 16.7) years; most were women (230/304, 75.7%) and white (271/304, 89.1%). Patients most commonly used the symptom checker to better understand the causes of their symptoms (232/304, 76.3%), followed by for deciding whether to seek care (101/304, 33.2%) or where (eg, primary or urgent care: 63/304, 20.7%), obtaining medical advice without going to a doctor (48/304, 15.8%), and understanding their diagnoses better (39/304, 12.8%). Most patients reported receiving useful information for their health problems (274/304, 90.1%), with half reporting positive health effects (154/302, 51.0%). Most patients perceived it to be useful as a diagnostic tool (253/301, 84.1%), as a tool providing insights leading them closer to correct diagnoses (231/303, 76.2%), and reported they would use it again (278/304, 91.4%). Patients who discussed findings with their physicians (103/213, 48.4%) more often felt physicians were interested (42/103, 40.8%) than not interested in learning about the tool’s results (24/103, 23.3%) and more often felt physicians were open (62/103, 60.2%) than not open (21/103, 20.4%) to discussing the results. Compared with patients who had not previously experienced diagnostic errors (missed or delayed diagnoses: 123/304, 40.5%), patients who had previously experienced diagnostic errors (181/304, 59.5%) were more likely to use the symptom checker to determine where they should seek care (15/123, 12.2% vs 48/181, 26.5%; P=.002), but they less often felt that physicians were interested in discussing the tool’s results (20/34, 59% vs 22/69, 32%; P=.04).

Conclusions

Despite ongoing concerns about symptom checker accuracy, a large patient-user group perceived an AI-assisted symptom checker as useful for diagnosis. Formal validation studies evaluating symptom checker accuracy and effectiveness in real-world practice could provide additional useful information about their benefit.

Pediatric clinician perspectives on communicating diagnostic uncertainty

OBJECTIVE

Diagnosis often evolves over time, involves uncertainty, and is vulnerable to errors. We examined pediatric clinicians' perspectives on communicating diagnostic uncertainty to patients' parents and how this occurs.

DESIGN

We conducted semi-structured interviews, which were audiotaped, transcribed, and analyzed using content analysis. Two researchers independently coded transcripts and then discussed discrepancies to reach consensus.

SETTING

A purposive sample of pediatric clinicians at two large academic medical institutions in Texas.

PARTICIPANTS

Twenty pediatric clinicians participated: 18 physicians, 2 nurse practitioners; 7 males, 13 females; 7 inpatient, 11 outpatient, and 2 practicing in mixed settings; with 0-16 years' experience post-residency.

INTERVENTION(S)

None.

MAIN OUTCOME MEASURE(S)

Pediatric clinician perspectives on communication of diagnostic uncertainty.

RESULTS

Pediatric clinicians commonly experienced diagnostic uncertainty and most were comfortable seeking help and discussing with colleagues. However, when communicating uncertainty to parents, clinicians used multiple considerations to adjust the degree to which they communicated. Considerations included parent characteristics (education, socioeconomic status, emotional response, and culture) and strength of parent-clinician relationships. Communication content included setting expectations, explaining the diagnostic process, discussing most relevant differentials, and providing reassurance. Responses to certain parent characteristics, however, were variable. For example, some clinicians were more open to discussing diagnostic uncertainty with more educated parents- others were less.

CONCLUSIONS

While pediatric clinicians are comfortable discussing diagnostic uncertainty with colleagues, how they communicate uncertainty to parents appears variable. Parent characteristics and parent-clinician relationships affect extent of communication and content discussed. Development and implementation of optimal strategies for managing and communicating diagnostic uncertainty can improve the diagnostic process.

An Exploration of Barriers, Facilitators, and Suggestions for Improving Electronic Health Record Inbox-Related Usability: A Qualitative Analysis

IMPORTANCE

Managing messages in the electronic health record (EHR) inbox consumes substantial amounts of physician time. Certain factors associated with inbox management, such as poor usability and excessive and unnecessary inbox messages, have been associated with physician burnout. Additionally, inbox design, usability, and workflows are associated with physicians' situational awareness (ie, perception, comprehension, and projection of clinical status) and efficiency of processing EHR inbox messages. Understanding factors associated with inbox usability could improve future EHR inbox designs and workflows, thus reducing risk of burnout while improving patient safety.

OBJECTIVE

To determine barriers, facilitators, and suggestions associated with EHR inbox-related usability.

DESIGN, SETTING, AND PARTICIPANTS

This qualitative study included cognitive walkthroughs of EHR inbox management with 25 physicians (17 primary care physicians and 8 specialists) at 6 large health care organizations using 4 different EHR systems between May 6, 2015, and September 19, 2016. While processing EHR inbox messages, participants identified facilitators and barriers associated with EHR inbox situational awareness and processing efficiency and potential interventions to address such barriers. A qualitative analysis was performed on transcribed recordings using an inductive thematic approach with an 8-dimension sociotechnical model as a theoretical lens from May 6, 2015, to August 15, 2019.

RESULTS

The cognitive walkthroughs identified 60 barriers, 32 facilitators, and 28 suggestions for improving the EHR inbox. Emergent data fit within 5 major themes: message processing complexity, inbox interface design, cognitive load, team communication, and inbox message content. Within these themes, similar barriers were identified across sites, such as poor usability due the high numbers of clicks needed to accomplish actions. In certain instances, an identified facilitator at one site provided the exact solution needed to address a barrier identified at another site.

CONCLUSIONS AND RELEVANCE

This qualitative study found that usability of the EHR inbox is often suboptimal and variable across sites, suggesting lack of shared best practices related to information management. Implementation of optimized design features and workflows will require EHR developers and health care organizations to collectively share this responsibility. Development of regional or national consortia to support collaborative sharing and implementation of EHR system best practices across EHR developers and health care organizations could also improve safety and efficiency and reduce physician burnout.

Practicing Clinicians' Recommendations to Reduce Burden from the Electronic Health Record Inbox: a Mixed-Methods Study

BACKGROUND

Workload from electronic health record (EHR) inbox notifications leads to information overload and contributes to job dissatisfaction and physician burnout. Better understanding of physicians' inbox requirements and workflows could optimize inbox designs, enhance efficiency, and reduce safety risks from information overload.

DESIGN

We conducted a mixed-methods study to identify strategies to enhance EHR inbox design and workflow. First, we performed a secondary analysis of national survey data of all Department of Veterans Affairs (VA) primary care practitioners (PCP) to identify major themes in responses to a free-text question soliciting suggestions to improve EHR inbox design and workflows. We then conducted expert interviews of clinicians at five health care systems (1 VA and 4 non-VA settings using 4 different EHRs) to understand existing optimal strategies to improve efficiency and situational awareness related to EHR inbox use. Themes from survey data were cross-validated with interview findings.

RESULTS

We analyzed responses from 2104 PCPs who completed the free-text inbox question (of 5001 PCPs who responded to survey) and used an inductive approach to identify five themes: (1) Inbox notification content should be actionable for patient care and relevant to recipient clinician, (2) Inboxes should reduce risk of losing messages, (3) Inbox functionality should be optimized to improve efficiency of processing notifications, (4) Team support should be leveraged to help with EHR inbox notification burden, (5) Sufficient time should be provided to all clinicians to process EHR inbox notifications. We subsequently interviewed 15 VA and non-VA clinicians and identified 11 unique strategies, each corresponding directly with one of these five themes.

CONCLUSION

Feedback from practicing end-user clinicians provides robust evidence to improve content and design of the EHR inbox and related clinical workflows and organizational policies. Several strategies we identified could improve clinicians' EHR efficiency and satisfaction as well as empower them to work with their local administrators, health IT personnel, and EHR developers to improve these systems.

Methods for Patient-Centered Interface Design of Test Result Display in Online Portals

OBJECTIVES

Patients have unique information needs to help them interpret and make decisions about laboratory test results they receive on web-based portals. However, current portals are not designed in a patient-centered way and little is known on how best to harness patients' information needs to inform user-centered interface design of portals. We designed a patient-facing laboratory test result interface prototype based on requirement elicitation research and used a mixed-methods approach to evaluate this interface.

METHODS

After designing an initial test result display prototype, we used multiple evaluation methods, including focus group review sessions, expert consultation, and user testing, to make iterative design changes. For the user testing component, we recruited 14 patient-users to collect and analyze three types of data: comments made during testing sessions, responses to post-session questionnaires, and system usability scores.

RESULTS

Our initial patient-centered interface design included visual ranges of laboratory values, nontechnical descriptions of the test and result, and access to features to help patients interpret and make decisions about their results. Findings from our evaluation resulted in 6 design iterations of the interface. Results from user testing indicate that the later versions of the interface fulfilled patient's information needs, were perceived as usable, and provided access to information and techniques that facilitated patient's ability to derive meaning from each test result.

CONCLUSIONS

Requirement elicitation studies can inform the design of a patient-facing test result interface, but considerable user-centered design efforts are necessary to create an interface that patients find useful. To promote patient engagement, health information technology designers and developers can use similar approaches to enhance user-centered software design in patient portals.

Methods for Patient-Centered Interface Design of Test Result Display in Online Portals

OBJECTIVES

Patients have unique information needs to help them interpret and make decisions about laboratory test results they receive on web-based portals. However, current portals are not designed in a patient-centered way and little is known on how best to harness patients' information needs to inform user-centered interface design of portals. We designed a patient-facing laboratory test result interface prototype based on requirement elicitation research and used a mixed-methods approach to evaluate this interface.

METHODS

After designing an initial test result display prototype, we used multiple evaluation methods, including focus group review sessions, expert consultation, and user testing, to make iterative design changes. For the user testing component, we recruited 14 patient-users to collect and analyze three types of data: comments made during testing sessions, responses to post-session questionnaires, and system usability scores.

RESULTS

Our initial patient-centered interface design included visual ranges of laboratory values, nontechnical descriptions of the test and result, and access to features to help patients interpret and make decisions about their results. Findings from our evaluation resulted in 6 design iterations of the interface. Results from user testing indicate that the later versions of the interface fulfilled patient's information needs, were perceived as usable, and provided access to information and techniques that facilitated patient's ability to derive meaning from each test result.

CONCLUSIONS

Requirement elicitation studies can inform the design of a patient-facing test result interface, but considerable user-centered design efforts are necessary to create an interface that patients find useful. To promote patient engagement, health information technology designers and developers can use similar approaches to enhance user-centered software design in patient portals.

Test results management and distributed cognition in electronic health record-enabled primary care

Managing abnormal test results in primary care involves coordination across various settings. This study identifies how primary care teams manage test results in a large, computerized healthcare system in order to inform health information technology requirements for test results management and other distributed healthcare services. At five US Veterans Health Administration facilities, we interviewed 37 primary care team members, including 16 primary care providers, 12 registered nurses, and 9 licensed practical nurses. We performed content analysis using a distributed cognition approach, identifying patterns of information transmission across people and artifacts (e.g. electronic health records). Results illustrate challenges (e.g. information overload) as well as strategies used to overcome challenges. Various communication paths were used. Some team members served as intermediaries, processing information before relaying it. Artifacts were used as memory aids. Health information technology should address the risks of distributed work by supporting awareness of team and task status for reliable management of results.

Patient perceptions of receiving test results via online portals: a mixed-methods study

Objective

Online portals provide patients with access to their test results, but it is unknown how patients use these tools to manage results and what information is available to promote understanding. We conducted a mixed-methods study to explore patients' experiences and preferences when accessing their test results via portals.

Materials and Methods

We conducted 95 interviews (13 semistructured and 82 structured) with adults who viewed a test result in their portal between April 2015 and September 2016 at 4 large outpatient clinics in Houston, Texas. Semistructured interviews were coded using content analysis and transformed into quantitative data and integrated with the structured interview data. Descriptive statistics were used to summarize the structured data.

Results

Nearly two-thirds (63%) did not receive any explanatory information or test result interpretation at the time they received the result, and 46% conducted online searches for further information about their result. Patients who received an abnormal result were more likely to experience negative emotions (56% vs 21%; P = .003) and more likely to call their physician (44% vs 15%; P = .002) compared with those who received normal results.

Discussion

Study findings suggest that online portals are not currently designed to present test results to patients in a meaningful way. Patients experienced negative emotions often with abnormal results, but sometimes even with normal results. Simply providing access via portals is insufficient; additional strategies are needed to help patients interpret and manage their online test results.

Conclusion

Given the absence of national guidance, our findings could help strengthen policy and practice in this area and inform innovations that promote patient understanding of test results.

Patient perspectives on how physicians communicate diagnostic uncertainty: An experimental vignette study

OBJECTIVE

We evaluated the effects of three different strategies for communicating diagnostic uncertainty on patient perceptions of physician competence and visit satisfaction.

DESIGN/SETTING

Experimental vignette-based study design involving pediatric cases presented to a convenience sample of parents living in a large US city.

PARTICIPANTS/INTERVENTION(S)

Three vignettes were developed, each describing one of three different ways physicians communicated diagnostic uncertainty to parents-(i) explicit expression of uncertainty ('not sure' about diagnosis), (ii) implicit expression of uncertainty using broad differential diagnoses and (iii) implicit expression of uncertainty using 'most likely' diagnoses. Participants were randomly assigned to one of the three vignettes and then answered a 37-item web-based questionnaire.

MAIN OUTCOME MEASURE(S)

Outcome variables included parent-perceived technical competence of physician, trust and confidence, visit satisfaction and adherence to physician instructions. Differences between the three groups were compared using analysis of variance, followed by individual post hoc analyses with Bonferroni correction.

RESULTS

Seventy-one participants completed the vignette questions. Demographic characteristics and scores on activation (parent activation measure [PAM]) and intolerance to uncertainty were similar across the three groups. Explicit expression of uncertainty was associated with lower perceived technical competence, less trust and confidence, and lower patient adherence as compared to the two groups with implicit communication. These latter two groups had comparable outcomes.

CONCLUSION

Parents may react less negatively in terms of perceived competence, physician confidence and trust, and intention to adhere when diagnostic uncertainty is communicated using implicit strategies, such as using broad differential diagnoses or most likely diagnoses. Evidence-based strategies to communicate diagnostic uncertainty to patients need further development.

Social Workers' Perceived Barriers to and Sources of Support for Integrating Clients' Religion and Spirituality in Practice

This article describes the qualitative responses from a national sample of licensed clinical social workers (LCSWs) on their views regarding integrating clients' religion and spirituality (RS) in practice. Two open-ended questions were asked to assess what helps or assists LCSWs in assessing and integrating clients' RS in practice and what hinders or prevents LCSWs from considering this area of clients' lives. A total of 329 responded to either item, with 319 responses to the first item and 279 responses to the second. The authors used open-coding procedures, developed a codebook to analyze the data, and reached consensus on each response. Overarching themes that emerged from LCSWs' responses to what helps them consider this area included personal religiosity, education, and having an RS-sensitive practice. Regarding what hinders RS integration, LCSWs reported that nothing hinders such integration; that it was not relevant; or listed various barriers, including a lack of training, client discouraging the discussion, or experiencing fear or perceiving RS as a taboo topic. The article concludes with a discussion of the implications for social work education and practice.

Safety huddles to proactively identify and address electronic health record safety

OBJECTIVE

Methods to identify and study safety risks of electronic health records (EHRs) are underdeveloped and largely depend on limited end-user reports. "Safety huddles" have been found useful in creating a sense of collective situational awareness that increases an organization's capacity to respond to safety concerns. We explored the use of safety huddles for identifying and learning about EHR-related safety concerns.

DESIGN

Data were obtained from daily safety huddle briefing notes recorded at a single midsized tertiary-care hospital in the United States over 1 year. Huddles were attended by key administrative, clinical, and information technology staff. We conducted a content analysis of huddle notes to identify what EHR-related safety concerns were discussed. We expanded a previously developed EHR-related error taxonomy to categorize types of EHR-related safety concerns recorded in the notes.

RESULTS

On review of daily huddle notes spanning 249 days, we identified 245 EHR-related safety concerns. For our analysis, we defined EHR technology to include a specific EHR functionality, an entire clinical software application, or the hardware system. Most concerns (41.6%) involved " EHR technology working incorrectly, " followed by 25.7% involving " EHR technology not working at all. " Concerns related to "EHR technology missing or absent" accounted for 16.7%, whereas 15.9% were linked to " user errors ."

CONCLUSIONS

Safety huddles promoted discussion of several technology-related issues at the organization level and can serve as a promising technique to identify and address EHR-related safety concerns. Based on our findings, we recommend that health care organizations consider huddles as a strategy to promote understanding and improvement of EHR safety.