Improving documentation of presenting problems in the emergency department using a domain-specific ontology and machine learning-driven user interfaces

Identifying Interventions to Improve Diagnostic Safety in Emergency Departments: Protocol for a Participatory Design Study

BACKGROUND

Emergency departments (EDs) are complex and fast-paced clinical settings where a diagnosis is made in a time-, information-, and resource-constrained context. Thus, it is predisposed to suboptimal diagnostic outcomes, leading to errors and subsequent patient harm. Arriving at a timely and accurate diagnosis is an activity that occurs after an effective collaboration between the patient or caregiver and the clinical team within the ED. Interventions such as novel sociotechnical solutions are needed to mitigate errors and risks.

OBJECTIVE

This study aims to identify challenges that frontline ED health care providers and patients face in the ED diagnostic process and involve them in co-designing technological interventions to enhance diagnostic excellence.

METHODS

We will conduct separate sessions with ED health care providers and patients, respectively, to assess various design ideas and use a participatory design (PD) approach for technological interventions to improve ED diagnostic safety. In the sessions, various intervention ideas will be presented to participants through storyboards. Based on a preliminary interview study with ED patients and health care providers, we created intervention storyboards that illustrate different care contexts in which ED health care providers or patients experience challenges and show how each intervention would address the specific challenge. By facilitating participant group discussion, we will reveal the overlap between the needs of the design research team observed during fieldwork and the needs perceived by target users (ie, participants) in their own experience to gain their perspectives and assessment on each idea. After the group discussions, participants will rank the ideas and co-design to improve our interventions. Data sources will include audio and video recordings, design sketches, and ratings of intervention design ideas from PD sessions. The University of Michigan Institutional Review Board approved this study. This foundational work will help identify the needs and challenges of key stakeholders in the ED diagnostic process and develop initial design ideas, specifically focusing on sociotechnological ideas for patient-, health care provider-, and system-level interventions for improving patient safety in EDs.

RESULTS

The recruitment of participants for ED health care providers and patients is complete. We are currently preparing for PD sessions. The first results from design sessions with health care providers will be reported in fall 2024.

CONCLUSIONS

The study findings will provide unique insights for designing sociotechnological interventions to support ED diagnostic processes. By inviting frontline health care providers and patients into the design process, we anticipate obtaining unique insights into the ED diagnostic process and designing novel sociotechnical interventions to enhance patient safety. Based on this study's collected data and intervention ideas, we will develop prototypes of multilevel interventions that can be tested and subsequently implemented for patients, health care providers, or hospitals as a system.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/55357.

The relationship between electronic health records user interface features and data quality of patient clinical information: an integrative review

OBJECTIVES

Electronic health records (EHRs) user interfaces (UI) designed for data entry can potentially impact the quality of patient information captured in the EHRs. This review identified and synthesized the literature evidence about the relationship of UI features in EHRs on data quality (DQ).

MATERIALS AND METHODS

We performed an integrative review of research studies by conducting a structured search in 5 databases completed on October 10, 2022. We applied Whittemore & Knafl's methodology to identify literature, extract, and synthesize information, iteratively. We adapted Kmet et al appraisal tool for the quality assessment of the evidence. The research protocol was registered with PROSPERO (CRD42020203998).

RESULTS

Eleven studies met the inclusion criteria. The relationship between 1 or more UI features and 1 or more DQ indicators was examined. UI features were classified into 4 categories: 3 types of data capture aids, and other methods of DQ assessment at the UI. The Weiskopf et al measures were used to assess DQ: completeness (n = 10), correctness (n = 10), and currency (n = 3). UI features such as mandatory fields, templates, and contextual autocomplete improved completeness or correctness or both. Measures of currency were scarce.

DISCUSSION

The paucity of studies on UI features and DQ underscored the limited knowledge in this important area. The UI features examined had both positive and negative effects on DQ. Standardization of data entry and further development of automated algorithmic aids, including adaptive UIs, have great promise for improving DQ. Further research is essential to ensure data captured in our electronic systems are high quality and valid for use in clinical decision-making and other secondary analyses.

Applications of natural language processing at emergency department triage: A narrative review

INTRODUCTION

Natural language processing (NLP) uses various computational methods to analyse and understand human language, and has been applied to data acquired at Emergency Department (ED) triage to predict various outcomes. The objective of this scoping review is to evaluate how NLP has been applied to data acquired at ED triage, assess if NLP based models outperform humans or current risk stratification techniques when predicting outcomes, and assess if incorporating free-text improve predictive performance of models when compared to predictive models that use only structured data.

METHODS

All English language peer-reviewed research that applied an NLP technique to free-text obtained at ED triage was eligible for inclusion. We excluded studies focusing solely on disease surveillance, and studies that used information obtained after triage. We searched the electronic databases MEDLINE, Embase, Cochrane Database of Systematic Reviews, Web of Science, and Scopus for medical subject headings and text keywords related to NLP and triage. Databases were last searched on 01/01/2022. Risk of bias in studies was assessed using the Prediction model Risk of Bias Assessment Tool (PROBAST). Due to the high level of heterogeneity between studies and high risk of bias, a metanalysis was not conducted. Instead, a narrative synthesis is provided.

RESULTS

In total, 3730 studies were screened, and 20 studies were included. The population size varied greatly between studies ranging from 1.8 million patients to 598 triage notes. The most common outcomes assessed were prediction of triage score, prediction of admission, and prediction of critical illness. NLP models achieved high accuracy in predicting need for admission, triage score, critical illness, and mapping free-text chief complaints to structured fields. Incorporating both structured data and free-text data improved results when compared to models that used only structured data. However, the majority of studies (80%) were assessed to have a high risk of bias, and only one study reported the deployment of an NLP model into clinical practice.

CONCLUSION

Unstructured free-text triage notes have been used by NLP models to predict clinically relevant outcomes. However, the majority of studies have a high risk of bias, most research is retrospective, and there are few examples of implementation into clinical practice. Future work is needed to prospectively assess if applying NLP to data acquired at ED triage improves ED outcomes when compared to usual clinical practice.

Drug-related causes attributed to acute kidney injury and their documentation in intensive care patients

PURPOSE

To investigate drug-related causes attributed to acute kidney injury (DAKI) and their documentation in patients admitted to the Intensive Care Unit (ICU).

METHODS

This study was conducted in an academic hospital in the Netherlands by reusing electronic health record (EHR) data of adult ICU admissions between November 2015 to January 2020. First, ICU admissions with acute kidney injury (AKI) stage 2 or 3 were identified. Subsequently, three modes of DAKI documentation in EHR were examined: diagnosis codes (structured data), allergy module (semi-structured data), and clinical notes (unstructured data).

RESULTS

n total 8124 ICU admissions were included, with 542 (6.7%) ICU admissions experiencing AKI stage 2 or 3. The ICU physicians deemed 102 of these AKI cases (18.8%) to be drug-related. These DAKI cases were all documented in the clinical notes (100%), one in allergy module (1%) and none via diagnosis codes. The clinical notes required the highest time investment to analyze.

CONCLUSIONS

Drug-related causes comprise a substantial part of AKI in the ICU patients. However, current unstructured DAKI documentation practice via clinical notes hampers our ability to gain better insights about DAKI occurrence. Therefore, both automating DAKI identification from the clinical notes and increasing structured DAKI documentation should be encouraged.

The Use and Structure of Emergency Nurses' Triage Narrative Data: Scoping Review

BACKGROUND

Emergency departments use triage to ensure that patients with the highest level of acuity receive care quickly and safely. Triage is typically a nursing process that is documented as structured and unstructured (free text) data. Free-text triage narratives have been studied for specific conditions but never reviewed in a comprehensive manner.

OBJECTIVE

The objective of this paper was to identify and map the academic literature that examines triage narratives. The paper described the types of research conducted, identified gaps in the research, and determined where additional review may be warranted.

METHODS

We conducted a scoping review of unstructured triage narratives. We mapped the literature, described the use of triage narrative data, examined the information available on the form and structure of narratives, highlighted similarities among publications, and identified opportunities for future research.

RESULTS

We screened 18,074 studies published between 1990 and 2022 in CINAHL, MEDLINE, Embase, Cochrane, and ProQuest Central. We identified 0.53% (96/18,074) of studies that directly examined the use of triage nurses' narratives. More than 12 million visits were made to 2438 emergency departments included in the review. In total, 82% (79/96) of these studies were conducted in the United States (43/96, 45%), Australia (31/96, 32%), or Canada (5/96, 5%). Triage narratives were used for research and case identification, as input variables for predictive modeling, and for quality improvement. Overall, 31% (30/96) of the studies offered a description of the triage narrative, including a list of the keywords used (27/96, 28%) or more fulsome descriptions (such as word counts, character counts, abbreviation, etc; 7/96, 7%). We found limited use of reporting guidelines (8/96, 8%).

CONCLUSIONS

The breadth of the identified studies suggests that there is widespread routine collection and research use of triage narrative data. Despite the use of triage narratives as a source of data in studies, the narratives and nurses who generate them are poorly described in the literature, and data reporting is inconsistent. Additional research is needed to describe the structure of triage narratives, determine the best use of triage narratives, and improve the consistent use of triage-specific data reporting guidelines.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1136/bmjopen-2021-055132.

Using less keystrokes to achieve high top-1 accuracy in Chinese clinical text entry

Background

As a routine task, physicians spend substantial time and keystrokes on text entry. Documentation burden is increasingly associated with physician burnout. Predicting at top-1 with less keystrokes (TLKs) is a hot topic for smart text entry. In Western countries, contextual autocomplete is deployed to alleviate the burden. Chinese text entry is intercepted by input method engines (IMEs), which cut off suggestions from electronic health records (EHRs).

Objective

To explore a user-friendly approach to make text entry easier and faster for Chinese physicians.

Methods

Physicians were shadowed to uncover the real-word input behaviors. System logs were collected for behavior validation and then used for context-based learning. An in-line web-based popup layer was proposed to hold the best suggestion from EHRs. Keystrokes per character and TLK rate were evaluated quantitatively. Questionnaires were used for qualitative assessment. Nine hundred fifty-two physicians were enrolled in a field testing.

Results

14 facilitators and 17 barriers related to IMEs were identified after shadowing. With system logs, physicians tended to split long words into short units, which were 1-4 in length. 81.7% of these units were disyllables. Compared to the control group, the intervention group improved TLK rate by 40.3% (p < .0001), and reduced keystrokes per character by 48.3% (p < .0001). Survey results also promised positive feedback from physicians.

Conclusions

Keystroke burden and frequent choice reaction time challenge Chinese physicians for text entry. The proposed system demonstrates an approach to alleviate the burden. Contextual information is easily retrieved and it further helps improve the top-1 accuracy, with a smaller number of keystrokes. While positive feedback is received, it promises a benefit to protect patient privacy.

Influence of artificial intelligence on the work design of emergency department clinicians a systematic literature review

Objective

This systematic literature review aims to demonstrate how Artificial Intelligence (AI) is currently used in emergency departments (ED) and how it alters the work design of ED clinicians. AI is still new and unknown to many healthcare professionals in emergency care, leading to unfamiliarity with its capabilities.

Method

Various criteria were used to establish the suitability of the articles to answer the research question. This study was based on 34 selected peer-reviewed papers on the use of Artificial Intelligence (AI) in the Emergency Department (ED), published in the last five years. Drawing on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, all articles were scanned, read full-text, and analyzed afterward.

Results

The majority of the AI applications consisted of AI-based tools to aid with clinical decisions and to relieve overcrowded EDs of their burden. AI support was mostly offered during triage, the moment that sets the patient trajectory. There is ample evidence that AI-based applications could improve the clinical decision-making process.

Conclusion

The use of AI in EDs is still in its nascent stages. Many studies focus on the question of whether AI has clinical utility, such as decision support, improving resource allocation, reducing diagnostic errors, and promoting proactivity. Some studies suggest that AI-based tools essentially have the ability to outperform human skills. However, it is evident from the literature that current technology does not have the aims or power to do so. Nevertheless, AI-based tools can impact clinician work design in the ED by providing support with clinical decisions, which could ultimately help alleviate a portion of the increasing clinical burden.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-08070-7.

Pharmacogenomic Clinical Decision Support: A Review, How-to Guide, and Future Vision

Clinical decision support (CDS) is an essential part of any pharmacogenomics (PGx) implementation. Increasingly, institutions have implemented CDS tools in the clinical setting to bring PGx data into patient care, and several have published their experiences with these implementations. However, barriers remain that limit the ability of some programs to create CDS tools to fit their PGx needs. Therefore, the purpose of this review is to summarize the types, functions, and limitations of PGx CDS currently in practice. Then, we provide an approachable step‐by‐step how‐to guide with a case example to help implementers bring PGx to the front lines of care regardless of their setting. Particular focus is paid to the five “rights” of CDS as a core around designing PGx CDS tools. Finally, we conclude with a discussion of opportunities and areas of growth for PGx CDS.

A Concept Analysis on the Use of Artificial Intelligence in Nursing

Artificial intelligence (AI) has a considerable present and future influence on healthcare. Nurses, representing the largest proportion of healthcare workers, are set to immensely benefit from this technology. However, the overall adoption of new technologies by nurses is quite slow, and the use of AI in nursing is considered to be in its infancy. The current literature on AI in nursing lacks conceptual clarity and consensus, which is affecting clinical practice, research activities, and theory development. Therefore, to set the foundations for nursing AI knowledge development, the purpose of this concept analysis is to clarify the conceptual components of AI in nursing and to determine its conceptual maturity. A concept analysis following Morse's approach was conducted, which examined definitions, characteristics, preconditions, outcomes, and boundaries on the state of AI in nursing. A total of 18 quantitative, qualitative, mixed-methods, and reviews related to AI in nursing were retrieved from the CINAHL and EMBASE databases using a Boolean search. Presently, the concept of AI in nursing is immature. The characteristics and preconditions of the use of AI in nursing are mixed between and within each other. The preconditions and outcomes on the use of AI in nursing are diverse and indiscriminately reported. As for boundaries, they can be more distinguished between robots, sensors, and clinical decision support systems, but these lines can become more blurred in the future. As of 2021, the use of AI in nursing holds much promise for the profession, but conceptual and theoretical issues remain.

Association of resident characteristics with patterns of patient self-assignment

OBJECTIVE

We hypothesized that resident characteristics impact patterns of patient self-assignment in the emergency department (ED). Our goal was to determine if male residents would be less likely than their female colleagues to see patients with sensitive (e.g. breast-related or gynecologic) chief complaints (CCs). We also investigated whether resident specialty was associated with preferentially choosing patients with more familiar chief complaints.

METHODS

We performed a retrospective cross-sectional study at a tertiary academic medical center using data from all adult patients presenting to the ED between 2010 and 2019 with one of six CC categories (vaginal bleeding, breast-related concerns, male genitourinary [GU] concerns, gastrointestinal bleeding, epistaxis, and laceration). These CCs were chosen as they each require either an invasive medical exam or procedure, and cannot easily be evaluated with an exam in a hallway bed. We used logistic regression to assess the likelihood of being treated by a male resident compared to a female resident for each CC, adjusting for candidate variables of patient age, race, primary language, ESI score, bed location, time of day, day of week, calendar month, and resident specialty. We also similarly analyzed patterns of patient self-assignment according to resident specialty.

RESULTS

Male residents were significantly less likely than female residents to treat patients with breast-related CCs (adjusted OR 0.67, 95% CI 0.54-0.83, p < 0.001) or vaginal bleeding (adjusted OR 0.73, 95% CI 0.63-0.84, p < 0.001, reference group: epistaxis). Off-service residents were more likely to assign themselves to familiar chief complaints, for example surgery residents were more likely to see patients with lacerations (adjusted OR 2.11, 95% CI 1.71-2.61, p < 0.001) and OB/GYN residents were less likely to see patients with male GU concerns (adjusted OR 0.21, 95% CI 0.05-0.85, p = 0.029), compared to emergency medicine residents.

CONCLUSION

In a single facility, resident characteristics were associated with preferential patient self-assignment. Further work is necessary to determine the underlying reasons for patient avoidance, and to create work environments in which preferentially choosing patients is discouraged.

Generating contextual embeddings for emergency department chief complaints

Objective

We learn contextual embeddings for emergency department (ED) chief complaints using Bidirectional Encoder Representations from Transformers (BERT), a state-of-the-art language model, to derive a compact and computationally useful representation for free-text chief complaints.

Materials and methods

Retrospective data on 2.1 million adult and pediatric ED visits was obtained from a large healthcare system covering the period of March 2013 to July 2019. A total of 355 497 (16.4%) visits from 65 737 (8.9%) patients were removed for absence of either a structured or unstructured chief complaint. To ensure adequate training set size, chief complaint labels that comprised less than 0.01%, or 1 in 10 000, of all visits were excluded. The cutoff threshold was incremented on a log scale to create seven datasets of decreasing sparsity. The classification task was to predict the provider-assigned label from the free-text chief complaint using BERT, with Long Short-Term Memory (LSTM) and Embeddings from Language Models (ELMo) as baselines. Performance was measured as the Top-k accuracy from k = 1:5 on a hold-out test set comprising 5% of the samples. The embedding for each free-text chief complaint was extracted as the final 768-dimensional layer of the BERT model and visualized using t-distributed stochastic neighbor embedding (t-SNE).

Results

The models achieved increasing performance with datasets of decreasing sparsity, with BERT outperforming both LSTM and ELMo. The BERT model yielded Top-1 accuracies of 0.65 and 0.69, Top-3 accuracies of 0.87 and 0.90, and Top-5 accuracies of 0.92 and 0.94 on datasets comprised of 434 and 188 labels, respectively. Visualization using t-SNE mapped the learned embeddings in a clinically meaningful way, with related concepts embedded close to each other and broader types of chief complaints clustered together.

Discussion

Despite the inherent noise in the chief complaint label space, the model was able to learn a rich representation of chief complaints and generate reasonable predictions of their labels. The learned embeddings accurately predict provider-assigned chief complaint labels and map semantically similar chief complaints to nearby points in vector space.

Conclusion

Such a model may be used to automatically map free-text chief complaints to structured fields and to assist the development of a standardized, data-driven ontology of chief complaints for healthcare institutions.