A rapid review on current and potential uses of large language models in nursing

BACKGROUND

The application of large language models across commercial and consumer contexts has grown exponentially in recent years. However, a gap exists in the literature on how large language models can support nursing practice, education, and research. This study aimed to synthesize the existing literature on current and potential uses of large language models across the nursing profession.

METHODS

A rapid review of the literature, guided by Cochrane rapid review methodology and PRISMA reporting standards, was conducted. An expert health librarian assisted in developing broad inclusion criteria to account for the emerging nature of literature related to large language models. Three electronic databases (i.e., PubMed, CINAHL, and Embase) were searched to identify relevant literature in August 2023. Articles that discussed the development, use, and application of large language models within nursing were included for analysis.

RESULTS

The literature search identified a total of 2028 articles that met the inclusion criteria. After systematically reviewing abstracts, titles, and full texts, 30 articles were included in the final analysis. Nearly all (93 %; n = 28) of the included articles used ChatGPT as an example, and subsequently discussed the use and value of large language models in nursing education (47 %; n = 14), clinical practice (40 %; n = 12), and research (10 %; n = 3). While the most common assessment of large language models was conducted by human evaluation (26.7 %; n = 8), this analysis also identified common limitations of large language models in nursing, including lack of systematic evaluation, as well as other ethical and legal considerations.

DISCUSSION

This is the first review to summarize contemporary literature on current and potential uses of large language models in nursing practice, education, and research. Although there are significant opportunities to apply large language models, the use and adoption of these models within nursing have elicited a series of challenges, such as ethical issues related to bias, misuse, and plagiarism.

CONCLUSION

Given the relative novelty of large language models, ongoing efforts to develop and implement meaningful assessments, evaluations, standards, and guidelines for applying large language models in nursing are recommended to ensure appropriate, accurate, and safe use. Future research along with clinical and educational partnerships is needed to enhance understanding and application of large language models in nursing and healthcare.

Identifying Barriers to The Implementation of Communicating Narrative Concerns Entered by Registered Nurses, An Early Warning System SmartApp

BACKGROUND

 Nurses are at the frontline of detecting patient deterioration. We developed Communicating Narrative Concerns Entered by Registered Nurses (CONCERN), an early warning system for clinical deterioration that generates a risk prediction score utilizing nursing data. CONCERN was implemented as a randomized clinical trial at two health systems in the Northeastern United States. Following the implementation of CONCERN, our team sought to develop the CONCERN Implementation Toolkit to enable other hospital systems to adopt CONCERN.

OBJECTIVE

 The aim of this study was to identify the optimal resources needed to implement CONCERN and package these resources into the CONCERN Implementation Toolkit to enable the spread of CONCERN to other hospital sites.

METHODS

 To accomplish this aim, we conducted qualitative interviews with nurses, prescribing providers, and information technology experts in two health systems. We recruited participants from July 2022 to January 2023. We conducted thematic analysis guided by the Donabedian model. Based on the results of the thematic analysis, we updated the α version of the CONCERN Implementation Toolkit.

RESULTS

 There was a total of 32 participants included in our study. In total, 12 themes were identified, with four themes mapping to each domain in Donabedian's model (i.e., structure, process, and outcome). Eight new resources were added to the CONCERN Implementation Toolkit.

CONCLUSIONS

 This study validated the α version of the CONCERN Implementation Toolkit. Future studies will focus on returning the results of the Toolkit to the hospital sites to validate the β version of the CONCERN Implementation Toolkit. As the development of early warning systems continues to increase and clinician workflows evolve, the results of this study will provide considerations for research teams interested in implementing early warning systems in the acute care setting.

Understanding the Technical Implementation of a Clinical Decision Support SmartApp: A Qualitative Analysis

CONCERN is a SmartApp that identifies patients at risk for deterioration. This study aimed to understand the technical components and processes that should be included in our Implementation Toolkit. In focus groups with technical experts five themes emerged: 1) implementation challenges, 2) implementation facilitators, 3) project management, 4) stakeholder engagement, and 5) security assessments. Our results may aid other teams in implementing healthcare SmartApps.

A Computational Framework to Evaluate Emergency Department Clinician Task Switching in the Electronic Health Record Using Event Logs

Workflow fragmentation, defined as task switching, may be one proxy to quantify electronic health record (EHR) documentation burden in the emergency department (ED). Few measures have been operationalized to evaluate task switching at scale. Theoretically grounded in the time-based resource-sharing model (TBRSM) which conceives task switching as proportional to the cognitive load experienced, we describe the functional relationship between cognitive load and the time and effort constructs previously applied for measuring documentation burden. We present a computational framework, COMBINE, to evaluate multilevel task switching in the ED using EHR event logs. Based on this framework, we conducted a descriptive analysis on task switching among 63 full-time ED physicians from one ED site using EHR event logs extracted between April-June 2021 (n=2,068,605 events) which were matched to scheduled shifts (n=952). On average, we found a high volume of event-level (185.8±75.3/hr) and within-(6.6±1.7/chart) and between-patient chart (27.5±23.6/hr) switching per shift worked.

Variability in Nursing Documentation Patterns across Patients' Hospital Stays

This study explores the variability in nursing documentation patterns in acute care and ICU settings, focusing on vital signs and note documentation, and examines how these patterns vary across patients' hospital stays, documentation types, and comorbidities. In both acute care and critical care settings, there was significant variability in nursing documentation patterns across hospital stays, by documentation type, and by patients' comorbidities. The results suggest that nurses adapt their documentation practices in response to their patients' fluctuating needs and conditions, highlighting the need to facilitate more individualized care and tailored documentation practices. The implications of these findings can inform decisions on nursing workload management, clinical decision support tools, and EHR optimizations.

Identifying Reuse and Redundancies in Respiratory Flowsheet Documentation: Implications for Clinician Documentation Burden

Documentation burden is experienced by clinical end-users of the electronic health record. Flowsheet measure reuse and clinical concept redundancy are two contributors to documentation burden. In this paper, we described nursing flowsheet documentation hierarchy and frequency of use for one month from two hospitals in our health system. We examined respiratory care management documentation in greater detail. We found 59 instances of reuse of respiratory care flowsheet measure fields over two or more templates and groups, and 5 instances of clinical concept redundancy. Flowsheet measure fields for physical assessment observations and measurements were the most frequently documented and most reused, whereas respiratory intervention documentation was less frequently reused. Further research should investigate the relationship between flowsheet measure reuse and redundancy and EHR information overload and documentation burden.

The Lived Experience of Ambulatory and Perioperative RNs Displaced During the COVID-19 Pandemic: A Phenomenological Study

During the patient surge associated with the onset of the COVID-19 pandemic in the spring of 2020, perioperative and ambulatory RNs at an acute-care specialty orthopedic hospital were redeployed to medical-surgical inpatient nursing units to care for patients with the disease. The purpose of this phenomenological study was to describe perioperative and ambulatory RNs' experiences during the redeployment. We used purposeful sampling to obtain representatives who worked routinely in perioperative (including postanesthesia care) and ambulatory settings before redeployment. Data saturation was reached after eight in-depth interviews that yielded rich descriptions of the nurses' experiences. Most participants indicated that the fundamental structure of the experience involved being "thrown into a war without weapons" and needing to find ways to fight. The results of this study provide a unique contribution to nursing literature and may assist nurses and leaders in the future.

Eliciting Insights From Chat Logs of the 25X5 Symposium to Reduce Documentation Burden: Novel Application of Topic Modeling

BACKGROUND

Addressing clinician documentation burden through "targeted solutions" is a growing priority for many organizations ranging from government and academia to industry. Between January and February 2021, the 25 by 5: Symposium to Reduce Documentation Burden on US Clinicians by 75% (25X5 Symposium) convened across 2 weekly 2-hour sessions among experts and stakeholders to generate actionable goals for reducing clinician documentation over the next 5 years. Throughout this web-based symposium, we passively collected attendees' contributions to a chat functionality-with their knowledge that the content would be deidentified and made publicly available. This presented a novel opportunity to synthesize and understand participants' perceptions and interests from chat messages. We performed a content analysis of 25X5 Symposium chat logs to identify themes about reducing clinician documentation burden.

OBJECTIVE

The objective of this study was to explore unstructured chat log content from the web-based 25X5 Symposium to elicit latent insights on clinician documentation burden among clinicians, health care leaders, and other stakeholders using topic modeling.

METHODS

Across the 6 sessions, we captured 1787 messages among 167 unique chat participants cumulatively; 14 were private messages not included in the analysis. We implemented a latent Dirichlet allocation (LDA) topic model on the aggregated dataset to identify clinician documentation burden topics mentioned in the chat logs. Coherence scores and manual examination informed optimal model selection. Next, 5 domain experts independently and qualitatively assigned descriptive labels to model-identified topics and classified them into higher-level categories, which were finalized through a panel consensus.

RESULTS

We uncovered ten topics using the LDA model: (1) determining data and documentation needs (422/1773, 23.8%); (2) collectively reassessing documentation requirements in electronic health records (EHRs) (252/1773, 14.2%); (3) focusing documentation on patient narrative (162/1773, 9.1%); (4) documentation that adds value (147/1773, 8.3%); (5) regulatory impact on clinician burden (142/1773, 8%); (6) improved EHR user interface and design (128/1773, 7.2%); (7) addressing poor usability (122/1773, 6.9%); (8) sharing 25X5 Symposium resources (122/1773, 6.9%); (9) capturing data related to clinician practice (113/1773, 6.4%); and (10) the role of quality measures and technology in burnout (110/1773, 6.2%). Among these 10 topics, 5 high-level categories emerged: consensus building (821/1773, 46.3%), burden sources (365/1773, 20.6%), EHR design (250/1773, 14.1%), patient-centered care (162/1773, 9.1%), and symposium comments (122/1773, 6.9%).

CONCLUSIONS

We conducted a topic modeling analysis on 25X5 Symposium multiparticipant chat logs to explore the feasibility of this novel application and elicit additional insights on clinician documentation burden among attendees. Based on the results of our LDA analysis, consensus building, burden sources, EHR design, and patient-centered care may be important themes to consider when addressing clinician documentation burden. Our findings demonstrate the value of topic modeling in discovering topics associated with clinician documentation burden using unstructured textual content. Topic modeling may be a suitable approach to examine latent themes presented in web-based symposium chat logs.

Using Time Series Clustering to Segment and Infer Emergency Department Nursing Shifts from Electronic Health Record Log Files

Few computational approaches exist for abstracting electronic health record (EHR) log files into clinically meaningful phenomena like clinician shifts. Because shifts are a fundamental unit of work recognized in clinical settings, shifts may serve as a primary unit of analysis in the study of documentation burden. We conducted a proof- of-concept study to investigate the feasibility of a novel approach using time series clustering to segment and infer clinician shifts from EHR log files. From 33,535,585 events captured between April-June 2021, we computationally identified 43,911 potential shifts among 2,285 (74.2%) emergency department nurses. On average, computationally-identified shifts were 10.6±3.1 hours long. Based on data distributions, we classified these shifts based on type: day, evening, night; and length: 12-hour, 8-hour, other. We validated our method through manual chart review of computationally-identified 12-hour shifts achieving 92.0% accuracy. Preliminary results suggest unsupervised clustering methods may be a reasonable approach for rapidly identifying clinician shifts.

25 × 5 Symposium to Reduce Documentation Burden: Report-out and Call for Action

BACKGROUND

 The widespread adoption of electronic health records and a simultaneous increase in regulatory demands have led to an acceleration of documentation requirements among clinicians. The corresponding burden from documentation requirements is a central contributor to clinician burnout and can lead to an increased risk of suboptimal patient care.

OBJECTIVE

 To address the problem of documentation burden, the 25 by 5: Symposium to Reduce Documentation Burden on United States Clinicians by 75% by 2025 (Symposium) was organized to provide a forum for experts to discuss the current state of documentation burden and to identify specific actions aimed at dramatically reducing documentation burden for clinicians.

METHODS

 The Symposium consisted of six weekly sessions with 33 presentations. The first four sessions included panel presentations discussing the challenges related to documentation burden. The final two sessions consisted of breakout groups aimed at engaging attendees in establishing interventions for reducing clinical documentation burden. Steering Committee members analyzed notes from each breakout group to develop a list of action items.

RESULTS

 The Steering Committee synthesized and prioritized 82 action items into Calls to Action among three stakeholder groups: Providers and Health Systems, Vendors, and Policy and Advocacy Groups. Action items were then categorized into as short-, medium-, or long-term goals. Themes that emerged from the breakout groups' notes include the following: accountability, evidence is critical, education and training, innovation of technology, and other miscellaneous goals (e.g., vendors will improve shared knowledge databases).

CONCLUSION

 The Symposium successfully generated a list of interventions for short-, medium-, and long-term timeframes as a launching point to address documentation burden in explicit action-oriented ways. Addressing interventions to reduce undue documentation burden placed on clinicians will necessitate collaboration among all stakeholders.

Clinician and Health Care Leaders' Experiences with-and Perceptions of-COVID-19 Documentation Reduction Policies and Practices

BACKGROUND

Substantial strategies to reduce clinical documentation were implemented by health care systems throughout the coronavirus disease-2019 (COVID-19) pandemic at national and local levels. This natural experiment provides an opportunity to study the impact of documentation reduction strategies on documentation burden among clinicians and other health professionals in the United States.

OBJECTIVES

The aim of this study was to assess clinicians' and other health care leaders' experiences with and perceptions of COVID-19 documentation reduction strategies and identify which implemented strategies should be prioritized and remain permanent post-pandemic.

METHODS

We conducted a national survey of clinicians and health care leaders to understand COVID-19 documentation reduction strategies implemented during the pandemic using snowball sampling through professional networks, listservs, and social media. We developed and validated a 19-item survey leveraging existing post-COVID-19 policy and practice recommendations proposed by Sinsky and Linzer. Participants rated reduction strategies for impact on documentation burden on a scale of 0 to 100. Free-text responses were thematically analyzed.

RESULTS

Of the 351 surveys initiated, 193 (55%) were complete. Most participants were informaticians and/or clinicians and worked for a health system or in academia. A majority experienced telehealth expansion (81.9%) during the pandemic, which participants also rated as highly impactful (60.1-61.5) and preferred that it remain (90.5%). Implemented at lower proportions, documenting only pertinent positives to reduce note bloat (66.1 ± 28.3), changing compliance rules and performance metrics to eliminate those without evidence of net benefit (65.7 ± 26.3), and electronic health record (EHR) optimization sprints (64.3 ± 26.9) received the highest impact scores compared with other strategies presented; support for these strategies widely ranged (49.7-63.7%).

CONCLUSION

The results of this survey suggest there are many perceived sources of and solutions for documentation burden. Within strategies, we found considerable support for telehealth, documenting pertinent positives, and changing compliance rules. We also found substantial variation in the experience of documentation burden among participants.