Measuring Collaboration Through Concurrent Electronic Health Record Usage: Network Analysis Study

Learning from Non-Routine Events and Teamwork in Intensive Care Units: Challenges and Opportunities

Patients admitted to intensive care units (ICUs) have profound and complex illnesses, often fraught with uncertainties in diagnoses, treatments, and care decisions. Clinicians often deviate from best practices to handle ICUs' myriad complexities and uncertainties. Non-routine events (NREs), defined as any aspect of care perceived by clinicians as deviations from optimal care, are latent and frequent safety threats that, if left unchecked, can be precursors to adverse events. Proper identification and analysis of NREs that represent latent safety threats have been proposed as a feasible and more effective approach for performance improvement than traditional root cause analysis for patient safety events. However, NRE studies to date have yet to show the holistic picture of NREs in the contexts of teamwork and time-dependent tasks that are frequently associated with NREs. NREs, an upstream interventional area to understand root causes, team performance, and human-computer interaction, still needs to be expanded. This article presents concepts of NREs, and the use of real-world data (RWD) and informatics methodology to investigate NREs in contexts and discusses the opportunities and challenges to enhance NREs research in teamwork and time-dependent tasks.

Assessing Intrahospital Care Transition Structures Before and During the COVID-19 Pandemic: Network Analysis Study

Transitions of care (TOC) is essential for patients with complex medical needs to maintain the continuity of care. The COVID-19 pandemic may result in unexpected pressure on healthcare organizations' routine work and may burden the TOC system. The objective of this study is to assess TOC structures in pre- and intra-COVID-19 and quantify changes in the structures through the lens of network analysis. We investigated a trauma registry repository consisting of care transitions of 5,674 (2,699 and 2,975 in pre- and intra-COVID-19) inpatients admitted to Vanderbilt University Medical Center (VUMC) between January 2019 and May 2021. Network metrics, including assortativity, homophily, and small-world-ness were leveraged to measure TOC structures and their changes. Our results showed both pre- and intra-COVID-19 TOC structures were disassortative, homophily, and small-world- ness, and the COVID-19 pandemic had limited influences on the three characteristics of the TOC structures. The disassortative TOC structure indicates patients can be efficiently transferred between triage centers (highly connected units) and receivers (lowly connected units); the homophily structure demonstrates two connected care units serve similar patients, and the small-world-ness reveals a patient can be transferred to highly collaborative care units with a short length of transfer path.

Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

OBJECTIVE

The aim of this article is to compare the aims, measures, methods, limitations, and scope of studies that employ vendor-derived and investigator-derived measures of electronic health record (EHR) use, and to assess measure consistency across studies.

MATERIALS AND METHODS

We searched PubMed for articles published between July 2019 and December 2021 that employed measures of EHR use derived from EHR event logs. We coded the aims, measures, methods, limitations, and scope of each article and compared articles employing vendor-derived and investigator-derived measures.

RESULTS

One hundred and two articles met inclusion criteria; 40 employed vendor-derived measures, 61 employed investigator-derived measures, and 1 employed both. Studies employing vendor-derived measures were more likely than those employing investigator-derived measures to observe EHR use only in ambulatory settings (83% vs 48%, P = .002) and only by physicians or advanced practice providers (100% vs 54% of studies, P < .001). Studies employing vendor-derived measures were also more likely to measure durations of EHR use (P < .001 for 6 different activities), but definitions of measures such as time outside scheduled hours varied widely. Eight articles reported measure validation. The reported limitations of vendor-derived measures included measure transparency and availability for certain clinical settings and roles.

DISCUSSION

Vendor-derived measures are increasingly used to study EHR use, but only by certain clinical roles. Although poorly validated and variously defined, both vendor- and investigator-derived measures of EHR time are widely reported.

CONCLUSION

The number of studies using event logs to observe EHR use continues to grow, but with inconsistent measure definitions and significant differences between studies that employ vendor-derived and investigator-derived measures.

Using electronic health record audit log data for research: insights from early efforts

Electronic health record audit logs capture a time-sequenced record of clinician activities while using the system. Audit log data therefore facilitate unobtrusive measurement at scale of clinical work activities and workflow as well as derivative, behavioral proxies (eg, teamwork). Given its considerable research potential, studies leveraging these data have burgeoned. As the field has matured, the challenges of using the data to answer significant research questions have come into focus. In this Perspective, we draw on our research experiences and insights from the broader audit log literature to advance audit log research. Specifically, we make 2 complementary recommendations that would facilitate substantial progress toward audit log-based measures that are: (1) transparent and validated, (2) standardized to allow for multisite studies, (3) sensitive to meaningful variability, (4) broader in scope to capture key aspects of clinical work including teamwork and coordination, and (5) linked to patient and clinical outcomes.

Characterizing the macrostructure of electronic health record work using raw audit logs: an unsupervised action embeddings approach

Raw audit logs provide a comprehensive record of clinicians' activities on an electronic health record (EHR) and have considerable potential for studying clinician behaviors. However, research using raw audit logs is limited because they lack context for clinical tasks, leading to difficulties in interpretation. We describe a novel unsupervised approach using the comparison and visualization of EHR action embeddings to learn context and structure from raw audit log activities. Using a dataset of 15 767 634 raw audit log actions performed by 88 intern physicians over 6 months of EHR use across inpatient and outpatient settings, we demonstrated that embeddings can be used to learn the situated context for EHR-based work activities, identify discrete clinical workflows, and discern activities typically performed across diverse contexts. Our approach represents an important methodological advance in raw audit log research, facilitating the future development of metrics and predictive models to measure clinician behaviors at the macroscale.