Modeling Care Team Structures in the Neonatal Intensive Care Unit through Network Analysis of EHR Audit Logs

A Scoping Review of Multilevel Patient-Sharing Network Measures in Health Services Research

Social network analysis is the study of the structure of relationships between social entities. Access to health care administrative datasets has facilitated use of "patient-sharing networks" to infer relationships between health care providers based on the extent to which they have encounters with common patients. The structure and nature of patient-sharing relationships can reflect observed or latent aspects of health care delivery systems, such as collaboration and influence. We conducted a scoping review of peer-reviewed studies that derived patient-sharing network measure(s) in the analyses. There were 134 papers included in the full-text review. We identified and created a centralized resource of 118 measures and uncovered three major themes captured by them: Influential and Key Players, Care Coordination and Teamwork, and Network Structure and Access to Care. Researchers may use this review to inform their use of patient-sharing network measures and to guide the development of novel measures.

Assessing Intrahospital Care Transition Structures Before and During the COVID-19 Pandemic: Network Analysis Study

Transitions of care (TOC) is essential for patients with complex medical needs to maintain the continuity of care. The COVID-19 pandemic may result in unexpected pressure on healthcare organizations' routine work and may burden the TOC system. The objective of this study is to assess TOC structures in pre- and intra-COVID-19 and quantify changes in the structures through the lens of network analysis. We investigated a trauma registry repository consisting of care transitions of 5,674 (2,699 and 2,975 in pre- and intra-COVID-19) inpatients admitted to Vanderbilt University Medical Center (VUMC) between January 2019 and May 2021. Network metrics, including assortativity, homophily, and small-world-ness were leveraged to measure TOC structures and their changes. Our results showed both pre- and intra-COVID-19 TOC structures were disassortative, homophily, and small-world- ness, and the COVID-19 pandemic had limited influences on the three characteristics of the TOC structures. The disassortative TOC structure indicates patients can be efficiently transferred between triage centers (highly connected units) and receivers (lowly connected units); the homophily structure demonstrates two connected care units serve similar patients, and the small-world-ness reveals a patient can be transferred to highly collaborative care units with a short length of transfer path.

Measuring the Cognitive Effort Associated with Task Switching in Routine EHR-based Tasks

OBJECTIVE

Clinical work involves performing overlapping, time-sensitive tasks that frequently require clinicians to switch their attention between multiple tasks. We developed a methodological approach using EHR-based audit logs to determine switch costs-the cognitive burden associated with task switching-and assessed its magnitude during routine EHR-based clinical tasks.

METHOD

Physician trainees (N=75) participated in a longitudinal study where they provided access to their EHR-based audit logs. Physicians' audit log actions were used to create a taxonomy of EHR tasks. These tasks were transformed into task sequences and the time spent on each task in a sequence was computed. Within these task sequences, instances of task switching (i.e., switching from one task to the next) and non-switching were identified. The primary outcome of interest was the time spent on a post-switch task. Using a mixed-effects regression model, we compared the durations of post-switch and non-switch tasks.

RESULTS

2,781,679 audit log events over 117,822 sessions from 75 physicians were analyzed. Physicians spent most time on chart review (Median (IQR)=5,439 (2,492-8,336) seconds), note review (1,936 (827-3,321) seconds), and navigating the EHR interface (1,048 (365.5-2,006) seconds) daily. Post task switch activity times were greater for documentation (Median increase=5 seconds), order entry (Median increase=3 seconds) and results review (Median increase=3 seconds). Mixed-effects regression showed that time spent on tasks were longer following a task switch (β=0.03; 95% CI_lower= 0.027, CI_upper=0.034), with greater post-swtich task times for imaging, order entry, note review, handoff, note entry, chart review and best practice advisory tasks.

DISCUSSION

Increased task switching time-an indicator of the cognitive burden associated with switching between tasks-is prevalent in routine EHR-based tasks. We discuss the cumulative impact of incremental switch costs have on overall EHR workload, wellness, and error rates. Relying on theoretical cognitive foundations, we suggest pragmatic design considerations for mitigating the effects of cognitive burden associated with task switching.

The Use of Electronic Health Record Metadata to Identify Nurse-Patient Assignments in the Intensive Care Unit: Algorithm Development and Validation

Background

Nursing care is a critical determinant of patient outcomes in the intensive care unit (ICU). Most studies of nursing care have focused on nursing characteristics aggregated across the ICU (eg, unit-wide nurse-to-patient ratios, education, and working environment). In contrast, relatively little work has focused on the influence of individual nurses and their characteristics on patient outcomes. Such research could provide granular information needed to create evidence-based nurse assignments, where a nurse’s unique skills are matched to each patient’s needs. To date, research in this area is hindered by an inability to link individual nurses to specific patients retrospectively and at scale.

Objective

This study aimed to determine the feasibility of using nurse metadata from the electronic health record (EHR) to retrospectively determine nurse-patient assignments in the ICU.

Methods

We used EHR data from 38 ICUs in 18 hospitals from 2018 to 2020. We abstracted data on the time and frequency of nurse charting of clinical assessments and medication administration; we then used those data to iteratively develop a deterministic algorithm to identify a single ICU nurse for each patient shift. We examined the accuracy and precision of the algorithm by performing manual chart review on a randomly selected subset of patient shifts.

Results

The analytic data set contained 5,479,034 unique nurse-patient charting times; 748,771 patient shifts; 87,466 hospitalizations; 70,002 patients; and 8,134 individual nurses. The final algorithm identified a single nurse for 97.3% (728,533/748,771) of patient shifts. In the remaining 2.7% (20,238/748,771) of patient shifts, the algorithm either identified multiple nurses (4,755/748,771, 0.6%), no nurse (14,689/748,771, 2%), or the same nurse as the prior shift (794/748,771, 0.1%). In 200 patient shifts selected for chart review, the algorithm had a 93% accuracy (ie, correctly identifying the primary nurse or correctly identifying that there was no primary nurse) and a 94.4% precision (ie, correctly identifying the primary nurse when a primary nurse was identified). Misclassification was most frequently due to patient transitions in care location, such as ICU transfers, discharges, and admissions.

Conclusions

Metadata from the EHR can accurately identify individual nurse-patient assignments in the ICU. This information enables novel studies of ICU nurse staffing at the individual nurse-patient level, which may provide further insights into how nurse staffing can be leveraged to improve patient outcomes.

Assessing Neonatal Intensive Care Unit Structures and Outcomes Before and During the COVID-19 Pandemic: Network Analysis Study

Background

Health care organizations (HCOs) adopt strategies (eg. physical distancing) to protect clinicians and patients in intensive care units (ICUs) during the COVID-19 pandemic. Many care activities physically performed before the COVID-19 pandemic have transitioned to virtual systems during the pandemic. These transitions can interfere with collaboration structures in the ICU, which may impact clinical outcomes. Understanding the differences can help HCOs identify challenges when transitioning physical collaboration to the virtual setting in the post–COVID-19 era.

Objective

This study aims to leverage network analysis to determine the changes in neonatal ICU (NICU) collaboration structures from the pre– to the intra–COVID-19 era.

Methods

In this retrospective study, we applied network analysis to the utilization of electronic health records (EHRs) of 712 critically ill neonates (pre–COVID-19, n=386; intra–COVID-19, n=326, excluding those with COVID-19) admitted to the NICU of Vanderbilt University Medical Center between September 1, 2019, and June 30, 2020, to assess collaboration between clinicians. We characterized pre–COVID-19 as the period of September-December 2019 and intra–COVID-19 as the period of March-June 2020. These 2 groups were compared using patients’ clinical characteristics, including age, sex, race, length of stay (LOS), and discharge dispositions. We leveraged the clinicians’ actions committed to the patients’ EHRs to measure clinician-clinician connections. We characterized a collaboration relationship (tie) between 2 clinicians as actioning EHRs of the same patient within the same day. On defining collaboration relationship, we built pre– and intra–COVID-19 networks. We used 3 sociometric measurements, including eigenvector centrality, eccentricity, and betweenness, to quantify a clinician’s leadership, collaboration difficulty, and broad skill sets in a network, respectively. We assessed the extent to which the eigenvector centrality, eccentricity, and betweenness of clinicians in the 2 networks are statistically different, using Mann-Whitney U tests (95% CI).

Results

Collaboration difficulty increased from the pre– to intra–COVID-19 periods (median eccentricity: 3 vs 4; P<.001). Nurses had reduced leadership (median eigenvector centrality: 0.183 vs 0.087; P<.001), and neonatologists with broader skill sets cared for more patients in the NICU structure during the pandemic (median betweenness centrality: 0.0001 vs 0.005; P<.001). The pre– and intra–COVID-19 patient groups shared similar distributions in sex (~0 difference), race (4% difference in White, and 3% difference in African American), LOS (interquartile range difference in 1.5 days), and discharge dispositions (~0 difference in home, 2% difference in expired, and 2% difference in others). There were no significant differences in the patient demographics and outcomes between the 2 groups.

Conclusions

Management of NICU-admitted patients typically requires multidisciplinary care teams. Understanding collaboration structures can provide fine-grained evidence to potentially refine or optimize existing teamwork in the NICU.

Measuring Collaboration Through Concurrent Electronic Health Record Usage: Network Analysis Study

BACKGROUND

Collaboration is vital within health care institutions, and it allows for the effective use of collective health care worker (HCW) expertise. Human-computer interactions involving electronic health records (EHRs) have become pervasive and act as an avenue for quantifying these collaborations using statistical and network analysis methods.

OBJECTIVE

We aimed to measure HCW collaboration and its characteristics by analyzing concurrent EHR usage.

METHODS

By extracting concurrent EHR usage events from audit log data, we defined concurrent sessions. For each HCW, we established a metric called concurrent intensity, which was the proportion of EHR activities in concurrent sessions over all EHR activities. Statistical models were used to test the differences in the concurrent intensity between HCWs. For each patient visit, starting from admission to discharge, we measured concurrent EHR usage across all HCWs, which we called temporal patterns. Again, we applied statistical models to test the differences in temporal patterns of the admission, discharge, and intermediate days of hospital stay between weekdays and weekends. Network analysis was leveraged to measure collaborative relationships among HCWs. We surveyed experts to determine if they could distinguish collaborative relationships between high and low likelihood categories derived from concurrent EHR usage. Clustering was used to aggregate concurrent activities to describe concurrent sessions. We gathered 4 months of EHR audit log data from a large academic medical center's neonatal intensive care unit (NICU) to validate the effectiveness of our framework.

RESULTS

There was a significant difference (P<.001) in the concurrent intensity (proportion of concurrent activities: ranging from mean 0.07, 95% CI 0.06-0.08, to mean 0.36, 95% CI 0.18-0.54; proportion of time spent on concurrent activities: ranging from mean 0.32, 95% CI 0.20-0.44, to mean 0.76, 95% CI 0.51-1.00) between the top 13 HCW specialties who had the largest amount of time spent in EHRs. Temporal patterns between weekday and weekend periods were significantly different on admission (number of concurrent intervals per hour: 11.60 vs 0.54; P<.001) and discharge days (4.72 vs 1.54; P<.001), but not during intermediate days of hospital stay. Neonatal nurses, fellows, frontline providers, neonatologists, consultants, respiratory therapists, and ancillary and support staff had collaborative relationships. NICU professionals could distinguish high likelihood collaborative relationships from low ones at significant rates (3.54, 95% CI 3.31-4.37 vs 2.64, 95% CI 2.46-3.29; P<.001). We identified 50 clusters of concurrent activities. Over 87% of concurrent sessions could be described by a single cluster, with the remaining 13% of sessions comprising multiple clusters.

CONCLUSIONS

Leveraging concurrent EHR usage workflow through audit logs to analyze HCW collaboration may improve our understanding of collaborative patient care. HCW collaboration using EHRs could potentially influence the quality of patient care, discharge timeliness, and clinician workload, stress, or burnout.

Mining tasks and task characteristics from electronic health record audit logs with unsupervised machine learning

OBJECTIVE

The characteristics of clinician activities while interacting with electronic health record (EHR) systems can influence the time spent in EHRs and workload. This study aims to characterize EHR activities as tasks and define novel, data-driven metrics.

MATERIALS AND METHODS

We leveraged unsupervised learning approaches to learn tasks from sequences of events in EHR audit logs. We developed metrics characterizing the prevalence of unique events and event repetition and applied them to categorize tasks into 4 complexity profiles. Between these profiles, Mann-Whitney U tests were applied to measure the differences in performance time, event type, and clinician prevalence, or the number of unique clinicians who were observed performing these tasks. In addition, we apply process mining frameworks paired with clinical annotations to support the validity of a sample of our identified tasks. We apply our approaches to learn tasks performed by nurses in the Vanderbilt University Medical Center neonatal intensive care unit.

RESULTS

We examined EHR audit logs generated by 33 neonatal intensive care unit nurses resulting in 57 234 sessions and 81 tasks. Our results indicated significant differences in performance time for each observed task complexity profile. There were no significant differences in clinician prevalence or in the frequency of viewing and modifying event types between tasks of different complexities. We presented a sample of expert-reviewed, annotated task workflows supporting the interpretation of their clinical meaningfulness.

CONCLUSIONS

The use of the audit log provides an opportunity to assist hospitals in further investigating clinician activities to optimize EHR workflows.

Collaboration Structures in COVID-19 Critical Care: Retrospective Network Analysis Study

Background

Few intensive care unit (ICU) staffing studies have examined the collaboration structures of health care workers (HCWs). Knowledge about how HCWs are connected to the care of critically ill patients with COVID-19 is important for characterizing the relationships among team structures, care quality, and patient safety.

Objective

We aimed to discover differences in the teamwork structures of COVID-19 critical care by comparing HCW collaborations in the management of critically ill patients with and without COVID-19.

Methods

In this retrospective study, we used network analysis methods to analyze the electronic health records (EHRs) of 76 critically ill patients (with COVID-19: n=38; without COVID-19: n=38) who were admitted to a large academic medical center, and to learn about HCW collaboration. We used the EHRs of adult patients who were admitted to the COVID-19 ICU at the Vanderbilt University Medical Center (Nashville, Tennessee, United States) between March 17, 2020, and May 31, 2020. We matched each patient according to age, gender, and their length of stay. Patients without COVID-19 were admitted to the medical ICU between December 1, 2019, and February 29, 2020. We used two sociometrics—eigencentrality and betweenness—to quantify HCWs’ statuses in networks. Eigencentrality characterizes the degree to which an HCW is a core person in collaboration structures. Betweenness centrality refers to whether an HCW lies on the path of other HCWs who are not directly connected. This sociometric was used to characterize HCWs’ broad skill sets. We measured patient staffing intensity in terms of the number of HCWs who interacted with patients’ EHRs. We assessed the statistical differences in the core and betweenness statuses of HCWs and the patient staffing intensities of COVID-19 and non–COVID-19 critical care, by using Mann-Whitney U tests and reporting 95% CIs.

Results

HCWs in COVID-19 critical care were more likely to frequently work with each other (eigencentrality: median 0.096) than those in non–COVID-19 critical care (eigencentrality: median 0.057; P<.001). Internal medicine physicians in COVID-19 critical care had higher core statuses than those in non–COVID-19 critical care (P=.001). Nurse practitioners in COVID-19 care had higher betweenness statuses than those in non–COVID-19 care (P<.001). Compared to HCWs in non–COVID-19 settings, the EHRs of critically ill patients with COVID-19 were used by a larger number of internal medicine nurse practitioners (P<.001), cardiovascular nurses (P<.001), and surgical ICU nurses (P=.002) and a smaller number of resident physicians (P<.001).

Conclusions

Network analysis methodologies and data on EHR use provide a novel method for learning about differences in collaboration structures between COVID-19 and non–COVID-19 critical care. Health care organizations can use this information to learn about the novel changes that the COVID-19 pandemic has imposed on collaboration structures in urgent care.

An electronic health record (EHR) log analysis shows limited clinician engagement with unsolicited genetic test results

How clinicians utilize medically actionable genomic information, displayed in the electronic health record (EHR), in medical decision-making remains unknown. Participating sites of the Electronic Medical Records and Genomics (eMERGE) Network have invested resources into EHR integration efforts to enable the display of genetic testing data across heterogeneous EHR systems. To assess clinicians' engagement with unsolicited EHR-integrated genetic test results of eMERGE participants within a large tertiary care academic medical center, we analyzed automatically generated EHR access log data. We found that clinicians viewed only 1% of all the eMERGE genetic test results integrated in the EHR. Using a cluster analysis, we also identified different user traits associated with varying degrees of engagement with the EHR-integrated genomic data. These data contribute important empirical knowledge about clinicians limited and brief engagements with unsolicited EHR-integrated genetic test results of eMERGE participants. Appreciation for user-specific roles provide additional context for why certain users were more or less engaged with the unsolicited results. This study highlights opportunities to use EHR log data as a performance metric to more precisely inform ongoing EHR-integration efforts and decisions about the allocation of informatics resources in genomic research.