Conceptual considerations for using EHR-based activity logs to measure clinician burnout and its effects

A Computational Framework to Evaluate Emergency Department Clinician Task Switching in the Electronic Health Record Using Event Logs

Workflow fragmentation, defined as task switching, may be one proxy to quantify electronic health record (EHR) documentation burden in the emergency department (ED). Few measures have been operationalized to evaluate task switching at scale. Theoretically grounded in the time-based resource-sharing model (TBRSM) which conceives task switching as proportional to the cognitive load experienced, we describe the functional relationship between cognitive load and the time and effort constructs previously applied for measuring documentation burden. We present a computational framework, COMBINE, to evaluate multilevel task switching in the ED using EHR event logs. Based on this framework, we conducted a descriptive analysis on task switching among 63 full-time ED physicians from one ED site using EHR event logs extracted between April-June 2021 (n=2,068,605 events) which were matched to scheduled shifts (n=952). On average, we found a high volume of event-level (185.8±75.3/hr) and within-(6.6±1.7/chart) and between-patient chart (27.5±23.6/hr) switching per shift worked.

Anesthesia Clinical Workload Estimated From Electronic Health Record Documentation vs Billed Relative Value Units

Importance

Accurate measurements of clinical workload are needed to inform health care policy. Existing methods for measuring clinical workload rely on surveys or time-motion studies, which are labor-intensive to collect and subject to biases.

Objective

To compare anesthesia clinical workload estimated from electronic health record (EHR) audit log data vs billed relative value units.

Design, Setting, and Participants

This cross-sectional study of anesthetic encounters occurring between August 26, 2019, and February 9, 2020, used data from 8 academic hospitals, community hospitals, and surgical centers across Missouri and Illinois. Clinicians who provided anesthetic services for at least 1 surgical encounter were included. Data were analyzed from January 2022 to January 2023.

Exposure

Anesthetic encounters associated with a surgical procedure were included. Encounters associated with labor analgesia and endoscopy were excluded.

Main Outcomes and Measures

For each encounter, EHR-derived clinical workload was estimated as the sum of all EHR actions recorded in the audit log by anesthesia clinicians who provided care. Billing-derived clinical workload was measured as the total number of units billed for the encounter. A linear mixed-effects model was used to estimate the relative contribution of patient complexity (American Society of Anesthesiology [ASA] physical status modifier), procedure complexity (ASA base unit value for the procedure), and anesthetic duration (time units) to EHR-derived and billing-derived workload. The resulting β coefficients were interpreted as the expected effect of a 1-unit change in each independent variable on the standardized workload outcome. The analysis plan was developed after the data were obtained.

Results

A total of 405 clinicians who provided anesthesia for 31 688 encounters were included in the study. A total of 8 288 132 audit log actions corresponding to 39 131 hours of EHR use were used to measure EHR-derived workload. The contributions of patient complexity, procedural complexity, and anesthesia duration to EHR-derived workload differed significantly from their contributions to billing-derived workload. The contribution of patient complexity toward EHR-derived workload (β = 0.162; 95% CI, 0.153-0.171) was more than 50% greater than its contribution toward billing-derived workload (β = 0.106; 95% CI, 0.097-0.116; P < .001). In contrast, the contribution of procedure complexity toward EHR-derived workload (β = 0.033; 95% CI, 0.031-0.035) was approximately one-third its contribution toward billing-derived workload (β = 0.106; 95% CI, 0.104-0.108; P < .001).

Conclusions and Relevance

In this cross-sectional study of 8 hospitals, reimbursement for anesthesiology services overcompensated for procedural complexity and undercompensated for patient complexity. This method for measuring clinical workload could be used to improve reimbursement valuations for anesthesia and other specialties.

Measuring the Cognitive Effort Associated with Task Switching in Routine EHR-based Tasks

OBJECTIVE

Clinical work involves performing overlapping, time-sensitive tasks that frequently require clinicians to switch their attention between multiple tasks. We developed a methodological approach using EHR-based audit logs to determine switch costs-the cognitive burden associated with task switching-and assessed its magnitude during routine EHR-based clinical tasks.

METHOD

Physician trainees (N=75) participated in a longitudinal study where they provided access to their EHR-based audit logs. Physicians' audit log actions were used to create a taxonomy of EHR tasks. These tasks were transformed into task sequences and the time spent on each task in a sequence was computed. Within these task sequences, instances of task switching (i.e., switching from one task to the next) and non-switching were identified. The primary outcome of interest was the time spent on a post-switch task. Using a mixed-effects regression model, we compared the durations of post-switch and non-switch tasks.

RESULTS

2,781,679 audit log events over 117,822 sessions from 75 physicians were analyzed. Physicians spent most time on chart review (Median (IQR)=5,439 (2,492-8,336) seconds), note review (1,936 (827-3,321) seconds), and navigating the EHR interface (1,048 (365.5-2,006) seconds) daily. Post task switch activity times were greater for documentation (Median increase=5 seconds), order entry (Median increase=3 seconds) and results review (Median increase=3 seconds). Mixed-effects regression showed that time spent on tasks were longer following a task switch (β=0.03; 95% CI_lower= 0.027, CI_upper=0.034), with greater post-swtich task times for imaging, order entry, note review, handoff, note entry, chart review and best practice advisory tasks.

DISCUSSION

Increased task switching time-an indicator of the cognitive burden associated with switching between tasks-is prevalent in routine EHR-based tasks. We discuss the cumulative impact of incremental switch costs have on overall EHR workload, wellness, and error rates. Relying on theoretical cognitive foundations, we suggest pragmatic design considerations for mitigating the effects of cognitive burden associated with task switching.

Relationships Between EHR-Based Audit Log Data and Physician Burnout and Clinical Practice Process Measures

OBJECTIVE

To explore the relationship of electronic health record (EHR)-based audit log data with physician burnout and clinical practice process measures.

METHODS

From September 4 to October 7, 2019, we surveyed physicians in a larger academic medical department and matched responses to August 1 through October 31, 2019, EHR-based audit log data. Multivariable regression analysis evaluated the relationship between log data and burnout and the interrelationship between log data and turnaround time for In Basket messages and percentage of encounters closed within 24 hours.

RESULTS

Of the 537 physicians surveyed, 413 (77%) responded. On multivariable analysis, number of In Basket messages received per day (each additional message: odds ratio, 1.04 [95% CI, 1.02 to 1.07]; P<.001) and time spent in the EHR outside scheduled patient care (each additional hour: odds ratio, 1.01 [95% CI, 1.00 to 1.02]; P=.04) were associated with burnout. Time spent doing In Basket work (each additional minute: parameter estimate, -0.11 [95% CI, -0.19 to -0.03]; P=.01) and in the EHR outside scheduled patient care (each additional hour: parameter estimate, 0.04 [95% CI, 0.01 to 0.06]; P=.002) were associated with turnaround time (days per message) for In Basket messages. None of the variables explored were independently associated with percentage of encounters closed within 24 hours.

CONCLUSION

Electronic health record-based audit log data of workload relate to odds of burnout and responsiveness to patient-related inquiries and results. Further study is needed to determine whether interventions that reduce the number of and time spent doing In Basket messages or time spent in the EHR outside scheduled patient care reduce physician burnout and improve clinical practice process measures.

Using event logs to observe interactions with electronic health records: an updated scoping review shows increasing use of vendor-derived measures

OBJECTIVE

The aim of this article is to compare the aims, measures, methods, limitations, and scope of studies that employ vendor-derived and investigator-derived measures of electronic health record (EHR) use, and to assess measure consistency across studies.

MATERIALS AND METHODS

We searched PubMed for articles published between July 2019 and December 2021 that employed measures of EHR use derived from EHR event logs. We coded the aims, measures, methods, limitations, and scope of each article and compared articles employing vendor-derived and investigator-derived measures.

RESULTS

One hundred and two articles met inclusion criteria; 40 employed vendor-derived measures, 61 employed investigator-derived measures, and 1 employed both. Studies employing vendor-derived measures were more likely than those employing investigator-derived measures to observe EHR use only in ambulatory settings (83% vs 48%, P = .002) and only by physicians or advanced practice providers (100% vs 54% of studies, P < .001). Studies employing vendor-derived measures were also more likely to measure durations of EHR use (P < .001 for 6 different activities), but definitions of measures such as time outside scheduled hours varied widely. Eight articles reported measure validation. The reported limitations of vendor-derived measures included measure transparency and availability for certain clinical settings and roles.

DISCUSSION

Vendor-derived measures are increasingly used to study EHR use, but only by certain clinical roles. Although poorly validated and variously defined, both vendor- and investigator-derived measures of EHR time are widely reported.

CONCLUSION

The number of studies using event logs to observe EHR use continues to grow, but with inconsistent measure definitions and significant differences between studies that employ vendor-derived and investigator-derived measures.

Using electronic health record audit log data for research: insights from early efforts

Electronic health record audit logs capture a time-sequenced record of clinician activities while using the system. Audit log data therefore facilitate unobtrusive measurement at scale of clinical work activities and workflow as well as derivative, behavioral proxies (eg, teamwork). Given its considerable research potential, studies leveraging these data have burgeoned. As the field has matured, the challenges of using the data to answer significant research questions have come into focus. In this Perspective, we draw on our research experiences and insights from the broader audit log literature to advance audit log research. Specifically, we make 2 complementary recommendations that would facilitate substantial progress toward audit log-based measures that are: (1) transparent and validated, (2) standardized to allow for multisite studies, (3) sensitive to meaningful variability, (4) broader in scope to capture key aspects of clinical work including teamwork and coordination, and (5) linked to patient and clinical outcomes.

Accuracy of Physician Electronic Health Record Usage Analytics using Clinical Test Cases

Usage log data are an important data source for characterizing the potential burden related to use of the electronic health record (EHR) system. However, the utility of this data source has been hindered by concerns related to the real-world validity and accuracy of the data. While time-motion studies have historically been used to address this concern, the restrictions caused by the pandemic have made it difficult to carry out these studies in-person. In this regard, we introduce a practical approach for conducting validation studies for usage log data in a controlled environment. By developing test runs based on clinical workflows and conducting them within a test EHR environment, it allows for both comparison of the recorded timings and retrospective investigation of any discrepancies. In this case report, we describe the utility of this approach for validating our physician EHR usage logs at a large academic teaching mental health hospital in Canada. A total of 10 test runs were conducted across 3 days to validate 8 EHR usage log metrics, finding differences between recorded measurements and the usage analytics platform ranging from 9 to 60%.

The effect of My Health Record use in the emergency department on clinician-assessed patient care: results from a survey

Background

The emergency department has been a major focus for the implementation of Australia’s national electronic health record, known as My Health Record. However, the association between use of My Health Record in the emergency department setting and patient care is largely unknown. The aim of this study was to explore the perspectives of emergency department clinicians regarding My Health Record use frequency, the benefits of My Health Record use (with a focus on patient care) and the barriers to use.

Methods

All 393 nursing, pharmacy, physician and allied health staff employed within the emergency department at a tertiary metropolitan public hospital in Melbourne were invited to participate in a web-based survey, between 1 May 2021 and 1 December 2021, during the height of the Delta and Omicron Covid-19 outbreaks in Victoria, Australia.

Results

Overall, the survey response rate was 18% (70/393). Approximately half of the sample indicated My Health Record use in the emergency department (n = 39, 56%, confidence interval [CI] 43–68%). The results showed that users typically only engaged with My Health Record less than once per shift (n = 15, 39%, CI 23–55%). Just over half (n = 19/39, 54%, CI 32–65%) of all participants who use My Health Record agreed they could remember a time when My Health Record had been critical to the care of a patient. Overall, clinicians indicated the biggest barrier preventing their use of My Health Record is that they forget to utilise the system.

Conclusion

The results suggest that My Health Record has not been adopted as routine practice in the emergency department, by the majority of participants. Close to half of self-identified users of My Health Record do not associate use as being critical to patient care. Instead, My Health Record may only be used in scenarios that clinicians perceive will yield the greatest benefit—which clinicians in this paper suggest is patients with chronic and complex conditions. Further research that explores the predictors to use and consumers most likely to benefit from use is recommended—and strategies to socialise this knowledge and educate clinicians is desperately required.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-022-01920-8.

Opportunities to use electronic health record audit logs to improve cancer care

The rapid adoption of electronic health records (EHRs) has created extensive repositories of digitized data that can be used to inform improvements in care delivery, processes, and patient outcomes. While the clinical data captured in EHRs are widely used for such efforts, EHRs also capture audit log data that reflect how users interact with the EHR to deliver care. Automatically collected audit log data provide a unique opportunity for new insights into EHR user behavior and decision‐making processes. Here, we provide an overview of audit log data and examples that could be used to improve oncology care and outcomes in four domains: diagnostic reasoning and consumption, care team collaboration and communication, patient outcomes and experience, and provider burnout/fatigue. This data source could identify gaps in performance and care, physician uptake of EHR features that enhance decision‐making, and integration of data trends for oncology. Ensuring researchers and oncologists are familiar with the data's potential and developing the data engineering capacity to utilize this rich data source, will expand the breadth of research to improve cancer care.

Tackling the Burden of Electronic Health Record Use Among Physicians in a Mental Health Setting: Physician Engagement Strategy

The burden associated with using the electronic health record system continues to be a critical issue for physicians and is potentially contributing to physician burnout. At a large academic mental health hospital in Canada, we recently implemented a Physician Engagement Strategy focused on reducing the burden of electronic health record use through close collaboration with clinical leadership, information technology leadership, and physicians. Built on extensive stakeholder consultation, this strategy highlights initiatives that we have implemented (or will be implementing in the near future) under four components: engage, inspire, change, and measure. In this viewpoint paper, we share our process of developing and implementing the Physician Engagement Strategy and discuss the lessons learned and implications of this work.

Predicting physician burnout using clinical activity logs: Model performance and lessons learned

BACKGROUND

Burnout is a significant public health concern affecting more than half of the healthcare workforce; however, passive screening tools to detect burnout are lacking. We investigated the ability of machine learning (ML) techniques to identify burnout using passively collected electronic health record (EHR)-based audit log data.

METHOD

Physician trainees participated in a longitudinal study where they completed monthly burnout surveys and provided access to their EHR-based audit logs. Using the monthly burnout scores as the target outcome, we trained ML models using combinations of features derived from audit log data-aggregate measures of clinical workload, time series-based temporal measures of EHR use, and the baseline burnout score. Five ML models were constructed to predict burnout as a continuous score: penalized linear regression, support vector machine, neural network, random forest, and gradient boosting machine.

RESULTS

88 trainee physicians participated and completed 416 surveys; greater than10 million audit log actions were collected (Mean [Standard Deviation] = 25,691 [14,331] actions per month, per physician). The workload feature set predicted burnout score with a mean absolute error (MAE) of 0.602 (95% Confidence Interval (CI), 0.412-0.826), and was able to predict burnout status with an average AUROC of 0.595 (95% CI 0.355-0.808) and average accuracy 0.567 (95% CI 0.393-0.742). The temporal feature set had a similar performance, with MAE 0.596 (95% CI 0.391-0.826), and AUROC 0.581 (95% CI 0.343-0.790). The addition of the baseline burnout score to the workload features improved the model performance to a mean AUROC of 0.829 (95% CI 0.607-0.996) and mean accuracy of 0.781 (95% CI 0.587-0.936); however, this performance was not meaningfully different than using the baseline burnout score alone.

CONCLUSIONS

Current findings illustrate the complexities of predicting burnout exclusively based on clinical work activities as captured in the EHR, highlighting its multi-factorial and individualized nature. Future prediction studies of burnout should account for individual factors (e.g., resilience, physiological measurements such as sleep) and associated system-level factors (e.g., leadership).

Burnout: A Review of Theory and Measurement

A growing body of empirical evidence shows that occupational health is now more relevant than ever due to the COVID-19 pandemic. This review focuses on burnout, an occupational phenomenon that results from chronic stress in the workplace. After analyzing how burnout occurs and its different dimensions, the following aspects are discussed: (1) Description of the factors that can trigger burnout and the individual factors that have been proposed to modulate it, (2) identification of the effects that burnout generates at both individual and organizational levels, (3) presentation of the main actions that can be used to prevent and/or reduce burnout, and (4) recapitulation of the main tools that have been developed so far to measure burnout, both from a generic perspective or applied to specific occupations. Furthermore, this review summarizes the main contributions of the papers that comprise the Special Issue on “Occupational Stress and Health: Psychological Burden and Burnout”, which represent an advance in the theoretical and practical understanding of burnout.

Predicting next-day discharge via electronic health record access logs

OBJECTIVE

Hospital capacity management depends on accurate real-time estimates of hospital-wide discharges. Estimation by a clinician requires an excessively large amount of effort and, even when attempted, accuracy in forecasting next-day patient-level discharge is poor. This study aims to support next-day discharge predictions with machine learning by incorporating electronic health record (EHR) audit log data, a resource that captures EHR users' granular interactions with patients' records by communicating various semantics and has been neglected in outcome predictions.

MATERIALS AND METHODS

This study focused on the EHR data for all adults admitted to Vanderbilt University Medical Center in 2019. We learned multiple advanced models to assess the value that EHR audit log data adds to the daily prediction of discharge likelihood within 24 h and to compare different representation strategies. We applied Shapley additive explanations to identify the most influential types of user-EHR interactions for discharge prediction.

RESULTS

The data include 26 283 inpatient stays, 133 398 patient-day observations, and 819 types of user-EHR interactions. The model using the count of each type of interaction in the recent 24 h and other commonly used features, including demographics and admission diagnoses, achieved the highest area under the receiver operating characteristics (AUROC) curve of 0.921 (95% CI: 0.919-0.923). By contrast, the model lacking user-EHR interactions achieved a worse AUROC of 0.862 (0.860-0.865). In addition, 10 of the 20 (50%) most influential factors were user-EHR interaction features.

CONCLUSION

EHR audit log data contain rich information such that it can improve hospital-wide discharge predictions.

Characterizing Multitasking and Workflow Fragmentation in Electronic Health Records among Emergency Department Clinicians: Using Time-Motion Data to Understand Documentation Burden

BACKGROUND

The impact of electronic health records (EHRs) in the emergency department (ED) remains mixed. Dynamic and unpredictable, the ED is highly vulnerable to workflow interruptions.

OBJECTIVES

The aim of the study is to understand multitasking and task fragmentation in the clinical workflow among ED clinicians using clinical information systems (CIS) through time-motion study (TMS) data, and inform their applications to more robust and generalizable measures of CIS-related documentation burden.

METHODS

Using TMS data collected among 15 clinicians in the ED, we investigated the role of documentation burden, multitasking (i.e., performing physical and communication tasks concurrently), and workflow fragmentation in the ED. We focused on CIS-related tasks, including EHRs.

RESULTS

We captured 5,061 tasks and 877 communications in 741 locations within the ED. Of the 58.7 total hours observed, 44.7% were spent on CIS-related tasks; nearly all CIS-related tasks focused on data-viewing and data-entering. Over one-fifth of CIS-related task time was spent on multitasking. The mean average duration among multitasked CIS-related tasks was shorter than non-multitasked CIS-related tasks (20.7 s vs. 30.1 s). Clinicians experienced 1.4 ± 0.9 task switches/min, which increased by one-third when multitasking. Although multitasking was associated with a significant increase in the average duration among data-entering tasks, there was no significant effect on data-viewing tasks. When engaged in CIS-related task switches, clinicians were more likely to return to the same CIS-related task at higher proportions while multitasking versus not multitasking.

CONCLUSION

Multitasking and workflow fragmentation may play a significant role in EHR documentation among ED clinicians, particularly among data-entering tasks. Understanding where and when multitasking and workflow fragmentation occurs is a crucial step to assessing potentially burdensome clinician tasks and mitigating risks to patient safety. These findings may guide future research on developing more scalable and generalizable measures of CIS-related documentation burden that do not necessitate direct observation techniques (e.g., EHR log files).

Digital and informatics competencies: Requirements for nursing leaders in Canada

The use of health information technologies continues to grow, especially with the increase in virtual care in response to COVID-19. As the largest health professional group in Canada, nurses are key stakeholders and their active engagement is essential for the meaningful adoption and use of digital health technologies to support patient care. Nurse leaders in particular are uniquely positioned to inform key technology decisions; therefore, enhancing their informatics capacity is paramount to the success of digital health initiatives and investments. The purpose of this commentary is to reflect on current projects relevant to the development of informatics competencies for nurse leaders in the Canadian context and offer our perspectives on ways to enhance current and future nurse leaders' readiness for participation in digital health initiatives. Addressing the digital health knowledge and abilities of nurse leaders will improve their capacity to champion and lead transformative health system changes through digital innovation.

Impact of Changes in EHR Use during COVID-19 on Physician Trainee Mental Health

OBJECTIVES

This article investigates the association between changes in electronic health record (EHR) use during the coronavirus disease 2019 (COVID-19) pandemic on the rate of burnout, stress, posttraumatic stress disorder (PTSD), depression, and anxiety among physician trainees (residents and fellows).

METHODS

A total of 222 (of 1,375, 16.2%) physician trainees from an academic medical center responded to a Web-based survey. We compared the physician trainees who reported that their EHR use increased versus those whose EHR use stayed the same or decreased on outcomes related to depression, anxiety, stress, PTSD, and burnout using univariable and multivariable models. We examined whether self-reported exposure to COVID-19 patients moderated these relationships.

RESULTS

Physician trainees who reported increased use of EHR had higher burnout (adjusted mean, 1.48 [95% confidence interval [CI] 1.24, 1.71] vs. 1.05 [95% CI 0.93, 1.17]; p = 0.001) and were more likely to exhibit symptoms of PTSD (adjusted mean = 15.09 [95% CI 9.12, 21.05] vs. 9.36 [95% CI 7.38, 11.28]; p = 0.035). Physician trainees reporting increased EHR use outside of work were more likely to experience depression (adjusted mean, 8.37 [95% CI 5.68, 11.05] vs. 5.50 [95% CI 4.28, 6.72]; p = 0.035). Among physician trainees with increased EHR use, those exposed to COVID-19 patients had significantly higher burnout (2.04, p < 0.001) and depression scores (14.13, p = 0.003).

CONCLUSION

Increased EHR use was associated with higher burnout, depression, and PTSD outcomes among physician trainees. Although preliminary, these findings have implications for creating systemic changes to manage the wellness and well-being of trainees.