The Association Between Perceived Electronic Health Record Usability and Professional Burnout Among US Physicians

Development, validation, and usability evaluation of machine learning algorithms for predicting personalized red blood cell demand among thoracic surgery patients

INTRODUCTION

Preparing appropriate red blood cells (RBCs) before surgery is crucial for improving both the efficacy of perioperative workflow and patient safety. In particular, thoracic surgery (TS) is a procedure that requires massive transfusion with high variability for each patient. Hence, the precise prediction of RBC requirements for individual patients is becoming increasingly important. This study aimed to 1) develop and validate a machine learning algorithm for personalized RBC predictions for TS patients and 2) assess the usability of a clinical decision support system (CDSS) integrating this artificial intelligence model.

METHODS

Adult patients who underwent TS between January 2016 and October 2021 were included in this study. Multiple models were developed by employing both traditional statistical- and machine-learning approaches. The primary outcome evaluated the model's performance in predicting RBC requirements through root mean square error and adjusted R2. Surgeons and informaticians determined the precision MSBOS-Thoracic Surgery (pMSBOS-TS) algorithm through a consensus process. The usability of the pMSBOS-TS was assessed using the System Usability Scale (SUS) survey with 60 clinicians.

RESULTS

We identified 7,843 cases (6,200 for training and 1,643 for test sets) of TSs. Among the models with variable performance indices, the extreme gradient boosting model was selected as the pMSBOS-TS algorithm. The pMSBOS-TS model showed statistically significant lower root mean square error (mean: 3.203 and 95% confidence interval [CI]: 3.186-3.220) compared to the calculated Maximum Surgical Blood Ordering Schedule (MSBOS) and a higher adjusted R2 (mean: 0.399 and 95% CI: 0.395-0.403) compared to the calculated MSBOS, while requiring approximately 200 fewer packs for RBC preparation compared to the calculated MSBOS. The SUS score of the pMSBOS-TS CDSS was 72.5 points, indicating good acceptability.

CONCLUSIONS

We successfully developed the pMSBOS-TS capable of predicting personalized RBC transfusion requirements for perioperative patients undergoing TS.

Virtual reality for preoperative patient education: Impact on satisfaction, usability, and burnout from the perspective of new nurses

BACKGROUND

Traditional paper-based preoperative patient education is a struggle for new nurses and requires extensive training. In this situation, virtual reality technology can help the new nurses. Despite its potential benefits, there are studies on patient satisfaction but there is limited information on the usability of virtual reality (VR) technology for new nurses in giving preoperative education to patients.

AIM

To investigate the impact on satisfaction, usability, and burnout of a system using VR technology in preoperative patient education.

METHODS

The study involved 20 nurses from the plastic surgery ward and 80 patients admitted between April and May 2019. Each nurse taught four patients: Two using traditional verbal education and two using virtual reality. The System Usability Scale, After-Scenario Questionnaire, and Maslach Burnout Inventory (MBI) were employed to evaluate the impact of these education methods.

RESULTS

The VR education groups showed a statistically higher satisfaction than the traditional verbal education groups. Among the three subscales of the MBI, emotional exhaustion and personal accomplishment improved statistically significantly. VR was also better in terms of usability.

CONCLUSION

This study suggests VR enhances usability and reduces burnout in nurses, but further research is needed to assess its impact on depersonalization and objective measures like stress and heart rate.

Electronic health record system use and documentation burden of acute and critical care nurse clinicians: a mixed-methods study

OBJECTIVES

Examine electronic health record (EHR) use and factors contributing to documentation burden in acute and critical care nurses.

MATERIALS AND METHODS

A mixed-methods design was used guided by Unified Theory of Acceptance and Use of Technology. Key EHR components included, Flowsheets, Medication Administration Records (MAR), Care Plan, Notes, and Navigators. We first identified 5 units with the highest documentation burden in 1 university hospital through EHR log file analyses. Four nurses per unit were recruited and engaged in interviews and surveys designed to examine their perceptions of ease of use and usefulness of the 5 EHR components. A combination of inductive/deductive coding was used for qualitative data analysis.

RESULTS

Nurses acknowledged the importance of documentation for patient care, yet perceived the required documentation as burdensome with levels varying across the 5 components. Factors contributing to burden included non-EHR issues (patient-to-nurse staffing ratios; patient acuity; suboptimal time management) and EHR usability issues related to design/features. Flowsheets, Care Plan, and Navigators were found to be below acceptable usability and contributed to more burden compared to MAR and Notes. The most troublesome EHR usability issues were data redundancy, poor workflow navigation, and cumbersome data entry based on unit type.

DISCUSSION

Overall, we used quantitative and qualitative data to highlight challenges with current nursing documentation features in the EHR that contribute to documentation burden. Differences in perceived usability across the EHR documentation components were driven by multiple factors, such as non-alignment with workflows and amount of duplication of prior data entries. Nurses offered several recommendations for improving the EHR, including minimizing redundant or excessive data entry requirements, providing visual cues (eg, clear error messages, highlighting areas where missing or incorrect information are), and integrating decision support.

CONCLUSION

Our study generated evidence for nurse EHR use and specific documentation usability issues contributing to burden. Findings can inform the development of solutions for enhancing multi-component EHR usability that accommodates the unique workflow of nurses. Documentation strategies designed to improve nurse working conditions should include non-EHR factors as they also contribute to documentation burden.

Deficiencies in Electronic Medical Record Inpatient List Capabilities Negatively Impact Patient Safety, Resident Education, and Wellness

OBJECTIVE

Electronic medical records (EMRs) have streamlined workflows for health care professionals, yet their full potential is not always actualized. Modern EMRs are often capable of generating automated prepopulated inpatient lists, however if these capabilities are not made available to inpatient teams or not designed with the end user in mind, resident physicians may be left to create alternative, manual solutions to ensure reliable and efficient care. The purpose of the current study was to longitudinally compare the impact of both manual and automated inpatient lists on resident education, wellness, and patient safety.

DESIGN

Retrospective standardized surveys were administered to resident physicians in the orthopedic surgery department at a level I trauma center over a 3-year period to evaluate the impact of various automated and manual list iterations coinciding with changes to the EMR. Data collected included post graduate year (PGY) status, arrival time to work, daily time spent preparing and updating the list, perceived impact on patient safety, resident workload, education, and sleep. We compared manual versus automated list data with unpaired t-tests and chi-squared tests.

SETTING

The study was conducted at Brooke Army Medical Center, a level 1 trauma center in San Antonio, Texas. It is an Academic Medical Center and the Department of Defense's largest medical facility.

PARTICIPANTS

A total of 71 surveys were collected from 33 orthopedic surgery residents in all levels of training.

RESULTS

Intern list prep time in the morning was 27 ± 16 minutes for the automated list (n = 17) vs 72 ± 21 minutes for the manual lists (n = 23) (p < 0.0001). Total time spent by interns updating the list for the entire day was on average 83 minutes for the automated list (n = 17) vs 196 minutes for the manual lists (n = 23) (p < 0.0001). In addition, 86% of interns felt the time spent on the manual list impacted their education, with 96% stating that it directly impacted the amount of time they had to study and 100% agreed that it negatively impacted their sleep (n = 23). Only 48% of interns (n = 23) were satisfied with the performance of the manual lists compared to 94% satisfaction (n = 17) with the automated list. Further, 87% of interns felt the manual list negatively impacted patient care and negatively affected their job satisfaction. In comparison, 59% of interns felt the automated list improved their job satisfaction. Ultimately, for an intern an automated list versus a manual list affords them an extra 106 minutes per day for education, sleep, or other activities. PGY2 residents and above noted similar trends regarding their experience with the lists.

CONCLUSIONS

The benefits of utilizing automated inpatient lists as determined by our study are improved efficiency in the morning with less preparation and maintenance throughout the day. Additionally, with automated lists there was more perceived time for education and sleep, with improved job satisfaction. Most importantly, respondents felt that automated lists were safer for patient care when compared to manual lists. This should compel further research and efforts into improving automated EMR tracking lists in hospitals. In summary, as compared to the automated electronic medical record inpatient list, manual lists resulted in substantially more preparation time and maintenance throughout the day thereby negatively impacting resident education and quality of life, while raising concerns for patient safety.

Medical and nursing clinician perspectives on the usability of the hospital electronic medical record: A qualitative analysis

BACKGROUND

Electronic medical records (EMRs) have been widely implemented in Australian hospitals. Their usability and design to support clinicians to effectively deliver and document care is essential, as is their impact on clinical workflow, safety and quality, communication, and collaboration across health systems. Perceptions of, and data about, usability of EMRs implemented in Australian hospitals are key to successful adoption.

OBJECTIVE

To explore perspectives of medical and nursing clinicians on EMR usability utilising free-text data collected in a survey.

METHOD

Qualitative analysis of one free-text optional question included in a web-based survey. Respondents included medical and nursing/midwifery professionals in Australian hospitals (85 doctors and 27 nurses), who commented on the usability of the main EMR used.

RESULTS

Themes identified related to the status of EMR implementation, system design, human factors, safety and risk, system response time, and stability, alerts, and supporting the collaboration between healthcare sectors. Positive factors included ability to view information from any location; ease of medication documentation; and capacity to access diagnostic test results. Usability concerns included lack of intuitiveness; complexity; difficulties communicating with primary and other care sectors; and time taken to perform clinical tasks.

CONCLUSION

If the benefits of EMRs are to be realised, there are good reasons to address the usability challenges identified by clinicians. Easy solutions that could improve the usability experience of hospital-based clinicians include resolving sign-on issues, use of templates, and more intelligent alerts and warnings to avoid errors.

IMPLICATIONS

These essential improvements to the usability of the EMR, which are the foundation of the digital health system, will enable hospital clinicians to deliver safer and more effective health care.

Predicting Primary Care Physician Burnout From Electronic Health Record Use Measures

OBJECTIVE

To evaluate the ability of routinely collected electronic health record (EHR) use measures to predict clinical work units at increased risk of burnout and potentially most in need of targeted interventions.

METHODS

In this observational study of primary care physicians, we compiled clinical workload and EHR efficiency measures, then linked these measures to 2 years of well-being surveys (using the Stanford Professional Fulfillment Index) conducted from April 1, 2019, through October 16, 2020. Physicians were grouped into training and confirmation data sets to develop predictive models for burnout. We used gradient boosting classifier and other prediction modeling algorithms to quantify the predictive performance by the area under the receiver operating characteristics curve (AUC).

RESULTS

Of 278 invited physicians from across 60 clinics, 233 (84%) completed 396 surveys. Physicians were 67% women with a median age category of 45 to 49 years. Aggregate burnout score was in the high range (≥3.325/10) on 111 of 396 (28%) surveys. Gradient boosting classifier of EHR use measures to predict burnout achieved an AUC of 0.59 (95% CI, 0.48 to 0.77) and an area under the precision-recall curve of 0.29 (95% CI, 0.20 to 0.66). Other models' confirmation set AUCs ranged from 0.56 (random forest) to 0.66 (penalized linear regression followed by dichotomization). Among the most predictive features were physician age, team member contributions to notes, and orders placed with user-defined preferences. Clinic-level aggregate measures identified the top quartile of clinics with 56% sensitivity and 85% specificity.

CONCLUSION

In a sample of primary care physicians, routinely collected EHR use measures demonstrated limited ability to predict individual burnout and moderate ability to identify high-risk clinics.

Expanding the Surgical Armamentarium Through Meaningful Use of the Electronic Health Record

This Viewpoint proposes several ways to innovate electronic health record (EHR) systems to address surgeon-specific needs and to improve clinicians’ experience and health care quality.

Re-Envisioning Electronic Health Records to Optimize Patient-Centered Cancer Care, Quality, Surveillance, and Research

Electronic health records (EHRs) are a significant advancement over paper records. However, the full potential of EHRs for improving care quality, patient outcomes, surveillance, and research in cancer care is yet to be realized. The organic evolution of EHRs has resulted in a number of unanticipated consequences including increased time spent by clinicians interfacing with the EHR for daily workflows. Patient access to clinicians and their records has been an important advancement in patient-centered care; however, this has brought to light additional gaps and challenges in EHRs meeting these needs. A significant challenge for EHR design and physician workflows is how best to meet the complex goals and priorities of various stakeholders including providers, researchers, patients, health systems, payors, and regulatory agencies. The National Cancer Policy Forum convened a 2022 workshop, "Innovations in Electronic Health Records for Oncology Care, Research and Surveillance," to address these challenges and to facilitate collaboration across all user groups with the goal of re-envisioning EHRs that will better support shared goals of improving patient outcomes and advancing cancer care and research without overburdening clinicians with administrative tasks. Here, we summarize the current EHR ecosystem as discussed at the workshop and highlight opportunities to improve EHR contributions to oncology evidence and care.

Electronic Health Record Usability, Satisfaction, and Burnout for Family Physicians

Importance

Electronic health record (EHR) work has been associated with decreased physician well-being. Understanding the association between EHR usability and physician satisfaction and burnout, and whether team and technology strategies moderate this association, is critical to informing efforts to address EHR-associated physician burnout.

Objectives

To measure family physician satisfaction with their EHR and EHR usability across functions and evaluate the association of EHR usability with satisfaction and burnout, as well as the moderating association of 4 team and technology EHR efficiency strategies.

Design, Setting, and Participants

This study uses data from a cross-sectional survey conducted from December 12, 2021, to October 17, 2022, of all family physicians seeking American Board of Family Medicine recertification in 2022.

Exposure

Physicians perceived EHR usability across 6 domains, as well as adoption of 4 EHR efficiency strategies: scribes, support from other staff, templated text, and voice recognition or transcription.

Main Outcomes and Measures

Physician EHR satisfaction and frequency of experiencing burnout measured with a single survey item ("I feel burned out from my work"), with answers ranging from "never" to "every day."

Results

Of the 2067 physicians (1246 [60.3%] younger than 50 years; 1051 men [50.9%]; and 1729 [86.0%] practicing in an urban area) who responded to the survey, 562 (27.2%) were very satisfied and 775 (37.5%) were somewhat satisfied, while 346 (16.7%) were somewhat dissatisfied and 198 (9.6%) were very dissatisfied with their EHR. Readability of information had the highest usability, with 543 physicians (26.3%) rating it as excellent, while usefulness of alerts had the lowest usability, with 262 physicians (12.7%) rating it as excellent. In multivariable models, good or excellent usability for entering data (β = 0.09 [95% CI, 0.05-0.14]; P < .001), alignment with workflow processes (β = 0.11 [95% CI, 0.06-0.16]; P < .001), ease of finding information (β = 0.14 [95% CI, 0.09-0.19]; P < .001), and usefulness of alerts (β = 0.11 [95% CI, 0.06-0.16]; P < .001) were associated with physicians being very satisfied with their EHR. In addition, being very satisfied with the EHR was associated with reduced frequency of burnout (β = -0.64 [95% CI, -1.06 to -0.22]; P < .001). In moderation analysis, only physicians with highly usable EHRs saw improvements in satisfaction from adopting efficiency strategies.

Conclusions and Relevance

In this survey study of physician EHR usability and satisfaction, approximately one-fourth of family physicians reported being very satisfied with their EHR, while another one-fourth reported being somewhat or very dissatisfied, a concerning finding amplified by the inverse association between EHR satisfaction and burnout. Electronic health record-based alerts had the lowest reported usability, suggesting EHR vendors should focus their efforts on improving alerts. Electronic health record efficiency strategies were broadly adopted, but only physicians with highly usable EHRs realized gains in EHR satisfaction from using these strategies, suggesting that EHR burden-reduction interventions are likely to have heterogenous associations across physicians with different EHRs.

Impact of response bias in three surveys on primary care providers' experiences with electronic health records

OBJECTIVE

To identify impacts of different survey methodologies assessing primary care physicians' (PCPs') experiences with electronic health records (EHRs), we compared three surveys: the 2022 Continuous Certification Questionnaire (CCQ) from the American Board of Family Medicine, the 2022 University of California San Francisco (UCSF) Physician Health IT Survey, and the 2021 National Electronic Health Records Survey (NEHRS).

MATERIALS AND METHODS

We evaluated differences between survey pairs using Rao-Scott corrected chi-square tests, which account for weighting.

RESULTS

CCQ received 3991 responses from PCPs (100% response rate), UCSF received 1375 (3.6% response rate), and NEHRS received 858 (18.2% response rate). Substantial, statistically significant differences in demographics were detected across the surveys. CCQ respondents were younger and more likely to work in a health system; NEHRS respondents were more likely to work in private practice; and UCSF respondents disproportionately practiced in larger academic settings. Many EHR experience indicators were similar between CCQ and NEHRS, but CCQ respondents reported higher documentation burden.

DISCUSSION

The UCSF approach is unlikely to supply reliable data. Significant demographic differences between CCQ and NEHRS raise response bias concerns, and while there were similarities in some reported EHR experiences, there were important, significant differences.

CONCLUSION

Federal EHR policy monitoring and maintenance require reliable data. This test of existing and alternative sources suggest that diversified data sources are necessary to understand physicians' experiences with EHRs and interoperability. Comprehensive surveys administered by specialty boards have the potential to contribute to these efforts, since they are likely to be free of response bias.

Measuring Documentation Burden in Healthcare

BACKGROUND

The enactment of the Health Information Technology for Economic and Clinical Health Act and the wide adoption of electronic health record (EHR) systems have ushered in increasing documentation burden, frequently cited as a key factor affecting the work experience of healthcare professionals and a contributor to burnout. This systematic review aims to identify and characterize measures of documentation burden.

METHODS

We integrated discussions with Key Informants and a comprehensive search of the literature, including MEDLINE, Embase, Scopus, and gray literature published between 2010 and 2023. Data were narratively and thematically synthesized.

RESULTS

We identified 135 articles about measuring documentation burden. We classified measures into 11 categories: overall time spent in EHR, activities related to clinical documentation, inbox management, time spent in clinical review, time spent in orders, work outside work/after hours, administrative tasks (billing and insurance related), fragmentation of workflow, measures of efficiency, EHR activity rate, and usability. The most common source of data for most measures was EHR usage logs. Direct tracking such as through time-motion analysis was fairly uncommon. Measures were developed and applied across various settings and populations, with physicians and nurses in the USA being the most frequently represented healthcare professionals. Evidence of validity of these measures was limited and incomplete. Data on the appropriateness of measures in terms of scalability, feasibility, or equity across various contexts were limited. The physician perspective was the most robustly captured and prominently focused on increased stress and burnout.

DISCUSSION

Numerous measures for documentation burden are available and have been tested in a variety of settings and contexts. However, most are one-dimensional, do not capture various domains of this construct, and lack robust validity evidence. This report serves as a call to action highlighting an urgent need for measure development that represents diverse clinical contexts and support future interventions.

Pharmacogenetics Clinical Decision Support Systems for Primary Care in England: Co-Design Study

BACKGROUND

Pharmacogenetics can impact patient care and outcomes through personalizing the selection of medicines, resulting in improved efficacy and a reduction in harmful side effects. Despite the existence of compelling clinical evidence and international guidelines highlighting the benefits of pharmacogenetics in clinical practice, implementation within the National Health Service in the United Kingdom is limited. An important barrier to overcome is the development of IT solutions that support the integration of pharmacogenetic data into health care systems. This necessitates a better understanding of the role of electronic health records (EHRs) and the design of clinical decision support systems that are acceptable to clinicians, particularly those in primary care.

OBJECTIVE

Explore the needs and requirements of a pharmacogenetic service from the perspective of primary care clinicians with a view to co-design a prototype solution.

METHODS

We used ethnographic and think-aloud observations, user research workshops, and prototyping. The participants for this study included general practitioners and pharmacists. In total, we undertook 5 sessions of ethnographic observation to understand current practices and workflows. This was followed by 3 user research workshops, each with its own topic guide starting with personas and early ideation, through to exploring the potential of clinical decision support systems and prototype design. We subsequently analyzed workshop data using affinity diagramming and refined the key requirements for the solution collaboratively as a multidisciplinary project team.

RESULTS

User research results identified that pharmacogenetic data must be incorporated within existing EHRs rather than through a stand-alone portal. The information presented through clinical decision support systems must be clear, accessible, and user-friendly as the service will be used by a range of end users. Critically, the information should be displayed within the prescribing workflow, rather than discrete results stored statically in the EHR. Finally, the prescribing recommendations should be authoritative to provide confidence in the validity of the results. Based on these findings we co-designed an interactive prototype, demonstrating pharmacogenetic clinical decision support integrated within the prescribing workflow of an EHR.

CONCLUSIONS

This study marks a significant step forward in the design of systems that support pharmacogenetic-guided prescribing in primary care settings. Clinical decision support systems have the potential to enhance the personalization of medicines, provided they are effectively implemented within EHRs and present pharmacogenetic data in a user-friendly, actionable, and standardized format. Achieving this requires the development of a decoupled, standards-based architecture that allows for the separation of data from application, facilitating integration across various EHRs through the use of application programming interfaces (APIs). More globally, this study demonstrates the role of health informatics and user-centered design in realizing the potential of personalized medicine at scale and ensuring that the benefits of genomic innovation reach patients and populations effectively.

Accounting for taste: preferences mediate the relationship between documentation time and ambulatory physician burnout

OBJECTIVES

Physician burnout in the US has reached crisis levels, with one source identified as extensive after-hours documentation work in the electronic health record (EHR). Evidence has illustrated that physician preferences for after-hours work vary, such that after-hours work may not be universally burdensome. Our objectives were to analyze variation in preferences for after-hours documentation and assess if preferences mediate the relationship between after-hours documentation time and burnout.

MATERIALS AND METHODS

We combined EHR active use data capturing physicians' hourly documentation work with survey data capturing documentation preferences and burnout. Our sample included 318 ambulatory physicians at MedStar Health. We conducted a mediation analysis to estimate if and how preferences mediated the relationship between after-hours documentation time and burnout. Our primary outcome was physician-reported burnout. We measured preferences for after-hours documentation work via a novel survey instrument (Burden Scenarios Assessment). We measured after-hours documentation time in the EHR as the total active time respondents spent documenting between 7 pm and 3 am.

RESULTS

Physician preferences varied, with completing clinical documentation after clinic hours while at home the scenario rated most burdensome (52.8% of physicians), followed by dealing with prior authorization (49.5% of physicians). In mediation analyses, preferences partially mediated the relationship between after-hours documentation time and burnout.

DISCUSSION

Physician preferences regarding EHR-based work play an important role in the relationship between after-hours documentation time and burnout.

CONCLUSION

Studies of EHR work and burnout should incorporate preferences, and operational leaders should assess preferences to better target interventions aimed at EHR-based contributors to burnout.

Evaluating the Prevalence of Burnout Among Health Care Professionals Related to Electronic Health Record Use: Systematic Review and Meta-Analysis

BACKGROUND

Burnout among health care professionals is a significant concern, with detrimental effects on health care service quality and patient outcomes. The use of the electronic health record (EHR) system has been identified as a significant contributor to burnout among health care professionals.

OBJECTIVE

This systematic review and meta-analysis aims to assess the prevalence of burnout among health care professionals associated with the use of the EHR system, thereby providing evidence to improve health information systems and develop strategies to measure and mitigate burnout.

METHODS

We conducted a comprehensive search of the PubMed, Embase, and Web of Science databases for English-language peer-reviewed articles published between January 1, 2009, and December 31, 2022. Two independent reviewers applied inclusion and exclusion criteria, and study quality was assessed using the Joanna Briggs Institute checklist and the Newcastle-Ottawa Scale. Meta-analyses were performed using R (version 4.1.3; R Foundation for Statistical Computing), with EndNote X7 (Clarivate) for reference management.

RESULTS

The review included 32 cross-sectional studies and 5 case-control studies with a total of 66,556 participants, mainly physicians and registered nurses. The pooled prevalence of burnout among health care professionals in cross-sectional studies was 40.4% (95% CI 37.5%-43.2%). Case-control studies indicated a higher likelihood of burnout among health care professionals who spent more time on EHR-related tasks outside work (odds ratio 2.43, 95% CI 2.31-2.57).

CONCLUSIONS

The findings highlight the association between the increased use of the EHR system and burnout among health care professionals. Potential solutions include optimizing EHR systems, implementing automated dictation or note-taking, employing scribes to reduce documentation burden, and leveraging artificial intelligence to enhance EHR system efficiency and reduce the risk of burnout.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42021281173; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021281173.

Use of the FallAkte Plus System as an IT Infrastructure for the North Rhine-Westphalian General Practice Research Network: Mixed Methods Usability Study

BACKGROUND

Primary care research networks can generate important information in the setting where most patients are seen and treated. However, this requires a suitable IT infrastructure (ITI), which the North Rhine-Westphalian general practice research network is looking to implement.

OBJECTIVE

This mixed methods research study aims to evaluate (study 1) requirements for an ITI and (study 2) the usability of an IT solution already available on the market, the FallAkte Plus (FA+) system for the North Rhine-Westphalian general practice research network, which comprises 8 primary care university institutes in Germany's largest state.

METHODS

In study 1, a survey was conducted among researchers from the institutes to identify the requirements for a suitable ITI. The questionnaire consisted of standardized questions with open-ended responses. In study 2, a mixed method approach combining a think-aloud approach and a quantitative survey was used to evaluate the usability and acceptance of the FA+ system among 3 user groups: researchers, general practitioners, and practice assistants. Respondents were asked to assess the usability with the validated system usability scale and to test a short questionnaire on vaccination management through FA+.

RESULTS

In study 1, five of 8 institutes participated in the requirements survey. A total of 32 user requirements related primarily to study management were identified, including data entry, data storage, and user access management. In study 2, a total of 36 participants (24 researchers and 12 general practitioners or practice assistants) were surveyed in the mixed methods study of an already existing IT solution. The tutorial video and handouts explaining how to use the FA+ system were well received. Researchers, unlike practice personnel, were concerned about data security and data protection regarding the system's emergency feature, which enables access to all patient data. The median overall system usability scale rating was 60 (IQR 33.0-85.0), whereby practice personnel (median 82, IQR 58.0-94.0) assigned higher ratings than researchers (median 44, IQR 14.0-61.5). Users appreciated the option to integrate data from practices and other health care facilities. However, they voted against the use of the FA+ system due to a lack of support for various study formats.

CONCLUSIONS

Usability assessments vary markedly by professional group and role. In its current stage of development, the FA+ system does not fully meet the requirements for a suitable ITI. Improvements in the user interface, performance, interoperability, security, and advanced features are necessary to make it more effective and user-friendly. Collaborating with end users and incorporating their feedback are crucial for the successful development of any practice network research ITI.

NAM National Plan for Health Workforce Well-being: Applications for Radiology

The National Academy of Medicine Plan for Health Workforce Well-Being identifies seven priority areas, including creating positive work environments, addressing burnout and stress, promoting transparency and equity in compensation, providing education and training to promote resilience, enhancing community and social support systems, addressing the stigma associated with seeking help for mental health and substance use disorders and fostering leadership commitment and accountability for workforce well-being. This paper will explore the National Plan for Health Workforce Well-Being, providing an overview of the seven priority areas and offering strategies for implementation in radiology.

Application of a user experience design approach for an EHR-based clinical decision support system

Objective

We applied a user experience (UX) design approach to clinical decision support (CDS) tool development for the specific use case of pediatric asthma. Our objective was to understand physicians' workflows, decision-making processes, barriers (ie, pain points), and facilitators to increase usability of the tool.

Materials and methods

We used a mixed-methods approach with semi-structured interviews and surveys. The coded interviews were synthesized into physician-user journey maps (ie, visualization of a process to accomplish goals) and personas (ie, user types). Interviews were conducted via video. We developed physician journey maps and user personas informed by their goals, systems interactions, and experiences with pediatric asthma management.

Results

The physician end-user personas identified were: efficiency, relationship, and learning. Features of a potential asthma CDS tool sought varied by physician practice type and persona. It was important to the physician end-user that the asthma CDS tool demonstrate value by lowering workflow friction (ie, difficulty or obstacles), improving the environment surrounding physicians and patients, and using it as a teaching tool. Customizability versus standardization were important considerations for uptake.

Discussion

Different values and motivations of physicians influence their use and interaction with the EHR and CDS tools. These different perspectives can be captured by applying a UX design approach to the development process. For example, with the importance of customizability, one approach may be to build a core module with variations depending on end-user preference.

Conclusion

A UX approach can drive design to help understand physician-users and meet their needs; ultimately with the goal of increased uptake.

Comparing "Meaningful Use" of Health Information Technology in Pennsylvania: Electronic Prescription Rates of Metropolitan and Rural Counties

The Health Information Technology for Economic and Clinical Health Act incentivized the adoption of electronic health records (EHRs). Health systems looked to leverage technology to assist in serving populations in health professional shortage areas. Qualitative research points to EHR usability as a source of health inequities in rural settings, making the challenges of EHR usage a subject of interest. Pennsylvania offers a model for investigating rural health infrastructure with it having the third largest rural population in the United States. This study analyzed the adoption of Electronic Prescribing in the 67 Pennsylvania (PA) counties. Physician adoption and usage data for PA and the United States were compared using a t-test to establish a basis for comparison. PA counties were categorized using the United States Department of Agriculture (USDA)'s Rural-Urban Commuting Areas (RUCAs) system. Surescript use percentages were plotted against the RUCA scores of each PA county to create a polynomial regression model. PA office-based physicians, on average, utilize e-prescription tools at the same rate as the national average with 59% of practices utilizing Surescripts as of 2013. There was no significant correlation between Surescript usage and the rural/urban classification of counties in Pennsylvania (R-squared value of 0.06). Pennsylvania was able to adopt health information technology (HIT) infrastructure at the same rate as the national average. Rural and metropolitan definitions do not correlate to meaningful use of HIT, thus usability of HIT cannot be tied to health outcomes. Future studies looking at specific forms of HIT and their ability to decrease the burden of administrative work for clinicians.

Improving healthcare quality by unifying the American electronic medical report system: time for change

BACKGROUND

In recent years, innovation in healthcare technology has significantly improved the efficiency of the healthcare system. Advancements have led to better patient care and more cost-effective services. The electronic medical record (EMR) system, in particular, has enhanced interoperability and collaboration across healthcare departments by facilitating the exchange and utilization of patient data. The COVID-19 pandemic further accelerated this trend, leading to a surge in telemedicine services, which rely on electronic communication to deliver healthcare remotely.

MAIN BODY

Integrating artificial intelligence (AI) and machine learning (ML) in healthcare have been instrumental in analyzing vast data sets, allowing for identifying patterns and trends that can improve care delivery and pinpoint potential issues. The proposal of a unified EMR system in the USA aims to capitalize on these technological advancements. Such a system would streamline the sharing of patient information among healthcare providers, improve the quality and efficiency of care, and minimize the likelihood of errors in patient treatment.

CONCLUSION

A unified electronic medical record system represents a promising avenue for enhancing interoperability within the US healthcare sector. By creating a more connected and accessible network of patient information, it sets the stage for a transformation in healthcare delivery. This change is imperative for maintaining the momentum of progress in healthcare technology and realizing the full potential of recent advancements in patient care and system efficiency.

Sprint-inspired One-on-One Post-Go-Live Training Session (Mini-Sprint) Improves Provider Electronic Health Record Efficiency and Satisfaction

BACKGROUND

 Inefficient electronic health record (EHR) usage increases the documentation burden on physicians and other providers, which increases cognitive load and contributes to provider burnout. Studies show that EHR efficiency sessions, optimization sprints, reduce burnout using a resource-intense five-person team. We implemented sprint-inspired one-on-one post-go-live efficiency training sessions (mini-sprints) as a more economical training option directed at providers.

OBJECTIVES

 We evaluated a post-go-live mini-sprint intervention to assess provider satisfaction and efficiency.

METHODS

 NorthShore University HealthSystem implemented one-on-one provider-to-provider mini-sprint sessions to optimize provider workflow within the EHR platform. The physician informaticist completed a 9-point checklist of efficiency tips with physician trainees covering schedule organization, chart review, speed buttons, billing, note personalization/optimization, preference lists, quick actions, and quick tips. We collected postsession survey data assessing for net promoter score (NPS) and open-ended feedback. We conducted financial analysis of pre- and post-mini-sprint efficiency levels and financial data.

RESULTS

 Seventy-six sessions were conducted with 32 primary care physicians, 28 specialty physicians, and 16 nonphysician providers within primary care and other areas. Thirty-seven physicians completed the postsession survey. The average NPS for the completed mini-sprint sessions was 97. The proficiency score had a median of 6.12 (Interquartile range (IQR): 4.71-7.64) before training, and a median of 7.10 (IQR: 6.25-8.49) after training. Financial data analysis indicates that higher level billing codes were used at a greater frequency post-mini-sprint. The revenue increase 12 months post-mini-sprint was $213,234, leading to a return of $75,559.50 for 40 providers, or $1,888.98 per provider in a 12-month period.

CONCLUSION

 Our data show that mini-sprint sessions were effective in optimizing efficiency within the EHR platform. Financial analysis demonstrates that this type of training program is sustainable and pays for itself. There was high satisfaction with the mini-sprint training modality, and feedback indicated an interest in further mini-sprint training sessions for physicians and nonphysician staff.

Prospective associations of technostress at work, burnout symptoms, hair cortisol, and chronic low-grade inflammation

BACKGROUND

Working conditions in the age of digitalization harbor risks for chronic stress and burnout. However, real-world investigations into biological effects of technostress, that is stress in the context of digital technology use, are sparse. This study prospectively assessed associations between technostress, general work stress, burnout symptoms, hair cortisol, and chronic low-grade inflammation.

METHODS

Hospital employees (N = 238, 182 females, Mage = 28.5 years) participated in a prospective cohort study with two follow-ups six months apart (T2, T3). Participants answered standardized questionnaires on general job strain (job demand-control ratio), technostressors (work interruptions, multitasking, information overload), burnout symptoms (exhaustion, mental distance), and relevant confounders. Moreover, they provided capillary blood samples for C-reactive protein (CRP) and hair strands for hair cortisol concentration (HCC) analysis. Structural equation modelling was performed.

RESULTS

The factorial structure of survey measures was confirmed. Burnout symptoms (MT2 = 2.17, MT3 = 2.33) and HCC (MT2 = 4.79, MT3 = 9.56; pg/mg) increased over time, CRP did not (MT2 = 1.15, MT3 = 1.21; mg/L). Adjusted path models showed that technostress was negatively associated with HCC (β = -0.16, p =.003), but not with burnout and CRP. General work stress in contrast, was not significantly associated with burnout, HCC or CRP. Furthermore, there were reciprocal effects of CRP on HCC (β = 0.28, p =.001) and of HCC on CRP (β = -0.10, p ≤.001). Associations were robust in additional analyses including further confounders.

CONCLUSION

This is the first study on prospective effects of technostress on employees' endocrine and inflammatory systems. Results suggest differential effects of technostress on the hypothalamic-pituitary-adrenocortical axis activity. Given its key role for long-term health, the findings have important implications for occupational health and safety in digitalized work environments.

We Shape Our Tools and Then Our Tools Shape Us: OPAT and the EHR

A commentary on Canterino et al. (2024) and Munsiff et al. (2024), articles where clinicians from two large OPAT programs.

Collecting patient-reported outcome measures in the electronic health record: Lessons from the NIH pragmatic trials Collaboratory

The NIH Pragmatic Trials Collaboratory supports the design and conduct of 27 embedded pragmatic clinical trials, and many of the studies collect patient reported outcome measures as primary or secondary outcomes. Study teams have encountered challenges in the collection of these measures, including challenges related to competing health care system priorities, clinician's buy-in for adoption of patient-reported outcome measures, low adoption and reach of technology in low resource settings, and lack of consensus and standardization of patient-reported outcome measure selection and administration in the electronic health record. In this article, we share case examples and lessons learned, and suggest that, when using patient-reported outcome measures for embedded pragmatic clinical trials, investigators must make important decisions about whether to use data collected from the participating health system's electronic health record, integrate externally collected patient-reported outcome data into the electronic health record, or collect these data in separate systems for their studies.

Consultants' and referrers' perceived barriers to closing the cross-institutional referral loop, and perceived impact on clinical care

BACKGROUND

Cross-institutional (external) referrals are prone to communication breakdowns, increasing patient safety risks, clinician burnout, and healthcare costs. To close these external referral loops, referring primary care physicians (PCPs) need to receive patient information from consultants at different healthcare institutions. Although existing studies investigated the early phases of external referral loops, we lack sufficient knowledge about the closing phases of these loops. This knowledge could allow health care institutions to improve care coordination and rates of closed referral loops by implementing socio-technical interventions for patient information exchange throughout a referral loop. Human factors engineering (HFE) provides a systematic approach to advance our understanding of barriers perceived by physicians. Using HFE, our objective was to characterize referring and consulting physicians' barriers to closing referral loops and implications for care.

METHODS

This qualitative cross-sectional study included semi-structured interviews with referrers and external consultants. We used the Systems Engineering Initiative for Patient Safety 2.0 framework to conduct rapid qualitative analyses, determining perceived barriers and related implications. Main measures were consultants' and referrers' perceptions of, and experiences with, barriers to external referrals.

RESULTS

Six referring PCPs and 12 consultants participated from two healthcare systems and four medical specialties. Physicians perceived three main barriers in external referrals: receipt of excessive and unnecessary faxed documents, missing or delayed documentation, and organizational policies regarding information privacy interfering with closing the loop. Compared to internal referrals, physicians reported increased staff burden, patient frustration, and delays in diagnosis with external referrals. Consultants reported the ability to provide the same level of care to patients with internal or external referrals. However, consultants described communication breakdowns that prohibited confirmation of follow-up plan retrieval, initiation, or effectiveness.

CONCLUSION

Physicians reported technological and organizational barriers to closing cross-institutional referral loops. Promises of HIE technology for external referrals have not fully materialized. Among physicians and patients, retrieval and exchange of medical information increases perceived workload, burden, and frustration. These increases are not accurately captured by traditional organizational metrics. This study provides evidence that informs future human factors engineering research to address perceived barriers and guide future HIE design or implementation.

Associations of perceived changes in work due to digitalization and the amount of digital work with job strain among physicians: a national representative sample

BACKGROUND

Physicians' work is often stressful. The digitalization of healthcare aims to streamline work, but not all physicians have experienced its realization. We examined associations of perceived changes in work due to digitalization and the amount of digital work with job strain among physicians. The moderating role of the length of work experience was investigated for these associations.

METHODS

We used representative survey data on Finnish physicians' (N = 4271) experiences of digitalization from 2021. The independent variables included perceptions on statements about work transformations aligned with digitalization goals, and the extent that information systems and teleconsultations were utilized. Stress related to information systems (SRIS), time pressure, and psychological stress were the dependent variables. We analyzed the associations using multivariable linear and logistic regressions.

RESULTS

Respondents had a mean SRIS score of 3.5 and a mean time pressure score of 3.7 on a scale of 1-5. Psychological stress was experienced by 60%. Perceptions associated with higher SRIS comprised disagreements with statements asserting that digitalization accelerates clinical encounters (b = .23 [95% CI: .16-.30]), facilitates access to patient information (b = .15 [.07-.23]), and supports decision-making (b = .11 [.05-.18]). Disagreement with accelerated clinical encounters (b = .12 [.04-.20]), and agreements with patients' more active role in care (b = .11 [.04-.19]) and interprofessional collaboration (b = .10 [.02-.18]) were opinions associated with greater time pressure. Disagreeing with supported decision-making (OR = 1.26 [1.06-1.48]) and agreeing with patients' active role (OR = 1.19 [1.02-1.40]) were associated with greater psychological stress. However, perceiving improvements in the pace of clinical encounters and access to patient information appeared to alleviate job strain. Additionally, extensive digital work was consistently linked to higher strain. Those respondents who held teleconsultations frequently and had less than 6 years of work experience reported the greatest levels of time pressure.

CONCLUSIONS

Physicians seem to be strained by frequent teleconsultations and work that does not meet the goals of digitalization. Improving physicians' satisfaction with digitalization through training specific to the stage of career and system development can be crucial for their well-being. Schedules for digital tasks should be planned and allocated to prevent strain related to achieving the digitalization goals.

National survey of user-reported usability of electronic medical record software in ophthalmology in Germany

PURPOSE

A nationwide, comparative survey of the physician-reported usability of electronic medical record (EMR) software used by ophthalmologists in Germany using the System Usability Scale (SUS) as a standardized metric.

METHODS

A cross-sectional survey of members of the German Ophthalmological Society (DOG) and professional association of ophthalmologists (BVA) was conducted in May 2022. All 7788 physician members of both societies were invited to participate in an anonymous online-survey by individualized links. User-reported usability of the participants main software used for electronic medical recordkeeping was assessed using the SUS (range 0-100).

RESULTS

A total of 881 participants with 51 different EMRs completed the entire questionnaire. Mean EMR-SUS score was 65.7 (SD ± 23.5). Significant differences in mean SUS of several EMR programs were observed with a range of 31.5 to 87.2 in programs with 10 or more responses. 31.8% of all main program SUS ratings were below 50 points. Female gender was associated with 4.02 higher SUS score (95% CI 0.46-7.59). Main program SUS was positively correlated with overall work-related satisfaction and work environment SUS but negatively correlated with the number of programs in the work environment. The SUS of the entire digital work environment including all programs used daily was closely correlated with the main EMR SUS, but not the number of programs used.

CONCLUSION

Our survey revealed a fragmented pattern of EMR use by ophthalmologists in Germany with many competing software products and widely diverging mean System Usability Scale scores. A considerable share of ophthalmologists report EMR usability below what is commonly considered acceptable.

Electronic health records perception among three healthcare providers specialties in Saudi Arabia: A cross-sectional study

Worldwide, more health care facilities are adapting the use of electronic health record (EHR). Healthcare providers (HCP) have different perceptions toward the use of EHR. To investigate the perception of three classes of HCP in Saudi Arabia toward using EHR, a questionnaire (targeting satisfaction, easiness, and benefits of use as major perception indicators) was prepared. The questionnaire was assessed by an expert panel for content validity. The questionnaire internal consistency was examined using Cronbach's alpha. 108 physicians, physical therapists (PT) and respiratory care therapists (RT) from different hospitals in Saudi Arabia answered the questionnaire. Most of respondents perceived EHR systems as beneficial and made work easier. Most HCP were satisfied with the use of EHR, however, with the use of EHR more time was needed to finish the work. Age, experience, job, and job rank of HCP are of different importance in determining responses, perception, and obstacles of using EHR. Moreover, the perception of using EHR seems to be field specific. There is a positive perception among Saudi Arabia HCP about EHR use.

Variation in Family Physicians' Experiences Across Different Electronic Health Record Platforms: a Descriptive Study

BACKGROUND

Electronic health records (EHRs) have been connected to excessive workload and physician burnout. Little is known about variation in physician experience with different EHRs, however.

OBJECTIVE

To analyze variation in reported usability and satisfaction across EHRs.

DESIGN

Internet-based survey available between December 2021 and October 2022 integrated into American Board of Family Medicine (ABFM) certification process.

PARTICIPANTS

ABFM-certified family physicians who use an EHR with at least 50 total responding physicians.

MEASUREMENTS

Self-reported experience of EHR usability and satisfaction.

KEY RESULTS

We analyzed the responses of 3358 physicians who used one of nine EHRs. Epic, athenahealth, and Practice Fusion were rated significantly higher across six measures of usability. Overall, between 10 and 30% reported being very satisfied with their EHR, and another 32 to 40% report being somewhat satisfied. Physicians who use athenahealth or Epic were most likely to be very satisfied, while physicians using Allscripts, Cerner, or Greenway were the least likely to be very satisfied. EHR-specific factors were the greatest overall influence on variation in satisfaction: they explained 48% of variation in the probability of being very satisfied with Epic, 46% with eClinical Works, 14% with athenahealth, and 49% with Cerner.

CONCLUSIONS

Meaningful differences exist in physician-reported usability and overall satisfaction with EHRs, largely explained by EHR-specific factors. User-centric design and implementation, and robust ongoing evaluation are needed to reduce physician burden and ensure excellent experience with EHRs.

Design, development, implementation, and evaluation of a severe drug-drug interaction alert system in the ICU: An analysis of acceptance and override rates

BACKGROUND AND OBJECTIVE

The override rate of Drug-Drug Interaction Alerts (DDIA) in Intensive Care Units (ICUs) is very high. Therefore, this study aimed to design, develop, implement, and evaluate a severe Drug-Drug Alert System (DDIAS) in a system of ICUs and measure the override rate of this system.

METHODS

This is a cross-sectional study that details the design, development, implementation, and evaluation of a DDIAS for severe interactions into a Computerized Provider Order Entry (CPOE) system in the ICUs of Nemazee general teaching hospitals in 2021. The patients exposed to the volume of DDIAS, acceptance and overridden of DDIAS, and usability of DDIAS have been collected. The study was approved by the local Institutional Review Board (IRB) and; the ethics committee of Shiraz University of Medical Science on date: 2019-11-23 (Approval ID: IR.SUMS.REC.1398.1046).

RESULTS

The knowledge base of the DDIAS contains 9,809 severe potential drug-drug interactions (pDDIs). A total of 2672 medications were prescribed in the population study. The volume and acceptance rate for the DDIAS were 81 % and 97.5 %, respectively. The override rate was 2.5 %. The mean System Usability Scale (SUS) score of the DDIAS was 75.

CONCLUSION

This study demonstrates that implementing high-risk DDIAS at the point of prescribing in ICUs improves adherence to alerts. In addition, the usability of the DDIAS was reasonable. Further studies are needed to investigate the establishment of severe DDIAS and measure the prescribers' response to DDIAS on a larger scale.

Data as symptom: Doctors' responses to patient-provided data in general practice

People are increasingly able to generate their own health data through new technologies such as wearables and online symptom checkers. However, generating data is one thing, interpreting them another. General practitioners (GPs) are likely to be the first to help with interpretations. Policymakers in the European Union are investing heavily in infrastructures to provide GPs access to patient measurements. But there may be a disconnect between policy ambitions and the everyday practices of GPs. To investigate this, we conducted semi-structured interviews with 23 Danish GPs. According to the GPs, patients relatively rarely bring data to them. GPs mostly remember three types of patient-generated data that patients bring to them for interpretation: heart and sleep measurements from wearables and results from online symptom checkers. However, they also spoke extensively about data work with patient queries concerning measurements from the GPs' own online Patient Reported Outcome system and online access to laboratory results. We juxtapose GP reflections on these five data types and between policy ambitions and everyday practices. These data require substantial recontextualization work before the GPs ascribe them evidential value and act on them. Even when they perceived as actionable, patient-provided data are not approached as measurements, as suggested by policy frameworks. Rather, GPs treat them as analogous to symptoms-that is to say, GPs treat patient-provided data as subjective evidence rather than authoritative measures. Drawing on Science and Technology Studies (STS) literature,we suggest that GPs must be part of the conversation with policy makers and digital entrepreneurs around when and how to integrate patient-generated data into healthcare infrastructures.

A Multiyear Survey Evaluating Clinician Electronic Health Record Satisfaction

OBJECTIVES

We assessed how clinician satisfaction with a vendor electronic health record (EHR) changed over time in the 4 years following the transition from a homegrown EHR system to identify areas for improvement.

METHODS

We conducted a multiyear survey of clinicians across a large health care system after transitioning to a vendor EHR. Eligible clinicians from the first institution to transition received a survey invitation by email in fall 2016 and then eligible clinicians systemwide received surveys in spring 2018 and spring 2019. The survey included items assessing ease/difficulty of completing tasks and items assessing perceptions of the EHR's value, usability, and impact. One item assessing overall satisfaction and one open-ended question were included. Frequencies and means were calculated, and comparison of means was performed between 2018 and 2019 on all clinicians. A multivariable generalized linear model was performed to predict the outcome of overall satisfaction.

RESULTS

Response rates for the surveys ranged from 14 to 19%. The mean response from 3 years of surveys for one institution, Brigham and Women's Hospital, increased for overall satisfaction between 2016 (2.85), 2018 (3.01), and 2019 (3.21, p < 0.001). We found no significant differences in mean response for overall satisfaction between all responders of the 2018 survey (3.14) and those of the 2019 survey (3.19). Systemwide, tasks rated the most difficult included "Monitoring patient medication adherence," "Identifying when a referral has not been completed," and "Making a list of patients based on clinical information (e.g., problem, medication)." Clinicians disagreed the most with "The EHR helps me focus on patient care rather than the computer" and "The EHR allows me to complete tasks efficiently."

CONCLUSION

Survey results indicate room for improvement in clinician satisfaction with the EHR. Usability of EHRs should continue to be an area of focus to ease clinician burden and improve clinician experience.

Perceived Impact of Digital Health Maturity on Patient Experience, Population Health, Health Care Costs, and Provider Experience: Mixed Methods Case Study

BACKGROUND

Health care organizations understand the importance of new technology implementations; however, the best strategy for implementing successful digital transformations is often unclear. Digital health maturity assessments allow providers to understand the progress made toward technology-enhanced health service delivery. Existing models have been criticized for their lack of depth and breadth because of their technology focus and neglect of meaningful outcomes.

OBJECTIVE

We aimed to examine the perceived impacts of digital health reported by health care staff employed in health care organizations across a spectrum of digital health maturity.

METHODS

A mixed methods case study was conducted. The digital health maturity of public health care systems (n=16) in Queensland, Australia, was examined using the quantitative Digital Health Indicator (DHI) self-assessment survey. The lower and upper quartiles of DHI scores were calculated and used to stratify sites into 3 groups. Using qualitative methods, health care staff (n=154) participated in interviews and focus groups. Transcripts were analyzed assisted by automated text-mining software. Impacts were grouped according to the digital maturity of the health care worker's facility and mapped to the quadruple aims of health care: improved patient experience, improved population health, reduced health care cost, and enhanced provider experience.

RESULTS

DHI scores ranged between 78 and 193 for the 16 health care systems. Health care systems in the high-maturity category (n=4, 25%) had a DHI score of ≥166.75 (the upper quartile); low-maturity sites (n=4, 25%) had a DHI score of ≤116.75 (the lower quartile); and intermediate-maturity sites (n=8, 50%) had a DHI score ranging from 116.75 to 166.75 (IQR). Overall, 18 perceived impacts were identified. Generally, a greater number of positive impacts were reported in health care systems of higher digital health maturity. For patient experiences, higher maturity was associated with maintaining a patient health record and tracking patient experience data, while telehealth enabled access and flexibility across all digital health maturity categories. For population health, patient journey tracking and clinical risk mitigation were reported as positive impacts at higher-maturity sites, and telehealth enabled health care access and efficiencies across all maturity categories. Limited interoperability and organizational factors (eg, strategy, policy, and vision) were universally negative impacts affecting health service delivery. For health care costs, the resource burden of ongoing investments in digital health and a sustainable skilled workforce was reported. For provider experiences, the negative impacts of poor usability and change fatigue were universal, while network and infrastructure issues were negative impacts at low-maturity sites.

CONCLUSIONS

This is one of the first studies to show differences in the perceived impacts of digital maturity of health care systems at scale. Higher digital health maturity was associated with more positive reported impacts, most notably in achieving outcomes for the population health aim.

Electronic Health Record Usability and Postsurgical Outcomes Among Older Adults With Dementia

INTRODUCTION

Electronic health record (EHR) usability, defined as the extent to which the system can be used to complete tasks, can influence patient outcomes. The aim of this study is to assess the relationship between EHR usability and postsurgical outcomes of older adults with dementia including 30-day readmission, 30-day mortality, and length of stay (LOS).

METHODS

A cross-sectional analysis of linked American Hospital Association, Medicare claims data, and nurse survey data was conducted using logistic regression and negative binominal models.

RESULTS

The dementia population who received care in hospitals with better EHR usability were less likely to die within 30 days of their admission following surgery compared to hospitals with poorer EHR usability (OR: 0.79, 95% CI: 0.68-0.91, p = 0.001). EHR usability was not associated with readmission or LOS.

DISCUSSION

Better nurse reported EHR usability has the potential to reduce mortality rates among older adults with dementia in hospitals.

Documentation dynamics: Note composition, burden, and physician efficiency

OBJECTIVE

To analyze how physician clinical note length and composition relate to electronic health record (EHR)-based measures of burden and efficiency that have been tied to burnout.

DATA SOURCES AND STUDY SETTING

Secondary EHR use metadata capturing physician-level measures from 203,728 US-based ambulatory physicians using the Epic Systems EHR between September 2020 and May 2021.

STUDY DESIGN

In this cross-sectional study, we analyzed physician clinical note length and note composition (e.g., content from manual or templated text). Our primary outcomes were three time-based measures of EHR burden (time writing EHR notes, time in the EHR after-hours, and EHR time on unscheduled days), and one measure of efficiency (percent of visits closed in the same day). We used multivariate regression to estimate the relationship between our outcomes and note length and composition.

DATA EXTRACTION

Physician-week measures of EHR usage were extracted from Epic's Signal platform used for measuring provider EHR efficiency. We calculated physician-level averages for our measures of interest and assigned physicians to overall note length deciles and note composition deciles from six sources, including templated text, manual text, and copy/paste text.

PRINCIPAL FINDINGS

Physicians in the top decile of note length demonstrated greater burden and lower efficiency than the median physician, spending 39% more time in the EHR after hours (p < 0.001) and closing 5.6 percentage points fewer visits on the same day (p < 0.001). Copy/paste demonstrated a similar dose/response relationship, with top-decile copy/paste users closing 6.8 percentage points fewer visits on the same day (p < 0.001) and spending more time in the EHR after hours and on days off (both p < 0.001). Templated text (e.g., Epic's SmartTools) demonstrated a non-linear relationship with burden and efficiency, with very low and very high levels of use associated with increased EHR burden and decreased efficiency.

CONCLUSIONS

"Efficiency tools" like copy/paste and templated text meant to reduce documentation burden and increase provider efficiency may have limited efficacy.

[Administrative burden, motivation and well-being among primary care physicians: comparison of management models]

OBJECTIVE

To analyze the relationship between administrative burden and intrinsic motivation and between administrative burden and professional well-being among family and community medicine physicians. To analyze the extent to which attention to three basic needs (autonomy, sense of competence and relationship capacity) that one of the main motivation theories identifies as a generator of intrinsic motivation compensates for the negative effect of the administrative burden. To compare management models.

METHOD

Data from a survey (542 valid questionnaires) carried out through the Catalan Society of Family and Community Medicine are used. Data analysis is based on multiple regression and ANOVA techniques.

RESULTS

The perception of administrative burden and "red tape" are clearly demotivating and reduce job satisfaction. On the contrary, individual autonomy, networking opportunities between professionals and performance feedback are motivating and feed the perception of professional well-being. The EBA (Association Based Entity) model presents results above the ICS (Catalan Health Institute) and OSI (Integrated Health Organizations) models both in controlling the negative effect of administrative burden and in the management of basic psychological needs.

CONCLUSIONS

The motivation and professional well-being of primary care physicians would benefit from reforms that limit the administrative burden of some procedures and red tape. At the primary health center level, greater attention to individual autonomy, improving relatedness between professionals and the sense of competence based on better information on individual performance can offset the negative effects of the administrative burden.

Associations of technostressors at work with burnout symptoms and chronic low-grade inflammation: a cross-sectional analysis in hospital employees

OBJECTIVE

Despite the increasing scholarly interest in the phenomenon technostress, associated biological effects on employee health are under-researched. Chronic low-grade inflammation is suggested as a central pathway linking stress experience to disease development. The aim of this study was to assess associations of technology-related work stressors (technostressors) with low-grade inflammation and burnout symptoms.

METHODS

N = 173 (74.6% women, M_age = 31.0 years) university hospital employees participated in a cross-sectional study. Self-report questionnaires were used for the assessment of general psychosocial working conditions (work overload, job control, social climate), a range of different technostressors, burnout symptoms, and relevant confounders. Participants provided capillary blood samples, and high-sensitivity C-reactive protein (hs-CRP) as an inflammatory biomarker was analyzed from dried blood spots.

RESULTS

Based on a factor analysis, we identified four underlying dimensions of technostressors: techno- and information overload, techno-complexity, interruptions and multitasking as well as usability and technical support. In multivariate linear regressions, techno-/information overload and techno-complexity were associated with core (exhaustion, mental distance) and secondary (psychosomatic complaints) symptoms of burnout. Techno-/information overload was a significant predictor of burnout core symptoms, even when general work overload was controlled for. The technostressors were not associated with hs-CRP.

CONCLUSION

This is the first study on technology-related stress at work and chronic low-grade inflammation. The results suggest that (information) overload caused by digital technology use is a distinct work stressor with genuine consequences for psychological health. To what extent these effects also manifest on a physiological level needs to be subjected to future studies, ideally with prospective designs.

Occupational Characteristics Associated With Professional Fulfillment and Burnout Among US Physiatrists

INTRODUCTION

Multiple national studies suggest that among physicians, physiatrists are at increased risk for occupational burnout.

OBJECTIVE

The aim of the study is to identify characteristics of the work environment associated with professional fulfillment and burnout among US physiatrists.

DESIGN

Between May and December 2021, a mixed qualitative and quantitative approach was used to identify factors contributing to professional fulfillment and burnout in physiatrists.

SETTING

Online interviews, focus groups, and survey were conducted.

PARTICIPANTS

The participants are physiatrists in the American Academy of Physical Medicine and Rehabilitation Membership Masterfile.

MAIN OUTCOME MEASURES

Burnout and professional fulfillment were assessed using the Stanford Professional Fulfillment Index.

RESULTS

Individual interviews with 21 physiatrists were conducted to identify domains that contributed to professional fulfillment followed by focus groups for further definition. Based on themes identified, scales were identified or developed to evaluate: control over schedule (6 items, Cronbach α = 0.86); integration of physiatry into patient care (3 items, Cronbach α = 0.71); personal-organizational values alignment (3 items, Cronbach α = 0.90); meaningfulness of physiatrist clinical work (6 items, Cronbach α = 0.90); teamwork and collaboration (3 items, Cronbach α = 0.89). Of 5760 physiatrists contacted in the subsequent national survey, 882 (15.4%) returned surveys (median age, 52 yrs; 46.1% women). Overall, 42.6% (336 of 788) experienced burnout and 30.6% (244 of 798) had high levels of professional fulfillment. In multivariable analysis, each one-point improvement in control over schedule (odds ratio = 1.96; 95% confidence interval = 1.45-2.69), integration of physiatry into patient care (odds ratio = 1.77; 95% confidence interval = 1.32-2.38), personal-organizational values alignment (odds ratio = 1.92; 95% confidence interval = 1.48-2.52), meaningfulness of physiatrist clinical work (odds ratio = 2.79; 95% confidence interval = 1.71-4.71), and teamwork and collaboration score (odds ratio = 2.11; 95% confidence interval = 1.48-3.03) was independently associated with higher likelihood of professional fulfillment.

CONCLUSIONS

Control over schedule, optimal integration of physiatry into clinical care, personal-organizational values alignment, teamwork, and meaningfulness of physiatrist clinical work are strong and independent drivers of occupational well-being in US physiatrists. Variation in these domains by practice setting and subspecialty suggests that tailored approaches are needed to promote professional fulfillment and reduce burnout among US physiatrists.

Classification of Health Information Technology Safety Events in a Pediatric Tertiary Care Hospital

OBJECTIVE

State agencies have developed reporting systems of safety events that include events related to health information technology (HIT). These data come from hospital reporting systems where staff submit safety reports and nurses, in the role of safety managers, review, and code events. Safety managers may have varying degrees of experience with identifying events related to HIT. Our objective was to review events potentially involving HIT and compare those with what was reported to the state.

METHODS

We performed a structured review of 1 year of safety events from an academic pediatric healthcare system. We reviewed the free-text description of each event and applied a classification scheme derived from the AHRQ Health IT Hazard Manager and compared the results with events reported to the state as involving HIT.

RESULTS

Of 33,218 safety events for a 1-year period, 1247 included key words related to HIT and/or were indicated by safety managers as involving HIT. Of the 1247 events, the structured review identified 769 as involving HIT. In comparison, safety managers only identified 194 of the 769 events (25%) as involving HIT. Most events, 353 (46%), not identified by safety managers were documentation issues. Of the 1247 events, the structured review identified 478 as not involving HIT while safety managers identified and reported 81 of these 478 events (17%) as involving HIT.

CONCLUSIONS

The current process of reporting safety events lacks standardization in identifying health technology contributions to safety events, which may minimize the effectiveness of safety initiatives.

Why is biomedical informatics hard? A fundamental framework

Building on previous work to define the scientific discipline of biomedical informatics, we present a framework that categorizes fundamental challenges into groups based on data, information, and knowledge, along with the transitions between these levels. We define each level and argue that the framework provides a basis for separating informatics problems from non-informatics problems, identifying fundamental challenges in biomedical informatics, and provides guidance regarding the search for general, reusable solutions to informatics problems. We distinguish between processing data (symbols) and processing meaning. Computational systems, that are the basis for modern information technology (IT), process data. In contrast, many important challenges in biomedicine, such as providing clinical decision support, require processing meaning, not data. Biomedical informatics is hard because of the fundamental mismatch between many biomedical problems and the capabilities of current technology.

Usability and Clinician Acceptance of a Deep Learning-Based Clinical Decision Support Tool for Predicting Glaucomatous Visual Field Progression

PRCIS

We updated a clinical decision support tool integrating predicted visual field (VF) metrics from an artificial intelligence model and assessed clinician perceptions of the predicted VF metric in this usability study.

PURPOSE

To evaluate clinician perceptions of a prototyped clinical decision support (CDS) tool that integrates visual field (VF) metric predictions from artificial intelligence (AI) models.

METHODS

Ten ophthalmologists and optometrists from the University of California San Diego participated in 6 cases from 6 patients, consisting of 11 eyes, uploaded to a CDS tool ("GLANCE", designed to help clinicians "at a glance"). For each case, clinicians answered questions about management recommendations and attitudes towards GLANCE, particularly regarding the utility and trustworthiness of the AI-predicted VF metrics and willingness to decrease VF testing frequency.

MAIN OUTCOMES AND MEASURES

Mean counts of management recommendations and mean Likert scale scores were calculated to assess overall management trends and attitudes towards the CDS tool for each case. In addition, system usability scale scores were calculated.

RESULTS

The mean Likert scores for trust in and utility of the predicted VF metric and clinician willingness to decrease VF testing frequency were 3.27, 3.42, and 2.64, respectively (1=strongly disagree, 5=strongly agree). When stratified by glaucoma severity, all mean Likert scores decreased as severity increased. The system usability scale score across all responders was 66.1±16.0 (43rd percentile).

CONCLUSIONS

A CDS tool can be designed to present AI model outputs in a useful, trustworthy manner that clinicians are generally willing to integrate into their clinical decision-making. Future work is needed to understand how to best develop explainable and trustworthy CDS tools integrating AI before clinical deployment.

Targeting repetitive laboratory testing with electronic health records-embedded predictive decision support: A pre-implementation study

INTRODUCTION

Unnecessary laboratory testing contributes to patient morbidity and healthcare waste. Despite prior attempts at curbing such overutilization, there remains opportunity for improvement using novel data-driven approaches. This study presents the development and early evaluation of a clinical decision support tool that uses a predictive model to help providers reduce low-yield, repetitive laboratory testing in hospitalized patients.

METHODS

We developed an EHR-embedded SMART on FHIR application that utilizes a laboratory test result prediction model based on historical laboratory data. A combination of semi-structured physician interviews, usability testing, and quantitative analysis on retrospective laboratory data were used to inform the tool's development and evaluate its acceptability and potential clinical impact.

KEY RESULTS

Physicians identified culture and lack of awareness of repeat orders as key drivers for overuse of inpatient blood testing. Users expressed an openness to a lab prediction model and 13/15 physicians believed the tool would alter their ordering practices. The application received a median System Usability Scale score of 75, corresponding to the 75th percentile of software tools. On average, physicians desired a prediction certainty of 85% before discontinuing a routine recurring laboratory order and a higher certainty of 90% before being alerted. Simulation on historical lab data indicates that filtering based on accepted thresholds could have reduced ∼22% of repeat chemistry panels.

CONCLUSIONS

The use of a predictive algorithm as a means to calculate the utility of a diagnostic test is a promising paradigm for curbing laboratory test overutilization. An EHR-embedded clinical decision support tool employing such a model is a novel and acceptable intervention with the potential to reduce low-yield, repetitive laboratory testing.

Utilizing Remote Access for Electronic Medical Records Reduces Overall EMR Time for Vascular Surgery Residents

INTRODUCTION

Survey data suggests that surgical residents spend 20-30% of training time using the electronic medical record (EMR), raising concerns about burnout and insufficient operative experience. We characterize trainee EMR activity in the vascular surgery service of a quaternary care center to identify modifiable factors associated with high EMR use.

METHODS

Resident activity while on the Vascular Surgery service was queried from the EMR. Weekends and holidays were excluded to focus on typical staffing periods. Variables including daily time spent, post-graduate year (PGY), remote access via mobile device or personal laptop, and patient census including operative caseload were extracted. Univariate analysis was performed with t-tests and chi-squared tests where appropriate. We then fit a linear mixed-effects model with normalized daily EMR time as the outcome variable, random slopes for resident and patient census, and fixed effects of PGY level, academic year, and fractional time spent using remote access.

RESULTS

EMR activity for 53 residents from July 2015 to June 2019 was included. The mean daily EMR usage was 1.6 hours, ranging from 3.6 hours per day in PGY1 residents to 1.1 hours in PGY4-5 residents. Across all post-graduate years, the most time-consuming EMR activities were chart review (43.0-46.6%) and notes review (22.4-27.0%). In the linear mixed-effects model, increased patient census was associated with increased daily EMR usage (Coefficient = 0.61, p-value < 0.001). Resident seniority (Coefficient = -1.2, p-value < 0.001) and increased remote access (Coefficient = -0.44, p-value < 0.001) were associated with reduced daily EMR usage. Over the study period, total EMR usage decreased significantly from the 2015-2016 academic year to 2018-2019 (mean difference 2.4 hours vs 1.78, p-value <0.001).

CONCLUSIONS

In an audit of EMR activity logs on a vascular surgery service, mean EMR time was 1.6 hours a day, which is lower than survey estimates. Resident seniority and remote access utilization were associated with reduced time spent on the EMR, independent of patient census. While increasing EMR accessibility via mobile devices and personal computers have been hypothesized to contribute to poor work-life balance, our study suggests a possible time-saving effect by enabling expedient access for data review, which constitutes the majority of resident EMR activity. Further research in other institutions and specialties is needed for external validation and exploring implications for resident wellness initiatives.

Muffins and Meditation: Combatting Burnout in Surgical Residents

OBJECTIVE

To quantify the prevalence of burnout in our surgical residency program and to assess the impact of a weekly wellness program for surgical residents through validated tools measuring mindfulness, self-compassion, flourishing, and burnout. Our hypothesis was that participants with more frequent attendance would: (1) be more mindful and self-compassionate and (2) experience less burnout and more flourishing.

DESIGN

An optional one-hour weekly breakfast conference was facilitated by a senior surgical faculty member with the time protected from all clinical duties. Following a guided meditation, participants were given time for reflection and dialogue about their training experiences or led in a wellness exercise. TRANCE (tolerance, respect, anonymity, nonretaliation, compassion, egalitarianism) principles were utilized to create a safe and open environment. Residents were surveyed at the end of the study period, which was from March 2017 through June 2018.

SETTING

The conference and data analysis was conducted at Denver Health Medical Center, affiliated with the University of Colorado School of Medicine.

PARTICIPANTS

This study analyzed survey responses from 85 surgical residents.

RESULTS

Following the wellness program, when answering the 2-question Maslach Burnout Inventory, 35.7% of residents reported feeling burned out by their work once a week or more, and 29.7% reported feeling more callous toward people once a week or more. After multivariate analysis, the only independent predictors of increased burnout were "not being married or in a committed relationship," lower positive affect, and higher negative affect. Written feedback was overwhelmingly positive, and residents expressed gratitude for the conference, the opportunity for self-reflection, and open dialogue with attendings and colleagues.

CONCLUSIONS

The prevalence of burnout is high among surgical residents. Allowing time to practice a mindfulness meditation while providing space for residents to share their experiences may be protective, and efforts should be made to reduce barriers to participation.