Clutter in electronic medical records: examining its performance and attentional costs using eye tracking

The mental workload of ICU nurses performing human-machine tasks and associated factors: A cross-sectional questionnaire survey

AIMS

To assess the level of mental workload (MWL) of intensive care unit (ICU) nurses in performing different human-machine tasks and examine the predictors of the MWL.

DESIGN

A cross-sectional questionnaire study.

METHODS

Between January and February 2021, data were collected from ICU nurses (n = 427) at nine tertiary hospitals selected from five (east, west, south, north, central) regions in China through an electronic questionnaire, including sociodemographic questions, the National Aeronautics and Space Administration Task Load Index, General Self-Efficacy Scale, Difficulty-assessing Index System of Nursing Operation Technique, and System Usability Scale. Descriptive statistics, t-tests, one-way ANOVA and multiple linear regression models were used.

RESULTS

ICU nurses experienced a medium level of MWL (score 52.04 on a scale of 0-100) while performing human-machine tasks. ICU nurses' MWL was notably higher in conducting first aid and life support tasks (using defibrillators or ventilators). Predictors of MWL were task difficulty, system usability, professional title, age, self-efficacy, ICU category, and willingness to study emerging technology actively. Task difficulty and system usability were the strongest predictors of nearly all typical tasks.

CONCLUSION

ICU nurses experience a medium MWL while performing human-machine tasks, but higher mental, temporal, and effort are perceived compared to physical demands. The MWL varied significantly across different human-machine tasks, among which are significantly higher: first aid and life support and information-based human-machine tasks. Task difficulty and system availability are decisive predictors of MWL.

IMPACT

This is the first study to investigate the level of MWL of ICU nurses performing different representative human-machine tasks and to explore its predictors, which provides a reference for future research. These findings suggest that healthcare organizations should pay attention to the MWL of ICU nurses and develop customized management strategies based on task characteristics to maintain a moderate level of MWL, thus enabling ICU nurses to perform human-machine tasks better.

PATIENT OR PUBLIC CONTRIBUTION

No patient or public contribution.

The influence of target layout and target graphic type on searching performance based on eye-tracking technology

With the development of various intelligent technologies, the application of interactive interfaces is becoming more and more widespread, and the related researches conducted for interactive interfaces are also increasing. The purpose of this study was to explore the influence of icon layout location, icon graphic type, and icon layout method on users' searching performance in interactive interfaces through eye-tracking technology. Participants were asked to perform search tasks for the search target (facet icon or linear icon) on each image. Thus, each trial consisted of a search task on a given image. In total, each participant had 36 trials to complete. Searching time, fixation duration, and fixation count were collected to evaluate the searching performance of participants. Results showed that when faced with familiar icons, whether the graphic type of icons was facet or linear did not affect the user's experience, but when other factors of the interaction interface changed, facet icons provided a more stable experience for users. And compared to the rectangular layout, the circular layout method provided a more stable experience for users when the location of icons in the interactive interface changed, but icons located in the top half of the interactive interface were easier to find than those located in the bottom half, regardless of whether the layout was circular or rectangular. These results could be used in the layout and icon design of the interactive interfaces to facilitate their optimization.

Time difference in retrieving clinical information in Patient-overview Prostate Cancer compared to electronic health records

BACKGROUND

Patients with advanced prostate cancer (PCa) typically undergo numerous lines of treatment leading to large amounts of information in Electronic Health Records (EHRs). The Patient-overview Prostate Cancer (PPC) presents clinical information in a graphical overview. The aim of this study was to measure time spent on retrieving clinical information in PPC compared to EHRs, to assess if retrieved data was correct and to explore usability of PPC.

MATERIAL AND METHODS

Oncologists, urologists and nurses in three hospitals in Sweden were timed when filling out questionnaires about patients using PPC and two different EHRs; Melior and COSMIC. Time and number of errors were analysed using linear mixed models (LMMs). Usability of PPC was measured with the System Usability Scale.

RESULTS

The LMM showed a significantly shorter time to retrieve information in PPC compared to EHRs. The estimated time to complete one questionnaire was 8 minutes (95% CI = 6-10, p < 0.001) in PPC compared to 25 minutes in Melior and 21 minutes in COSMIC. Compared to PPC, the estimated time difference was 17 minutes longer in Melior (95% CI = 14-20, p < 0.001) and 13 minutes longer in COSMIC (95% CI = 10-17, p < 0.001). The LMM showed significantly fewer errors in PPC compared to Melior. No significant difference in the number of errors was found between PPC and COSMIC. The usability of PPC was rated as excellent by oncologists, urologists and nurses.

CONCLUSION

A graphical overview of a patient's medical history, as in PPC, gives health staff rapid access to relevant information with a high degree of usability.

How do we react to cluttered displays? Evidence from the first seconds of visual search in websites

Display clutter is known to degrade search performance and lead to differences in eye movement measures in different contexts. The goal of this study was to determine whether these differences in eye movements could be detected in the first few seconds of a search task using a realistic display, both with or without time pressure. Participants were asked to search for image or word targets in 40 website screenshots. Time pressure was introduced for half the trials. Clutter algorithms were used to classify the websites as low- or high-clutter. Performance, subjective, and eye-tracking metrics were collected. Results showed that people's attention allocation within the first 3 s of search is different when viewing high-clutter websites. In particular, people's spread of attention was larger in high-clutter websites. The results can be used to detect whether a person is struggling with clutter early on after they view a display. Practitioner summary: Eye-tracking metrics showed that people react differently to a cluttered website in a variety of conditions. These differences were evident within the first 3 s of the search. The eye-tracking metrics identified can be used to detect people struggling with clutter as soon as they look at a website.

Feeling and thinking: can theories of human motivation explain how EHR design impacts clinician burnout?

The psychology of motivation can help us understand the impact of electronic health records (EHRs) on clinician burnout both directly and indirectly. Informatics approaches to EHR usability tend to focus on the extrinsic motivation associated with successful completion of clearly defined tasks in clinical workflows. Intrinsic motivation, which includes the need for autonomy, sense-making, creativity, connectedness, and mastery is not well supported by current designs and workflows. This piece examines existing research on the importance of 3 psychological drives in relation to healthcare technology: goal-based decision-making, sense-making, and agency/autonomy. Because these motives are ubiquitous, foundational to human functioning, automatic, and unconscious, they may be overlooked in technological interventions. The results are increased cognitive load, emotional distress, and unfulfilling workplace environments. Ultimately, we hope to stimulate new research on EHR design focused on expanding functionality to support intrinsic motivation, which, in turn, would decrease burnout and improve care.

Misdiagnosis: Burnout, moral injury, and implications for the electronic health record

Burnout is a long-term stress reaction marked by emotional exhaustion, depersonalization, and a lack of sense of personal accomplishment. Burnout in clinicians is receiving significant attention. Some have proposed that clinicians are experiencing symptoms of moral injury, defined as "perpetrating, failing to prevent, bearing witness to, or learning about acts that transgress deeply held moral beliefs and expectations." Current efforts to improve the electronic health record (EHR) have focused on improving the user experience to reduce burden that has been identified as a contributing factor to provider burnout. However, if EHRs are contributing to moral injury, improvements to user experience will not eliminate the effects on providers. Current research has not evaluated the risk for moral injury resulting from the use of EHRs. This Perspective reviews the differences between burnout and moral injury, discusses the implications for clinicians using EHRs, and highlights the need for research to better define the problem.

Leveraging Eye Tracking to Prioritize Relevant Medical Record Data: Comparative Machine Learning Study

BACKGROUND

Electronic medical record (EMR) systems capture large amounts of data per patient and present that data to physicians with little prioritization. Without prioritization, physicians must mentally identify and collate relevant data, an activity that can lead to cognitive overload. To mitigate cognitive overload, a Learning EMR (LEMR) system prioritizes the display of relevant medical record data. Relevant data are those that are pertinent to a context-defined as the combination of the user, clinical task, and patient case. To determine which data are relevant in a specific context, a LEMR system uses supervised machine learning models of physician information-seeking behavior. Since obtaining information-seeking behavior data via manual annotation is slow and expensive, automatic methods for capturing such data are needed.

OBJECTIVE

The goal of the research was to propose and evaluate eye tracking as a high-throughput method to automatically acquire physician information-seeking behavior useful for training models for a LEMR system.

METHODS

Critical care medicine physicians reviewed intensive care unit patient cases in an EMR interface developed for the study. Participants manually identified patient data that were relevant in the context of a clinical task: preparing a patient summary to present at morning rounds. We used eye tracking to capture each physician's gaze dwell time on each data item (eg, blood glucose measurements). Manual annotations and gaze dwell times were used to define target variables for developing supervised machine learning models of physician information-seeking behavior. We compared the performance of manual selection and gaze-derived models on an independent set of patient cases.

RESULTS

A total of 68 pairs of manual selection and gaze-derived machine learning models were developed from training data and evaluated on an independent evaluation data set. A paired Wilcoxon signed-rank test showed similar performance of manual selection and gaze-derived models on area under the receiver operating characteristic curve (P=.40).

CONCLUSIONS

We used eye tracking to automatically capture physician information-seeking behavior and used it to train models for a LEMR system. The models that were trained using eye tracking performed like models that were trained using manual annotations. These results support further development of eye tracking as a high-throughput method for training clinical decision support systems that prioritize the display of relevant medical record data.

A Framework for Contracted Health IT Performance Assessments

Limited research exists which aids in structuring health IT contracts in an era of performance-based payments. We provide an assessment of common approaches to contracting and measuring of performance in practice. We conducted a review of existing literature and compliment this approach with a survey of healthcare professionals directly involved with health IT systems to further understand and classify current approaches. We identified architypes for structuring healthcare IT performance contracts to include: (1) internal operations, (2) external evaluation and (3) joint agreement for the delivery of value-based care.

I-CATCH: A Novel Bundle to Improve Postcall Morning Handoffs

BACKGROUND

Conducting postcall morning handoffs using a resident handoff bundle such as I-PASS can prove challenging. This may delay recognizing and acting on clinically important patient issues that arose overnight.

OBJECTIVE

We developed and implemented the I-CATCH morning handoff bundle and evaluated its impact on the proportion of overnight patient issues handed off from the on-call resident to the daytime team.

METHODS

We evaluated the I-CATCH (Identify patient; Characterize situation; Action-what was done overnight?; To do for the team in the morning; Confirm the Handoff) handoff bundle from November 2015 to May 2016 on general internal medicine wards at 1 academic teaching hospital. The bundle entailed staff/resident training, structured communication, and dedicated handoff space and time. We compared handoffs of overnight on-call issues by evening resident to daytime medical team before and after implementation, and used statistical process control to analyze adherence to the mnemonic.

RESULTS

We observed 435 handoffs (242 pre- and 193 postimplementation) over 63 days. There was no significant association between I-CATCH implementation and proportion of on-call overnight issues handed off (OR = 0.96; 95% confidence interval [CI] 0.52-1.47; P = .85). Running the list by going through patients one-by-one (OR = 1.74; 95% CI 1.1-2.77; P = .019), progress note documentation (OR = 3.80; 95% CI 2.19-6.60; P < .001), and direct handoff (OR = 4.84; 95% CI 1.43-16.42; P = .011) correlated with an increased likelihood of morning handoff.

CONCLUSIONS

Implementing the I-CATCH bundle did not improve handoff of overnight issues to the daytime team.

Too much or too little? Investigating the usability of high and low data displays of the same electronic medical record

The high data density on electronic medical record screens is touted as a major usability issue. However, it may not be a problem if the data is relevant and well-organized. Our objective was to test this assumption using a comprehensive set of measures that assess the three pillars of usability: efficiency (both physical and cognitive), effectiveness, and satisfaction. Physicians were asked to go through a series of tasks using two versions of the same electronic medical record: one where all the display items were separated into tabs (the original display), and one where important display items were grouped logically in one tab (the redesigned display). Results supported the hypothesis that combining relevant data in organized fashion into a smaller location would improve usability. The findings highlight the role of good display organization to mitigate the effects of high data density, as well as the importance of assessing cognitive load as part of usability studies.

Comparative usability evaluation of consultation order templates in a simulated primary care environment

Communication breakdowns in the referral process negatively impact clinical workflow and patient safety. There is a lack of evidence demonstrating the impact of published design recommendations addressing contributing issues with consultation order templates. This study translated the recommendations into a computer-based prototype and conducted a comparative usability evaluation. With a scenario-based simulation, 30 clinicians (referrers) participated in a within-group, counterbalanced experiment comparing the prototype with their present electronic order entry system. The prototype significantly increased satisfaction (Cohen's d = 1.80, 95% CI [1.19, 2.41], p < .001), and required significantly less mental effort (d = 0.67 [0.14, 1.20], p < .001). Regarding efficiency, the prototype required significantly fewer mouse clicks (mean difference = 29 clicks, p < .001). Although overall task time did not differ significantly (d = -0.05 [-0.56, 0.47]), the prototype significantly quickened identification of the appropriate specialty clinic (mean difference = 12 s, d = 0.98 [0.43, 1.52], p < .001). The experimental evidence demonstrated that clinician-centered interfaces significantly improved system usability during ordering of consultations.

The effect of navigation display clutter on performance and attention allocation in presentation- and simulator-based driving experiments

Display clutter can have differential effects based on environmental factors, such as workload, stress, and experiment paradigm. The objectives of the current study were to assess the effects of display clutter on driver performance and attention allocation and compare results across two experimental paradigms. Forty-two participants searched high- and low-clutter in-car navigation displays for routine information either during a static, presentation-based experiment or in a dynamic, driving simulator experiment. Results revealed display clutter to significantly alter attention allocation and degrade performance in the presentation experiment, but had little to no effect on driver performance or attention allocation in the driving simulator experiment. Results suggest that display clutter may have its greatest effect on performance and attention allocation in domains requiring extended attention to the cluttered display compared to tasks in which the cluttered display acts as a support tool for secondary tasks.

Usability Evaluation and Implementation of a Health Information Technology Dashboard of Evidence-Based Quality Indicators

Health information technology dashboards that integrate evidence-based quality indicators can efficiently and accurately display patient risk information to promote early intervention and improve overall quality of patient care. We describe the process of developing, evaluating, and implementing a dashboard designed to promote quality care through display of evidence-based quality indicators within an electronic health record. Clinician feedback was sought throughout the process. Usability evaluations were provided by three nurse pairs and one physician from medical-surgical areas. Task completion times, error rates, and ratings of system usability were collected to compare the use of quality indicators displayed on the dashboard to the indicators displayed in a conventional electronic health record across eight experimental scenarios. Participants rated the dashboard as "highly usable" following System Usability Scale (mean, 87.5 [SD, 9.6]) and Poststudy System Usability Questionnaire (mean, 1.7 [SD, 0.5]) criteria. Use of the dashboard led to reduced task completion times and error rates in comparison to the conventional electronic health record for quality indicator-related tasks. Clinician responses to the dashboard display capabilities were positive, and a multifaceted implementation plan has been used. Results suggest application of the dashboard in the care environment may lead to improved patient care.

Electronic Health Records and Improved Patient Care: Opportunities for Applied Psychology

Healthcare is undergoing an unprecedented technology transition from paper medical records to electronic health records (EHRs). While the adoption of EHRs holds tremendous promise for improving efficiency, quality and safety, there have been numerous challenges that have been largely centered on the technology not meeting the cognitive needs of the clinical end-users. Clinicians are experiencing increased stress and frustration, and new safety hazards have been introduced. There is a significant opportunity for applied psychologists to address many of these challenges. I highlight three key areas: studying and modeling clinician needs, applying theoretically grounded design principles, and developing technology to support teamwork and communication.

Turning Electromyography Reports Upside Down: A Pilot Study Surveying Referring Providers

Providers are expressing a desire for more efficient ways to retrieve relevant clinical data from the Electronic Health Record. In an effort to improve our Electromyography and Nerve Conduction Study reports, we surveyed referring providers on the effects of having the IMPRESSION at the start of the report. Our survey respondents felt that using this format for an Electromyography and Nerve Conduction Study report significantly improved the quality of the report while saving them time and/or mouse clicks when interpreting the report. Electro diagnosticians might consider using this format for their Electromyography and Nerve Conduction Study reports to improve referring provider satisfaction.