Eye tracking metrics of orthopedic surgeons with different competency levels who practice simulation-based hip arthroscopic procedures

Objective Assessment of Cognitive Workload in Surgery: A Systematic Review

OBJECTIVE

To systematically review technologies that objectively measure cognitive workload (CWL) in surgery, assessing their psychometric and methodological characteristics.

BACKGROUND

Surgical tasks involving concurrent clinical decision-making and the safe application of technical and non-technical skills require a substantial cognitive demand and resource utilization. Cognitive overload leads to impaired clinical decision-making and performance decline. Assessing CWL could enable interventions to alleviate burden and improve patient safety.

METHODS

Ovid MEDLINE, OVID Embase, the Cochrane Library, and IEEE Xplore databases were searched from inception to August 2023. Full-text, peer-reviewed original studies in a population of surgeons, anesthesiologists or interventional radiologists were considered, with no publication date constraints. Study population, task paradigm, stressor, cognitive load theory (CLT) domain, objective and subjective parameters, statistical analysis, and results were extracted. Studies were assessed for (1) definition of CWL; (2) details of the clinical task paradigm; and (3) objective CWL assessment tool. Assessment tools were evaluated using psychometric and methodological characteristics.

RESULTS

A total of 10,790 studies were identified; 9004 were screened; 269 full studies were assessed for eligibility, of which 67 met inclusion criteria. The most widely used assessment modalities were autonomic (32 eye studies and 24 cardiac). Intrinsic workload (eg, task complexity) and germane workload (effect of training or expertize) were the most prevalent designs investigated. CWL was not defined in 30 of 67 studies (44.8%). Sensitivity was greatest for neurophysiological instruments (100% EEG, 80% fNIRS); and across modalities accuracy increased with multisensor recordings. Specificity was limited to cardiac and ocular metrics, and was found to be suboptimal (50% and 66.67%). Cardiac sensors were the least intrusive, with 54.2% of studies conducted in naturalistic clinical environments (higher ecological validity).

CONCLUSIONS

Physiological metrics provide an accessible, objective assessment of CWL, but dependence on autonomic function negates selectivity and diagnosticity. Neurophysiological measures demonstrate favorable sensitivity, directly measuring brain activation as a correlate of cognitive state. Lacking an objective gold standard at present, we recommend the concurrent use of multimodal objective sensors and subjective tools for cross-validation. A theoretical and technical framework for objective assessment of CWL is required to overcome the heterogeneity of methodological reporting, data processing, and analysis.

User Experience Study of the Patient Monitoring Systems Based on Usability Testing and Eye Tracking

BACKGROUND/OBJECTIVES

The patient monitoring system is a critical tool commonly used in hospitals, making it essential to assess caregivers' user requirements and satisfaction with its usability. In intensive care units (ICUs), the usability of these systems is closely linked to the work efficiency of key users, such as nurses, and directly impacts patient safety and treatment outcomes. This study evaluates the usability of patient monitoring systems in intensive care units (ICUs), focusing on user requirements and satisfaction among nurses, the primary users. Usability is directly linked to work efficiency and patient safety, with post-marketing surveillance (PMS) data from overseas, highlighting issues such as unrecognized alarms, leading to worsened patient conditions.

METHODS

This study involved 22 ICU nurses who had used the system for over a year, assessing usability through testing, satisfaction surveys, the Health-ITUES, and eye-tracking analysis.

RESULTS

The results showed a high success rate (94%) and positive satisfaction scores (4.15, SD = 0.88), with a Health-ITUES score of 4.13 (SD = 0.78). Eye tracking revealed that some functions, including alarms, were overlooked or not recognized.

CONCLUSIONS

Recommendations include improving the interface for alarm messages and recording deletion functions to enhance user satisfaction and patient safety.

Exploring eye-tracking data as an indicator of situational awareness in nursing students during a cardiorespiratory arrest simulation

AIM

To examine the components of visual attention that maintain situational awareness during simulation training in undergraduate nursing students with different instruction levels.

BACKGROUND

Eye-tracking can provide deep insight into the nurses' attention during simulated practice. Knowing which gaze patterns promote situational awareness can significantly improve nurse instruction.

DESIGN

A comparative observational study investigated the role of visual attention on the performance quality, psychophysiological parameters (vital signs, anxiety and stress) and socioemotional competencies (cognitive workload, motivation and self-efficacy) of nursing students with various experience levels.

METHODS

Thirty nursing students divided into two groups according to their academic level: first cycle (n=14) and second-cycle (n=16) faced a clinical simulation scenario to resolve a cardiorespiratory arrest event. Eye tracking-based analysis required the selection of six areas of interest. The monitorization of vital signs included measuring blood pressure, heart rate, respiratory rate and oxygen saturation before and after the simulation practice. Participants completed the socioemotional questionnaire (NASA-TLX). They answered the state subscale of the State-Trait Anxiety Inventory (STAI), the Visual Analogue Scale (VAS) of stress, the Situational Motivation Scale (SIMS) and the Baessler and Schwarzer General Self-Efficacy Scale.

RESULTS

The first-cycle group displayed higher vital sign scores than the second cycle, apart from the post-simulation respiratory rate. All physiological parameters increased in mean value after the clinical simulation, except oxygen saturation. Anxiety was the only parameter in the socioemotional domain to present a statistically significant difference between the groups. First-year nursing students showed greater anxiety, stress, mental workload, identified regulation and intrinsic motivation, while second-year students showed higher levels of amotivation, external regulation and perceived self-efficacy. Eye-tracking data (revisits, gaze and duration of fixations) exhibited statistically significant differences depending on the area of interest in both groups (p =. 05). The performance outcomes showed a negative and moderate association with gaze the total number of gazes in the second-cycle group (rho = -0.640, p = 0.010).

CONCLUSION

Eye-tracking-based analysis can help to predict performance quality while maintaining situational awareness during nursing instruction.

Gaze behavior is related to objective technical skills assessment during virtual reality simulator-based surgical training: a proof of concept

PURPOSE

Simulation-based training allows surgical skills to be learned safely. Most virtual reality-based surgical simulators address technical skills without considering non-technical skills, such as gaze use. In this study, we investigated surgeons' visual behavior during virtual reality-based surgical training where visual guidance is provided. Our hypothesis was that the gaze distribution in the environment is correlated with the simulator's technical skills assessment.

METHODS

We recorded 25 surgical training sessions on an arthroscopic simulator. Trainees were equipped with a head-mounted eye-tracking device. A U-net was trained on two sessions to segment three simulator-specific areas of interest (AoI) and the background, to quantify gaze distribution. We tested whether the percentage of gazes in those areas was correlated with the simulator's scores.

RESULTS

The neural network was able to segment all AoI with a mean Intersection over Union superior to 94% for each area. The gaze percentage in the AoI differed among trainees. Despite several sources of data loss, we found significant correlations between gaze position and the simulator scores. For instance, trainees obtained better procedural scores when their gaze focused on the virtual assistance (Spearman correlation test, N = 7, r = 0.800, p = 0.031).

CONCLUSION

Our findings suggest that visual behavior should be quantified for assessing surgical expertise in simulation-based training environments, especially when visual guidance is provided. Ultimately visual behavior could be used to quantitatively assess surgeons' learning curve and expertise while training on VR simulators, in a way that complements existing metrics.