Differences in gaze behaviour of expert and junior surgeons performing open inguinal hernia repair

Effects of a training system that tracks the operator's gaze pattern during endoscopic submucosal dissection on hemostasis

BACKGROUND

The early acquisition of skills required to perform hemostasis during endoscopy may be hindered by the lack of tools that allow assessments of the operator's viewpoint. Understanding the operator's viewpoint may facilitate the skills.

AIM

To evaluate the effects of a training system using operator gaze patterns during gastric endoscopic submucosal dissection (ESD) on hemostasis.

METHODS

An eye-tracking system was developed to record the operator's viewpoints during gastric ESD, displaying the viewpoint as a circle. In phase 1, videos of three trainees' viewpoints were recorded. After reviewing these, trainees were recorded again in phase 2. The videos from both phases were retrospectively reviewed, and short clips were created to evaluate the hemostasis skills. Outcome measures included the time to recognize the bleeding point, the time to complete hemostasis, and the number of coagulation attempts.

RESULTS

Eight cases treated with ESD were reviewed, and 10 video clips of hemostasis were created. The time required to recognize the bleeding point during phase 2 was significantly shorter than that during phase 1 (8.3 ± 4.1 seconds vs 23.1 ± 19.2 seconds; P = 0.049). The time required to complete hemostasis during phase 1 and that during phase 2 were not significantly different (15.4 ± 6.8 seconds vs 31.9 ± 21.7 seconds; P = 0.056). Significantly fewer coagulation attempts were performed during phase 2 (1.8 ± 0.7 vs 3.2 ± 1.0; P = 0.004).

CONCLUSION

Short-term training did not reduce hemostasis completion time but significantly improved bleeding point recognition and reduced coagulation attempts. Learning from the operator's viewpoint can facilitate acquiring hemostasis skills during ESD.

A scoping review of participant reported outcome measures assessed during live and simulated surgical procedures

BACKGROUND

The study aims to describe current use of participant-reported outcome measures (PROM) to assess stress in surgeons that may impact patient and surgeon wellbeing.

METHODS

Medline, Embase, Cochrane library, and clinical trial registries were searched. Articles reporting PROM of stress in live or simulated surgery were included.

RESULTS

Of 10,160 articles screened, 67 were included in the review. PROM tools identified include task load index (TLX), State-Trait Anxiety Inventory (STAI), Dundee Stress State Questionnaire (DSSQ), and single question scales. PROM scores increased from baseline to procedure in single question scales and DSSQ. STAI increased by ​≥ ​1 point in 37-75 ​%. TLX scores were greater at laparoscopy vs robot, single incision laparoscopy and open surgery vs conventional laparoscopy. Residents showed higher mental demand than attendings. Music lowered TLX scores.

CONCLUSIONS

Anxiety, stress, and workload have been measured in surgeons although the most appropriate to monitor surgeon wellbeing is not clear.

Comparison Between Scene-Independent and Scene-Dependent Eye Metrics in Assessing Psychomotor Skills

OBJECTIVE

This study aims to compare the relative sensitivity between scene-independent and scene-dependent eye metrics in assessing trainees' performance in simulated psychomotor tasks.

BACKGROUND

Eye metrics have been extensively studied for skill assessment and training in psychomotor tasks, including aviation, driving, and surgery. These metrics can be categorized as scene-independent or scene-dependent, based on whether predefined areas of interest are considered. There is a paucity of direct comparisons between these metric types, particularly in their ability to assess performance during early training.

METHOD

Thirteen medical students practiced the peg transfer task in the Fundamentals of Laparoscopic Surgery. Scene-independent and scene-dependent eye metrics, completion time, and tool motion metrics were derived from eye-tracking data and task videos. K-means clustering of nine eye metrics identified three groups of practice trials with similar gaze behaviors, corresponding to three performance levels verified by completion time and tool motion metrics. A random forest model using eye metrics estimated classification accuracy and determined the feature importance of the eye metrics.

RESULTS

Scene-dependent eye metrics demonstrated a clearer linear trend with performance levels than scene-independent metrics. The random forest model achieved 88.59% accuracy, identifying the top four predictors of performance as scene-dependent metrics, whereas the two least effective predictors were scene-independent metrics.

CONCLUSION

Scene-dependent eye metrics are overall more sensitive than scene-independent ones for assessing trainee performance in simulated psychomotor tasks.

APPLICATION

The study's findings are significant for advancing eye metrics in psychomotor skill assessment and training, enhancing operator competency, and promoting safe operations.

Surgical Sabermetrics: A Scoping Review of Technology-enhanced Assessment of Nontechnical Skills in the Operating Room

OBJECTIVE

To evaluate the current evidence for surgical sabermetrics: digital methods of assessing surgical nontechnical skills and investigate the implications for enhancing surgical performance.

BACKGROUND

Surgeons need high-quality, objective, and timely feedback to optimize performance and patient safety. Digital tools to assess nontechnical skills have the potential to reduce human bias and aid scalability. However, we do not fully understand which of the myriad of digital metrics of performance assessment have efficacy for surgeons.

METHODS

A systematic review was conducted by searching PubMed, EMBASE, CINAHL, and PSYCINFO databases following PRISMA-ScR guidelines. MeSH terms and keywords included "Assessment," "Surgeons," and "Technology". Eligible studies included a digital assessment of nontechnical skills for surgeons, residents, and/or medical students within an operative context.

RESULTS

From 19,229 articles screened, 81 articles met the inclusion criteria. The studies varied in surgical specialties, settings, and outcome measurements. A total of 122 distinct objective, digital metrics were utilized. Studies digitally measured at least 1 category of surgical nontechnical skill using a single (n=54) or multiple objective measures (n=27). The majority of studies utilized simulation (n=48) over live operative settings (n=32). Surgical Sabermetrics has been demonstrated to be beneficial in measuring cognitive load (n=57), situation awareness (n=24), communication (n=3), teamwork (n=13), and leadership (n=2). No studies measured intraoperative decision-making.

CONCLUSIONS

The literature detailing the intersection between surgical data science and operative nontechnical skills is diverse and growing rapidly. Surgical Sabermetrics may provide a promising modifiable technique to achieve desirable outcomes for both the surgeon and the patient. This study identifies a diverse array of measurements possible with sensor devices and highlights research gaps, including the need for objective assessment of decision-making. Future studies may advance the integration of physiological sensors to provide a holistic assessment of surgical performance.

Eye-tracking in surgery: a systematic review

BACKGROUND

Surgery is constantly evolving with the assistance of rapidly developing novel technology. Eye-tracking devices provide opportunities to monitor the acquisition of surgical skills, gain insight into performance, and enhance surgical practice. The aim of this review was to consolidate the available evidence for the use of eye-tracking in the surgical disciplines.

METHODS

A systematic literature review was conducted in accordance with PRISMA guidelines. A search of OVID Medline, EMBASE, Cochrane library, Scopus, and Science Direct was conducted January 2000 until December 2022. Studies involving eye-tracking in surgical training, assessment and technical innovation were included in the review. Non-surgical procedures, animal studies, and studies not involving surgical participants were excluded from the review.

RESULTS

The search returned a total of 12 054 articles, 80 of which were included in the final analysis and review. Seventeen studies involved eye-tracking in surgical training, 48 surgical assessment, and 20 were focussing on technical aspects of this technology. Twenty-six different eye-tracking devices were used in the included studies. Metrics such as the number of fixations, duration of fixations, dwell time, and cognitive workload were able to differentiate between novice and expert performance. Eight studies demonstrated the effectiveness of gaze-training for improving surgical skill.

CONCLUSION

The current literature shows a broad range of utility for a variety of eye-tracking devices in surgery. There remains a lack of standardization for metric parameters and gaze analysis techniques. Further research is required to validate its use to establish reliability and create uniform practices.

Changes in Radiologists' Gaze Patterns Against Lung X-rays with Different Abnormalities: a Randomized Experiment

The workload of some radiologists increased dramatically in the last several, which resulted in a potentially reduced quality of diagnosis. It was demonstrated that diagnostic accuracy of radiologists significantly reduces at the end of work shifts. The study aims to investigate how radiologists cover chest X-rays with their gaze in the presence of different chest abnormalities and high workload. We designed a randomized experiment to quantitatively assess how radiologists' image reading patterns change with the radiological workload. Four radiologists read chest X-rays on a radiological workstation equipped with an eye-tracker. The lung fields on the X-rays were automatically segmented with U-Net neural network allowing to measure the lung coverage with radiologists' gaze. The images were randomly split so that each image was shown at a different time to a different radiologist. Regression models were fit to the gaze data to calculate the treads in lung coverage for individual radiologists and chest abnormalities. For the study, a database of 400 chest X-rays with reference diagnoses was assembled. The average lung coverage with gaze ranged from 55 to 65% per radiologist. For every 100 X-rays read, the lung coverage reduced from 1.3 to 7.6% for the different radiologists. The coverage reduction trends were consistent for all abnormalities ranging from 3.4% per 100 X-rays for cardiomegaly to 4.1% per 100 X-rays for atelectasis. The more image radiologists read, the smaller part of the lung fields they cover with the gaze. This pattern is very stable for all abnormality types and is not affected by the exact order the abnormalities are viewed by radiologists. The proposed randomized experiment captured and quantified consistent changes in X-ray reading for different lung abnormalities that occur due to high workload.

Telestration with augmented reality improves surgical performance through gaze guidance

BACKGROUND

In minimally invasive surgery (MIS), trainees need to learn how to interpret the operative field displayed on the laparoscopic screen. Experts currently guide trainees mainly verbally during laparoscopic procedures. A newly developed telestration system with augmented reality (iSurgeon) allows the instructor to display hand gestures in real-time on the laparoscopic screen in augmented reality to provide visual expert guidance (telestration). This study analysed the effect of telestration guided instructions on gaze behaviour during MIS training.

METHODS

In a randomized-controlled crossover study, 40 MIS naive medical students performed 8 laparoscopic tasks with telestration or with verbal instructions only. Pupil Core eye-tracking glasses were used to capture the instructor's and trainees' gazes. Gaze behaviour measures for tasks 1-7 were gaze latency, gaze convergence and collaborative gaze convergence. Performance measures included the number of errors in tasks 1-7 and trainee's ratings in structured and standardized performance scores in task 8 (ex vivo porcine laparoscopic cholecystectomy).

RESULTS

There was a significant improvement 1-7 on gaze latency [F(1,39) = 762.5, p < 0.01, ηp2 = 0.95], gaze convergence [F(1,39) = 482.8, p < 0.01, ηp2 = 0.93] and collaborative gaze convergence [F(1,39) = 408.4, p < 0.01, ηp2 = 0.91] upon instruction with iSurgeon. The number of errors was significantly lower in tasks 1-7 (0.18 ± 0.56 vs. 1.94 ± 1.80, p < 0.01) and the score ratings for laparoscopic cholecystectomy were significantly higher with telestration (global OSATS: 29 ± 2.5 vs. 25 ± 5.5, p < 0.01; task-specific OSATS: 60 ± 3 vs. 50 ± 6, p < 0.01).

CONCLUSIONS

Telestration with augmented reality successfully improved surgical performance. The trainee's gaze behaviour was improved by reducing the time from instruction to fixation on targets and leading to a higher convergence of the instructor's and the trainee's gazes. Also, the convergence of trainee's gaze and target areas increased with telestration. This confirms augmented reality-based telestration works by means of gaze guidance in MIS and could be used to improve training outcomes.

Eye-tracking during simulated endotracheal newborn intubation: a prospective, observational multi-center study

BACKGROUND

The aim was to assess health care providers' (HCPs) visual attention (VA) by using eye-tracking glasses during a simulated neonatal intubation.

METHODS

HCPs from three pediatric and neonatal departments (Feldkirch and Vienna, Austria, and Edmonton, Canada) completed a simulated neonatal intubation scenario while wearing eye-tracking glasses (Tobii Pro Glasses 2^®, Tobii, Stockholm, Sweden) to record their VA. Main outcomes included duration of intubation, success rate, and VA. We further compared orotracheal and nasotracheal intubations.

RESULTS

30 participants were included. 50% completed the intubation within 30 s (M = 35.40, SD = 16.01). Mostly nasotracheal intubations exceeded the limit. Experience was an important factor in reducing intubation time. VA differed between more and less experienced HCPs as well as between orotracheal and nasotracheal intubations. Participants also focused on different areas of interest (AOIs) depending on the intubator's experience. More experience was associated with a higher situational awareness (SA) and fewer distractions, which, however, did not transfer to significantly better intubation performance.

CONCLUSION

Half of the intubations exceeded the recommended time limit. Differences in intubation duration depending on type of intubation were revealed. VA differed between HCPs with different levels of experience and depended on duration and type of intubation.

IMPACT

Simulated neonatal intubation duration differs between orotracheal and nasotracheal intubation. Visual attention during simulated neonatal intubation shows differences depending on intubation duration, intubator experience, type of intubation, and level of distraction. Intubator experience is a vital parameter for reducing intubation duration and improving intubator focus on task-relevant stimuli.

Objective psychomotor laparoscopic skills evaluation using a low-cost wearable device based on accelerometry: construct and concurrent validity study

BACKGROUND

Motion analysis of surgical maneuvers provides useful quantitative information for the objective evaluation of the surgeons. However, surgical simulation laboratories for laparoscopic training do not usually integrate devices that help quantify the level of skills of the surgeons due to their limited resources and the high costs of new technologies. The purpose of this study is to present the construct and concurrent validity of a low-cost motion tracking system, based on a wireless triaxial accelerometer, employed to objectively evaluate psychomotor skills of surgeons during laparoscopic training.

METHODS

An accelerometry system, a wireless three-axis accelerometer with appearance of wristwatch, was placed on the dominant hand of the surgeons to register the motion during the laparoscopy practice with the EndoViS simulator, which simultaneously recorded the motion of the laparoscopic needle driver. This study included the participation of 30 surgeons (6 experts, 14 intermediates and 10 novices) who performed the task of intracorporeal knot-tying suture. Using 11 motion analysis parameters (MAPs), the performance of each participant was assessed. Subsequently, the scores of the three groups of surgeons were statistically analyzed. In addition, a validity study was conducted comparing the metrics between the accelerometry-tracking system and the EndoViS hybrid simulator.

RESULTS

Construct validity was achieved for 8 of the 11 metrics examined with the accelerometry system. Concurrent validity demonstrated that there is a strong correlation between the results of the accelerometry system and the EndoViS simulator in 9 of 11 parameters, showing reliability of the accelerometry system as an objective evaluation method.

CONCLUSION

The accelerometry system was successfully validated. This method is potentially useful to complement the objective evaluation of surgeons during laparoscopic practice in training environments such as box-trainers and simulators.

Training and proficiency level in endoscopic sinus surgery change residents' eye movements

Nose surgery is challenging and needs a lot of training for safe and efficient treatments. Eye tracking can provide an objective assessment to measure residents' learning curve. The aim of the current study was to assess residents' fixation duration and other dependent variables over the course of a dedicated training in functional endoscopic sinus surgery (FESS). Sixteen residents performed a FESS training over 18 sessions, split into three surgical steps. Eye movements in terms of percent fixation on the screen and average fixation duration were measured, in addition to residents' completion time, cognitive load, and surgical performance. Results indicated performance improvements in terms of completion time and surgical performance. Cognitive load and average fixation duration showed a significant change within the last step of training. Percent fixation on screen increased within the first step, and then stagnated. Results showed that eye movements and cognitive load differed between residents of different proficiency levels. In conclusion, eye tracking is a helpful objective measuring tool in FESS. It provides additional insights of the training level and changes with increasing performance. Expert-like gaze was obtained after half of the training sessions and increased proficiency in FESS was associated with increased fixation duration.

Eye tracking metrics of orthopedic surgeons with different competency levels who practice simulation-based hip arthroscopic procedures

Objective

This study aimed to investigate the feasibility of using eye tracking data to identify orthopedic trainees' technical proficiency in hip arthroscopic procedures during simulation-based training.

Design

A cross sectional study.

Setting

A simulation-based training session for hip arthroscopy was conducted. Eye tracking devices were used to record participants' eye movements while performing simulated operations. The NASA Task Load Index survey was then used to measure subjective opinions on the perceived workload of the training. Statistical analyses were performed to determine the significance of the eye metrics and survey data.

Participants

A total of 12 arthroscopic trainees, including resident doctors, junior specialist surgeons, and consultant surgeons from the Department of Orthopedics in five hospitals, participated in this study. They were divided into three subgroups based on their prior clinical experience.

Results

Significant differences, including those for dwell time, number of fixations, number of saccades, saccade duration, peak velocity of the saccade, and pupil entropy, were observed among the three subgroups. Additionally, there were clear trends in the perceived workload of the simulation-based training based on feedback from the participants.

Conclusion

Based on this preliminary study, a correlation was identified between the eye tracking metrics and participants' experience levels. Hence, it is feasible to apply eye tracking data as a supplementary objective assessment tool to benchmark the technical proficiency of surgical trainees in hip arthroscopy, and enhance simulation-based training.

Stress in surgical educational environments: a systematic review

BACKGROUND

The effects of stress on surgical residents and how stress management training can prepare residents to effectively manage stressful situations is a relevant topic. This systematic review aimed to analyze the literature regarding (1) the current stress monitoring tools and their use in surgical environments, (2) the current methods in surgical stress management training, and (3) how stress affects surgical performance.

METHODS

A search strategy was implemented to retrieve relevant articles from Web of Science, Scopus, and PubMed. The 787 initially retrieved articles were reviewed for further evaluation according to the inclusion/exclusion criteria (Prospero registration number CRD42021252682).

RESULTS

Sixty-one articles were included in the review. The stress monitoring methods found in the articles showed heart rate analysis as the most used monitoring tool for physiological parameters while the STAI-6 scale was preferred for psychological parameters. The stress management methods found in the articles were mental-, simulation- and feedback-based training, with the mental-based training showing clear positive effects on participants. The studies analyzing the effects of stress on surgical performance showed both negative and positive effects on technical and non-technical performance.

CONCLUSIONS

The impact of stress responses presents an important factor in surgical environments, affecting residents' training and performance. This study identified the main methods used for monitoring stress parameters in surgical educational environments. The applied surgical stress management training methods were diverse and demonstrated positive effects on surgeons' stress levels and performance. There were negative and positive effects of stress on surgical performance, although a collective pattern on their effects was not clear.

Assessment of changes in gaze patterns during training in point-of-care ultrasound

BACKGROUND

Point-of-care ultrasound (POCUS) is a core skill in emergency medicine (EM), however, there is a lack of objective competency measures. Eye-tracking technology is a potentially useful assessment tool, as gaze patterns can reliably discriminate between experts and novices across medical specialties. We aim to determine if gaze metrics change in an independent and predictable manner during ultrasound training.

METHODS

A convenience sample of first-year residents from a single academic emergency department was recruited. Participants interpreted 16 ultrasound videos of the focused assessment with sonography for trauma (FAST) scan while their gaze patterns were recorded using a commercially available eye-tracking device. The intervention group then completed an introductory ultrasound course whereas the control group received no additional education. The gaze assessment was subsequently repeated. The primary outcome was total gaze duration on the area of interest (AOI). Secondary outcomes included time to fixation, mean duration of first fixation and mean number of fixations on the AOI.

RESULTS

10 EM residents in the intervention group and 10 non-EM residents in the control group completed the study. After training, there was an 8.8 s increase in the total gaze time on the AOI in the intervention group compared to a 4.0 s decrease in the control group (p = .03). EM residents were also 3.8 s quicker to fixate on the AOI whereas the control group became 2.5 s slower (p = .04). There were no significant interactions on the number of fixations (0.43 vs. 0.18, p = .65) or duration of first fixation on the AOI (0.02 s vs. 0.06 s, p = .63).

CONCLUSIONS

There are significant and quantifiable changes in gaze metrics, which occur with incremental learning after an ultrasound course. Further research is needed to validate the serial use of eye-tracking technology in following a learner's progress toward competency in point-of-care ultrasound image interpretation.

Visual gaze patterns in trainee endoscopists - a novel assessment tool

BACKGROUND

Colonoscopy proficiency is significantly influenced by skills achieved during training. Although assessment scores exist, they do not evaluate the impact of visual search strategies and their use is time and labour intensive. Eye-tracking has shown significant differences in visual gaze patterns (VGPs) between expert endoscopists with varying polyp detection rates, so may provide a means of automated assessment and guidance for trainees. This study aimed to assess the feasibility of eye-tracking as a novel assessment method for trainee endoscopists.

METHODS

Eye-tracking glasses were used to record 26 colonoscopies from 12 endoscopy trainees who were assessed with directly observed procedural scores (DOPS), devised by the Joint Advisory Group (JAG) on GI endoscopy, and a visual analogue score of overall competence. A 'total weighted procedure score' (TWPS) was calculated from 1 to 20. Primary outcomes of fixation duration (FixD) and fixation frequency (FixF) were analysed according to areas of interest (AOIs) with the bowel surface and lumen represented by three concentric rings. Correlation was assessed using Pearson's coefficient. Significance was set at p<.050.

RESULTS

Trainees displayed a significant positive correlation between TWPS and FixD (R = 0.943, p<.0001) and FixF (R = 0.936, p<.0001) in the anatomical bowel mucosa peripheries. Conversely, they had significant negative correlations between TWPS and the anatomical bowel lumen (FixD: R= -0.546, p=.004; FixF: R= -0.568, p=.002).

CONCLUSIONS

Higher objective performance scores were associated with VGPs focussing on bowel mucosa. This is consistent with prior analysis showing peripheral VGPs correspond with higher polyp detection rates. Analysis of VGPs, therefore, has potential for training and assessment in colonoscopy.

The Measurement of Cognitive Workload in Surgery Using Pupil Metrics: A Systematic Review and Narrative Analysis

INTRODUCTION

Increased cognitive workload (CWL) is a well-established entity that can impair surgical performance and increase the likelihood of surgical error. The use of pupil and gaze tracking data is increasingly being used to measure CWL objectively in surgery. The aim of this review is to summarize and synthesize the existing evidence that surrounds this.

METHODS

A systematic review was undertaken in accordance with Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. A search of OVID MEDLINE, IEEE Xplore, Web of Science, Google Scholar, APA PsychINFO, and EMBASE was conducted for articles published in English between 1990 and January 2021. In total, 6791 articles were screened and 32 full-text articles were selected based on the inclusion criteria. A narrative analysis was undertaken in view of the heterogeneity of studies.

RESULTS

Seventy-eight percent of selected studies were deemed high quality. The most frequent surgical environment and task studied was surgical simulation (75%) and performance of laparoscopic skills (56%) respectively. The results demonstrated that the current literature can be broadly categorized into pupil, blink, and gaze metrics used in the assessment of CWL. These can be further categorized according to their use in the context of CWL: (1) direct measurement of CWL (n = 16), (2) determination of expertise level (n = 14), and (3) predictors of performance (n = 2).

CONCLUSIONS

Eye-tracking data provide a wealth of information; however, there is marked study heterogeneity. Pupil diameter and gaze entropy demonstrate promise in CWL assessment. Future work will entail the use of artificial intelligence in the form of deep learning and the use of a multisensor platform to accurately measure CWL.

Eye Tracking Use in Surgical Research: A Systematic Review

INTRODUCTION

Eye tracking (ET) is a popular tool to study what factors affect the visual behaviour of surgical team members. To our knowledge, there have been no reviews to date that evaluate the broad use of ET in surgical research. This review aims to identify and assess the quality of this evidence, to synthesize how ET can be used to inform surgical practice, and to provide recommendations to improve future ET surgical studies.

METHODS

In line with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, a systematic literature review was conducted. An electronic search was performed in MEDLINE, Cochrane Central, Embase, and Web of Science databases up to September 2020. Included studies used ET to measure the visual behaviour of members of the surgical team during surgery or surgical tasks. The included studies were assessed by two independent reviewers.

RESULTS

A total of 7614 studies were identified, and 111 were included for data extraction. Eleven applications were identified; the four most common were skill assessment (41%), visual attention assessment (22%), workload measurement (17%), and skills training (10%). A summary was provided of the various ways ET could be used to inform surgical practice, and three areas were identified for the improvement of future ET studies in surgery.

CONCLUSIONS

This review provided a comprehensive summary of the various applications of ET in surgery and how ET could be used to inform surgical practice, including how to use ET to improve surgical education. The information provided in this review can also aid in the design and conduct of future ET surgical studies.

Methods and Measures for Mental Stress Assessment in Surgery: A Systematic Review of 20 Years of Literature

Real-time mental stress monitoring from surgeons and surgical staff in operating rooms may reduce surgical injuries, improve performance and quality of medical care, and accelerate implementation of stress-management strategies. Motivated by the increase in usage of objective and subjective metrics for cognitive monitoring and by the gap in reviews of experimental design setups and data analytics, a systematic review of 71 studies on mental stress and workload measurement in surgical settings, published in 2001-2020, is presented. Almost 61% of selected papers used both objective and subjective measures, followed by 25% that only administered subjective tools - mostly consisting of validated instruments and customized surveys. An overall increase in the total number of publications on intraoperative stress assessment was observed from mid-2010 s along with a momentum in the use of both subjective and real-time objective measures. Cardiac activity, including heart-rate variability metrics, stress hormones, and eye-tracking metrics were the most frequently and electroencephalography (EEG) was the least frequently used objective measures. Around 40% of selected papers collected at least two objective measures, 41% used wearable devices, 23% performed synchronization and annotation, and 76% conducted baseline or multi-point data acquisition. Furthermore, 93% used a variety of statistical techniques, 14% applied regression models, and only one study released a public, anonymized dataset. This review of data modalities, experimental setups, and analysis techniques for intraoperative stress monitoring highlights the initiatives of surgical data science and motivates research on computational techniques for mental and surgical skills assessment and cognition-guided surgery.

Eye-Tracking Indicators of Workload in Surgery: A Systematic Review

BackgroundEye tracking is a powerful tool for unobtrusive and real time assessment of workload in clinical settings. Before the complex eye tracking derived surrogates can be proactively utilized to improve surgical safety, the indications, validity and reliability requires careful evaluation.MethodsWe conducted a systematic review of literature from 2010 to 2020 according to PRISMA guidelines. A search on PubMed, Cochrane, Scopus, Web of science, PsycInfo and Google scholar databases was conducted on July 2020. The following search query was used" ("eye tracking" OR "gaze tracking") AND (surgery OR surgical OR operative OR intraoperative) AND (workload OR stress)". Short papers, no peer reviewed or papers in which eye-tracking methodology was not used to investigate workload or stress factors in surgery, were omitted.ResultsA total of 17 (N = 17) studies were identified eligible to this review. Most of the studies (n = 15) measured workload in simulated setting. Task difficulty and expertise were the most studied factors. Studies consistently showed surgeon's eye movements such as pupil responses, gaze patterns, blinks were associated with the level of perceived workload. However, differences between measurements in operational room and simulated environments have been found.ConclusionPupil responses, blink rate and gaze indices are valid indicators of workload. However, the effect of distractions and non-technical factors on workload is underrepresented aspect in the literature even though recognized as underlying factors in successful surgery.

Brain substrates for automatic retrieval of value memory in the primate basal ganglia

Our behavior is often carried out automatically. Automatic behavior can be guided by past experiences, such as learned values associated with objects. Passive-viewing and free-viewing tasks with no immediate outcomes provide a testable condition in which monkeys and humans automatically retrieve value memories and perform habitual searching. Interestingly, in these tasks, caudal regions of the basal ganglia structures are involved in automatic retrieval of learned object values and habitual gaze. In contrast, rostral regions do not participate in these activities but instead monitor the changes in outcomes. These findings indicate that automatic behaviors based on the value memories are processed selectively by the caudal regions of the primate basal ganglia system. Understanding the distinct roles of the caudal basal ganglia may provide insight into finding selective causes of behavioral disorders in basal ganglia disease.

The Influence of Experience on Gazing Patterns during Endovascular Treatment: Eye-Tracking Study

Objective

In various fields, differences in eye-gazing patterns during tasks between experts and novices have been evaluated. The aim of this study was to investigate gazing patterns during neuro-endovascular treatment using an eye-tracking device and assess whether gazing patterns depend on the physician's experience or skill.

Methods

Seven physicians performed coil embolization for a cerebral aneurysm in a silicone vessel model under biplane X-ray fluoroscopy, and their gazing patterns were recorded using an eye-tracking device. The subjects were divided into three groups according to experience, highly experienced (Expert) group, intermediately experienced (Trainee) group, and less experienced (Novice) group. The duration of fixation on the anterior-posterior (AP) view screen, lateral (LR) view, and out-of-screen were compared between each group.

Results

During microcatheter navigation, the Expert and Trainee groups spent a long time on fixation to AP, while the Novice group split their attention between each location. In coil insertion, the Expert group gazed at both the AP and the LR views with more saccades between screens. In contrast, the Trainee group spent most of their time only on the AP view screen and the Novice group spent longer out-of-screen.

Conclusion

An eye-tracking device can detect different gazing patterns among physicians with several experiences and skill levels of neuroendovascular treatment. Learning the gazing patterns of experts using eye tracking may be a good educational tool for novices and trainees.

An algorithmic approach to determine expertise development using object-related gaze pattern sequences

Eye tracking (ET) technology is increasingly utilized to quantify visual behavior in the study of the development of domain-specific expertise. However, the identification and measurement of distinct gaze patterns using traditional ET metrics has been challenging, and the insights gained shown to be inconclusive about the nature of expert gaze behavior. In this article, we introduce an algorithmic approach for the extraction of object-related gaze sequences and determine task-related expertise by investigating the development of gaze sequence patterns during a multi-trial study of a simplified airplane assembly task. We demonstrate the algorithm in a study where novice (n = 28) and expert (n = 2) eye movements were recorded in successive trials (n = 8), allowing us to verify whether similar patterns develop with increasing expertise. In the proposed approach, AOI sequences were transformed to string representation and processed using the k-mer method, a well-known method from the field of computational biology. Our results for expertise development suggest that basic tendencies are visible in traditional ET metrics, such as the fixation duration, but are much more evident for k-mers of k > 2. With increased on-task experience, the appearance of expert k-mer patterns in novice gaze sequences was shown to increase significantly (p < 0.001). The results illustrate that the multi-trial k-mer approach is suitable for revealing specific cognitive processes and can quantify learning progress using gaze patterns that include both spatial and temporal information, which could provide a valuable tool for novice training and expert assessment.

Machine learning-based analysis of operator pupillary response to assess cognitive workload in clinical ultrasound imaging

Introduction

Pupillometry, the measurement of eye pupil diameter, is a well-established and objective modality correlated with cognitive workload. In this paper, we analyse the pupillary response of ultrasound imaging operators to assess their cognitive workload, captured while they undertake routine fetal ultrasound examinations. Our experiments and analysis are performed on real-world datasets obtained using remote eye-tracking under natural clinical environmental conditions.

Methods

Our analysis pipeline involves careful temporal sequence (time-series) extraction by retrospectively matching the pupil diameter data with tasks captured in the corresponding ultrasound scan video in a multi-modal data acquisition setup. This is followed by the pupil diameter pre-processing and the calculation of pupillary response sequences. Exploratory statistical analysis of the operator pupillary responses and comparisons of the distributions between ultrasonographic tasks (fetal heart versus fetal brain) and operator expertise (newly-qualified versus experienced operators) are performed. Machine learning is explored to automatically classify the temporal sequences into the corresponding ultrasonographic tasks and operator experience using temporal, spectral, and time-frequency features with classical (shallow) models, and convolutional neural networks as deep learning models.

Results

Preliminary statistical analysis of the extracted pupillary response shows a significant variation for different ultrasonographic tasks and operator expertise, suggesting different extents of cognitive workload in each case, as measured by pupillometry. The best-performing machine learning models achieve receiver operating characteristic (ROC) area under curve (AUC) values of 0.98 and 0.80, for ultrasonographic task classification and operator experience classification, respectively.

Conclusion

We conclude that we can successfully assess cognitive workload from pupil diameter changes measured while ultrasound operators perform routine scans. The machine learning allows the discrimination of the undertaken ultrasonographic tasks and scanning expertise using the pupillary response sequences as an index of the operators’ cognitive workload. A high cognitive workload can reduce operator efficiency and constrain their decision-making, hence, the ability to objectively assess cognitive workload is a first step towards understanding these effects on operator performance in biomedical applications such as medical imaging.

•

Machine learning-based pupillary response analysis is performed to assess operator cognitive workload in clinical ultrasound.

•

A systematic multi-modal data analysis pipeline is proposed using eye-tracking, pupillometry, and sonography data science.

•

Pertinent challenges of natural or real-world clinical datasets are addressed.

•

Pupillary responses around event triggers, different ultrasonographic tasks, and different operator experiences are studied.

•

Machine learning models are learnt to classify undertaken tasks or operator expertise from pupillometric time-series data.

Using Eye Tracking for Measuring Cognitive Workload During Clinical Simulations: Literature Review and Synthesis

High-fidelity clinical simulations can be used by clinicians to acquire technical (physical ability and knowledge) and non-technical (cognitive and social processes) skills. Excessive cognitive workload contributes to medical errors because of the impact on both technical and non-technical skills. Many studies measure cognitive workload with psychometric instruments that limit the assessment of cognitive workload to a single time period and may involve response bias. Using eye tracking to measure task-evoked pupillary responses allows the measurement of changes in pupil diameter related to the cognitive workload associated with a specific activity. Incorporating eye tracking with high-fidelity clinical simulations provides a reliable and continuous assessment of cognitive workload. The purpose of this literature review is to summarize the use of eye-tracking technology to measure cognitive workload of healthcare providers to generate evidence-based guidelines for measuring cognitive workload during high-fidelity clinical simulations. What this manuscript adds to the body of literature is a summary of best practices related to the different methods of measuring cognitive workload, benefits and limitations of using eye tracking, and high-fidelity clinical simulation design considerations for successful integration of eye tracking.

Exploring Eye-tracking Technology as an Assessment Tool for Point-of-care Ultrasound Training

OBJECTIVES

Eye-tracking technology has emerged as a potentially useful learner assessment tool in several medical specialties. In the fields of general surgery and anesthesiology, it has been shown to reliably differentiate between different levels of expertise in procedural skills. In the field of radiology, it has been shown to be a valid assessment tool for diagnostic test interpretation. Current methods of competency assessment in point-of-care ultrasound (POCUS) remain a challenge, because they require significant direct observation time by an instructor. The purpose of this study was to determine if eye-tracking technology can accurately distinguish between novice and experts in the interpretation of POCUS clips, specifically of the focused assessment using sonography in trauma (FAST) scan.

METHODS

A convenience sample of medical students, residents, and emergency physicians from a single academic emergency department were invited to participate. Participants included both novices and experts in POCUS. Each participant completed a baseline questionnaire and viewed 16 video clips of a FAST ultrasound examination while their gaze patterns were recorded by a commercially available eye-tracking device. The primary outcome was total gaze time on the area of interest (AOI). Secondary outcomes included total time to fixation, mean number of fixations, and mean duration of first fixation on the AOI.

RESULTS

Fifteen novices and 15 experts completed this study. For total gaze time on the AOI, experts fixated their gaze significantly longer than novices (75.8 ± 16.2 seconds vs. 56.6 ± 12.8 seconds, p = 0.001). Similarly, experts were significantly faster to fixate on the AOI and had a higher fixation count on the AOI (8.5 ± 4.0 seconds vs. 15.1 ± 6.8 seconds, p = 0.003; and 170 ± 30 vs. 143 ± 28 seconds, p = 0.016). There were no differences on the mean duration of first fixation on the AOI (0.42 ± 0.12 seconds vs. 0.39 ± 0.09 seconds, p = 0.467).

CONCLUSION

Eye-tracking technology shows the potential to differentiate between experts and novices by their gaze patterns on video clips of FAST examinations. The total gaze time on the AOI may be a useful metric to help in the assessment of competency in POCUS image interpretation. In addition, the evaluation of gaze patterns may help educators identify causes of interpretation errors. Future studies are needed to further validate these metrics in a larger cohort.

Eye Tracking Supported Human Factors Testing Improving Patient Training

The handling of left ventricular assist devices (LVADs) can be challenging for patients and requires appropriate training. The devices' usability impacts patients' safety and quality of life. In this study, an eye tracking supported human factors testing was performed to reveal problems during use and test the trainings' effectiveness. In total 32 HeartWare HVAD patients (including 6 pre-VAD patients) and 3 technical experts as control group performed a battery change (BC) and a controller change (CC) as an everyday and emergency scenario on a training device. By tracking the patients' gaze point, task duration and pump-off time were evaluated. Patients with LVAD support ≥1 year showed significantly shorter BC task duration than patients with LVAD support <1 year (p = 0.008). In contrast their CC task duration (p = 0.002) and pump-off times (median = 12.35 s) were higher than for LVAD support patients <1 year (median = 5.3 s) with p = 0.001. The shorter BC task duration for patients with LVAD support ≥1 year indicate that with time patients establish routines and gain confidence using their device. The opposite effect was found for CC task duration and pump-off times. This implies the need for intermittent re-training of less frequent tasks to increase patients' safety.

Comparing the Visual Perception According to the Performance Using the Eye-Tracking Technology in High-Fidelity Simulation Settings

Introduction: We used eye-tracking technology to explore the visual perception of clinicians during a high-fidelity simulation scenario. We hypothesized that physicians who were able to successfully manage a critical situation would have a different visual focus compared to those who failed. Methods: A convenience sample of 18 first-year emergency medicine residents were enrolled voluntarily to participate in a high-fidelity scenario involving a patient in shock with a 3rd degree atrioventricular block. Their performance was rated as pass or fail and depended on the proper use of the pacing unit. Participants were wearing pre-calibrated eye-tracking glasses throughout the 9-min scenario and infrared (IR) markers installed in the simulator were used to define various Areas of Interest (AOI). Total View Duration (TVD) and Time to First Fixation (TFF) by the participants were recorded for each AOI and the results were used to produce heat maps. Results: Twelve residents succeeded while six failed the scenario. The TVD for the AOI containing the pacing unit was significantly shorter (median [quartile]) for those who succeeded compared to the ones who failed (42 [31–52] sec vs. 70 [61–90] sec, p = 0.0097). The TFF for the AOI containing the ECG and vital signs monitor was also shorter for the participants who succeeded than for those who failed (22 [6–28] sec vs. 30 [27–77] sec, p = 0.0182). Discussion: There seemed to be a connection between the gaze pattern of residents in a high-fidelity bradycardia simulation and their performance. The participants who succeeded looked at the monitor earlier (diagnosis). They also spent less time fixating the pacing unit, using it promptly to address the bradycardia. This study suggests that eye-tracking technology could be used to explore how visual perception, a key information-gathering element, is tied to decision-making and clinical performance.

Eye tracking metrics and leader's behavioral performance during a post-partum hemorrhage high-fidelity simulated scenario

BACKGROUND

The use of eye tracking in the simulated setting can help improve our understanding of what sources of information clinicians are using as they deliver routine patient care. The aim of this simulation study was to observe the differences, if any, between the eye tracking patterns of leaders who performed best in a simulated postpartum hemorrhage (PPH) high-fidelity scenario, in comparison with those who performed worst.

METHODS

Forty anesthesia trainees from the University of Catania Medical School were divided into eight teams, to enact four times the same scenario of a patient with postpartum hemorrhage following vaginal delivery. Trainees who were assigned the leader's role wore the eye tracking glasses during the scenario, and their behavioral skills were evaluated by two observers, who reviewed the video recordings of the scenarios using a standardized checklist. The leader's eye tracking metrics, extracted from 27 selected areas of interest (AOI), were recorded by a Tobii Pro Glasses 50 Hz wearable wireless eye tracker. Team performance was evaluated using a PPH checklist. After completion of the study, the leaders were divided into two groups, based on the scores they had received (High-Performance Leader group, HPL, and Low-Performance Leader group, LPL).

RESULTS

In the HPL group, the duration and number of fixations were greater, and the distribution of gaze was uniformly distributed among the various members of the team as compared with the LPL group (with the exception of the participant who performed the role of the obstetrician). The HPL group also looked both at the patient's face and established eye contact with their team members more often and for longer (P < .05). The team performance (PPH checklist) score was greater in the HPL group (P < .001). The LPL group had more and/or longer fixations of technical areas of interest (P < .05).

CONCLUSIONS

Our findings suggest that the leaders who perform the best distribute their gaze across all members of their team and establish direct eye contact. They also look longer at the patient's face and dwell less on areas that are more relevant to technical skills. In addition, the teams led by these best performing leaders fulfilled their clinical task better. The information provided by the eye behaviors of "better-performing physicians" may lay the foundation for the future development of both the assessment process and the educational tools used in simulation.

TRIAL REGISTRATION

Clinical.Trial.Gov ID n. NCT04395963 .

See Like an Expert: Gaze-Augmented Training Enhances Skill Acquisition in a Virtual Reality Robotic Suturing Task

Introduction: The da Vinci Skills Simulator (DVSS) is an effective platform for robotic skills training. Novel training methods using expert gaze patterns to guide trainees have demonstrated superiority to traditional instruction. Portable head-mounted eye-trackers (HMET) offer the opportunity for eye tracking technology to enhance surgical robotic simulation training. Objective: To evaluate if training guided by expert gaze patterns can improve trainee performance over standard movement training techniques during robotic simulation. Methods: Medical students were recruited and randomized into gaze training (GT, n = 9) and movement training (MT, n = 8) groups. First, the participants reviewed an instructional video, with the GT group emulating expert gaze patterns and the MT group (n = 8) standard movement-based instruction. Training consisted of 10 repetitions of "Suture Sponge 3" on the DVSS while wearing HMET; the first three repetitions were followed by group-appropriate video coaching (gaze vs movement feedback), while the remaining repetitions were without feedback. Finally, two multitasking repetitions with a secondary bell-counting task were completed. Primary outcomes included DVSS scores during training and multitasking. Secondary outcomes included metrics collected from the HMET (gaze patterns and gaze entropy). Results: Total score, efficiency, and penalties improved significantly over the training in both groups; the GT group achieved higher scores on every attempt. Total scores in the GT group were higher than the MT group postvideo review (20.3 ± 21.8 vs 3.0 ± 6.2, p = 0.047), after coaching repetitions (61.8 ± 18.8 vs 30.1 ± 26.2, p = 0.01), and at the last training attempt (73.0 ± 16.5 vs 63.1 ± 17.4, p = 0.247). During multitasking, the GT group maintained higher total scores (75 ± 10.1 vs 63.3 ± 15.3, p = 0.01), efficiency (86.3 ± 7.4 vs 77.4 ± 11.2, p = 0.009), and superior secondary task performance (error: 6.3% ± 0.06 vs 10.7% ± 0.11, p = 0.20). Gaze entropy (cognitive-load indicator) and gaze pattern analysis showed similar trends. Conclusion: Gaze-augmented training leads to more efficient movements through adoption of expert gaze patterns that withstand additional stressors.

Objective and automated assessment of surgical technical skills with IoT systems: A systematic literature review

The assessment of surgical technical skills to be acquired by novice surgeons has been traditionally done by an expert surgeon and is therefore of a subjective nature. Nevertheless, the recent advances on IoT (Internet of Things), the possibility of incorporating sensors into objects and environments in order to collect large amounts of data, and the progress on machine learning are facilitating a more objective and automated assessment of surgical technical skills. This paper presents a systematic literature review of papers published after 2013 discussing the objective and automated assessment of surgical technical skills. 101 out of an initial list of 537 papers were analyzed to identify: 1) the sensors used; 2) the data collected by these sensors and the relationship between these data, surgical technical skills and surgeons' levels of expertise; 3) the statistical methods and algorithms used to process these data; and 4) the feedback provided based on the outputs of these statistical methods and algorithms. Particularly, 1) mechanical and electromagnetic sensors are widely used for tool tracking, while inertial measurement units are widely used for body tracking; 2) path length, number of sub-movements, smoothness, fixation, saccade and total time are the main indicators obtained from raw data and serve to assess surgical technical skills such as economy, efficiency, hand tremor, or mind control, and distinguish between two or three levels of expertise (novice/intermediate/advanced surgeons); 3) SVM (Support Vector Machines) and Neural Networks are the preferred statistical methods and algorithms for processing the data collected, while new opportunities are opened up to combine various algorithms and use deep learning; and 4) feedback is provided by matching performance indicators and a lexicon of words and visualizations, although there is considerable room for research in the context of feedback and visualizations, taking, for example, ideas from learning analytics.

Patient-specific mental rehearsal with three-dimensional models before low anterior resection: randomized clinical trial

Background

It was hypothesized that preparing for a surgical procedure, taking into account individual patient characteristics, may facilitate the procedure and improve surgical quality. The aim of this study was to compare different case-specific, preoperative mental rehearsal methods before minimally invasive rectal cancer surgery.

Methods

In this RCT, patients were allocated in a 1 : 1 : 1 : 1 ratio to four groups: systematic mental rehearsal (SMR) using MRI scans; SMR and three-dimensional (3D) virtual models; SMR and synthetic 3D printed models; and routine practice (control group). Surgeons operating on all but the control group underwent mental rehearsal with the visual aids, including axial MRI scans of the pelvis, interactive 3D virtual models reconstructed from axial MRIs, and synthetic models, manufactured by 3D printing. Operations were video-recorded and assessed by two experts blinded to allocation using two validated scores, the Competency Assessment Tool (CAT) and Objective Clinical Human Reliability Analysis (OCHRA). The primary outcome of the study was surgical performance, measured by the CAT.

Results

Forty-nine patients were randomized and allocated to the four groups. There were 12 participants in each of the control, MRI and SMR, and virtual and SMR groups, whereas the SMR using physical models and simulation group included 13. No difference was observed between groups in median CAT scores (control 30.50, MRI 34.25, virtual 31.75, physical 34.00; P = 0.748, partial η² <0.001, where _pη² is indicative of effect size) or OCHRA scores (anterior, posterior, right and left lateral planes, transection P>0.200, _pη² =0.052–0.088). Time spent not performing dissection was significantly shorter for the SMR with MRI group than for the control (57.5 versus 42 respectively; P < 0.001, _pη² =0.212).

Conclusion

Mental rehearsal did not affect CAT and OCHRA scores of consultant surgeons. Reference number: ISRCTN 75603704 (https://www.isrctn.com).

Distinguishing Intermediate and Novice Surgeons by Eye Movements

Surgical skill-level assessment is key to collecting the required feedback and adapting the educational programs accordingly. Currently, these assessments for the minimal invasive surgery programs are primarily based on subjective methods, and there is no consensus on skill level classifications. One of the most detailed of these classifications categorize skill levels as beginner, novice, intermediate, sub-expert, and expert. To properly integrate skill assessment into minimal invasive surgical education programs and provide skill-based training alternatives, it is necessary to classify the skill levels in as detailed a way as possible and identify the differences between all skill levels in an objective manner. Yet, despite the existence of very encouraging results in the literature, most of the studies have been conducted to better understand the differences between novice and expert surgical skill levels leaving out the other crucial skill levels between them. Additionally, there are very limited studies by considering the eye-movement behaviors of surgical residents. To this end, the present study attempted to distinguish novice- and intermediate-level surgical residents based on their eye movements. The eye-movement data was recorded from 23 volunteer surgical residents while they were performing four computer-based simulated surgical tasks under different hand conditions. The data was analyzed using logistic regression to estimate the skill levels of both groups. The best results of the estimation revealing a 91.3% recognition rate of predicting novice and intermediate surgical residents on one scenario were selected from four under the dominant hand condition. These results show that the eye-movements can be potentially used to identify surgeons with intermediate and novice skills. However, the results also indicate that the order in which the scenarios are provided, and the design of the scenario, the tasks, and their appropriateness with the skill levels of the participants are all critical factors to be considered in improving the estimation ratio, and hence require thorough assessment for future research.

The Quiet Eye and Expertise: Sustained Fixations Do Not Transfer to Unpracticed Throws Among Expert Dart Players

The length of the last visual fixation before the critical final phase of a movement-the quiet eye (QE) fixation-is positively correlated with expertise and success. The present study tested the potential for intraskill transfer of QE durations in order to determine whether it is intrinsically linked to expertise development or is a separable skill that may be employed to improve performance under novel circumstances. The authors tracked highly skilled dart throwers' gazes while they executed familiar (highly practiced) and familiar yet novel (distance/effector-modified) sport-specific actions. QE duration was significantly reduced when performing in unfamiliar conditions, suggesting that QE does not transfer to atypical conditions and may therefore be a result of-rather than a contributor to-expertise development. These results imply that intraskill transfer of QE is limited and, consistent with the inhibition hypothesis of QE development, argue against the value of teaching QE as an independent means of improving performance.

Combined Gaze Metrics as Stress-Sensitive Indicators of Microsurgical Proficiency

Background. Evaluation of microsurgical proficiency is conventionally subjective, time consuming, and unreliable. Eye movement–based metrics have been promising not only in detection of surgical expertise but also in identifying actual cognitive stress and workload. We investigated if pupil dilations and blinks could be utilized in parallel to accurately classify microsurgical proficiency and its moderating features, especially task-related stress. Methods. Participants (n = 11) were divided into groups based on prior experience in microsurgery: novices (n = 6) with no experience and trained microsurgeons (n = 5). All participants conducted standardized suturing tasks with authentic instruments and a surgical microscope. A support vector machine classifier was used to classify features of microsurgical expertise based on percentage changes in pupil size. Results. A total of 109 successful sutures with 1090 segments were recorded. Classification of expertise from sutures achieved accuracies between 74.3% and 76.0%. Classification from individual segments based on these same features was not feasible. Conclusions. Combined gaze metrics are applicable for classifying surgical proficiency during a defined task. Pupil dilation is also sensitive to external stress factors; however, the usefulness of blinks is impaired by low blink rates. The results can be translated to surgical education to improve feedback and should be investigated individually in the context of actual performance and in real patient operations. Combined gaze metrics may be ultimately utilized to help microsurgeons monitor their performance and workload in real time—which may lead to prevention of errors.

Feasibility and Utility of an Eye-Tracking Device for Assessing Teachers of Invasive Bedside Procedures

Patient-related complications from invasive bedside procedures (IBPs) are attributed to the experience and proficiency of the operator. Furthermore, IBP complications by trainees may be due to practice variability and competency among IBP teachers. The use of gaze metrics technology to better understand the behaviors of IBP teachers may aid in the creation of faculty development checklists and, ultimately, reduce procedural complications. Prior research on gaze patterns has focused on the individual performing the procedure, but the goal of this pilot study was to assess gaze behaviors of supervising teachers of IBPs, which is a paradigm shift within procedural education. In this study, pulmonary and critical care medicine fellows placed a central venous catheter on a simulated task trainer as pulmonary and critical care medicine faculty supervised while wearing an eye-tracking device. Both quantitative and qualitative data were obtained. Gaze analysis was divided into 2 areas of interest (ultrasonography and procedure site) and 3 procedural tasks (venous puncture, dilation, and flushing the line). Study findings included the following: (1) calibration was easy and took seconds to complete, (2) the device is relatively comfortable and did not interfere with tasks, (3) a trend toward a higher fixation frequency and dwell time on the ultrasound images during the puncture segment, and (4) variations in fixation frequency on the ultrasound images among supervising IBP teachers. This study documents the feasibility of the eye-tracking device for assessing behaviors of supervisory IBP teachers. There may be a signal suggesting differences in gaze patterns among supervisory teachers, which warrants further study.

Pupil diameter differentiates expertise in dental radiography visual search

Expert behavior is characterized by rapid information processing abilities, dependent on more structured schemata in long-term memory designated for their domain-specific tasks. From this understanding, expertise can effectively reduce cognitive load on a domain-specific task. However, certain tasks could still evoke different gradations of load even for an expert, e.g., when having to detect subtle anomalies in dental radiographs. Our aim was to measure pupil diameter response to anomalies of varying levels of difficulty in expert and student dentists’ visual examination of panoramic radiographs. We found that students’ pupil diameter dilated significantly from baseline compared to experts, but anomaly difficulty had no effect on pupillary response. In contrast, experts’ pupil diameter responded to varying levels of anomaly difficulty, where more difficult anomalies evoked greater pupil dilation from baseline. Experts thus showed proportional pupillary response indicative of increasing cognitive load with increasingly difficult anomalies, whereas students showed pupillary response indicative of higher cognitive load for all anomalies when compared to experts.

Joint Attention in a Laparoscopic Simulation-Based Training: A Pilot Study on Camera Work, Gaze Behavior, and Surgical Performance in Laparoscopic Surgery

Introduction: Training in laparoscopic surgery seems to be an important aspect in gaining and maintaining professional competency. In experimental settings, camera navigation skills improved after simulation-based training, but the effect of camera work on the surgeon's performance is not well studied. The aim of this study was to investigate whether a fixed camera or an operated camera, as well as the experience of the camera operator has an effect on the performance of the surgeon. Materials and Methods: The study was performed on the LapSim laparoscopic training system. The task was to tie an intracorporal knot in a static surgical environment with three different camera conditions: fixed camera, camera operated by an inexperienced person (inexpert camera), and camera operated by an experienced surgeon (expert camera). The camera conditions were counterbalanced across trials. Performance variables were completion time in seconds and the extend of movements in path length and angular pathway. Gaze behavior was measured with eye-tracking glasses worn by the surgeon as well as the camera operator and was evaluated for performance-harming effects. Results: Completion time varied across conditions, with participants performing significantly longer in the fixed camera condition than in the expert or the inexpert condition. The expert and inexpert conditions did not differ. The performance-harming effect of non-focusing on the tissue was especially visible in the fixed camera condition but disappeared in the expert camera condition. Neither the camera operators' gaze behavior nor the surgeon-camera operator fixation agreement predicted task performance. Discussion: A camera operator can potentially eliminate performance-harming effects of maladaptive gaze behavior and promote optimal visual behavior of a surgeon. In our experimental task, there was no significant difference in whether the camera operator had previous training in laparoscopic surgery or not.

Eye-tracking during simulation-based neonatal airway management

BACKGROUND

Eye-tracking devices help to understand provider behavior during medical tasks. The aim of this study was to assess participants' gaze behavior and usability of eye-tracking glasses during airway management in a simulated neonatal resuscitation.

METHODS

This study was an observational simulation-based study. The team member assigned to airway management wore head-mounted eye-tracking glasses. Main outcome measures were airway providers' gaze, dwell time (total amount of time a participant fixates certain areas of interest), and usability of eye-tracking glasses.

RESULTS

Data from 13 participants were included. There were significant differences in dwell time during the scenario (p < 0.001), with participants spending twice as much time on the newborn and instruments as on the monitor and other staff. Participants spent about 25% more time focusing on another provider while the provider was inserting the umbilical vein catheter than in all other times of interest (intervals of time with meaningful events) (p = 0.04). The use of the glasses was perceived easy and not disturbing.

CONCLUSIONS

Eye-tracking glasses enhance our understanding of providers' gaze and perspective during simulated neonatal airway management. Future studies will better characterize the ideal use in real situations.

Entropy-Based Effect Evaluation of Delineators in Tunnels on Drivers' Gaze Behavior

Driving safety in tunnels has always been an issue of great concern. Establishing delineators to improve drivers’ instantaneous cognition of the surrounding environment in tunnels can effectively enhance driver safety. Through a simulation study, this paper explored how delineators affect drivers’ gaze behavior (including fixation and scanpath) in tunnels. In addition to analyzing typical parameters, such as fixation position and fixation duration in areas of interest (AOIs), by modeling drivers’ switching process as Markov chains and calculating Shannon’s entropy of the fit Markov model, this paper quantified the complexity of individual switching patterns between AOIs under different delineator configurations and with different road alignments. A total of 25 subjects participated in this research. The results show that setting delineators in tunnels can attract drivers’ attention and make them focus on the pavement. When driving in tunnels equipped with delineators, especially tunnels with both wall delineators and pavement delineators, the participants exhibited a smaller transition entropy Ht and stationary entropy Hs, which can greatly reduce drivers’ visual fatigue. Compared with left curve and right curve, participants obtained higher Ht and Hs values in the straight section.

Visual behaviour in robotic surgery-Demonstrating the validity of the simulated environment

BACKGROUND

Eye metrics provide insight into surgical behaviour allowing differentiation of performance, however have not been used in robotic surgery. This study explores eye metrics of robotic surgeons in training in simulated and real tissue environments.

METHODS

Following the Fundamentals of Robotic Surgery (FRS), training curriculum novice robotic surgeons were trained to expert-derived benchmark proficiency using real tissue on the da Vinci Si and the da Vinci skills simulator (dVSS) simulator. Surgeons eye metrics were recorded using eye-tracking glasses when both "novice" and "proficient" in both environments. Performance was assessed using Global Evaluative Assessment of Robotic skills (GEARS) and numeric psychomotor test score (NPMTS) scores.

RESULTS

Significant (P ≤ .05) correlations were seen between pupil size, rate of change and entropy, and associated GEARS/NPMTS in "novice" and "proficient" surgeons. Only number of blinks per minute was significantly different between pupilometrics in the simulated and real tissue environments.

CONCLUSIONS

This study illustrates the value of eye tracking as an objective physiological tool in the robotic setting. Pupilometrics significantly correlate with established assessment methods and could be incorporated into robotic surgery assessments.

Visual attention on a respiratory function monitor during simulated neonatal resuscitation: an eye-tracking study

OBJECTIVE

A respiratory function monitor (RFM) may improve positive pressure ventilation (PPV) technique, but many providers do not use RFM data appropriately during delivery room resuscitation. We sought to use eye-tracking technology to identify RFM parameters that neonatal providers view most commonly during simulated PPV.

DESIGN

Mixed methods study. Neonatal providers performed RFM-guided PPV on a neonatal manikin while wearing eye-tracking glasses to quantify visual attention on displayed RFM parameters (ie, exhaled tidal volume, flow, leak). Participants subsequently provided qualitative feedback on the eye-tracking glasses.

SETTING

Level 3 academic neonatal intensive care unit.

PARTICIPANTS

Twenty neonatal resuscitation providers.

MAIN OUTCOME MEASURES

Visual attention: overall gaze sample percentage; total gaze duration, visit count and average visit duration for each displayed RFM parameter. Qualitative feedback: willingness to wear eye-tracking glasses during clinical resuscitation.

RESULTS

Twenty providers participated in this study. The mean gaze sample captured wa s 93% (SD 4%). Exhaled tidal volume waveform was the RFM parameter with the highest total gaze duration (median 23%, IQR 13-51%), highest visit count (median 5.17 per 10 s, IQR 2.82-6.16) and longest visit duration (median 0.48 s, IQR 0.38-0.81 s). All participants were willing to wear the glasses during clinical resuscitation.

CONCLUSION

Wearable eye-tracking technology is feasible to identify gaze fixation on the RFM display and is well accepted by providers. Neonatal providers look at exhaled tidal volume more than any other RFM parameter. Future applications of eye-tracking technology include use during clinical resuscitation.

Visual scanning behavior during distracted walking in healthy young adults

BACKGROUND

An epidemic of pedestrian accidents when walking while texting suggests that people are less aware of their surroundings during distracted walking, and highlights the importance of visual scanning for pedestrian safety. Quantitative examination of visual scanning during distracted walking is still lacking.

RESEARCH QUESTION

Is visual scanning behavior altered by distracted walking in healthy young adults?

METHODS

We compared visual scanning behavior in 20 young adults during usual (single-task) walking, walking while performing a letter-fluency task, and walking while texting. Visual scanning behavior was measured by fixation count and dwell time percentage in specific areas of interest. Dual-task effects on gait speed, letter fluency, texting speed and accuracy, and situational awareness were also examined.

RESULT

Visual scanning behavior differed between the three walking conditions. During dual-task letter fluency, participants had significantly more non-walking path fixations than either of the other two conditions (i.e., more frequent, broader visual scanning). Conversely, during dual-task texting, gaze was focused predominantly on the phone, with little visual scanning of the walking path and surrounding environment. When walking without texting or talking, gaze was directed equally to far walking path and surrounding environment.

SIGNIFICANCE

Texting while walking is associated with a considerable reduction in overt visual attention to the walking path and surrounding areas. Whether this translates to reduced conscious awareness of environmental stimuli remains unclear. Performing a verbal task while walking was associated with more frequent, wider visual scanning behavior, which may be specific to the nature of the verbal task in this study.

Looks can be deceiving: Gaze pattern differences between novices and experts during placement of central lines

BACKGROUND

The objective of this study was to determine whether gaze patterns could differentiate expertise during simulated ultrasound-guided Internal Jugular Central Venous Catheterization (US-IJCVC) and if expert gazes were different between simulators of varying functional and structural fidelity.

METHODS

A 2017 study compared eye gaze patterns of expert surgeons (n = 11), senior residents (n = 4), and novices (n = 7) during CVC needle insertions using the dynamic haptic robotic trainer (DHRT), a system which simulates US-IJCVC. Expert gaze patterns were also compared between a manikin and the DHRT.

RESULTS

Expert gaze patterns were consistent between the manikin and DHRT environments (p = 0.401). On the DHRT system, CVC experience significantly impacted the percent of time participants spent gazing at the ultrasound screen (p < 0.0005) and the needle and ultrasound probe (p < 0.0005).

CONCLUSION

Gaze patterns differentiate expertise during ultrasound-guided IJCVC placement and the fidelity of the simulator does not impact gaze patterns.

Glance behavior as design indices of in-vehicle visual support system: A study using crane simulators

A prediction model is used to predict subjective responses of crane operators with respect to different designs of In-Vehicle Visual Support (IVVS). Selected gaze metrics are used as objective metrics to minimize prejudice, which is commonly caused by subjective measures. Experiments are carried out using crane simulator to measure glance behavior of novice operators and the 3D perspective projection method is used for autonomous mapping of gaze fixations to dynamic Area-of-Interests (AOIs). Subjective responses, such as operators' emotion and usability of IVVS, are evaluated using the Likert scale of the Semantic Differential method. Correlation between gaze metrics and subjective responses is established using multiple linear regression. Glance behavior exhibits a statistically significant difference when information on IVVS is perceived as useful to ease operation and reduce tension. Despite this, there are no significant signs of distraction. Glance behavior is found to be a reliable sub-conscious indicator of subjective response and distraction. More importantly, the proposed gaze metrics are found to be a good representation of glance behavior, such as randomness and distribution of attention. The methods and findings are useful to evaluate impact of IVVS, which is becoming more common in many other applications.

Gaze Behaviors During Serve Returns in Tennis: A Comparison Between Intermediate- and High-Skill Players

The authors studied gaze behaviors in high- and intermediate-skill tennis players while they performed tennis serve returns. Participants returned 40 serves in 4 serve locations while wearing a mobile eye tracker. The ball's flight path was deconstructed into 3 distinct locations (i.e., ball before bouncing on surface, the bounce area, and ball after bouncing on surface), and gaze behaviors along with quiet-eye (QE) onset and durations were recorded. Results revealed that (a) high-skill players exhibited better return shots than their lower skill counterparts, (b) high-skill players and high-score shots were characterized by longer fixation durations on the ball at prebounce, and (c) longer QE durations were observed for high-skill players and high-score shots. Findings provide valuable insight into the relationship between gaze behaviors, QE, and performance in fast-pace interceptive sports.

The HerniaSurge Group. International guidelines for groin hernia management

INTRODUCTION

Worldwide, more than 20 million patients undergo groin hernia repair annually. The many different approaches, treatment indications and a significant array of techniques for groin hernia repair warrant guidelines to standardize care, minimize complications, and improve results. The main goal of these guidelines is to improve patient outcomes, specifically to decrease recurrence rates and reduce chronic pain, the most frequent problems following groin hernia repair. They have been endorsed by all five continental hernia societies, the International Endo Hernia Society and the European Association for Endoscopic Surgery.

METHODS

An expert group of international surgeons (the HerniaSurge Group) and one anesthesiologist pain expert was formed. The group consisted of members from all continents with specific experience in hernia-related research. Care was taken to include surgeons who perform different types of repair and had preferably performed research on groin hernia surgery. During the Group's first meeting, evidence-based medicine (EBM) training occurred and 166 key questions (KQ) were formulated. EBM rules were followed in complete literature searches (including a complete search by The Dutch Cochrane database) to January 1, 2015 and to July 1, 2015 for level 1 publications. The articles were scored by teams of two or three according to Oxford, SIGN and Grade methodologies. During five 2-day meetings, results were discussed with the working group members leading to 136 statements and 88 recommendations. Recommendations were graded as "strong" (recommendations) or "weak" (suggestions) and by consensus in some cases upgraded. In the Results and summary section below, the term "should" refers to a recommendation. The AGREE II instrument was used to validate the guidelines. An external review was performed by three international experts. They recommended the guidelines with high scores. The risk factors for inguinal hernia (IH) include: family history, previous contra-lateral hernia, male gender, age, abnormal collagen metabolism, prostatectomy, and low body mass index. Peri-operative risk factors for recurrence include poor surgical techniques, low surgical volumes, surgical inexperience and local anesthesia. These should be considered when treating IH patients. IH diagnosis can be confirmed by physical examination alone in the vast majority of patients with appropriate signs and symptoms. Rarely, ultrasound is necessary. Less commonly still, a dynamic MRI or CT scan or herniography may be needed. The EHS classification system is suggested to stratify IH patients for tailored treatment, research and audit. Symptomatic groin hernias should be treated surgically. Asymptomatic or minimally symptomatic male IH patients may be managed with "watchful waiting" since their risk of hernia-related emergencies is low. The majority of these individuals will eventually require surgery; therefore, surgical risks and the watchful waiting strategy should be discussed with patients. Surgical treatment should be tailored to the surgeon's expertise, patient- and hernia-related characteristics and local/national resources. Furthermore, patient health-related, life style and social factors should all influence the shared decision-making process leading up to hernia management. Mesh repair is recommended as first choice, either by an open procedure or a laparo-endoscopic repair technique. One standard repair technique for all groin hernias does not exist. It is recommended that surgeons/surgical services provide both anterior and posterior approach options. Lichtenstein and laparo-endoscopic repair are best evaluated. Many other techniques need further evaluation. Provided that resources and expertise are available, laparo-endoscopic techniques have faster recovery times, lower chronic pain risk and are cost effective. There is discussion concerning laparo-endoscopic management of potential bilateral hernias (occult hernia issue). After patient consent, during TAPP, the contra-lateral side should be inspected. This is not suggested during unilateral TEP repair. After appropriate discussions with patients concerning results tissue repair (first choice is the Shouldice technique) can be offered. Day surgery is recommended for the majority of groin hernia repair provided aftercare is organized. Surgeons should be aware of the intrinsic characteristics of the meshes they use. Use of so-called low-weight mesh may have slight short-term benefits like reduced postoperative pain and shorter convalescence, but are not associated with better longer-term outcomes like recurrence and chronic pain. Mesh selection on weight alone is not recommended. The incidence of erosion seems higher with plug versus flat mesh. It is suggested not to use plug repair techniques. The use of other implants to replace the standard flat mesh in the Lichtenstein technique is currently not recommended. In almost all cases, mesh fixation in TEP is unnecessary. In both TEP and TAPP it is recommended to fix mesh in M3 hernias (large medial) to reduce recurrence risk. Antibiotic prophylaxis in average-risk patients in low-risk environments is not recommended in open surgery. In laparo-endoscopic repair it is never recommended. Local anesthesia in open repair has many advantages, and its use is recommended provided the surgeon is experienced in this technique. General anesthesia is suggested over regional in patients aged 65 and older as it might be associated with fewer complications like myocardial infarction, pneumonia and thromboembolism. Perioperative field blocks and/or subfascial/subcutaneous infiltrations are recommended in all cases of open repair. Patients are recommended to resume normal activities without restrictions as soon as they feel comfortable. Provided expertise is available, it is suggested that women with groin hernias undergo laparo-endoscopic repair in order to decrease the risk of chronic pain and avoid missing a femoral hernia. Watchful waiting is suggested in pregnant women as groin swelling most often consists of self-limited round ligament varicosities. Timely mesh repair by a laparo-endoscopic approach is suggested for femoral hernias provided expertise is available. All complications of groin hernia management are discussed in an extensive chapter on the topic. Overall, the incidence of clinically significant chronic pain is in the 10-12% range, decreasing over time. Debilitating chronic pain affecting normal daily activities or work ranges from 0.5 to 6%. Chronic postoperative inguinal pain (CPIP) is defined as bothersome moderate pain impacting daily activities lasting at least 3 months postoperatively and decreasing over time. CPIP risk factors include: young age, female gender, high preoperative pain, early high postoperative pain, recurrent hernia and open repair. For CPIP the focus should be on nerve recognition in open surgery and, in selected cases, prophylactic pragmatic nerve resection (planned resection is not suggested). It is suggested that CPIP management be performed by multi-disciplinary teams. It is also suggested that CPIP be managed by a combination of pharmacological and interventional measures and, if this is unsuccessful, followed by, in selected cases (triple) neurectomy and (in selected cases) mesh removal. For recurrent hernia after anterior repair, posterior repair is recommended. If recurrence occurs after a posterior repair, an anterior repair is recommended. After a failed anterior and posterior approach, management by a specialist hernia surgeon is recommended. Risk factors for hernia incarceration/strangulation include: female gender, femoral hernia and a history of hospitalization related to groin hernia. It is suggested that treatment of emergencies be tailored according to patient- and hernia-related factors, local expertise and resources. Learning curves vary between different techniques. Probably about 100 supervised laparo-endoscopic repairs are needed to achieve the same results as open mesh surgery like Lichtenstein. It is suggested that case load per surgeon is more important than center volume. It is recommended that minimum requirements be developed to certify individuals as expert hernia surgeon. The same is true for the designation "Hernia Center". From a cost-effectiveness perspective, day-case laparoscopic IH repair with minimal use of disposables is recommended. The development and implementation of national groin hernia registries in every country (or region, in the case of small country populations) is suggested. They should include patient follow-up data and account for local healthcare structures. A dissemination and implementation plan of the guidelines will be developed by global (HerniaSurge), regional (international societies) and local (national chapters) initiatives through internet websites, social media and smartphone apps. An overarching plan to improve access to safe IH surgery in low-resource settings (LRSs) is needed. It is suggested that this plan contains simple guidelines and a sustainability strategy, independent of international aid. It is suggested that in LRSs the focus be on performing high-volume Lichtenstein repair under local anesthesia using low-cost mesh. Three chapters discuss future research, guidelines for general practitioners and guidelines for patients.

CONCLUSIONS

The HerniaSurge Group has developed these extensive and inclusive guidelines for the management of adult groin hernia patients. It is hoped that they will lead to better outcomes for groin hernia patients wherever they live. More knowledge, better training, national audit and specialization in groin hernia management will standardize care for these patients, lead to more effective and efficient healthcare and provide direction for future research.