The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model

Development of a Sensor Technology to Objectively Measure Dexterity for Cardiac Surgical Proficiency

BACKGROUND

Technical skill is essential for good outcomes in cardiac surgery. However, no objective methods exist to measure dexterity while performing surgery. The purpose of this study was to validate sensor-based hand motion analysis (HMA) of technical dexterity while performing a graft anastomosis within a validated simulator.

METHODS

Surgeons at various training levels performed an anastomosis while wearing flexible sensors (BioStamp nPoint, MC10 Inc) with integrated accelerometers and gyroscopes on each hand to quantify HMA kinematics. Groups were stratified as experts (n = 8) or novices (n = 18). The quality of the completed anastomosis was scored using the 10 Point Microsurgical Anastomosis Rating Scale (MARS10). HMA parameters were compared between groups and correlated with quality. Logistic regression was used to develop a predictive model from HMA parameters to distinguish experts from novices.

RESULTS

Experts were faster (11 ± 6 minutes vs 21 ± 9 minutes; P = .012) and used fewer movements in both dominant (340 ± 166 moves vs 699 ± 284 moves; P = .003) and nondominant (359 ± 188 moves vs 567 ± 201 moves; P = .02) hands compared with novices. Experts' anastomoses were of higher quality compared with novices (9.0 ± 1.2 MARS10 vs 4.9 ± 3.2 MARS10; P = .002). Higher anastomosis quality correlated with 9 of 10 HMA parameters, including fewer and shorter movements of both hands (dominant, r = -0.65, r = -0.46; nondominant, r = -0.58, r = -0.39, respectively).

CONCLUSIONS

Sensor-based HMA can distinguish technical dexterity differences between experts and novices, and correlates with quality. Objective quantification of hand dexterity may be a valuable adjunct to training and education in cardiac surgery training programs.

Motion capture device reveals a quick learning curve in vascular anastomosis training

PURPOSE

Surgical procedures are often evaluated subjectively, and an objective evaluation has been considered difficult to make and rarely reported, especially in open surgery, where the range of motion is wide. This study evaluated the effectiveness of surgical suturing training as an educational tool using the Leap Motion Controller (LMC), which can capture hand movements and reproduce them as data comprising parametric elements.

METHODS

We developed an off-the-job training system (Off-JT) in our department, mainly using prosthetic grafts and various anastomotic methodologies with graded difficulty levels. We recruited 50 medical students (novice group) and 6 vascular surgeons (expert group) for the study. We evaluated four parameters for intraoperative skills: suturing time, slope of the roll, smoothness, and rate of excess motion.

RESULTS

All 4 parameters distinguished the skill of the novice group at 1 and 10 h off-JT. After 10 h of off-JT, all 4 parameters of the novices were comparable to those of the expert group.

CONCLUSION

Our education system using the LMC is relatively inexpensive and easy to set up, with a free application for analyses, serving as an effective and ubiquitous educational tool for young surgeons.

Enhancing surgical performance in cardiothoracic surgery with innovations from computer vision and artificial intelligence: a narrative review

When technical requirements are high, and patient outcomes are critical, opportunities for monitoring and improving surgical skills via objective motion analysis feedback may be particularly beneficial. This narrative review synthesises work on technical and non-technical surgical skills, collaborative task performance, and pose estimation to illustrate new opportunities to advance cardiothoracic surgical performance with innovations from computer vision and artificial intelligence. These technological innovations are critically evaluated in terms of the benefits they could offer the cardiothoracic surgical community, and any barriers to the uptake of the technology are elaborated upon. Like some other specialities, cardiothoracic surgery has relatively few opportunities to benefit from tools with data capture technology embedded within them (as is possible with robotic-assisted laparoscopic surgery, for example). In such cases, pose estimation techniques that allow for movement tracking across a conventional operating field without using specialist equipment or markers offer considerable potential. With video data from either simulated or real surgical procedures, these tools can (1) provide insight into the development of expertise and surgical performance over a surgeon's career, (2) provide feedback to trainee surgeons regarding areas for improvement, (3) provide the opportunity to investigate what aspects of skill may be linked to patient outcomes which can (4) inform the aspects of surgical skill which should be focused on within training or mentoring programmes. Classifier or assessment algorithms that use artificial intelligence to 'learn' what expertise is from expert surgical evaluators could further assist educators in determining if trainees meet competency thresholds. With collaborative efforts between surgical teams, medical institutions, computer scientists and researchers to ensure this technology is developed with usability and ethics in mind, the developed feedback tools could improve cardiothoracic surgical practice in a data-driven way.

Change in economy of ultrasound probe motion among general medicine trainees

OBJECTIVES

To observe change in economy of 9 ultrasound probe movement metrics among internal medicine trainees during a 5-day training course in cardiac point of care ultrasound (POCUS).

METHODS

We used a novel probe tracking device to record nine features of ultrasound probe movement, while trainees and experts optimized ultrasound clips on the same volunteer patients. These features included translational movements, gyroscopic movements (titling, rocking, and rotation), smoothness, total path length, and scanning time. We determined the adjusted difference between each trainee's movements and the mean value of the experts' movements for each patient. We then used a mixed effects model to trend average the adjusted differences between trainees and experts throughout the 5 days of the course.

RESULTS

Fifteen trainees were enrolled. Three echocardiographer technicians and the course director served as experts. Across 16 unique patients, 294 ultrasound clips were acquired. For all 9 movements, the adjusted difference between trainees and experts narrowed day-to-day (p value < 0.05), suggesting ongoing improvement during training. By the last day of the course, there were no statistically significant differences between trainees and experts in translational movement, gyroscopic movement, smoothness, or total path length; yet on average trainees took 28 s (95% CI [14.7-40.3] seconds) more to acquire a clip.

CONCLUSIONS

We detected improved ultrasound probe motion economy among internal medicine trainees during a 5-day training course in cardiac POCUS using an inexpensive probe tracking device. Objectively quantifying probe motion economy may help assess a trainee's level of proficiency in this skill and individualize their POCUS training.

Deep learning-based video-analysis of instrument motion in microvascular anastomosis training

PURPOSE

Attaining sufficient microsurgical skills is paramount for neurosurgical trainees. Kinematic analysis of surgical instruments using video offers the potential for an objective assessment of microsurgical proficiency, thereby enhancing surgical training and patient safety. The purposes of this study were to develop a deep-learning-based automated instrument tip-detection algorithm, and to validate its performance in microvascular anastomosis training.

METHODS

An automated instrument tip-tracking algorithm was developed and trained using YOLOv2, based on clinical microsurgical videos and microvascular anastomosis practice videos. With this model, we measured motion economy (procedural time and path distance) and motion smoothness (normalized jerk index) during the task of suturing artificial blood vessels for end-to-side anastomosis. These parameters were validated using traditional criteria-based rating scales and were compared across surgeons with varying microsurgical experience (novice, intermediate, and expert). The suturing task was deconstructed into four distinct phases, and parameters within each phase were compared between novice and expert surgeons.

RESULTS

The high accuracy of the developed model was indicated by a mean Dice similarity coefficient of 0.87. Deep learning-based parameters (procedural time, path distance, and normalized jerk index) exhibited correlations with traditional criteria-based rating scales and surgeons' years of experience. Experts completed the suturing task faster than novices. The total path distance for the right (dominant) side instrument movement was shorter for experts compared to novices. However, for the left (non-dominant) side, differences between the two groups were observed only in specific phases. The normalized jerk index for both the right and left sides was significantly lower in the expert than in the novice groups, and receiver operating characteristic analysis showed strong discriminative ability.

CONCLUSION

The deep learning-based kinematic analytic approach for surgical instruments proves beneficial in assessing performance in microvascular anastomosis. Moreover, this methodology can be adapted for use in clinical settings.

Objective assessment for open surgical suturing training by finger tracking can discriminate novices from experts

It is difficult, time consuming and expensive to assess manual skills in open surgery. The aim of this study is to investigate the construct validity of a low-cost, easily accessible tracking technique for basic open suturing tasks. Medical master students, surgical residents, and surgeons at the Radboud University Medical Center were recruited between September 2020 until September 2021. The participants were divided, according to experience, in a novice group (≤10 sutures performed) and an expert group (>50 sutures performed). For objective tracking, a tablet with SurgTrac software was used, which tracked a blue and a red tag placed on respectively their left and right index finger. The participants executed four basic tasks on a suturing model: 1) knot tying by hand, 2) transcutaneous suturing with an instrument knot, 3) 'Donati' (vertical mattress suture) with an instrument knot and 4) continuous intracutaneous suturing without a knot. In total 76 participants were included: 57 novices and 19 experts. All four tasks showed significant differences between the novice group and expert group for the parameters time (p<0.001), distance (p<0.001 for Task 1, 2 and 3 and p=0.034 for Task 4) and smoothness (p<0.001). Additionally, Task 3 showed a significant difference for the parameter handedness (p=0.006) and Task 4 for speed (p=0.033). Tracking index finger movements using SurgTrac software on a tablet while executing basic open suturing skills on a simulator shows excellent construct validity for time, distance and motion smoothness in all four suturing tasks.

Measuring hand movement for suturing skill assessment: A simulation-based study

BACKGROUND

To maximize patient safety, surgical skills education is increasingly adopting simulation-based curricula for formative skills assessment and training. However, many standardized assessment tools rely on human raters for performance assessment, which is resource-intensive and subjective. Simulators that provide automated and objective metrics from sensor data can address this limitation. We present an instrumented bench suturing simulator, patterned after the clock face radial suturing model from the Fundamentals of Vascular Surgery, for automated and objective assessment of open suturing skills.

METHODS

For this study, 97 participants (35 attending surgeons, 32 residents, and 30 novices) were recruited at national vascular conferences. Automated hand motion metrics, especially focusing on rotational motion analysis, were developed from the inertial measurement unit attached to participants' hands, and the proposed suite of metrics was used to differentiate between the skill levels of the 3 groups.

RESULTS

Attendings' and residents' performances were found to be significantly different from novices for all metrics. Moreover, most of our novel metrics could successfully distinguish between finer skill differences between attending and resident groups. In contrast, traditional operative skill metrics, such as time and path length, were unable to distinguish attendings from residents.

CONCLUSION

This study provides evidence for the effectiveness of rotational motion analysis in assessing suturing skills. The suite of inertial measurement unit-based hand motion metrics introduced in this study allows for the incorporation of hand movement data for suturing skill assessment.

Development and Validation of a Laryngeal Microsurgery Simulation Training System for Otolaryngology Residents

OBJECTIVE

This study aims to create a synthetic laryngeal microsurgery simulation model and training program; to assess its face, content, and construct validity; and to review the available phonomicrosurgery simulation models in the literature.

STUDY DESIGN

Nonrandomly assigned control study.

SETTING

Simulation training course for the otolaryngology residency program at Pontificia Universidad Católica de Chile.

METHODS

Resident (postgraduate year 1 [PGY1]/PGY2) and expert groups were recruited. A laryngeal microsurgery synthetic model was developed. Nine tasks were designed and assessed through a set of programmed exercises with increasing difficulty, to fulfill 5 surgical competencies. Imperial College Surgical Assessment Device sensors applied to the participants' hands measured time and movements. The activities were video-recorded and blindly assessed by 2 laryngologists using a specific and global rating scale (SRS and GRS). A 5-point Likert survey assessing validity was completed by experts.

RESULTS

Eighteen participants were recruited (14 residents and 4 experts). Experts performed significantly better than residents in the SRS (p = .003), and GRS (p = .004). Internal consistency was demonstrated for the SRS (α = .972, p < .001). Experts had a shorter execution time (p = .007), and path length with the right hand (p = .04). The left hand did not show significant differences. The survey assessing validity resulted in a median 36 out of 40 points score for face validity; and 43 out of 45 points score, for global content validity. The literature review revealed 20 available phonomicrosurgery simulation models, only 6 with construct validity.

CONCLUSION

The face, content, and construct validity of the laryngeal microsurgery simulation training program were established. It could be replicated and incorporated into residents' curricula.

Use of Virtual Reality in the Education of Orthopaedic Procedures: A Randomised Control Study in Early Validation of a Novel Virtual Reality Simulator

Background Virtual reality (VR) simulation is a potential solution to the barriers surgical trainees are facing. There needs to be validation for its implementation within current training. We aimed to compare VR simulation to traditional methods in acquiring surgical skills for a TFN-ADVANCED™ Proximal Femoral Nailing System (TFNA; DePuy Synthes, Auckland, New Zealand) femoral nailing system. Methods Thirty-one surgical trainees were randomised to two groups: traditional-training group (control group) and a VR-training group (intervention group) for insertion of a short cephalomedullary TFNA nail. Both groups then inserted the same TFNA system into saw-bone femurs. Surveys evaluated validity of the relevant activities, perception of simulation, confidence, stress and anxiety. The primary outcomes were tip-apex distance (TAD) and user anxiety/confidence levels. Secondary outcomes included number of screw- and nail-guidewire insertion attempts, the time taken to complete and user validity of the VR system. Results There was no statistical difference in TAD between the intervention and control groups (9mm vs 15mm, p=0.0734). The only TAD at risk of cut-out was in the control group (25mm). There was no statistical difference in time taken (2547.5ss vs 2395ss, p=0.668), nail guide-wire attempts (two for both groups, p=0.355) and screw guide-wire attempts (one for both groups, p=0.702). The control group versus intervention had higher anxiety levels (50% vs 33%) and had lower confidence (61% vs 84%). Interpretation There was no objective difference in performance on a saw-bone model between groups. However, this VR simulator resulted in more confidence and lower anxiety levels whilst performing a simulated TFNA. Whilst further studies with larger sample sizes and exploration of transfer validity to the operating theatre are required, this study does indicate potential benefits of VR within surgical training.

Using the language of surgery to enhance ophthalmology surgical education

Background

Currently, surgical education utilizes a combination of the apprentice model, wet-lab training, and simulation, but due to reliance on subjective data, the quality of teaching and assessment can be variable. The "language of surgery," an established concept in engineering literature whose incorporation into surgical education has been limited, is defined as the description of each surgical maneuver using quantifiable metrics. This concept is different from the traditional notion of surgical language, generally thought of as the qualitative definitions and terminology used by surgeons.

Methods

A literature search was conducted through April 2023 using MEDLINE/PubMed using search terms to investigate wet-lab, virtual simulators, and robotics in ophthalmology, along with the language of surgery and surgical education. Articles published before 2005 were mostly excluded, although a few were included on a case-by-case basis.

Results

Surgical maneuvers can be quantified by leveraging technological advances in virtual simulators, video recordings, and surgical robots to create a language of surgery. By measuring and describing maneuver metrics, the learning surgeon can adjust surgical movements in an appropriately graded fashion that is based on objective and standardized data. The main contribution is outlining a structured education framework that details how surgical education could be improved by incorporating the language of surgery, using ophthalmology surgical education as an example.

Conclusion

By describing each surgical maneuver in quantifiable, objective, and standardized terminology, a language of surgery can be created that can be used to learn, teach, and assess surgical technical skill with an approach that minimizes bias.

Key message

The "language of surgery," defined as the quantification of each surgical movement's characteristics, is an established concept in the engineering literature. Using ophthalmology surgical education as an example, we describe a structured education framework based on the language of surgery to improve surgical education.

Classifications

Surgical education, robotic surgery, ophthalmology, education standardization, computerized assessment, simulations in teaching.

Competencies

Practice-Based Learning and Improvement.

Training Program for Orthopedic Residents in Forefoot Osteotomy Skills: Transference From a Simulator to a Cadaveric Surgical Scenario

INTRODUCTION

An effective simulation program allows both the acquisition of surgical skills on the simulated model and the transfer of these skills to a surgical scenario. We designed a forefoot osteotomy training program and sought to determine the transferability to a cadaveric surgical scenario.

METHODS

Eleven orthopedic residents and 2 foot and ankle surgeons were included. A foot simulator was used. All residents were instructed on the surgical techniques of Chevron, Akin, and triple Weil osteotomies. Eight junior residents (trainees) were enrolled in a supervised simulation program. Baseline assessment was performed on the simulator with the Objective Structured Assessment of Technical Skills (OSATS) and the Imperial College Surgical Assessment Device (ICSAD). After baseline, trainees completed a training program and had a final evaluation of proficiency on the simulator and on cadaveric specimens. Three senior residents with no simulated training (controls) and experts were assessed for comparison.

RESULTS

All trainees improved from a baseline OSATS score of 11 points (9-20) to a final score of 35 points (33-35) in the simulator and 34 points (32-34) in the cadaveric specimen ( P < 0.01). Compared with baseline, the ICSAD results improved in path length (391 [205-544] to 131 [73-278] meters, P < 0.01) and number of movements (2756 [1258-3338] to 992 [478-1908], P < 0.01). The final OSATS and ICSAD scores did not differ from experts ( P = 0.1) and were significantly different from untrained residents ( P = 0.02).

CONCLUSIONS

Simulated training of Chevron, Akin, and triple Weil osteotomies in orthopedic residents improved procedural proficiency, enabling successful skill transfer to a surgical scenario in cadavers.

LEVEL OF EVIDENCE

II (Prospective Cohort Study).

Relating process and outcome metrics for meaningful and interpretable cannulation skill assessment: A machine learning paradigm

BACKGROUND AND OBJECTIVES

The quality of healthcare delivery depends directly on the skills of clinicians. For patients on hemodialysis, medical errors or injuries caused during cannulation can lead to adverse outcomes, including potential death. To promote objective skill assessment and effective training, we present a machine learning approach, which utilizes a highly-sensorized cannulation simulator and a set of objective process and outcome metrics.

METHODS

In this study, 52 clinicians were recruited to perform a set of pre-defined cannulation tasks on the simulator. Based on data collected by sensors during their task performance, the feature space was then constructed based on force, motion, and infrared sensor data. Following this, three machine learning models- support vector machine (SVM), support vector regression (SVR), and elastic net (EN)- were constructed to relate the feature space to objective outcome metrics. Our models utilize classification based on the conventional skill classification labels as well as a new method that represents skill on a continuum.

RESULTS

With less than 5% of trials misplaced by two classes, the SVM model was effective in predicting skill based on the feature space. In addition, the SVR model effectively places both skill and outcome on a fine-grained continuum (versus discrete divisions) that is representative of reality. As importantly, the elastic net model enabled the identification of a set of process metrics that highly impact outcomes of the cannulation task, including smoothness of motion, needle angles, and pinch forces.

CONCLUSIONS

The proposed cannulation simulator, paired with machine learning assessment, demonstrates definite advantages over current cannulation training practices. The methods presented here can be adopted to drastically increase the effectiveness of skill assessment and training, thereby potentially improving clinical outcomes of hemodialysis treatment.

Automatic assessment of performance in the FLS trainer using computer vision

BACKGROUND

Fundamentals of Laparoscopic Surgery (FLS) box trainer is a well-accepted method for training and evaluating laparoscopic skills. It mandates an observer that will measure and evaluate the trainee's performance. Measuring performance in the Peg Transfer task includes time and penalty for dropping pegs. This study aimed to assess whether computer vision (CV) may be used to automatically measure performance in the FLS box trainer.

METHODS

Four groups of metrics were defined and measured automatically using CV. Validity was assessed by dividing participants to 3 groups of experience levels. Twenty-seven participants were recorded performing the Peg Transfer task 2-4 times, amounting to 72 videos. Frames were sampled from the videos and labeled to create an image dataset. Using these images, we trained a deep neural network (YOLOv4) to detect the different objects in the video. We developed an evaluation system that tracks the transfer of the triangles and produces a feedback report with the metrics being the main criteria. The metric groups were Time, Grasper Movement Speed, Path Efficiency, and Grasper Coordination. The performance was compared based on their last video (3 participants were excluded due to technical issues).

RESULTS

The ANOVA tests show that for all metrics except one, the variance in performance can be explained by the experience level of participants. Senior surgeons and residents significantly outperform students and interns on almost every metric. Senior surgeons usually outperform residents, but the gap is not always significant.

CONCLUSION

The statistical analysis shows that the metrics can differentiate between the experts and novices performing the task in several aspects. Thus, they may provide a more detailed performance analysis than is currently used. Moreover, these metrics calculation is automatic and relies solely on the video camera of the FLS trainer. As a result, they allow independent training and assessment.

Objective psychomotor laparoscopic skills evaluation using a low-cost wearable device based on accelerometry: construct and concurrent validity study

BACKGROUND

Motion analysis of surgical maneuvers provides useful quantitative information for the objective evaluation of the surgeons. However, surgical simulation laboratories for laparoscopic training do not usually integrate devices that help quantify the level of skills of the surgeons due to their limited resources and the high costs of new technologies. The purpose of this study is to present the construct and concurrent validity of a low-cost motion tracking system, based on a wireless triaxial accelerometer, employed to objectively evaluate psychomotor skills of surgeons during laparoscopic training.

METHODS

An accelerometry system, a wireless three-axis accelerometer with appearance of wristwatch, was placed on the dominant hand of the surgeons to register the motion during the laparoscopy practice with the EndoViS simulator, which simultaneously recorded the motion of the laparoscopic needle driver. This study included the participation of 30 surgeons (6 experts, 14 intermediates and 10 novices) who performed the task of intracorporeal knot-tying suture. Using 11 motion analysis parameters (MAPs), the performance of each participant was assessed. Subsequently, the scores of the three groups of surgeons were statistically analyzed. In addition, a validity study was conducted comparing the metrics between the accelerometry-tracking system and the EndoViS hybrid simulator.

RESULTS

Construct validity was achieved for 8 of the 11 metrics examined with the accelerometry system. Concurrent validity demonstrated that there is a strong correlation between the results of the accelerometry system and the EndoViS simulator in 9 of 11 parameters, showing reliability of the accelerometry system as an objective evaluation method.

CONCLUSION

The accelerometry system was successfully validated. This method is potentially useful to complement the objective evaluation of surgeons during laparoscopic practice in training environments such as box-trainers and simulators.

The frequency of assessment tools in arthroscopic training: a systematic review

BACKGROUND

Multiple assessment tools are used in arthroscopic training and play an important role in feedback. However, it is not fully recognized as to the standard way to apply these tools. Our study aimed to investigate the use of assessment tools in arthroscopic training and determine whether there is an optimal way to apply various assessment tools in arthroscopic training.

METHODS

A search was performed using PubMed, Embase and Cochrane Library electronic databases for articles published in English from January 2000 to July 2021. Eligible for inclusion were primary research articles related to using assessment tools for the evaluation of arthroscopic skills and training environments. Studies that focussed only on therapeutic cases, did not report outcome measures of technical skills, or did not mention arthroscopic skills training were excluded.

RESULTS

A total of 28 studies were included for review. Multiple assessment tools were used in arthroscopic training. The most common objective metric was completion time, reported in 21 studies. Technical parameters based on simulator or external equipment, such as instrument path length, hand movement, visual parameters and injury, were also widely used. Subjective assessment tools included checklists and global rating scales (GRS). Among these, the most commonly used GRS was the Arthroscopic Surgical Skill Evaluation Tool (ASSET). Most of the studies combined objective metrics and subjective assessment scales in the evaluation of arthroscopic skill training.

CONCLUSIONS

Overall, both subjective and objective assessment tools can be used as feedback for basic arthroscopic skill training, but there are still differences in the frequency of application in different contexts. Despite this, combined use of subjective and objective assessment tools can be applied to more situations and skills and can be the optimal way for assessment.

LEVEL OF EVIDENCE

Level III, systematic review of level I to III studies. Key messagesBoth subjective and objective assessment tools can be used as feedback for basic arthroscopic skill training.Combined use of subjective and objective assessment tools can be applied to more situations and skills and can be the optimal way for assessment.

Assessment of Core Surgical Skills Using a Mixed Reality Headset - The MoTOR Study

INTRODUCTION

Surgical skill assessment utilises direct observation and feedback by an expert which is potentially subjective, therefore obtaining objective data for hand and eye tracking is essential. Our aim was to evaluate a wearable mixed reality (MR) headset in these domains.

METHODS

Participants with differing levels of surgical expertise [novice (N), intermediate (I) & expert (E)] performed 4 simulated surgical tasks; 2 general dexterity (tasks 1&2) and 2 surgical skills (tasks 3&4) wearing the MR headset capturing their hand and eye movements (median & range). Metrics included hand path length and the speed of each index or thumb tip. Gaze data were also captured. Participant demographics, prior expertise and current experience were captured with an electronic survey. Data were analysed with a Shapiro-Wilk test or ANOVA as appropriate. A p-value of < 0.05 was significant.

RESULTS

Thirty-six participants were analysed (N = 18, I = 8, E = 8). Tasks 1&2 revealed 2 speed outcomes (left index and left-hand speed) which were significant. For tasks 3&4, various outcomes were significant: path length for left hand (N:45 cm vs. I:31 cm vs. E:27 cm, p = 0.03) and right hand (N:48 cm vs. I:29 cm vs. E:28 cm, p = 0.01) and total time (N:456s vs. I:292 vs. E: 245, p = 0.0002). With left-hand-tying, average path length (N:61 cm vs. I:39 vs. E:36, p = 0.04), average speed (N:11 cm/s vs. I:23 vs. E:24, p = 0.03), and total time (N:156s vs. I:43 vs. E:37, p = 0.003) were significant. The gaze-tracking was not statistically significant.

CONCLUSION

The MR headset can be utilised as a valid tool for surgical performance assessment. Outcomes including path length and speed can be valuable metrics captured by the MR Headset during the task completion for detecting surgical proficiency.

Feasibility of tracking in open surgical simulation

The aim of this study was to develop an adequate tracking method for open surgical training, using tracking of the instrument or hand motions.

An open surgical training model and the SurgTrac application were used to track four separate suturing tasks. These tasks were performed with colour markings of either instruments or fingers, to find the most promising setting for reliable tracking.

Four experiments were used to find the optimal settings for the tracking system. Tracking of instruments was not usable for knot tying by hand. Tracking of fingers seemed to be a more promising method. Tagging the fingers with a coloured balloon-tube, seemed to be a more promising method (1.2–3.0% right hand vs. 9.2–17.9% left hand off-screen) than covering the nails with coloured tape (1.5–3.5% right hand vs. 25.5–55.4% left hand off-screen). However, analysis of the videos showed that redness of the hand was seen as red tagging as well. To prevent misinterpreting of the red tag by redness of the hand, white surgical gloves were worn underneath in the last experiment. The off-screen percentage of the right side decreased from 1.0 to 1.2 without gloves to 0.8 with gloves and the off-screen percentage of the left side decreased from 16.9–17.9 to 6.6–7.2, with an adequate tracking mark on the video images.

This study shows that tagging of the index fingers with a red (right) and blue (left) balloon-tube while wearing surgical gloves is a feasible method for tracking movements during basic open suturing tasks.

Septoplasty Training During the COVID-19 Era: Development and Validation of a Novel Low-Cost Simulation Model

OBJECTIVE

In a context of increasingly limited surgical exposition, enhanced by the coronavirus disease 2019 (COVID-19) pandemic context, the objective of this article is to explain the development of a novel low-cost and simple replication animal-based septoplasty training model for otolaryngology residents, to assess its face and construct validity, and to validate a specific rating scale for each task.

STUDY DESIGN

Experimental study.

SETTING

Surgical simulation laboratory.

METHODS

Septoplasty experts divided the procedure into key tasks. A simulator model to perform tasks was developed using pig ears to imitate human nasal septum cartilage, and a Specific Rating Scale was constructed. Trainees and faculty performed all tasks in the model. The participants were videotaped, and operative time, hand movements, and path length were recorded using a motion sensor device. Two blinded experts evaluated the videos with Global and Specific Rating Scales. All participants answered a satisfaction survey.

RESULTS

Fifteen subjects were recruited (7 trainees and 8 faculty). Significantly higher Global Rating Scale score, shorter operative time and path length, and fewer hand movements were observed in the faculty group. The satisfaction survey showed high applicability to a real scenario (mean score of 4.6 out of 5). Specific Rating Scale showed construct and concurrent validity and high reliability.

CONCLUSION

This simulation model and its specific rating scale can be accurately used as a validated surgical assessment tool for endonasal septoplasty skills. Its low cost and simple replicability make it a potentially useful tool in any otolaryngology surgical training program.

Gaze Behaviors of Neurosurgeon in Performing Gross Movements under Microscope

BACKGROUND

Frequent intermittent bleeding control and suction are often necessary during microneurosurgical procedures. We compared the visual searching strategy that guides these types of gross hand movements between expert surgeons and neurosurgical residents.

METHODS

A total of 70 trials of a microsurgical task consisting of moving a cotton ball with bipolar forceps and a suction instrument under the microscope were recorded. Based on surgical videos, we extracted total task time, boarding time, gaze-tool time gap, number of fixations during the boarding time, and target-locked fixation duration. A comparison of these measures between experts and resident surgeons were performed.

RESULTS

No significant difference was found for total task time, boarding time, and number of fixations during the boarding time between the 2 groups of surgeons. However, we found significant differences in the measure of gaze-tool time gap (P < 0.001, partial η² = 0.54) and the target-locked fixation (P < 0.001, partial η² = 0.24). Specifically, expert microsurgeons' eyes move to the target board earlier before their tools in hands move (1.01 ± 0.30 seconds) compared with resident microsurgeons (0.44 ± 0.17 seconds). The target-locked fixation duration was also longer among experts comparing to residents (experts: 1.16 ± 0.82 seconds, residents: 0.40 ± 0.30 seconds).

CONCLUSIONS

Task time analysis for this basic microsurgical task is not sufficient to evaluate participants' level of expertise. Gaze behavior analysis helps to reveal hidden differences between experts and residents. This research provides more evidence that supports the use of gaze analysis for assessing surgeons' skills in microsurgery.

Do Individual Surgeon Preferences Affect Procedural Outcomes?

OBJECTIVES

Surgeon preferences such as instrument and suture selection and idiosyncratic approaches to individual procedure steps have been largely viewed as minor differences in the surgical workflow. We hypothesized that idiosyncratic approaches could be quantified and shown to have measurable effects on procedural outcomes.

METHODS

At the American College of Surgeons (ACS) Clinical Congress, experienced surgeons volunteered to wear motion tracking sensors and be videotaped while evaluating a loop of porcine intestines to identify and repair 2 preconfigured, standardized enterotomies. Video annotation was used to identify individual surgeon preferences and motion data was used to quantify surgical actions. χ 2 analysis was used to determine whether surgical preferences were associated with procedure outcomes (bowel leak).

RESULTS

Surgeons' (N=255) preferences were categorized into 4 technical decisions. Three out of the 4 technical decisions (repaired injuries together, double-layer closure, corner-stitches vs no corner-stitches) played a significant role in outcomes, P <0.05. Running versus interrupted did not affect outcomes. Motion analysis revealed significant differences in average operative times (leak: 6.67 min vs no leak: 8.88 min, P =0.0004) and work effort (leak-path length=36.86 cm vs no leak-path length=49.99 cm, P =0.001). Surgeons who took the riskiest path but did not leak had better bimanual dexterity (leak=0.21/1.0 vs no leak=0.33/1.0, P =0.047) and placed more sutures during the repair (leak=4.69 sutures vs no leak=6.09 sutures, P =0.03).

CONCLUSIONS

Our results show that individual preferences affect technical decisions and play a significant role in procedural outcomes. Future analysis in more complex procedures may make major contributions to our understanding of contributors to procedure outcomes.

Validation of a Newly Developed Competency Assessment Tool for the Posterior Sagittal Anorectoplasty

INTRODUCTION

 The correction of an anorectal malformation (ARM) is complex and relatively infrequent. Simulation training and subsequent assessment may result in better clinical outcomes. Assessment can be done using a competency assessment tool (CAT). This study aims to develop and validate a CAT for the posterior sagittal anorectoplasty (PSARP) on a simulation model.

MATERIALS AND METHODS

 The CAT-PSARP was developed after consultation with experts in the field. The PSARP was divided into five steps, while tissue and instrument handling were scored separately. Participants of pediatric colorectal hands-on courses in 2019 and 2020 were asked to participate. They performed one PSARP procedure on an ARM simulation model, while being assessed by two objective observers using the CAT-PSARP.

RESULTS

 A total of 82 participants were enrolled. A fair interobserver agreement was found for general skills (intraclass correlation coefficient [ICC] = 0.524, p < 0.001), a good agreement for specific skills (ICC = 0.646, p < 0.001), and overall performance (ICC = 0.669, p < 0.001). The experienced group scored higher on all steps (p < 0.001), except for "anoplasty (p = 0.540)," compared with an inexperienced group.

CONCLUSION

 The CAT-PSARP is a suitable objective assessment tool for the overall performance of the included steps of the PSARP for repair of an ARM on a simulation model.

Contextual interference for skills development and transfer in laparoscopic surgery: a randomized controlled trial

INTRODUCTION

Past education literature has shown benefits for random practice schedules (termed contextual interference) for skills retention and transfer to novel tasks. The purpose of fundamentals of laparoscopic surgery (FLS) training is to develop skills in simulation and transfer to new in vivo intraoperative experiences. The study objective was to assess whether individuals trained over a fixed number of trials in the FLS tasks would outperform untrained controls on an unpracticed previously validated bile duct cannulation task and scoring system and to determine whether random training schedules conferred any relative advantage.

METHODS

44 trainees with no laparoscopic experience were recruited to participate. 35 were randomized to practice the FLS tasks using either a blocked or random training schedule. Nine were randomized to no additional training (controls). Participant performance was measured throughout training to monitor skills acquisition and were then tested on an unpracticed bile duct cannulation simulation task 4 to 6 weeks later. Outcomes included previously validated FLS scores and hand-motion analyses.

RESULTS

All 44 participants completed the study. Trained individuals in both groups showed significant improvements in all FLS tasks after training. There were no differences between groups in performance on the cannulation task median scores (Blocked: 89.8 [IQR:37.6]; Random: 83.2 [32.3]; Control: 83.6 [19.1]; p = 0.955), number of hand motions (Blocked: 42.5 [IQR:130.3]; Random: 75.3 [111.3]; Control: 63.0 [71.8]; p = 0.912), or distance traveled by participants hands (Blocked: 2.0 m [IQR:5.8]; Random: 3.8 [8.9]; Control: 2.6 [2.5]; p = 0.816). Cannulation task performance had no correlation with total FLS performance, R2 linear = 0.014, p = 0.445.

CONCLUSIONS

Skills acquired from conventional FLS tasks did not effectively transfer to a laparoscopic bile duct cannulation task. Neither blocked nor random practice schedules conferred a relative advantage. These findings provide evidence that cannulation is a distinct skill from what is taught and assessed in FLS.

Real-Time 3D Tracking of Laparoscopy Training Instruments for Assessment and Feedback

Assessment of minimally invasive surgical skills is a non-trivial task, usually requiring the presence and time of expert observers, including subjectivity and requiring special and expensive equipment and software. Although there are virtual simulators that provide self-assessment features, they are limited as the trainee loses the immediate feedback from realistic physical interaction. The physical training boxes, on the other hand, preserve the immediate physical feedback, but lack the automated self-assessment facilities. This study develops an algorithm for real-time tracking of laparoscopy instruments in the video cues of a standard physical laparoscopy training box with a single fisheye camera. The developed visual tracking algorithm recovers the 3D positions of the laparoscopic instrument tips, to which simple colored tapes (markers) are attached. With such system, the extracted instrument trajectories can be digitally processed, and automated self-assessment feedback can be provided. In this way, both the physical interaction feedback would be preserved and the need for the observance of an expert would be overcome. Real-time instrument tracking with a suitable assessment criterion would constitute a significant step towards provision of real-time (immediate) feedback to correct trainee actions and show them how the action should be performed. This study is a step towards achieving this with a low cost, automated, and widely applicable laparoscopy training and assessment system using a standard physical training box equipped with a fisheye camera.

Design and Validity Evidence for a Unique Endoscopy Simulator Using a Commercial Video Game

Background Procedural simulation enhances early endoscopy training. Multiple commercial simulators are available; however, their application is limited by cost and poor user compliance. First-person "shooter" (FPS) video games are popular. In this study, we aimed to show that a novel in-house designed colonoscope controller used to play an FPS video game shares similar constructs with real-life endoscopy. Methodology Participants completed the first three levels on an FPS video game, Portal (Valve Corporation, Bellevue, WA), first using a conventional controller and then the modified endoscope controller. A total of 12 expert endoscopists and 12 surgical residents with minimal endoscopy experience were evaluated based on completion time, button presses, and hand motion analyses. Results Experts outperformed novices for completion time (expert: 944 seconds; novice: 1,515 seconds; p = 0.006) and hand movements (expert: 1,263.1; novice: 2,052.6; p = 0.004) in using the novel colonoscope controller. There was no difference in button presses or total path length traveled. Furthermore, performance did not differ using conventional game controls. Conclusions Experts outperformed novices using the endoscope but not the conventional controller with respect to the economy of movement and completion time. This result confirms that our endoscope-controlled video game shares similar paradigms with real-life endoscopy and serves as a first step toward creating a more enjoyable and cheaper alternative to commercially available endoscopy simulators.

Identifying Kinematic Markers Associated with Intraoperative Stress during Surgical Training Tasks

Increased levels of stress can impair surgeon performance and patient safety during surgery. The aim of this study is to investigate the effect of short term stressors on laparoscopic performance through analysis of kinematic data. Thirty subjects were randomly assigned into two groups in this IRB-approved study. The control group was required to finish an extended-duration peg transfer task (6 minutes) using the FLS trainer while listening to normal simulated vital signs and while being observed by a silent moderator. The stressed group finished the same task but listened to a period of progressively deteriorating simulated patient vitals, as well as critical verbal feedback from the moderator, which culminated in 30 seconds of cardiac arrest and expiration of the simulated patient. For all subjects, video and position data using electromagnetic trackers mounted on the handles of the laparoscopic instruments were recorded. A statistical analysis comparing time-series velocity, acceleration, and jerk data, as well as path length and economy of volume was conducted. Clinical stressors lead to significantly higher velocity, acceleration, jerk, and path length as well as lower economy of volume. An objective evaluation score using a modified OSATS technique was also significantly worse for the stressed group than the control group. This study shows the potential feasibility and advantages of using the time-series kinematic data to identify the stressful conditions during laparoscopic surgery in near-real-time. This data could be useful in the design of future robot-assisted algorithms to reduce the unwanted effects of stress on surgical performance.

Rethinking Autonomous Surgery: Focusing on Enhancement over Autonomy

CONTEXT

As robot-assisted surgery is increasingly used in surgical care, the engineering research effort towards surgical automation has also increased significantly. Automation promises to enhance surgical outcomes, offload mundane or repetitive tasks, and improve workflow. However, we must ask an important question: should autonomous surgery be our long-term goal?

OBJECTIVE

To provide an overview of the engineering requirements for automating control systems, summarize technical challenges in automated robotic surgery, and review sensing and modeling techniques to capture real-time human behaviors for integration into the robotic control loop for enhanced shared or collaborative control.

EVIDENCE ACQUISITION

We performed a nonsystematic search of the English language literature up to March 25, 2021. We included original studies related to automation in robot-assisted laparoscopic surgery and human-centered sensing and modeling.

EVIDENCE SYNTHESIS

We identified four comprehensive review papers that present techniques for automating portions of surgical tasks. Sixteen studies relate to human-centered sensing technologies and 23 to computer vision and/or advanced artificial intelligence or machine learning methods for skill assessment. Twenty-two studies evaluate or review the role of haptic or adaptive guidance during some learning task, with only a few applied to robotic surgery. Finally, only three studies discuss the role of some form of training in patient outcomes and none evaluated the effects of full or semi-autonomy on patient outcomes.

CONCLUSIONS

Rather than focusing on autonomy, which eliminates the surgeon from the loop, research centered on more fully understanding the surgeon's behaviors, goals, and limitations could facilitate a superior class of collaborative surgical robots that could be more effective and intelligent than automation alone.

PATIENT SUMMARY

We reviewed the literature for studies on automation in surgical robotics and on modeling of human behavior in human-machine interaction. The main application is to enhance the ability of surgical robotic systems to collaborate more effectively and intelligently with human surgeon operators.

Otoskills training during covid-19 pandemic: a before-after study

BACKGROUND

The ongoing COVID-19 pandemic has disrupted the surgical training of residents. There is a real concern that trainees will not be able to meet their training requirements. Low-fidelity surgical simulation appears to be an alternative for surgical training. The educational benefits of repeating ossiculoplasty simulations under a microscope have never been evaluated. With this study we aimed to evaluate the differences in performance scores and on a global rating scale before and after training on an ossiculoplasty simulator.

METHODS

In this quasi-experimental, prospective, single-centre, before-after study with blinded rater evaluation, residents performed five microscopic ossiculoplasty tasks with a difficulty gradient (sliding beads onto rods, the insertion of a partial prosthesis, the insertion of a total prosthesis, and the insertion of a stapedotomy piston under microscopic or endoscopic surgery) before and after training on the same simulator. Performance scores were defined for each task, and total performance scores (score/min) were calculated. All data were collected prospectively.

RESULTS

Six out of seven intermediate residents and 8/9 novices strongly agreed that the simulator was an effective training device and should be included in the ENT residency program. The mean effect of training was a significant increase in the total performance score (+ 0.52 points/min, [95 % CI, 0.40-0.64], p < 0.001), without a significant difference between novice and intermediate residents.

CONCLUSIONS

This preliminary study shows that techniques for middle-ear surgery can be acquired using a simulator, avoiding any risk for patients, even under lockdown measures.

Surgical Hand Gesture Recognition Utilizing Electroencephalogram as Input to the Machine Learning and Network Neuroscience Algorithms

Surgical gestures detection can provide targeted, automated surgical skill assessment and feedback during surgical training for robot-assisted surgery (RAS). Several sources including surgical videos, robot tool kinematics, and an electromyogram (EMG) have been proposed to reach this goal. We aimed to extract features from electroencephalogram (EEG) data and use them in machine learning algorithms to classify robot-assisted surgical gestures. EEG was collected from five RAS surgeons with varying experience while performing 34 robot-assisted radical prostatectomies over the course of three years. Eight dominant hand and six non-dominant hand gesture types were extracted and synchronized with associated EEG data. Network neuroscience algorithms were utilized to extract functional brain network and power spectral density features. Sixty extracted features were used as input to machine learning algorithms to classify gesture types. The analysis of variance (ANOVA) F-value statistical method was used for feature selection and 10-fold cross-validation was used to validate the proposed method. The proposed feature set used in the extra trees (ET) algorithm classified eight gesture types performed by the dominant hand of five RAS surgeons with an accuracy of 90%, precision: 90%, sensitivity: 88%, and also classified six gesture types performed by the non-dominant hand with an accuracy of 93%, precision: 94%, sensitivity: 94%.

Global versus task-specific postoperative feedback in surgical procedure learning

BACKGROUND

Task-specific checklists and global rating scales are both recommended assessment tools to provide constructive feedback on surgical performance. This study evaluated the most effective feedback tool by comparing the effects of the Observational Clinical Human Reliability Analysis (OCHRA) and the Objective Structured Assessment of Technical Skills (OSATS) on surgical performance in relation to the visual-spatial ability of the learners.

METHODS

In a randomized controlled trial, medical students were allocated to either the OCHRA (n = 25) or OSATS (n = 25) feedback group. Visual-spatial ability was measured by a Mental Rotation Test. Participants performed an open inguinal hernia repair procedure on a simulation model twice. Feedback was provided after the first procedure. Improvement in performance was evaluated blindly using a global rating scale (performance score) and hand-motion analysis (time and path length).

RESULTS

Mean improvement in performance score was not significantly different between the OCHRA and OSATS feedback groups (P = .100). However, mean improvement in time (371.0 ± 223.4 vs 274.6 ± 341.6; P = .027) and path length (53.5 ± 42.4 vs 34.7 ± 39.0; P = .046) was significantly greater in the OCHRA feedback group. When stratified by mental rotation test scores, the greater improvement in time (P = .032) and path length (P = .053) was observed only among individuals with low visual-spatial abilities.

CONCLUSION

A task-specific (OCHRA) feedback is more effective in improving surgical skills in terms of time and path length in novices compared to a global rating scale (OSATS). The effects of a task-specific feedback are present mostly in individuals with lower visual-spatial abilities.

Utilization of Flexible-Wearable Sensors to Describe the Kinematics of Surgical Proficiency

BACKGROUND

Traditional assessment (e.g., checklists, videotaping) for surgical proficiency may lead to subjectivity and does not predict performance in the clinical setting. Hand motion analysis is evolving as an objective tool for grading technical dexterity; however, most devices accompany with technical limitations or discomfort. We purpose the use of flexible wearable sensors to evaluate the kinematics of surgical proficiency.

METHODS

Surgeons were recruited and performed a vascular anastomosis task in a single institution. A modified objective structured assessment of technical skills (mOSATS) was used for technical qualification. Flexible wearable sensors (BioStamp RCTM, mc10 Inc., Lexington, MA) were placed on the dorsum of the dominant hand (DH) and nondominant hand (nDH) to measure kinematic parameters: path length (T_path), mean (V_mean) and peak (V_peak) velocity, number of hand movements (N_move), ratio of DH to nDH movements (rMov), and time of task (tTask) and further compared with the mOSATS score.

RESULTS

Participants were categorized as experts (n = 12) and novices (n = 8) based on a cutoff mean mOSATS score. Significant differences for tTask (P = 0.02), rMov (P = 0.07), DH T_path (P = 0.04), V_mean (P = 0.07), V_peak (P = 0.04), and nDH N_move (P = 0.02) were in favor of the experts. Overall, mOSATS had significant correlation with tTask (r = -0.69, P = 0.001), N_move of DH (r = -0.44, P = 0.047) and nDH (r = -0.66, P = 0.001), and rMov (r = 0.52, P = 0.017).

CONCLUSIONS

Hand motion analysis evaluated by flexible wearable sensors is feasible and informative. Experts utilize coordinated two-handed motion, whereas novices perform one-handed tasks in a hastily jerky manner. These tendencies create opportunity for improvement in surgical proficiency among trainees.

Automatic and Objective Assessment of Motor Skills Performance in Flexible Bronchoscopy

BACKGROUND

Motor skills have been identified as a useful measure to evaluate competency in bronchoscopy. However, no automatic assessment system of motor skills with a clear pass/fail criterion in flexible bronchoscopy exists.

OBJECTIVES

The objective of the study was to develop an objective and automatic measure of motor skills in bronchoscopy and set a pass/fail criterion.

METHODS

Participants conducted 3 bronchoscopies each in a simulated setting. They were equipped with a Myo Armband that measured lower arm movements through an inertial measurement unit, and hand and finger motions through electromyography sensors. These measures were composed into an objective and automatic composite score of motor skills, the motor bronchoscopy skills score (MoBSS).

RESULTS

Twelve novices, eleven intermediates, and ten expert bronchoscopy operators participated, resulting in 99 procedures available for assessment. MoBSS was correlated with a higher diagnostic completeness (Pearson's correlation, r = 0.43, p < 0.001) and a lower procedure time (Pearson's correlation, r = -0.90, p < 0.001). MoBSS was able to differentiate operator performance based on the experience level (one-way ANOVA, p < 0.001). Using the contrasting groups' method, a passing score of -0.08 MoBSS was defined that failed 30/36 (83%) novice, 5/33 (15%) intermediate, and 1/30 (3%) expert procedures.

CONCLUSIONS

MoBSS can be used as an automatic and unbiased assessment tool for motor skills performance in flexible bronchoscopy. MoBSS has the potential to generate automatic feedback to help guide trainees toward expert performance.

Quantifying the scientific interest in surgical training and education: numerical evidence of a PubMed analysis

The scientific interest (SI) for a given field can be ascertained by quantifying the volume of published research. We quantified the SI in surgical education to clarify the extent of worldwide efforts on this crucial factor required to improve health-care systems. A set of Medical Subject Headings (MeSH) was defined for the PubMed search. The number of Pubmed Indexed Papers (nPIP) relevant to the SI was extracted from database conception to December 2016 and their distribution and evolution by country were analyzed at 10-year intervals. Population Adjusted Index (PAI) and Medical School Adjusted Index (MSAI) analyses were performed for countries with the nPIP > 30. We identified 51,713 articles written in 33 different languages related to surgical education; 87.6% of these were written in English. General surgery was the leading surgical specialty. The overall nPIP doubled every 10 years from 1987 (from 6009 to 13,501, to 26,272) but stabilized at 3707, 3800 and 3433 in the past 3 years, respectively. The PAI and MSAI analyses showed that the USA, United Kingdom, New Zealand, Canada, Australia and Ireland are top producers of published research in surgical education, constituting a combined 62.88% of the nPIP. Our quantification of the change in SI in surgical education and training gives a clear picture of evolution, efforts and leadership worldwide over time. This picture mirrors an international academic society that should encourage all those involved in surgical education to improve efforts in educational research.

Is Experience in Hemodialysis Cannulation Related to Expertise? A Metrics-based Investigation for Skills Assessment

Cannulation is not only one of the most common medical procedures but also fraught with complications. The skill of the clinician performing cannulation directly impacts cannulation outcomes. However, current methods of teaching this skill are deficient, relying on subjective demonstrations and unrealistic manikins that have limited utility for skills training. Furthermore, of the factors that hinders effective continuing medical education is the assumption that clinical experience results in expertise. In this work, we examine if objective metrics acquired from a novel cannulation simulator are able to distinguish between experienced clinicians and established experts, enabling the measurement of true expertise. Twenty-two healthcare professionals, who practiced cannulation with varying experience, performed a simulated arteriovenous fistula cannulation task on the simulator. Four clinicians were peer-identified as experts while the others were designated to the experienced group. The simulator tracked the motion of the needle (via an electromagnetic sensor), rendered blood flashback function (via an infrared light sensor), and recorded pinch forces exerted on the needle (via force sensing elements). Metrics were computed based on motion, force, and other sensor data. Results indicated that, with near 80% of accuracy using both logistic regression and linear discriminant analysis, the objective metrics differentiated between experts and the experienced, including identifying needle motion and finger force as two prominent features that distinguished between the groups. Furthermore, results indicated that expertise was not correlated with years of experience, validating the central hypothesis of the study. These insights contribute to structured and standardized medical skills training by enabling a meaningful definition of expertise and could potentially lead to more effective skills training methods.

Enhanced Training Benefits of Video Recording Surgery With Automated Hand Motion Analysis

Background

Hand motion analysis by video recording during surgery has potential for evaluation of surgical performance. The aim was to identify how technical skill during open surgery can be measured unobtrusively by video recording during a surgical procedure. We hypothesized that procedural-step timing, hand movements, instrument use and Shannon entropy differ with expertise and training and are concordant with a performance-based validated individual procedure score.

Methods

Surgeon and non-surgeon participants with varying training and levels of expertise were video recorded performing axillary artery exposure and control (AA) on un-preserved cadavers. Color-coded gloves permitted motion-tracking and automated extraction of entropy data from recordings. Timing and instrument-use metrics were obtained through observational video reviews. Shannon entropy measured speed, acceleration and direction by computer-vision algorithms. Findings were compared with individual procedure score for AA performance

Results

Experts had lowest entropy values, idle time, active time and shorter time to divide pectoralis minor, using fewer instruments. Residents improved with training, without reaching expert levels, and showed deterioration 12–18 months later. Individual procedure scores mirrored these results. Non-surgeons differed substantially.

Conclusions

Hand motion entropy and timing metrics discriminate levels of surgical skill and training, and these findings are congruent with individual procedure score evaluations. These measures can be collected using consumer-level cameras and analyzed automatically with free software. Hand motion with video timing data may have widespread application to evaluate resident performance and can contribute to the range of evaluation and testing modalities available to educators, training course designers and surgical quality assurance programs.

Supplementary Information

The online version of this article (10.1007/s00268-020-05916-1) contains supplementary material, which is available to authorized users.

Immersive virtual reality enables technical skill acquisition for scrub nurses in complex revision total knee arthroplasty

INTRODUCTION

Immersive Virtual Reality (iVR) is a novel technology which can enhance surgical training in a virtual environment without supervision. However, it is untested for the training to select, assemble and deliver instrumentation in orthopaedic surgery-typically performed by scrub nurses. This study investigates the impact of an iVR curriculum on this facet of the technically demanding revision total knee arthroplasty.

MATERIALS AND METHODS

Ten scrub nurses completed training in four iVR sessions over a 4-week period. Initially, nurses completed a baseline real-world assessment, performing their role with real equipment in a simulated operation assessment. Each subsequent iVR session involved a guided mode, where the software taught participants the procedural choreography and assembly of instrumentation in a simulated operating room. In the latter three sessions, nurses also undertook an assessment in iVR. Outcome measures were related to procedural sequence, duration of surgery and efficiency of movement. Transfer of skills from iVR to the real world was assessed in a post-training simulated operation assessment. A pre- and post-training questionnaire assessed the participants knowledge, confidence and anxiety.

RESULTS

Operative time reduced by an average of 47% across the 3 unguided sessions (mean 55.5 ± 17.6 min to 29.3 ± 12.1 min, p > 0.001). Assistive prompts reduced by 75% (34.1 ± 16.8 to 8.6 ± 8.8, p < 0.001), dominant hand motion by 28% (881.3 ± 178.5 m to 643.3 ± 119.8 m, p < 0.001) and head motion by 36% (459.9 ± 99.7 m to 292.6 ± 85.3 m, p < 0.001). Real-world skill improved from 11% prior to iVR training to 84% correct post-training. Participants reported increased confidence and reduced anxiety in scrubbing for rTKA procedures (p < 0.001).

CONCLUSIONS

For scrub nurses, unfamiliarity with complex surgical procedures or equipment is common. Immersive VR training improved their understanding, technical skills and efficiency. These iVR-learnt skills transferred into the real world.

Analysis of Kinematic Differences in Hand Motion between Novice and Experienced Operators in IR: A Pilot Study

PURPOSE

To prospectively validate electromagnetic hand motion tracking in interventional radiology to detect differences in operator experience using simulation.

METHODS

Sheath task: Six attending interventional radiologists (experts) and 6 radiology trainees (trainees) placed a wire through a sheath and performed a "pin-pull" maneuver, while an electromagnetic motion detection system recorded the hand motion. Radial task: Eight experts and 12 trainees performed palpatory radial artery access task on a radial access simulator. The trainees repeated the task with the nondominant hand. The experts were classified by their most frequent radial artery access technique as having either palpatory, ultrasound, or overall limited experience. The time, path length, and number of movements were calculated. Mann-Whitney U tests were used to compare the groups, and P < .05 was considered significant.

RESULTS

Sheath task: The experts took less time, had shorter path lengths, and used fewer movements than the trainees (11.7 seconds ± 3.3 vs 19.7 seconds ± 6.5, P < .01; 1.1 m ± 0.3 vs 1.4 m ± 0.4, P < .01; and 19.5 movements ± 8.5 vs 31.0 movements ± 8.0, P < .01, respectively). Radial task: The experts took less time, had shorter path lengths, and used fewer movements than the trainees (24.2 seconds ± 10.6 vs 33.1 seconds ± 16.9, P < .01; 2.0 m ± 0.5 vs 3.0 m ± 1.9, P < .001; and 36.5 movements ± 15.0 vs 54.5 movements ± 28.0, P < .001, respectively). The trainees had a shorter path length for their dominant hand than their nondominant hand (3.0 m ± 1.9 vs 3.5 m ± 1.9, P < .05). The expert palpatory group had a shorter path length than the ultrasound and limited experience groups (1.8 m ± 0.4 vs 2.0 m ± 0.4 and 2.3 m ± 1.2, respectively, P < .05).

CONCLUSIONS

Electromagnetic hand motion tracking can differentiate between the expert and trainee operators for simulated interventional tasks.

Navigation-assisted anchor insertion in shoulder arthroscopy: a validity study

BACKGROUND

This study aimed to compare conventional and navigation-assisted arthroscopic rotator cuff repair in terms of anchor screw insertion.

METHODS

The surgical performance of five operators while using the conventional and proposed navigation-assisted systems in a phantom surgical model and cadaveric shoulders were compared. The participating operators were divided into two groups, the expert group (n = 3) and the novice group (n = 2). In the phantom model, the experimental tasks included anchor insertion in the rotator cuff footprint and sutures retrieval. A motion analysis camera system was used to track the surgeons' hand movements. The surgical performance metric included the total path length, number of movements, and surgical duration. In cadaveric experiments, the repeatability and reproducibility of the anchor insertion angle were compared among the three experts, and the feasibility of the navigation-assisted anchor insertion was validated.

RESULTS

No significant differences in the total path length, number of movements, and time taken were found between the conventional and proposed systems in the phantom model. In cadaveric experiments, however, the clustering of the anchor insertion angle indicated that the proposed system enabled both novice and expert operators to reproducibly insert the anchor with an angle close to the predetermined target angle, resulting in an angle error of < 2° (P = 0.0002).

CONCLUSION

The proposed navigation-assisted system improved the surgical performance from a novice level to an expert level. All the experts achieved high repeatability and reproducibility for anchor insertion. The navigation-assisted system may help surgeons, including those who are inexperienced, easily familiarize themselves to of suture anchors insertion in the right direction by providing better guidance for anchor orientation.

LEVEL OF EVIDENCE

A retrospective study (level 2).

A Review of Orthopaedic Surgical Set-Up and Introduction of the TULIPS Mnemonic - Six Simple Steps for Optimising Set-Up in Orthopaedic Surgery

Conducting a thorough check to ensure that all equipment and personnel are positioned correctly at the start of any operation is essential for both the safety of the surgical team and the patient outcome. Orthopaedic surgery in particular carries a high risk of occupational injury and this group could benefit greatly from ergonomic improvements. This review highlights multiple factors that can influence safety of surgeons, surgical efficiency and patient outcomes. "TULIPS" is a mnemonic that lists six key steps in optimising the surgical procedure through effective positioning of equipment and personnel pre-operatively. This was trialled by distribution amongst orthopaedic registrars regionally and it received excellent feedback, with the majority changing their current practice. Here we report that using this simple and memorable checklist can assist orthopaedic surgeons in setting up the operating theatre, facilitating ergonomic improvements that can reduce the risk of musculoskeletal injury and radiation exposure.

Towards Development of an Open Surgery Competency Assessment for Residents (OSCAR) Tool - A Systematic Review of the Literature and Delphi Consensus

OBJECTIVE

Competency-based education has mandated accurate intra-operative assessment tools. We aimed to define consensus-based open surgical skills perceived by experts as critical for assessment.

DESIGN

A mixed-method design was employed: systematic review and e-Delphi methodology.

SETTING

The study was performed at McGill University-affiliated large tertiary academic centers in Montreal, Quebec, Canada.

PARTICIPANTS

Per PRISMA guidelines, a peer-reviewed search strategy was employed. Studies published in English and those describing technical skill assessment of open abdominal surgery were included; subspecialty-specific skills, conference abstracts, academic memoirs were excluded. Most-cited skills were subjected to e-Delphi methodology to identify those deemed essential by experts, based a 3-point Likert scale. Eighteen McGill University-affiliated general surgeons, representing a variety of subspecialties of General Surgery, were invited to answer the questionnaire.

RESULTS

Around 120 of 4285 references were retained for analysis. The 12 most cited skills included suturing, tissue and instrument handling, movement economy, instrument knowledge, knot tying, flow, knowledge of procedure, completion time, dissection technique, knowledge of anatomy and sterile technique; 6 of these achieved high or perfect scores and agreement after 2 rounds of survey: suturing, sterile technique, knot tying, knowledge of anatomy, knowledge of procedure, and tissue handling. Median standard deviation decreased (0.495 to 0.450) from first to second round, indicating improvement in consensus.

CONCLUSION

These results will help develop and validate the OSCAR (objective structured clinical assessment rubric) assessment tool for immediate intra-operative feedback of open technical skills for surgical trainees.

Measurement of Three-Dimensional Force Applied to Elastic Suture Training Pads for Laparoscopic Suturing

Laparoscopic surgery requires highly trained skills to manipulate the laparoscopic instruments. The effectiveness and efficiency of manipulation training are expected to increase by quantitatively evaluating how a series of trainee's manipulations are different from those conducted by skilled surgeons and providing feedback. However, such detailed feed- back is not available because adequate measurement systems have not been proposed to measure the precise forces applied to suture training pads without disturbing the delicate manipulations. Therefore, we proposed a sensorized suture training pad satisfying the above requirements. Three surgical residents participated in an experiment to measure time sequences of three-dimensional forces applied to the pad when executing a transfixion suture. As a result, the differences in manipulation patterns among the residents were found by dividing the transfixion operation based on the sequential force data and recorded videos.

Low-Cost Haptic Simulation Using Material Fracture

Medical simulation training is widely used to effectively train for invasive medical procedures such as peripheral nerve blocks. Traditionally, accurate haptic training relies on expensive cadavers, manikins, or advanced haptic robots. Proposed herein is a novel concept for haptic training called the low-cost haptic force needle insertion simulator (LCNIS), which uses material fracture inside disposable cartridges to accurately replicate the force of inserting a needle into tissue. Cadaver and material fracture experiments were performed to develop and determine the accuracy of the LCNIS. The material testing showed that polycarbonate had the highest maximum needle puncture force of the materials tested, 9.85 N, and that fluorinated ethylene propylene had the lowest maximum puncture force, 0.84 N. The cadaver results showed that the error between the three peak forces in a cadaver and a cadaver mimicking cartridge was 1.00 N, 0.01 N, and 1.54 N. The standard deviation of these peaks was 0.60 N, 0.55 N, and 0.41 N. This novel method of haptic simulation can easily be adapted to recreate any type of force and, therefore, could be utilized to train for a wide variety of medical procedures.

Effects of Sleep Deprivation on Surgeons Dexterity

Sleep deprivation is an ordinary aspect in the global society and its prevalence is increasing. Chronic and acute sleep deprivation have been linked to diabetes and heart diseases as well as depression and enhanced impulsive behaviors. Surgeons are often exposed to long hour on call and few hours of sleep in the previous days. Nevertheless, few studies have focused their attention on the effects of sleep deprivation on surgeons and more specifically on the effects of sleep deprivation on surgical dexterity, often relying on virtual surgical simulators. A better understanding of the consequences of sleep loss on the key surgical skill of dexterity can shed light on the possible risks associated to a sleepy surgeon. In this paper, the authors aim to provide a comprehensive review of the relationship between sleep deprivation and surgical dexterity.

Can Haptic Simulators Distinguish Expert Performance? A Case Study in Central Venous Catheterization in Surgical Education

INTRODUCTION

High-tech simulators are gaining popularity in surgical training programs because of their potential for improving clinical outcomes. However, most simulators are static in nature and only represent a single anatomical patient configuration. The Dynamic Haptic Robotic Training (DHRT) system was developed to simulate these diverse patient anatomies during Central Venous Catheterization (CVC) training. This article explores the use of the DHRT system to evaluate objective metrics for CVC insertion by comparing the performance of experts and novices.

METHODS

Eleven expert surgeons and 13 first-year surgical residents (novices) performed multiple needle insertion trials on the DHRT system. Differences between expert and novice performance on the following five metrics were assessed using a multivariate analysis of variance: path length, standard deviation of deviations (SDoD), average velocity, distance to the center of the vessel, and time to complete (TtC) the needle insertion. A regression analysis was performed to identify if expertise could be predicted using these metrics. Then, a curve fit was conducted to identify whether learning curves were present for experts or novices on any of these five metrics.

RESULTS

Time to complete the insertion and SDoD of the needle tip from an ideal path were significantly different between experts and novices. Learning curves were not present for experts but indicated a significant decrease in path length and TtC for novices.

CONCLUSIONS

The DHRT system was able to identify significant differences in TtC and SDoD between experts and novices during CVC needle insertion procedures. In addition, novices were shown to improve their skills through DHRT training.

Surgery Tutor for Computational Assessment of Technical Proficiency in Soft-Tissue Tumor Resection in a Simulated Setting

BACKGROUND

In competency-based medical education, progression between milestones requires reliable and valid methods of assessment. Surgery Tutor is an open-source motion tracking platform developed to objectively assess technical proficiency during open soft-tissue tumor resections in a simulated setting. The objective of our study was to provide evidence in support of construct validity of the scores obtained by Surgery Tutor. We hypothesized that Surgery Tutor would discriminate between novice, intermediate, and experienced operators.

METHODS

Thirty participants were assigned to novice, intermediate, or experienced groups, based on the number of prior soft-tissue resections performed. Each participant resected 2 palpable and 2 nonpalpable lesions from a soft-tissue phantom. Surgery Tutor was used to track hand and instrument motions, number of tumor breaches, and time to perform each resection. Mass of excised specimens and margin status were also recorded.

RESULTS

Surgery Tutor scores demonstrated "moderate" to "good" internal structure (test-retest reliability) for novice, intermediate, and experienced groups (interclass correlation coefficient = 0.596, 0.569, 0.737; p < 0.001). Evidence in support of construct validity (consequences) was demonstrated by comparing scores of novice, intermediate, and experienced participantsfor number of hand and instrument motions (690 ± 190, 597 ± 169, 469 ± 110; p < 0.001), number of tumor breaches (29 ± 34, 16 ± 11, 9 ± 6; p < 0.001), time per resection (677 ± 331 seconds, 561 ± 210 seconds, 449 ± 148 seconds; p < 0.001), mass of completely excised specimens (22 ± 7g, 21 ± 11g, 17 ± 6 g; p = 0.035), and rate of positive margin (68%, 50%, 28%; p < 0.001). There was "strong" and "moderate" relationships between motion scores and Objective Structured Assessment of Technical Skill scores, and time per resection and Objective Structured Assessment of Technical Skill scores respectively (r = -0.60, p < 0.001; r = -0.54, p < 0.001).

CONCLUSION

Surgery Tutor scores demonstrate evidenceof construct validity with regards to good internal structure, consequences, and relationship to other variables in the assessment of technical proficiency duringopen soft-tissue tumor resections in a simulated setting. Utilization of Surgery Tutor can provide formative feedback and objective assessment of surgical proficiency in a simulated setting.

Machine learning methods for automated technical skills assessment with instructional feedback in ultrasound-guided interventions

OBJECTIVE

Currently, there is a worldwide shift toward competency-based medical education. This necessitates the use of automated skills assessment methods during self-guided interventions training. Making assessment methods that are transparent and configurable will allow assessment to be interpreted into instructional feedback. The purpose of this work is to develop and validate skills assessment methods in ultrasound-guided interventions that are transparent and configurable.

METHODS

We implemented a method based upon decision trees and a method based upon fuzzy inference systems for technical skills assessment. Subsequently, we validated these methods for their ability to predict scores of operators on a 25-point global rating scale in ultrasound-guided needle insertions and their ability to provide useful feedback for training.

RESULTS

Decision tree and fuzzy rule-based assessment performed comparably to state-of-the-art assessment methods. They produced median errors (on a 25-point scale) of 1.7 and 1.8 for in-plane insertions and 1.5 and 3.0 for out-of-plane insertions, respectively. In addition, these methods provided feedback that was useful for trainee learning. Decision tree assessment produced feedback with median usefulness 7 out of 7; fuzzy rule-based assessment produced feedback with median usefulness 6 out of 7.

CONCLUSION

Transparent and configurable assessment methods are comparable to the state of the art and, in addition, can provide useful feedback. This demonstrates their value in self-guided interventions training curricula.