Objective metrics for hand-sewn bowel anastomoses can differentiate novice from expert surgeons

Validity of a virtual reality-based straight coloanal anastomosis simulator

PURPOSE

Current training methods for surgical trainees are inadequate because they are costly, low-fidelity, or have a low skill ceiling. This work aims to expand available virtual reality training options by developing a VR trainer for straight coloanal anastomosis (SCA), one of the Colorectal Objective Structured Assessment of Technical Skills (COSATS) tasks.

METHODS

We developed a VR-based SCA simulator to evaluate trainees based on their performance. To increase the immersiveness, alongside the VR headset, we used haptics as the primary method of interaction with the simulation. We also implemented objective performance metrics to evaluate trainee performance throughout the simulation.

RESULTS

We presented our performance metrics to 27 participants for an Expert Consensus Survey (5-point Likert scale) and created weights for our metrics. The weighted average scores for the 24 task-specific metrics ranged from 3.5 to 5. Additionally, for the general metrics, the scores spanned from 3.3 to 4.6. In the second phase of our study, we conducted a study with 16 participants (novice n = 9, expert n = 7). Based on the performance, experts outperformed novices by 8.56% when referring to the total score (p = 0.0041). Three of the measurable metrics, purse suture (p = 0.0797), retracting the anvil (p = 0.0738), and inserting the colonoscope (p = 0.0738) showed a significant difference between experts and novices. Experts were smoother with their hand motions by 3.67% per second and took 70.77% longer paths to complete the same tasks.

CONCLUSION

We created a high-fidelity coloanal anastomosis VR simulator. The simulator runs in real-time while allowing high immersion with a VR headset, deformable bodies, and a haptic device while providing objective feedback through performance metrics. Experts obtained higher scores throughout the simulation, including the quiz to demonstrate procedural knowledge, the metrics to demonstrate experience in steps/procedure, and control of their basic surgical skills and hand movements.

From the simulation lab to the operating room: simulation performance predicts intraoperative performance in robotic gastrojejunostomy

BACKGROUND

Simulation and video-based assessment (VBA) offer residents the opportunity to develop operative skills while ensuring patient safety. This study aims to determine whether simulation training can predict residents' operative performance, focusing on the gastrojejunal (GJ) anastomosis during robotic pancreatoduodenectomy.

METHODS

Twenty-seven general surgery residents completed simulated robotic GJ drills and subsequently performed GJs in the operating room (OR). Both simulated and intraoperative performances were video recorded and retrospectively assessed by two blinded graders using the Objective Structural Assessment of Technical Skills (OSATS) scale, time to completion, and occurrence of errors. Intraoperative GJ OSATS scores were compared in cases with and without Clinically Relevant Delayed Gastric Emptying (CRDGE). Statistical analysis was performed using Spearman's rho, Chi-square, and Kruskal-Wallis tests.

RESULTS

For simulated GJs, the median OSATS score was 29 (IQR 27-33), time to completion was 30 min (IQR 27-35), and 11 cases had at least one error. Intraoperative GJs had a median OSATS of 30 (IQR 27-31), time to completion of 41 min (IQR 36-51), and errors occurred in nine cases. The OSATS score on the simulated GJs demonstrated a significant positive correlation to the OSATS score on the operative GJs (r = 0.74; p < 0.001) and less time to completion (r =  - 0.68; p < 0.001). A shorter simulated GJ completion time significantly correlated with a higher intraoperative OSATS score (r =  - 0.52; p < 0.01). Residents with at least one error in the simulated GJs had lower OSATS scores and higher times intraoperatively. Those cases with CRDGE had significantly lower intraoperative OSATS scores than those without CRDGE.

CONCLUSION

Performance on a simulated robotic GJ environment is a robust predictor of OR GJ performance, demonstrating predictive validity. VBA of residents' operative GJ performance is associated with the presentation of CRDGE. Simulation-based training may be crucial to optimizing surgical outcomes before operating on patients.

Validity of task-specific metrics for assessment in perineal proctectomy

BACKGROUND

Perineal proctectomy is a complex procedure that requires advanced skills. Currently, there are no simulators for training in this procedure. As part of our objective of developing a virtual reality simulator, our goal was to develop and validate task-specific metrics for the assessment of performance for this procedure. We conducted a three-phase study to establish task-specific metrics, obtain expert consensus on the appropriateness of the developed metrics, and establish the discriminant validity of the developed metrics.

METHODS

In phase I, we utilized hierarchical task analysis to formulate the metrics. In phase II, a survey involving expert colorectal surgeons determined the significance of the developed metrics. Phase III was aimed at establishing the discriminant validity for novices (PGY1-3) and experts (PGY4-5 and faculty). They performed a perineal proctectomy on a rectal prolapse model. Video recordings were independently assessed by two raters using global ratings and task-specific metrics for the procedure. Total scores for both metrics were computed and analyzed using the Kruskal-Wallis test. A Mann-Whitney U test with Benjamini-Hochberg correction was used to evaluate between-group differences. Spearman's rank correlation coefficient was computed to assess the correlation between global and task-specific scores.

RESULTS

In phase II, a total of 23 colorectal surgeons were recruited and consensus was obtained on all the task-specific metrics. In phase III, participants (n = 22) included novices (n = 15) and experts (n = 7). There was a strong positive correlation between the global and task-specific scores (rs = 0.86; P < 0.001). Significant between-group differences were detected for both global (χ2 = 15.38; P < 0.001; df = 2) and task-specific (χ2 = 11.38; P = 0.003; df = 2) scores.

CONCLUSIONS

Using a biotissue rectal prolapse model, this study documented high IRR and significant discriminant validity evidence in support of video-based assessment using task-specific metrics.

Validity and reliability evidence support task-specific metrics for laparoscopic fundoplication

BACKGROUND

Laparoscopic hiatal hernia repair (LHHR) is a complex operation requiring advanced surgical training. Surgical simulation offers a potential solution for learning complex operations without the need for high surgical volume. Our goal is to develop a virtual reality (VR) simulator for LHHR; however, data supporting task-specific metrics for this procedure are lacking. The purpose of this study was to develop and assess validity and reliability evidence of task-specific metrics for the fundoplication phase of LHHR.

METHODS

In phase I, structured interviews with expert foregut surgeons were conducted to develop task-specific metrics (TSM). In phase II, participants with varying levels of surgical expertise performed a laparoscopic Nissen fundoplication procedure on a porcine stomach explant. Video recordings were independently assessed by two blinded graders using global and TSM. An intraclass correlation coefficient (ICC) was used to assess interrater reliability (IRR). Performance scores were compared using a Kruskal-Wallis test. Spearman's rank correlation was used to evaluate the association between global and TSM.

RESULTS

Phase I of the study consisted of 12 interviews with expert foregut surgeons. Phase II engaged 31 surgery residents, a fellow, and 6 attendings in the simulation. Phase II results showed high IRR for both global (ICC = 0.84, p < 0.001) and TSM (ICC = 0.75, p < 0.001). Significant between-group differences were detected for both global (χ2 = 24.01, p < 0.001) and TSM (χ2 = 18.4, p < 0.001). Post hoc analysis showed significant differences in performance between the three groups for both metrics (p < 0.05). There was a strong positive correlation between the global and TSM (rs = 0.86, p < 0.001).

CONCLUSION

We developed task-specific metrics for LHHR and using a fundoplication model, we documented significant reliability and validity evidence. We anticipate that these LHHR task-specific metrics will be useful in our planned VR simulator.