ACOG Simulation Working Group: A Needs Assessment of Simulation Training in OB/GYN Residencies and Recommendations for Future Research

Impact of a virtual reality-based simulation training for shoulder dystocia on human and technical skills among caregivers: a randomized-controlled trial

This study analyzed the adherence to the modified Advanced Life Support in Obstetrics (ALSO) algorithm (HELP-RER) for handling shoulder dystocia (SD) using a virtual reality (VR) training modality. Secondary outcomes were improvements in the post-training diagnosis-to-delivery time, human skills factors (HuFSHI), and perceived task-load index (TLX). Prospective, case-control, single-blind, 1:1 randomized crossover study. Participants were shown a 360° VR video of SD management. The control group was briefed theoretically. Both groups underwent HuFSHI and HELP-RER score assessments at baseline and after the manikin-based training. The TLX questionnaire was then administered. After a washout phase of 12 weeks, we performed a crossover, and groups were switched. There were similar outcomes between groups during the first training session. However, after crossover, the control group yielded significantly higher HELP-RER scores [7 vs. 6.5; (p = 0.01)], with lower diagnosis-to-delivery-time [85.5 vs. 99 s; (p = 0.02)], and TLX scores [57 vs. 68; (p = 0.04)]. In the multivariable linear regression analysis, VR training was independently associated with improved HELP-RER scores (p = 0.003). The HuFSHI scores were comparable between groups. Our data demonstrated the feasibility of a VR simulation training of SD management for caregivers. Considering the drawbacks of common high-fidelity trainings, VR-based simulations offer new perspectives.

Validation of a Simulation Model for Robotic Myomectomy

STUDY OBJECTIVE

Several simulation models have been evaluated for gynecologic procedures such as hysterectomy, but there are limited published data for myomectomy. This study aimed to assess the validity of a low-cost robotic myomectomy model for surgical simulation training.

DESIGN

Prospective cohort simulation study.

SETTING

Surgical simulation laboratory.

PARTICIPANTS

Twelve obstetrics and gynecology residents and 4 fellowship-trained minimally invasive gynecologic surgeons were recruited for a 3:1 novice-to-expert ratio.

INTERVENTIONS

A robotic myomectomy simulation model was constructed using <$5 worth of materials: a foam cylinder, felt, a stress ball, bandage wrap, and multipurpose sealing wrap. Participants performed a simulation task involving 2 steps: fibroid enucleation and hysterotomy repair. Video-recorded performances were timed and scored by 2 blinded reviewers using the validated Global Evaluative Assessment of Robotic Skills (GEARS) scale (5-25 points) and a modified GEARS scale (5-40 points), which adds 3 novel domains specific to robotic myomectomy. Performance was also scored using predefined task errors. Participants completed a post-task questionnaire assessing the model's realism and utility.

MEASUREMENTS AND MAIN RESULTS

Median task completion time was shorter for experts than novices (9.7 vs 24.6 min, p = .001). Experts scored higher than novices on both the GEARS scale (median 23 vs 12, p = .004) and modified GEARS scale (36 vs 20, p = .004). Experts made fewer task errors than novices (median 15.5 vs 37.5, p = .034). For interrater reliability of scoring, the intraclass correlation coefficient was calculated to be 0.91 for the GEARS assessment, 0.93 for the modified GEARS assessment, and 0.60 for task errors. Using the contrasting groups method, the passing mark for the simulation task was set to a minimum modified GEARS score of 28 and a maximum of 28 errors. Most participants agreed that the model was realistic (62.5%) and useful for training (93.8%).

CONCLUSION

We have demonstrated evidence supporting the validity of a low-cost robotic myomectomy model. This simulation model and the performance assessments developed in this study provide further educational tools for robotic myomectomy training.

Surgical education in urogynecology from low fidelity to virtual reality: Systematic review

IMPORTANCE

Increase dissemination of educational tools in urogynecology.

OBJECTIVE

Describe the effectiveness and public availability of published educational tools for urogynecologic surgery.

STUDY DESIGN

A systematic review was conducted by searching MEDLINE, EMBASE, Cochrane Library, and Web of Science from 1946 to 2023 for articles describing educational tools in urogynecology. There were no restrictions on study design or language. Data were extracted in duplicate using a standardized piloted extraction form, and outcomes were combined descriptively.

RESULTS

2997 titles, 457 abstracts, and 97 full-text articles were analyzed. Of the 97 interventions included, 43 were manuscripts and 54 were conference abstracts. The median study quality was low, with a moderate risk of bias. Six intervention categories were identified: didactics, animal models, cadavers, static models, extended reality (XR), and multimodal workshops. Didactics were subjectively useful for teaching pelvic anatomy and diseases and improving surgical techniques. If good quality, animal models and cadavers provided visual and tactile learning and assessed performance in real-time. Animal models were also anatomically realistic and useful at half the cost of cadavers. Static models and XR improved confidence, knowledge, skills, and error rates despite lack of realism and accurate tissue texture in some models and steep learning curve with XR. Only four models were commercially accessible. Most studies did not assess long-term (>6 months) retention.

CONCLUSION

All educational modalities for urogynecologic surgery are largely realistic and increase participant satisfaction, preparedness, knowledge, skills, and likelihood of use. But only 40% advanced to manuscripts, and even fewer (<5%) were widely available.

Comparison of practice vs theory model training for vacuum-assisted delivery: A randomized controlled trial

OBJECTIVE

To assess the effect of two modalities of simulation training on acquisition/retention of skills for performing operative vaginal delivery.

DESIGN

Randomized, controlled, single-centre study.

SETTING

A tertiary referral hospital in Italy.

PARTICIPANTS

Twenty residents from a single university programme and two young specialists.

INTERVENTION

Group 1 had an individual training session with a single senior specialist using a fixed simulator model. After the session, trainees watched a pre-recorded 1-h lecture on vacuum-assisted operative vaginal delivery. Group 1 repeated the simulator session using the same test after 8-12 weeks and 12 months. Group 2 watched the pre-recorded lecture then undertook the same sessions as Group 1. Video recordings of all test performances were evaluated by five specialists in a blinded manner.

MAIN OUTCOME MEASURE

Each procedure was evaluated using a Global Rating Scale (GRS), scored with 0-5 points for each item on an Objective Structured Assessment of Technical Skills (OSATS) dataset with seven items (total 35 points). OSATS were evaluated over time, compared for the whole population, and weighted for route and year of residency. The primary outcome was comparison of the effectiveness of training between the two groups based on year of residency by assessing videos of the baseline test and GRS for OSATS scores. The secondary outcome was overall retention of skills at 8-12 weeks and 12 months.

RESULTS

Twenty-two participants were recruited and randomized to either Group 1 (n = 11) or Group 2 (n = 11). Five participants did not complete follow-up. The primary outcome of GRS for OSATS scores at time 0 differed significantly between groups for total GRS score, and 'Call for help' and 'Explanation to woman and relatives' item scores (p = 0.002 and p = 0.007, respectively). In a multiple linear regression analysis, OSATS scores were not independently influenced by year of residency. At 8-12-week follow-up, the 'Call for help' item score showed a significant improvement in Group 1 (p = 0.018), although this was not confirmed when year of residency was included as an independent variable. At 12-month follow-up, none of the item scores demonstrated a significant change (p = 0.033). Year of residency did not influence the difference between groups. One-way analysis of variance found significant differences between the groups for 'Localization of the flexion point' (p = 0.005), 'Traction of vacuum cup' (p = 0.039) and 'Use of second hand of the operator' (p = 0.009) item scores and total GRS score (p = 0.007). The values weighted by year of residency did not demonstrate any significant difference. The secondary outcomes evaluated all the candidates of both groups for retention of technical skills over time. A significant effect of time was found for total GRS score (p < 0.001) and OSATS item scores.

CONCLUSIONS

Independent of the sequence of theoretical teaching and simulation training, trainees demonstrated high retention - and, actually, improvement - of technical skills for operative vaginal delivery at 12-month follow-up.

[Simulation training for French obstetrics and gynecology residents: A national survey in 2022]

OBJECTIVES

To evaluate the use of simulation among French Obstetrics and Gynecology residency programs.

METHODS

A survey was conducted with all 28 French residency program directors. The questionnaire covered equipment and human resources, training programs, types of simulation tools and time spent.

RESULTS

Of the cities hosting a residency program, 93% (26/28) responded regarding equipment and human resources, and 75% (21/28) responded regarding training program details. All respondents declared having at least one structure dedicated to simulation. A formal training program was reported by 81% (21/26) of cities. This training program was mandatory in 73% of the cases. There was a median number of seven senior trainers involved, three of whom had received a specific training in medical education. Most of declared simulation activities concerned technical skills in obstetrics and surgery. Simulations to practice breaking bad news were offered by 62% (13/21) of cities. The median number of half-days spent annually on simulation training was 55 (IQR: 38-83).

CONCLUSION

Simulation training is now widely available among French residency programs. There remains heterogeneity between centers regarding equipment, time spent and content of simulation curricula. The French College of Teachers of Gynecology and Obstetrics has proposed a roadmap for the content of simulation-based training based on the results of this survey. An inventory of all existing "train the trainers" simulation programs in France is also provided.

Second-Trimester Dilation and Evacuation: A Simulation-Based Team Training Curriculum

Introduction

Despite the need for providers skilled in second-trimester dilation and evacuation (D&E) procedures, there are few second-trimester abortion training opportunities for OB/GYN residents and other health care trainees. Barriers to such training include restrictive state laws and institutional policies, lack of trained faculty, and limited procedural volume. Simulation-based D&E training is, therefore, a critical tool for OB/GYN residents and other medical professionals to achieve clinical competency.

Methods

This simulation for OB/GYN residents centers on a 29-year-old woman at 18 weeks gestation with intrauterine fetal demise, requiring learners to perform a second-trimester D&E and manage an unexpected postprocedural hemorrhage. We designed the simulation to be used with a high-fidelity mannequin. Personnel roles required for the simulation included an anesthesiologist, medical assistant, OR nurse, and two OB/GYN faculty. Learner performance was assessed using a pre- and postsimulation learner evaluation, a critical action checklist, and a focus group with simulation facilitators.

Results

Forty-nine residents participated over an 8-year period. Learners demonstrated improved competency performing a second-trimester D&E and increased confidence managing postprocedural hemorrhage after participating in this simulation. In addition, focus group participants reported that a majority of learners demonstrated confidence and effective communication with team members while performing in a decision-making role.

Discussion

In addition to improving learners' clinical competency and surgical confidence for second-trimester D&E procedures, this simulation serves as a valuable instrument for the standardized assessment of learners' performance, as well as an opportunity for all participants to practice teamwork and communication in a high-acuity setting.

Simulation-Based Outreach Program Improves Rural Hospitals’ Team Confidence in Neonatal Resuscitation

Introduction: Neonatal resuscitation is a high acuity, low occurrence event (HALO), and in rural community hospitals, low birth rates prevent providers from regular opportunities to maintain essential resuscitation skills. Simulation is an effective training modality for medical education, although resources for simulation are often limited in rural hospitals. Our primary objective was to test the hypothesis that in situ neonatal resuscitation simulation training improves rural hospitals' delivery room team confidence in performing key Neonatal Resuscitation Program^® (NRP^®) skills. Our secondary objective was to compare confidence to performance as measured by adherence to NRP® guidelines.

Methods: We conducted a quasi-experimental pre-training survey and post-training survey of delivery room team confidence in NRP^® skills at five level one delivery hospitals before and after an in situ simulation training program. Participants included rural hospitals’ usual delivery room team members. Participants rated their confidence on a five-point Likert scale. Simulations were analyzed using an adapted version of a validated scoring tool for NRP^® adherence and presented as overall percentage scores.

Results: Our data demonstrate a significant improvement in self-assessed confidence levels pre- and post-simulation training in key areas of neonatal resuscitation. Participants reported higher confidence in airway management (4 vs. 3, p=0.003), emergency intravenous access (3 vs. 2, p=0.007), and the ability to manage a code in the delivery room (4 vs. 3, p=0.013) and the operating room (4 vs. 3, p=0.028). Improvements were also noted in their team member’s knowledge and skills to perform neonatal resuscitation. While improvements were appreciated in confidence, the performance of skills (NRP^® adherence scores) was often in the sub-optimal performance range.

Conclusions: An in situ-based neonatal resuscitation outreach simulation program improves self-confidence among rural delivery room teams. Additional research is needed to understand how to translate improved confidence into actual improved performance. 

Surgical Science - Simbionix Robotic Hysterectomy Simulator: Validating a New Tool

STUDY OBJECTIVE

To gather validity evidence for and determine acceptability of Surgical Science - Simbionix Hysterectomy Modules for the DaVinci Xi console simulation system and evaluate performance benchmarks between novice and experienced or expert surgeons.

DESIGN

Prospective education study (Messick validity framework) SETTING: Multi-center, academic medical institutions PARTICIPANTS: Residents, fellows, and faculty in Obstetrics and Gynecology were invited to participate at 3 institutions. Participants were categorized by experience level: less than 10 hysterectomies (novice), 10 to 50 hysterectomies (experienced), and greater than 50 hysterectomies (expert). A total of 10 novice, 10 experienced, and 14 expert surgeons were included.

INTERVENTIONS

Participants completed 4 simulator modules (ureter identification, bladder flap development, colpotomy, complete hysterectomy) and a qualitative survey. Simulator recordings were reviewed in duplicate by educators in minimally invasive gynecologic surgery using the Modified Global Evaluative Assessment of Robotic Skills (GEARS) rating scale.

MEASUREMENTS AND MAIN RESULTS

Most participants felt the simulator realistically simulated robotic hysterectomy (64.7%) and that feedback provided by the simulator was as or more helpful than feedback from previous simulators (88.2%) but less helpful than feedback provided in the OR (73.5%). Participants felt this simulator would be helpful for teaching junior residents. Simulator-generated metrics correlated with GEARS performance for bladder flap and ureter identification modules in multiple domains including total movements and total time for completion. GEARS performance for the bladder flap module correlated with experience level (novice vs experienced/expert) in domains of interest and total score but did not consistently correlate for the other procedural modules. Performance benchmarks were evaluated for the bladder flap module for each GEARS domain and total score.

CONCLUSION

The modules were well received by participants of all experience levels. Individual simulation modules appear to better discriminate between novice and experienced/expert users than overall simulator performance. Based on these data and participant feedback, use of individual modules in early residency education may be helpful for providing feedback and may ultimately serve as one component of determining readiness to perform robotic hysterectomy.

Association of Simulation Training With Rates of Medical Malpractice Claims Among Obstetrician-Gynecologists

OBJECTIVE

To compare malpractice claim rates before and after participation in simulation training, which focused on team training during a high-acuity clinical case.

METHODS

We performed a retrospective analysis comparing the claim rates before and after simulation training among 292 obstetrician-gynecologists, all of whom were insured by the same malpractice insurer, who attended one or more simulation training sessions from 2002 to 2019. The insurer provided malpractice claims data involving study physicians, along with durations of coverage, which we used to calculate claim rates, expressed as claims per 100 physician coverage years. We used three different time periods in our presimulation and postsimulation training claim rates comparisons: the entire study period, 2 years presimulation and postsimulation training, and 1 year presimulation and postsimulation training. Secondary outcomes included indemnity payment amounts, percent of claims paid, and injury severity.

RESULTS

Compared with presimulation training, malpractice claim rates were significantly lower postsimulation training for the full study period (11.2 vs 5.7 claims per 100 physician coverage years; P<.001) and the 2 years presimulation and postsimulation training (9.2 vs 5.4 claims per 100 physician coverage years; P=.043). For the 1 year presimulation and postsimulation training comparison, the decrease in claim rates was nonsignificant (8.8 vs 5.3 claims per 100 physician coverage years; P=.162). Attending more than one simulation session was associated with a greater reduction in claim rates. Postsimulation claim rates for physicians who attended one, two, or three or more simulation sessions were 6.3, 2.1, and 1.3 claims per 100 physician coverage years, respectively (P<.001). Compared with presimulation training, there was no significant difference in the median or mean indemnity paid, percent of claims on which an indemnity payment was made, or median severity of injury after simulation training.

CONCLUSION

We observed a significant reduction in malpractice claim rates after simulation training. Wider use of simulation training within obstetrics and gynecology should be considered.

Residents' Confidence in Performing Robotic Hysterectomy in Obstetrics and Gynecologic Training Programs

STUDY OBJECTIVE

To compare residents' perceptions of readiness to perform robotic-assisted laparoscopic hysterectomy with the perceptions of residency program directors in obstetrics and gynecology programs throughout the United States.

DESIGN

A survey was administered to all residents taking the 2019 Council on Resident Education in Obstetrics and Gynecology Exam and concurrently to program directors in all Accreditation Council for Graduate Medical Education-accredited training programs.

SETTING

The survey was designed to assess resident confidence to perform robotic hysterectomies by the time of graduation.

PATIENTS

No patients were included in the study.

INTERVENTIONS

The only intervention was administration of the survey.

MEASUREMENTS AND MAIN RESULTS

De-identified survey data were analyzed using chi-squared and Fisher's exact tests. A total of 5473 resident respondents and 241 residency program directors were included in the study. Fifty-two percent of graduating residents reported that they felt they were given surgical autonomy to perform robotic hysterectomies, and 53.7% reported that they could perform one independently (if it was an "emergency" and they had to). By the time of graduation, only 59% of residents reported confidence performing a robotic hysterectomy, and only 56% reported they felt that it would be an important procedure for their future career. Program directors were significantly more likely to report that their residents were given autonomy to perform robotic hysterectomy by graduation (61.0% [95% confidence interval (CI), 54.3-67.3]), could perform a robotic hysterectomy independently (60.9% [95% CI, 53.9-67.6]), or could perform a robotic hysterectomy by graduation (70.2% [95% CI, 63.5-76.3]) than residents themselves (38.6% [95% CI, 37.2-40.0], 22.8% [95% CI, 21.6-24.0], 62.6% [95% CI, 61.2-64.0], respectively).

CONCLUSION

At the time of graduation, residents' confidence in performing robotic hysterectomy independently is lower than their confidence in performing all other approaches to hysterectomy.

Surgical simulation supplements reproductive endocrinology and infertility fellowship training

Objective

To assess if a surgical boot camp improves laparoscopic skill among reproduction endocrinology and infertility (REI) fellows and increases fellow desire to incorporate surgical skills into practice and to examine whether fellowship in vitro fertilization (IVF) volume correlates with surgical efficiency.

Design

Prospective evaluation.

Setting

Simulation Center.

Patients

Forty REI fellows.

Interventions

Fellows were timed before and after training in laparoscopic suturing and knot tying and while using virtual simulators. Fellows were surveyed before boot camp on prior experience with IVF and reproductive surgery, and immediately and 1 month after boot camp on their desire to incorporate surgical skills into practice.

Main Outcome Measures

Efficiency of laparoscopic suturing and knot tying before and after boot camp; likelihood and persistence of incorporating surgical skills into practice immediately and 1 month after boot camp; and correlation between fellowship IVF volume and fellow surgical efficiency.

Results

Fellows experienced significant improvement in laparoscopic suturing (44 sec), intracorporeal knot tying (82 sec), and extracorporeal knot tying (71 sec). Fellows reported being more likely to incorporate operative hysteroscopy (89%), operative laparoscopy (87%), and laparoscopic suturing (84%) into practice immediately following boot camp with no difference 1 month later. Fifty-four percent of fellows reported being more likely to perform robotic surgery after the boot camp, increasing to 70% 1 month later. There were weak correlations between IVF case volume and efficiency in laparoscopic suturing or hysteroscopic polypectomy (Spearman correlation coefficients, -0.14 and -0.03).

Conclusions

An intensive surgical boot camp enhances surgical skill among REI fellows.