Deliberate practice improves obstetrics and gynecology residents' hysteroscopy skills

EndoTrainer: a novel hybrid training platform for endoscopic surgery

PURPOSE

Endoscopy implies high demanding procedures, and their practice requires structured formation curricula supported by adequate training platforms. Physical platforms are the most standardised solution for surgical training, but over the last few years, virtual platforms have been progressively introduced. This research work presents a new hybrid, physic-virtual, endoscopic training platform that exploits the benefits of the two kind of platforms combining realistic tools and phantoms together with the capacity of measuring all relevant parameters along the execution of the exercises and of providing an objective assessment performance.

METHODS

The developed platform, EndoTrainer, has been designed to train and assess surgical skills in hysteroscopy and cystoscopy following a structured curricula. The initial development and validation is focused on hysteroscopic exercises proposed in the Gynaecological Endoscopic Surgical Education and Assessment (GESEA) Certification Programme from The Academy and European Society for Gynaecological Endoscopy (ESGE) and analyses the obtained results of an extensive study with 80 gynaecologists executing 30 trials of the standard 30 degree endoscope navigation exercise.

RESULTS

The experiments demonstrate the benefits of the presented hybrid platform. Multi-variable statistical analysis points out that all subjects have obtained statistically significant improvement in all relevant parameters: shorter and safer trajectories, improved 30-degree endoscope navigation, accurate positioning over the targets and reduction of the execution time.

CONCLUSION

This paper presents a new hybrid approach for training, and evaluating whether it provides an objectivable improvement of camera navigation endoscopic basic skills. The obtained results demonstrate the initial hypothesis: all subjects have improved their camera handling and navigation skills.

Deliberate Practice in Simulation-Based Surgical Skills Training: A Scoping Review

BACKGROUND

In recent years there has been a shift from traditional Halstedian methods toward more simulation-based medical education (SBME) for developing surgical skills. Questions remain about the role and value of SBME, although feedback and engagement in repetitive practice have been associated with positive learning outcomes. Regardless of approach, the principles of deliberate practice align with both the Halstedian traditions and ways of implementing SBME. Whilst deliberate practice is well described in the wider literature, the extent to which it is an effective instructional approach in surgical training remains unknown.

OBJECTIVE

To explore the effectiveness of deliberate practice as an instructional design for developing surgical skills through SBME interventions, as assessed by improvements in trainee performance and/or patient outcomes.

METHODS

A combined search was conducted in PUBMED, CINAHL, EMBASE, MEDLINE, PSYCHINFO, and Google Scholar. Three hundred one articles were screened and 17 met the inclusion criteria for analysis.

RESULTS

There was heterogeneity of study methods with 6 randomized control trials, 7 pretest/post-test design, 2 nonrandomized comparisons and 2 observational studies. All articles demonstrated positive learner outcomes following SBME with deliberate practice, although there was no direct comparison to another instructional method. Two studies demonstrated skill transfer to the clinical environment and 1 demonstrated improved patient outcomes.

CONCLUSION

Deliberate practice informed SBME interventions appeared effective for developing surgical skills among trainee surgeons, however the reliability of these conclusions was limited by the modest quality of the research studies and the design elements of deliberate practice were inconsistently applied. There was little evidence that deliberate practice led to skills retention beyond 30 days, although participant numbers were low and the quality of studies was modest.

Does Performance on Evidence-Based Medicine and Urgent Clinical Scenarios Assessments Deteriorate During the Fourth Year of Medical School? Findings From One Institution

PURPOSE

The fourth year of medical school (M4) should prepare students for residency yet remains generally unstructured, with ill-defined goals. The primary aim of this study was to determine whether there were performance changes in evidence-based medicine (EBM) and urgent clinical scenarios (UCS) assessments before and after M4 year.

METHOD

University of Michigan Medical School graduates who matched into internship at Michigan Medicine completed identical assessments on EBM and UCS at the beginning of M4 year and 13 months later during postgraduate year 1 (PGY1) orientation. Individual scores on these assessments were compared using paired t test analysis. The associations of academic performance, residency specialty classification, and initial performance on knowledge changes were analyzed.

RESULTS

During academic years 2014 and 2015, 76 students matched into a Michigan Medicine internship; 52 completed identical EBM stations and 53 completed UCS stations. Learners' performance on the EBM assessment decreased from M4 to PGY1 (mean 93% [SD = 7%] vs. mean 80% [SD = 13%], P < .01), while performance on UCS remained stable (mean 80% [SD = 9%] vs. mean 82% [SD = 8%], P = .22). High M4 performers experienced a greater rate of decline in knowledge level compared with low M4 performers for EBM (-20% vs. -4%, P = .01). Residency specialty and academic performance did not affect performance.

CONCLUSIONS

This study demonstrated degradation of performance in EBM during the fourth year and adds to the growing literature that highlights the need for curricular reform during this year.

Using virtual-reality simulation to ensure basic competence in hysteroscopy

BACKGROUND

Hysteroscopy is a technically challenging procedure. Specialty curricula of obstetrics and gynaecology appraise hysteroscopy for trainees but there is no present evidence-based training program that certifies the fundamental technical skills before performance on patients. The objectives of this study were to develop and gather validity evidence for a simulation-based test that can ensure basic competence in hysteroscopy.

METHODS

We used the virtual-reality simulator HystMentor™. Six experts evaluated the feasibility and clinical relevance of the simulator modules. Six modules were selected for the test and a pilot study was carried out. Subsequently, medical students, residents, and experienced gynaecologists were enrolled for testing. Outcomes were based on generated simulator metrics. Validity evidence was explored for all five sources of evidence (content, response process, internal structure, relations to other variables, consequences of testing).

RESULTS

Inter-case reliability was high for four out of five metrics (Cronbach's alpha ≥ 0.80). Significant differences were identified when comparing the three groups' performances (p values < 0.05). Participants' clinical experience was significantly correlated to their simulator test score (Pearson's r = 0.49, p < 0.001). A single medical student managed to achieve the established pass/fail score (6.7% false positive) and three experienced gynaecologists failed the test (27.3% false negative).

CONCLUSIONS

We developed a virtual-reality simulation-based test in hysteroscopy with supporting validity evidence. The test is intended to ensure competency in a mastery learning program where trainees practise on the simulator until they are able to pass before they proceed to supervised training on patients.

Simulation for Training and Assessment in Hysteroscopy: A Systematic Review

Hysteroscopy simulation complements conventional training on patients, yet evidence-based recommendations about its implementation and use are lacking. This systematic review analyzes and critically discusses hysteroscopy simulation literature published over the last 30 years. Systematic searches on PubMed, Embase, PsychINFO, ERIC, and the Cochrane Library produced 27 original articles published through 2017. Strategies based on different simulation models (e.g., animal organs, vegetables, synthetic uteri, virtual reality) were evaluated by users and appeared to facilitate learning. Observational studies have suggested a large impact on the knowledge and technical skills of novices for a wide range of hysteroscopic procedures, including for diagnosis, resection, and sterilization. Pretest/posttest studies show large improvements in performance time (6 studies; pooled effect size, 1.45; 95% confidence interval, 1.06-1.85) and overall performance scores (4 studies; pooled effect size, 3.19; 95% confidence interval, 1.45-4.94). Additionally, performance assessment on simulated models distinguishes novices from experts. Caution should be exercised because the available evidence largely originates from heterogeneous studies with weak designs, conducted in experimental settings with nonclinical participants (i.e., medical students). Moreover, neither clinical outcomes nor the clinical value of simulation-based assessment has been addressed. Hysteroscopy simulation may be supported ethically and pedagogically, but its role should be evaluated in pragmatic contexts, with robust interventional studies and broader competence-defining outcomes that include nontechnical skills.

Incorporating simulation into gynecologic surgical training

Today's educational environment has made it more difficult to rely on the Halstedian model of "see one, do one, teach one" in gynecologic surgical training. There is decreased surgical volume, but an increased number of surgical modalities. Fortunately, surgical simulation has evolved to fill the educational void. Whether it is through skill generalization or skill transfer, surgical simulation has shifted learning from the operating room back to the classroom. This article explores the principles of surgical education and ways to introduce simulation as an adjunct to residency training. We review high- and low-fidelity surgical simulators, discuss the progression of surgical skills, and provide options for skills competency assessment. Time and money are major hurdles when designing a simulation curriculum, but low-fidelity models, intradepartmental cost sharing, and utilizing local experts for simulation proctoring can aid in developing a simulation program.

Assessing Surgical Training and Deliberate Practice Methods in Dermatology Residency: A Survey of Dermatology Program Directors

BACKGROUND

Resident surgical education and technical skills may be enhanced with deliberate practice-based learning. Deliberate practice methods, such as simulation-based training and formal skills-based assessments, allow for trainees to repeatedly practice a defined task with expert supervision and feedback.

OBJECTIVE

The authors sought to characterize how surgical skills are taught and assessed in dermatology residency, with an emphasis on whether deliberate practice methods are incorporated in the surgical curriculum.

MATERIALS AND METHODS

A survey was administered to program directors at 117 Accreditation Council for Graduate Medical Education-approved dermatology residency programs during 2013 to 2014.

RESULTS

A total of 42 responses (36%) were collected. Over half of programs (57%) devote 10 to 30 hours each year to surgical didactics. Sixty-nine percent of programs use simulation models, and 62% of programs use formal assessment-guided feedback in evaluating surgical skills. Residents most commonly assume the role of primary surgeon in excisional surgery (100%) and less commonly in graft and flap reconstruction (52% and 52%, respectively). Twenty-nine percent of residents are the primary surgeons in Mohs micrographic surgery.

CONCLUSION

Dermatology residency programs are incorporating deliberate practice-based tenets in the surgical curriculum. These results provide a benchmark for programs to assess and improve the quality of dermatologic surgery training.

Embryo transfer simulation improves pregnancy rates and decreases time to proficiency in Reproductive Endocrinology and Infertility fellow embryo transfers

OBJECTIVE

To design and evaluate an ET simulator to train Reproductive Endocrinology and Infertility (REI) fellows' techniques of ET.

DESIGN

Simulation model development and retrospective cohort analysis.

SETTING

Not applicable.

PATIENT(S)

Patients undergoing IVF.

INTERVENTION(S)

Simulation model evaluation and implementation of ET simulation training.

MAIN OUTCOME MEASURE(S)

Pregnancy rates.

RESULT(S)

The REI fellow and faculty evaluation responses (n = 19/21 [90%]) of the model demonstrated realistic characteristics, with evaluators concluding the model was suitable for training in almost all evaluated areas. A total of 12 REI fellows who performed ET were analyzed: 6 before ET trainer and 6 after ET trainer. Pregnancy rates were 31% in the initial 10 ETs per fellow before simulator vs. 46% after simulator. One of six pre-ET trainer fellows (17%) had pregnancy rates ≥40% in their first 10 ETs; whereas four of six post-ET trainer fellows had pregnancy rates ≥40% in their first 10 ETs. The average number of ETs to obtain >40% pregnancy efficiency was 27 ETs before trainer vs. 15 ETs after trainer. Pregnancy rates were similar in the two groups after 20 ETs, and collective terminal pregnancy rates were >50% after 40 ETs.

CONCLUSION(S)

Embryo transfer simulation improved REI fellow pregnancy rates in their first 10 transfers and led to a more rapid ET proficiency. These data suggest potential value in adopting ET simulation, even in programs with a robust history of live ET in fellowship training.

Training and Assessment of Hysteroscopic Skills: A Systematic Review

OBJECTIVE

The aim of this systematic review was to identify studies on hysteroscopic training and assessment.

DESIGN

PubMed, Excerpta Medica, the Cochrane Library, and Web of Science were searched in January 2015. Manual screening of references and citation tracking were also performed. Studies on hysteroscopic educational interventions were selected without restrictions on study design, populations, language, or publication year. A qualitative data synthesis including the setting, study participants, training model, training characteristics, hysteroscopic skills, assessment parameters, and study outcomes was performed by 2 authors working independently. Effect sizes were calculated when possible. Overall, 2 raters independently evaluated sources of validity evidence supporting the outcomes of the hysteroscopy assessment tools.

RESULTS

A total of 25 studies on hysteroscopy training were identified, of which 23 were performed in simulated settings. Overall, 10 studies used virtual-reality simulators and reported effect sizes for technical skills ranging from 0.31 to 2.65; 12 used inanimate models and reported effect sizes for technical skills ranging from 0.35 to 3.19. One study involved live animal models; 2 studies were performed in clinical settings. The validity evidence supporting the assessment tools used was low. Consensus between the 2 raters on the reported validity evidence was high (94%).

CONCLUSIONS

This systematic review demonstrated large variations in the effect of different tools for hysteroscopy training. The validity evidence supporting the assessment of hysteroscopic skills was limited.

[Contribution of a virtual hysteroscopic simulator in the learning of hysteroscopic myoma resection]

OBJECTIVES

Uterine myomas are a frequent pathology affecting 20% of women of reproductive age. Myomas induce abnormal uterine bleeding, pelvic pain and increase the risk of infertility and obstetrical complications. Symptomatic sub-mucosal myomas are classically treated by hysteroscopic resection. Simulation is a method of education and training. It could improve quality and security of cares. The aim of this study is to assess the interest of a hysteroscopic simulator for the resection of myoma by novice surgeons.

METHODS

Twenty medical students were recruited, in a prospective study, in august 2014. The virtual-reality simulator VirtaMed HystSim™ (VirtaMed AG, Zurich, Switzerland) was used to perform the hysteroscopic training. All students received a short demonstration of myoma resection. The practice consists of a submucous myoma type 0 resection. The procedure and the evaluation were performed before and after a specific training in hysteroscopic resection of sixty minutes long. The main outcome criteria were time for the resection before and after training. The second criteria were fluid quantity used, number of contact between optic and uterine cavity and uterine perforation.

RESULTS

Twenty students aged from 22 to 24 years were included. The time for the procedure was significantly reduced after training (170s versus 335s, P<0.01). There is the same for fluid quantity used (335 mL versus 717mL, P<0.01) and the number of contact between optic and uterine cavity (0.2 contact versus 3, P=0.012). No perforation occurred in the simulation.

CONCLUSION

The results suggest that hysteroscopic simulator enhances and facilitates hysteroscopic resection for novice surgeons.

Loop electrosurgical excision procedure: an effective, inexpensive, and durable teaching model

The effectiveness of simulation training for enhancing operative skills is well established. Here we describe the construction of a simple, low-cost model for teaching the loop electrosurgical excision procedure. Composed of common materials such as polyvinyl chloride pipe and sausages, the simulation model, shown in the accompanying figure, can be easily reproduced by other training programs. In addition, we also present an instructional video that utilizes this model to review loop electrosurgical excision procedure techniques, highlighting important steps in the procedure and briefly addressing challenging situations and common mistakes as well as strategies to prevent them. The video and model can be used in conjunction with a simulation skills laboratory to teach the procedure to students, residents, and new practitioners.

Advanced hysteroscopic surgery training

Hysteroscopic surgery is pivotal in management of many gynecological pathologies. The skills required for performing advanced hysteroscopic surgery (AHS), eg, transcervical hysteroscopic endometrial resection (TCRE), hysteroscopic polypectomy and myomectomy in the management of menorrhagia, hysteroscopic septulysis in fertility-related gynecological problems and hysteroscopic removal of chronically retained products of conception and excision of intramural ectopic pregnancy ought to be practiced by contemporary gynecological surgeons in their day-to-day clinical practice. AHS is a minimally invasive procedure that preserves the uterus in most cases. Whilst the outcome is of paramount importance, proper training should be adopted and followed through so that doctors, nurses, and institutions may deliver the highest standard of patient care.

Validity and reliability of the robotic Objective Structured Assessment of Technical Skills

OBJECTIVE

Objective Structured Assessments of Technical Skills have been developed to measure the skill of surgical trainees. Our aim was to develop an Objective Structured Assessments of Technical Skills specifically for trainees learning robotic surgery.

METHODS

This is a multiinstitutional study conducted in eight academic training programs. We created an assessment form to evaluate robotic surgical skill through five inanimate exercises. Gynecology, general surgery, and urology residents, Fellows, and faculty completed five robotic exercises on a standard training model. Study sessions were recorded and randomly assigned to three blinded judges who scored performance using the assessment form. Construct validity was evaluated by comparing scores between participants with different levels of surgical experience; interrater and intrarater reliability were also assessed.

RESULTS

We evaluated 83 residents, nine Fellows, and 13 faculty totaling 105 participants; 88 (84%) were from gynecology. Our assessment form demonstrated construct validity with faculty and Fellows performing significantly better than residents (mean scores 89±8 faculty, 74±17 Fellows, 59±22 residents; P<.01). In addition, participants with more robotic console experience scored significantly higher than those with fewer prior console surgeries (P<.01). Robotic Objective Structured Assessments of Technical Skills demonstrated good interrater reliability across all five drills (mean Cronbach's α 0.79±0.02). Intrarater reliability was also high (mean Spearman's correlation 0.91±0.11).

CONCLUSION

We developed a valid and reliable assessment form for robotic surgical skill. When paired with standardized robotic skill drills, this form may be useful to distinguish between levels of trainee performance.

LEVEL OF EVIDENCE

II.

Validity and reliability of the robotic Objective Structured Assessment of Technical Skills

OBJECTIVE

Objective Structured Assessments of Technical Skills have been developed to measure the skill of surgical trainees. Our aim was to develop an Objective Structured Assessments of Technical Skills specifically for trainees learning robotic surgery.

METHODS

This is a multiinstitutional study conducted in eight academic training programs. We created an assessment form to evaluate robotic surgical skill through five inanimate exercises. Gynecology, general surgery, and urology residents, Fellows, and faculty completed five robotic exercises on a standard training model. Study sessions were recorded and randomly assigned to three blinded judges who scored performance using the assessment form. Construct validity was evaluated by comparing scores between participants with different levels of surgical experience; interrater and intrarater reliability were also assessed.

RESULTS

We evaluated 83 residents, nine Fellows, and 13 faculty totaling 105 participants; 88 (84%) were from gynecology. Our assessment form demonstrated construct validity with faculty and Fellows performing significantly better than residents (mean scores 89±8 faculty, 74±17 Fellows, 59±22 residents; P<.01). In addition, participants with more robotic console experience scored significantly higher than those with fewer prior console surgeries (P<.01). Robotic Objective Structured Assessments of Technical Skills demonstrated good interrater reliability across all five drills (mean Cronbach's α 0.79±0.02). Intrarater reliability was also high (mean Spearman's correlation 0.91±0.11).

CONCLUSION

We developed a valid and reliable assessment form for robotic surgical skill. When paired with standardized robotic skill drills, this form may be useful to distinguish between levels of trainee performance.

LEVEL OF EVIDENCE

II.