Impact of Virtual Reality-Simulated Training on Urology Residents' Performance of Transurethral Resection of the Prostate

Current Standards for Training in Robot-assisted Surgery and Endourology: A Systematic Review

BACKGROUND AND OBJECTIVE

Different training programs have been developed to improve trainee outcomes in urology. However, evidence on the optimal training methodology is sparse. Our aim was to provide a comprehensive description of the training programs available for urological robotic surgery and endourology, assess their validity, and highlight the fundamental elements of future training pathways.

METHODS

We systematically reviewed the literature using PubMed/Medline, Embase, and Web of Science databases. The validity of each training model was assessed. The methodological quality of studies on metrics and curricula was graded using the MERSQI scale. The level of evidence (LoE) and level of recommendation for surgical curricula were awarded using the educational Oxford Centre for Evidence-Based Medicine classification.

KEY FINDINGS AND LIMITATIONS

A total of 75 studies were identified. Many simulators have been developed to aid trainees in mastering skills required for both robotic and endourology procedures, but only four demonstrated predictive validity. For assessment of trainee proficiency, we identified 18 in robotics training and six in endourology training; however, the majority are Likert-type scales. Although proficiency-based progression (PBP) curricula demonstrated superior outcomes to traditional training in preclinical settings, only four of six (67%) in robotics and three of nine (33%) in endourology are PBP-based. Among these, the Fundamentals of Robotic Surgery and the SIMULATE curricula have the highest LoE (level 1b). The lack of a quantitative synthesis is the main limitation of our study.

CONCLUSIONS AND CLINICAL IMPLICATIONS

Training curricula that integrate simulators and PBP methodology have been introduced to standardize trainee outcomes in robotics and endourology. However, evidence regarding their educational impact remains restricted to preclinical studies. Efforts should be made to expand these training programs to different surgical procedures and assess their clinical impact.

PATIENT SUMMARY

Simulation-based training and programs in which progression is based on proficiency represent the new standard of quality for achieving surgical proficiency in urology. Studies have demonstrated the educational impact of these approaches. However, there are still no standardized training pathways for several urology procedures.

Residency Surgical BPH Training Paradigms from MIST to HOLEP

PURPOSE OF REVIEW

Benign prostatic hyperplasia (BPH) is a common medical condition of older men that often requires medical or surgical therapy. Surgical options for BPH have grown exponentially over the last two decades. The numerous options and/or lack of access to them can make it challenging for new trainees to gain proficiency. We examine the literature for available BPH surgical simulators, learning curves, and training pathways.

RECENT FINDINGS

Each BPH surgical therapy has a learning curve which must be overcome. There is an abundance of TURP simulators which have shown face, content, and construct validity in the literature. Similarly, laser therapies have validated simulators. Newer technologies do have available simulators, but they have not been validated. There are strategies to improve learning and outcomes, such as having a structured training program. Simulators are available for BPH surgical procedures and some have been implemented in urology residencies. It is likely that such simulation may make urologists more facile on their learning curves for newer technologies. Further studies are needed. Future directions may include integration of simulator technology into training pathways that include surgical observation and proctorship.

[Impact of simulation-based training in endourology: A systematic review of the literature]

INTRODUCTION

The impact of simulation on the acquisition of surgical skills as well as their transfer to the operating room is still debated. The objective was to assess these two specific points, focusing on the field of endourology.

METHODS

A systematic review of the literature, following the PRIMA statement, was performed using Medline database through September 2021 without time limit. Studies focusing on the impact of simulators on the acquisition of surgical technical skills as well as their transfer to the operating room in the field of endourology were included. The endourological procedures were: cystoscopy, ureteroscopy, percutaneous nephrolithotomy, endoscopic treatment of benign prostatic hyperplasia, endoscopic bladder resection.

RESULTS

Among the 11,442 publications identified, fifty-two studies were included in the analysis. The majority reported an improvement in procedure time of the requested tasks and dexterity of participants, regardless of the type of simulator and procedure. The level of evidence of included studies was often low. Few studies evaluated the transfer of acquired skills from the simulator to the operating room.

CONCLUSION

This review showed the positive impact of simulation on the acquisition of technical skills in endourology. However, in order to include proficiency-based progression in the curriculum of trainees, some parameters such as the choice of reference simulators, choice of tasks, and method of validation of acquired skills must be validated in a consensual manner to offer a quality training.

Simulation training in transurethral resection/laser vaporization of the prostate; evidence from a systematic review by the European Section of Uro-Technology

PURPOSE

Transurethral resection (TURP) and photoselective vaporization of the prostate (PVP) constitute established surgical options to treat benign prostate hyperplasia. We investigated the current literature for simulators that could be used as a tool for teaching urologists alone or within the boundaries of a course or a curriculum.

METHODS

A literature search was performed using PubMed, Scopus, EMBASE, and Cochrane Central Register of Controlled Trials-CENTRAL. Search terms included: Simulat*, train*, curricull*, transurethral, TUR*, vaporesect*, laser. The efficacy of different simulators and the impact of different devices, curricula and courses in training and trainee learning curves were the primary endpoints.

RESULTS

Thirty-one studies are selected and presented. Validated virtual reality TURP simulators are the UW VR, PelvicVision, Uro-Trainer, and TURPsim™. Validated synthetic TURP models are Dr. K. Forke's TURP trainer, Bristol TURP trainer, different tissue prostate models, and 3D-printed phantoms. The Myo Sim PVP and the GreenLightTM are sufficiently validated PVP simulators. Several TURP and PVP training curricula have been developed and judged as applicable. Finally, the TURP modules of the European Urology Residents Education Programme (EUREP) Hands-on Training course and the Urology Simulation Bootcamp Course (USBC) are the most basic annual TURP courses identified in the international literature.

CONCLUSIONS

Simulators and courses or curricula are valuable learning and training TURP/PVP tools. The existent models seem efficient, are not always adequately evaluated and accepted. As part of training curricula and training courses, the use of training simulators can significantly improve quality for young urologists' education and clinical practice.

Endoscopic surgical simulation using low-fidelity and virtual reality transurethral resection simulators in urology simulation boot camp course: trainees feedback assessment study

OBJECTIVES

The objective of our study was to study trainees' feedback and rating of models for training transurethral resection of bladder lesions (TURBT) and prostate (TURP) during simulation.

METHODS

The study was performed during the ''Transurethral resection (TUR) module" at the boot camp held in 2019. Prior to the course, all trainees were required to evaluate their experience in performing TURBT and TURP procedures. Trainees simulated resection on two different models; low-fidelity tissue model (Samed, GmBH, Dresden, Germany) and virtual reality simulator (TURPMentor, 3D Systems, Littleton, US). Following the completion of the module, trainees completed a questionnaire using a 5-point Likert scale to evaluate their assessment of the models for surgical training.

RESULTS

In total, 174 simulation assessments were performed by 56 trainees (Samed Bladder-40, Prostate-45, TURPMentor Bladder-51, Prostate-37). All trainees reported that they had performed < 50 TUR procedures. The Samed model median scores were for appearance (4/5), texture (5/5), feel (5/5) and conductibility (5/5). The TURPMentor median score was for appearance (4/5), texture and feel (4/5) and conductibility (4/5). The most common criticism of the Samed model was that it failed to mimic bleeding. In contrast, trainees felt that the TURPMentor haptic feedback was inadequate to allow for close resection and did not calibrate movements accurately.

CONCLUSIONS

Our results demonstrate that both forms of simulators (low-fidelity and virtual reality) were rated highly by urology trainees and improve their confidence in performing transurethral resection and in fact complement each other in providing lower tract endoscopic resection simulation.

Benefits and Limitations of Transurethral Resection of the Prostate Training With a Novel Virtual Reality Simulator

PURPOSE

Profound endourological skills are required for optimal postoperative outcome parameters after transurethral resection of the prostate (TURP). We investigated the Karl Storz (Tuttlingen, Germany) UroTrainer for virtual simulation training of the TURP.

MATERIALS AND METHODS

Twenty urologists underwent a virtual reality (VR) TURP training. After a needs analysis, performance scores and self-rated surgical skills were compared before and after the curriculum, the realism of the simulator was assessed, and the optimal level of experience for VR training was evaluated. Statistical testing was done with SPSS 25.

RESULTS

Forty percent of participants indicated frequent intraoperative overload during real-life TURP and 80% indicated that VR training might be beneficial for endourological skills development, underlining the need to advance classical endourological training. For the complete cohort, overall VR performance scores (P = 0.022) and completeness of resection (P < 0.001) significantly improved. Self-rated parameters including identification of anatomical structures (P = 0.046), sparing the sphincter (P = 0.002), and handling of the resectoscope (P = 0.033) became significantly better during the VR curriculum. Participants indicated progress regarding handling of the resectoscope (70%), bleeding control (55%), and finding the correct resection depth (50%). Although overall realism and handling of the resectoscope was good, virtual bleeding control and correct tissue feedback should be improved in future VR simulators. Seventy percent of participants indicated 10 to 50 virtual TURP cases to be optimal and 80% junior residents to be the key target group for VR TURP training.

CONCLUSIONS

There is a need to improve training the TURP and VR simulators might be a valuable supplement, especially for urologists beginning with the endourological desobstruction of the prostate.

Robotic simulation training for urological trainees: a comprehensive review on cost, merits and challenges

Simulation in surgery is a safe and cost-effective way of training. Operating room performance is improved after simulation training. The necessary attributes of surgical simulators are acceptability and cost-effectiveness. It is also necessary for a simulator to demonstrate face, content, predictive, construct and concurrent validity. Urologists have embraced robot-assisted surgery. These procedures require steep learning curves. There are 6 VR simulators available for robot-assisted surgery; the daVinci Skills Simulator (dVSS), the Mimic dV Trainer (MdVT), the ProMIS simulator, the Simsurgery Educational Platform (SEP) simulator, the Robotic Surgical Simulator (RoSS) and the RobotiX Mentor (RM). Their efficacy is limited by the lack of comparative studies, standardisation of validation and high cost. There are a number of robotic surgery training curricula developed in recent years which successfully include simulation training. There are growing calls for these simulators to be incorporated into the urology training curriculum globally to shorten the learning curve without compromising patient safety. Surgical educators in urology should aim to develop a cost-effective, acceptable, validated simulator that can be incorporated into a standardised, validated robot-assisted surgery training curriculum for the next generation of robotic surgeons.

The changing practice of transurethral resection of the prostate

Introduction Transurethral resection of the prostate (TURP) is considered the gold standard surgical treatment for lower urinary tract symptoms (LUTS) secondary to benign prostatic hyperplasia. The number of TURPs performed has declined significantly over the last three decades owing to pharmaceutical therapy. TURP data from a single institution for the years 1990, 2000 and 2010 were compared to assess the difference in performance. Methods A retrospective analysis was undertaken of all patients who underwent TURP between January and December 2010. These findings were compared with historical data for the years 1990 and 2000: 100 sets of case notes were selected randomly from each of these years. Results The number of TURPs performed fell from 326 in 1990 to 113 in 2010. The mean age of patients increased from 70.6 years to 74.0 years. There was also a significant increase in the mean ASA grade from 1.9 to 2.3. The most common indication for TURP shifted from LUTS to acute urinary retention. No significant change in operating time was observed. The mean resection weight remained constant (22.95g in 1990, 22.55g in 2000, 20.76g in 2010). A reduction in transfusion rates was observed but there were higher rates of secondary haematuria and bladder neck stenosis. There was an increase from 2% to 11.5% of patients with long-term failure to void following TURP. Conclusions The number of TURPs performed continues to decline, which could lead to potential training issues. Urinary retention is still by far the most common indication. However, there has been a significant rise in the percentage of men presenting for TURP with high pressure chronic retention. The number of patients with bladder dysfunction who either have persistent storage LUTS or eventually require long-term catheterisation or intermittent self-catheterisation has increased markedly, which raises the question of what the long-term real life impact of medical therapy is on men with LUTS secondary to benign prostatic hyperplasia who eventually require surgery.

Simulation in urological training: Where are we in 2017?

Traditional Halstedian principles are no longer compatible with modern day urological training. Changing patterns of healthcare provision and the advanced nature of minimally invasive urological surgery have resulted in trainees receiving reduced exposure to independent operative experience. The drive towards consultant-delivered care, coupled with the introduction of the European working time directive, necessitate the urology trainee to meet end-of-training competencies with fewer learning opportunities compared with previous generations. Simulation provides an opportunity to compensate for reduced operative experience by augmenting both technical and non-technical skills development. A variety of simulation models can be used from cadaveric, synthetic and animal models to advanced virtual reality to aid surgical training. Whilst simulation is not without limitations, it has the potential to significantly enhance procedural competency and non-technical skills in a risk-free environment. Clear benefits, including equipment familiarity and trainee confidence have been demonstrated. However, direct transferability of these benefits to an operating-theatre is not clearly evident. Whilst not a substitute for clinical practice, simulation is likely to prove itself as a valuable adjunct to urological training. In this narrative review, we examine the current literature on simulation training in relation to urological surgery.

Development and Validation of a Novel and Cost-Effective Animal Tissue Model for Training Transurethral Resection of the Prostate

OBJECTIVES

To develop and validate a new and cost-effective animal tissue training model for practicing resection skills of transurethral resection of the prostate (TURP).

METHODS AND MATERIALS

A porcine kidney was prepared and restructured to simulate the relevant anatomy of the human prostate. The restructured prostate was connected to an artificial urethra and bladder. Face, content, and construct validity of the model was carried out using a 5-point Likert scale questionnaire, and comparison in task performance between participants and experts was made using observational clinical human reliability analysis.

RESULTS

A total of 24 participants and 11 experts who practiced TURP skills on this model from October 2014 to December 2015 were recruited. The mean score on specific feature of the anatomy and color, sensation of texture and feeling of resection, conductibility of current, and efficacy and safety of the model were 4.34 ± 0.37, 4.51 ± 0.63, 4.13 ± 0.53, and 4.35 ± 0.71, respectively, by participants whereas they were 4.22 ± 0.23, 4.30 ± 0.48, 4.11 ± 0.62, and 4.56 ± 0.77, respectively, by the experts on a scale of 1 (unrealistic) to 5 (very realistic). Participants committed more technical errors than the experts (11 vs 7, p < 0.001), produced more movements of the instruments (51 vs 33, p < 0.001), and required longer operating time (11.4 vs 6.2min, p < 0.001).

CONCLUSIONS

A newly developed restructured animal tissue model for training TURP was reported. Validation study on the model demonstrates that this is a very realistic and effective model for skills training of TURP. Trainees committed more technical errors, more unproductive movements, and required longer operating time.

Beyond the simulation laboratory: a realist synthesis review of clinical outcomes of simulation-based mastery learning

PURPOSE

Translational educational outcomes have been defined as starting in simulation laboratories (T1) and moving downstream to improved patient care practices (T2), patient outcomes (T3), and cost/other value outcomes (T4). The authors conducted a realist synthesis review of the literature to evaluate the translational effect of simulation-based mastery learning (SBML) principles beyond the laboratory. They also sought to address future directions in SBML to improve patient care processes and outcomes and, thus, the quality of health care delivery.

METHOD

The authors searched multiple databases for simulation-based medical education (SBME) studies published through April 2013. They screened articles using the PICO method-population (P), intervention (I), control (C), outcome (O)-to answer the research question: For (P) any health care providers, does the (I) implementation of SBML training, compared with (C) other training methodologies or no extra training, result in (O) a change in patient care practices or T2-T4 outcomes? Studies implementing SBME interventions with training methodologies that met all SBML principles and reporting T2-T4 outcomes were identified.

RESULTS

The 14 included studies used pre/post or cohort study designs; the majority were limited to individual performance and procedural competency. They reported improvement after SBML training in procedure performance, task success, patient discomfort, procedure time, complication rates, or T4 impacts (e.g., cost reduction).

CONCLUSIONS

Findings suggest health professions education conducted using SBML methodology can improve patient care processes and outcomes. Further research is needed to understand the translational impact of SBML for nontechnical skills, including teamwork, and skill retention.

The current role of simulation in urological training

Introduction

Simulation is becoming an increasingly popular educational tool in numerous surgical specialities, including urology. This article reviews the current role of urological simulators; discussing their need, availability, incorporation and current limitations.

Material and methods

A literature review of the electronic databases Medline, Embase and Google Scholar was performed.

Results

For increasingly limited urological training programs, simulation can act as a valuable adjunct to clinical training. Evidence suggests that simulation enables the trainee to bypass the early, error-prone part of the surgical learning curve. It should be incorporated into proficiency-based curricula, with junior trainees initially beginning with low fidelity simulators to grasp basic surgical skills before moving onto full-procedural simulation as they progress through their training. A wide variety of simulators of differing fidelity are currently available, teaching both technical (eg. cystoscopy) and non-technical (eg. communication) urological surgical skills. Whist numerous studies have assessed the face, content and construct validity of various urological simulators, further work needs to be undertaken to determine whether the skills learnt actually improve trainee performance in the operating room. Then, educators will be able to make informed decisions about whether these resource demanding (financially and in terms of demands on faculty) simulators are a worthwhile educational tool.

Conclusions

Although further investigation is required, urological simulators appear to have a considerable role for developing both technical and non-technical urological skills in an increasingly restricted educational environment in modern urogynecology.

Development and incorporation of hybrid simulation OSCE into in-training examinations to assess multiple CanMEDS competencies in urologic trainees

INTRODUCTION

As residency training requirements increasingly emphasize a competency-based approach, novel tools to directly evaluate Canadian Medical Education Directives for Specialists (CanMEDS) competencies must be developed. Incorporating simulation allows residents to demonstrate knowledge and skills in a safe, standardized environment. We describe a novel hybrid simulation station for use in a urology resident in-training Objective Structured Clinical Exam (OSCE) to assess multiple CanMEDS competencies.

METHODS

An OSCE station was developed to assess Communicator, Health Advocate, Manager, and Medical Expert (including technical skills) CanMEDS roles. Residents interviewed a standardized patient, interacted with a nurse, performed flexible cystoscopy and attempted stent removal using a novel bladder/stent model. Communication was assessed using the Calgary-Cambridge Observational Guide, knowledge was assessed using a checklist, and technical skills were assessed using a previously validated global rating scale. Video debriefing allowed residents to review their performance. Face and discriminative validity were assessed, and feasibility was determined through qualitative post-examination interviews and cost analysis.

RESULTS

All 9 residents (postgraduate years [PGY] 3, 4, 5) completed the OSCE in 15 minutes. Communicator and knowledge scores were similar among all PGYs. Scores in technical skills were higher in PGY-5 compared with PGY-3/4 reside nts (mean score 79% vs. 73%). Residents and exam personnel felt the OSCE station allowed for realistic demonstration of competencies. Equipment cost was $218 for the exam station.

CONCLUSIONS

We developed and implemented a hybrid simulation-based OSCE station to assess multiple CanMEDS roles. This approach was feasible and cost-effective; it also provided a framework for future development of similar OSCE stations to assess resident competencies across multiple domains.

Face, content, and construct validation of the Bristol TURP trainer

INTRODUCTION

Validation studies are an important part of simulator evaluation and are considered necessary to establish the effectiveness of simulation-based training. The widely used Bristol transurethral resection of prostate (TURP) simulator has not been formally validated.

OBJECTIVES

Evaluation of the face, content, and construct validities of the Bristol TURP simulator as an endourology training tool.

DESIGN

Using established validation methodology, face, content, and construct validities were evaluated. Face and content validities were assessed using a structured quantitative survey. Construct validity was assessed by comparing the performance of experts and novices using a validated performance scale and resection efficiency.

PARTICIPANTS AND SETTING

Overall, 8 novice urologists and 8 expert urologists participated in the study. The study was conducted in a dedicated surgical simulation training facility.

RESULTS

All 16 participants felt the model was a good training tool and should be used as an essential part of urology training (face validity). Content validity evaluation showed that most aspects of the simulator were adequately realistic (mean Likert scores 3.38-3.57/5); however, the model does not simulate bleeding. Experts significantly outperformed novices (p < 0.001) across all measures of performance, therefore establishing construct validity.

CONCLUSIONS

The Bristol TURP simulator shows face, content, and construct validities, although some aspects of the simulator were not very realistic (e.g., bleeding). This study provides evidence for the continuing use of this simulator in endourology training.

Systematic review of skills transfer after surgical simulation-based training

BACKGROUND

Simulation-based training assumes that skills are directly transferable to the patient-based setting, but few studies have correlated simulated performance with surgical performance.

METHODS

A systematic search strategy was undertaken to find studies published since the last systematic review, published in 2007. Inclusion of articles was determined using a predetermined protocol, independent assessment by two reviewers and a final consensus decision. Studies that reported on the use of surgical simulation-based training and assessed the transferability of the acquired skills to a patient-based setting were included.

RESULTS

Twenty-seven randomized clinical trials and seven non-randomized comparative studies were included. Fourteen studies investigated laparoscopic procedures, 13 endoscopic procedures and seven other procedures. These studies provided strong evidence that participants who reached proficiency in simulation-based training performed better in the patient-based setting than their counterparts who did not have simulation-based training. Simulation-based training was equally as effective as patient-based training for colonoscopy, laparoscopic camera navigation and endoscopic sinus surgery in the patient-based setting.

CONCLUSION

These studies strengthen the evidence that simulation-based training, as part of a structured programme and incorporating predetermined proficiency levels, results in skills transfer to the operative setting.

Development and verification of a taxonomy of assessment metrics for surgical technical skills

PURPOSE

To create and empirically verify a taxonomy of metrics for assessing surgical technical skills, and to determine which types of metrics, skills, settings, learners, models, and instruments were most commonly reported in the technical skills assessment literature.

METHOD

In 2011-2012, the authors used a rational analysis of existing and emerging metrics to create the taxonomy, and used PubMed to conduct a systematic literature review (2001-2011) to test the taxonomy's comprehensiveness and verifiability. Using 202 articles identified from the review, the authors classified metrics according to the taxonomy and coded data concerning their context and use. Frequencies (counts, percentages) were calculated for all variables.

RESULTS

The taxonomy contained 12 objective and 4 subjective categories. Of 567 metrics identified in the literature, 520 (92%) were classified using the new taxonomy. Process metrics outnumbered outcome metrics by 8:1. The most frequent metrics were "time," "manual techniques" (objective and subjective), "errors," and "procedural steps." Only one new metric, "learning curve," emerged. Assessments of basic motor skills and skills germane to laparoscopic surgery dominated the literature. Novices, beginners, and intermediate learners were the most frequent subjects, and box trainers and virtual reality simulators were the most frequent models used for assessing performance.

CONCLUSIONS

Metrics convey what is valued in human performance. This taxonomy provides a common nomenclature. It may help educators and researchers in procedurally oriented disciplines to use metrics more precisely and consistently. Future assessments should focus more on bedside tasks and open surgical procedures and should include more outcome metrics.

An update and review of simulation in urological training

Simulation, if appropriately integrated into surgical training, may provide a time efficient, cost effective and safe method of training. The use of simulation in urology training is supported by a growing evidence base for its use, leading many authors to call for it to be integrated into the curriculum. There is growing evidence for the utilisation of part task (technical skills) simulators to shorten the learning curve in an environment that does not compromise patient safety. There is also evidence that non-technical skills affect patient outcomes in the operating room and that high fidelity team based simulation training can improve non-technical skills and surgical team performance. This evidence has strengthened the argument of surgical educators who feel that simulation should be formally incorporated into the urology training curriculum to develop both technical and non-technical skills with the aim of optimising performance and patient safety.

Patient outcomes in simulation-based medical education: a systematic review

OBJECTIVES

Evaluating the patient impact of health professions education is a societal priority with many challenges. Researchers would benefit from a summary of topics studied and potential methodological problems. We sought to summarize key information on patient outcomes identified in a comprehensive systematic review of simulation-based instruction.

DATA SOURCES

Systematic search of MEDLINE, EMBASE, CINAHL, PsychINFO, Scopus, key journals, and bibliographies of previous reviews through May 2011.

STUDY ELIGIBILITY

Original research in any language measuring the direct effects on patients of simulation-based instruction for health professionals, in comparison with no intervention or other instruction.

APPRAISAL AND SYNTHESIS

Two reviewers independently abstracted information on learners, topics, study quality including unit of analysis, and validity evidence. We pooled outcomes using random effects.

RESULTS

From 10,903 articles screened, we identified 50 studies reporting patient outcomes for at least 3,221 trainees and 16,742 patients. Clinical topics included airway management (14 studies), gastrointestinal endoscopy (12), and central venous catheter insertion (8). There were 31 studies involving postgraduate physicians and seven studies each involving practicing physicians, nurses, and emergency medicine technicians. Fourteen studies (28 %) used an appropriate unit of analysis. Measurement validity was supported in seven studies reporting content evidence, three reporting internal structure, and three reporting relations with other variables. The pooled Hedges' g effect size for 33 comparisons with no intervention was 0.47 (95 % confidence interval [CI], 0.31-0.63); and for nine comparisons with non-simulation instruction, it was 0.36 (95 % CI, -0.06 to 0.78).

LIMITATIONS

Focused field in education; high inconsistency (I(2) > 50 % in most analyses).

CONCLUSIONS

Simulation-based education was associated with small-moderate patient benefits in comparison with no intervention and non-simulation instruction, although the latter did not reach statistical significance. Unit of analysis errors were common, and validity evidence was infrequently reported.

Face validity, construct validity and training benefits of a virtual reality TURP simulator

OBJECTIVE

To assess face validity, construct validity and the training benefits of a virtual reality TURP simulator.

METHOD

11 novices (no TURP experience) and 7 experts (>200 TURP's) completed a virtual reality median lobe prostate resection task on the TURPsim™ (Simbionix USA Corp., Cleveland, OH). Performance indicators (percentage of prostate resected (PR), percentage of capsular resection (CR) and time diathermy loop active without tissue contact (TAWC) were recorded via the TURPsim™ and compared between novices and experts to assess construct validity. Verbal comments provided by experts following task completion were used to assess face validity. Repeated attempts of the task by the novices were analysed to assess the training benefits of the TURPsim™.

RESULTS

Experts resected a significantly greater percentage of prostate per minute (p < 0.01) and had significantly less active diathermy time without tissue contact (p < 0.01) than novices. After practice, novices were able to perform the simulation more effectively, with significant improvement in all measured parameters. Improvement in performance was noted in novices following repetitive training, as evidenced by improved TAWC scores that were not significantly different from the expert group (p = 0.18).

CONCLUSIONS

This study has established face and construct validity for the TURPsim™. The potential benefit in using this tool to train novices has also been demonstrated.

Face, content and construct validity of a novel robotic surgery simulator

PURPOSE

We evaluated the face, content and construct validity of the novel da Vinci® Skills Simulator™ using the da Vinci Si™ Surgeon Console as the surgeon interface.

MATERIALS AND METHODS

We evaluated a novel robotic surgical simulator for robotic surgery using the da Vinci Si Surgeon Console and Mimic™ virtual reality. Subjects were categorized as novice-no surgical training, intermediate-surgical training with fewer than 100 robotic cases or expert-100 or more primary surgeon robotic cases. Each participant completed 10 virtual reality exercises with 3 repetitions and a questionnaire with a 1 to 10 visual analog scale to assess simulator realism (face validity) and training usefulness (content validity). The simulator recorded performance based on specific metrics. The performance of experts, intermediates and novices was compared (construct validity) using the Kruskal-Wallis test.

RESULTS

We studied 16 novices, 32 intermediates with a median surgical experience of 6 years (range 1 to 37) and a median of 0 robotic cases (range 0 to 50), and 15 experts with a median of 315 robotic cases (range 100 to 800). Participants rated the virtual reality and console experience as very realistic (median visual analog scale score 8/10) while expert surgeons rated the simulator as a very useful training tool for residents (10/10) and fellows (9/10). Experts outperformed intermediates and novices in almost all metrics (median overall score 88.3% vs 75.6% and 62.1%, respectively, between group p<0.001).

CONCLUSIONS

We confirmed the face, content and construct validity of a novel robotic skill simulator that uses the da Vinci Si Surgeon Console. Although it is currently limited to basic skill training, this device is likely to influence robotic surgical training across specialties.