Consensus guidelines for validation of virtual reality surgical simulators

Colonoscopy Training on Virtual-Reality Simulators or Physical Model Simulators: A Randomized Controlled Trial

OBJECTIVE

This study employed a randomized controlled trial to assess the efficacy of virtual-reality (VR) simulators and physical model simulators on colonoscopy training to explore the optimal and evidence-based simulation training.

DESIGN

Forty participants were divided into 2 groups and randomized as dyads: the VR simulator group and the physical model simulator group. All the participants performed a baseline test through porcine colonoscopy. After a 6 h simulation training, each participant underwent a post-test on a pig after bowel preparation, and the procedures were video-recorded. Both the baseline test and the post-test were blindly assessed by 2 experienced assistant director physicians based on the GAGES-C scoring system.

SETTING

Simulation center, Renji Hospital, Shanghai Jiaotong University School of Medicine, Shanghai.

PARTICIPANTS

Forty surgical residents without colonoscopy experience.

RESULTS

Both the VR simulator group and the physical model simulator group improved significantly over the baseline test. The VR simulator group performed significantly better than the physical model simulator group, p=0.042. The participants in both groups expressed a high level of simulator satisfaction.

CONCLUSIONS

Novice residents can benefit from both VR simulators and physical model simulators. The VR simulator was shown to be more effective for colonoscopy training. VR simulators were more recommended for novices conducting basic colonoscopy training.

Three-dimensional simulation for interventional cardiology procedures: Face and content validity

INTRODUCTION AND OBJECTIVES

Three-dimensional (3D) model simulation provides the opportunity to manipulate real devices and learn intervention skills in a realistic, controlled, and safe environment. To ensure that simulators provide a realistic surrogate to real procedures they must undergo scientific validation. We aimed to evaluate the 3D-printed simulator SimulHeart® for face and content validity to demonstrate its value as a training tool in interventional cardiology (IC).

METHODS

Health professionals were recruited from sixteen Portuguese IC units. All participants received a 30-minute theoretical introduction, 10-minute demonstration of each task and then performed the intervention on a 3D-printed simulator (SimulHeart®). Finally, a post-training questionnaire focusing on the appearance of the simulation, simulation content, and satisfaction/self-efficacy was administered.

RESULTS

We included 56 participants: 16 "experts" (general and interventional cardiologists), 26 "novices" (cardiology residents), and 14 nurses and allied professionals. On a five-point Likert scale, the overall mean score of face validity was 4.38±0.35 and the overall mean score of content validity was 4.69±0.32. There was no statistically significant difference in the scores provided by "experts" and "novices". Participants reported a high level of satisfaction/self-efficacy with 60.7% considering it strongly improved their skills. The majority (82.1%) "agreed" or "strongly agreed" that after the simulation they felt confident to perform the procedure on a patient.

CONCLUSION

The 3D-printed simulator (SimulHeart®) showed excellent face and content validity. 3D simulation may play an important role in future IC training programs. Further research is required to correlate simulator performance with clinical performance in real patients.

Face, content, and construct validity of a novel VR/AR surgical simulator of a minimally invasive spine operation

Mixed-reality surgical simulators are seen more objective than conventional training. The simulators' utility in training must be established through validation studies. Establish face-, content-, and construct-validity of a novel mixed-reality surgical simulator developed by McGill University, CAE-Healthcare, and DePuy Synthes. This study, approved by a Research Ethics Board, examined a simulated L4-L5 oblique lateral lumbar interbody fusion (OLLIF) scenario. A 5-point Likert scale questionnaire was used. Chi-square test verified validity consensus. Construct validity investigated 276 surgical performance metrics across three groups, using ANOVA, Welch-ANOVA, or Kruskal-Wallis tests. A post-hoc Dunn's test with a Bonferroni correction was used for further analysis on significant metrics. Musculoskeletal Biomechanics Research Lab, McGill University, Montreal, Canada. DePuy Synthes, Johnson & Johnson Family of Companies, research lab. Thirty-four participants were recruited: spine surgeons, fellows, neurosurgical, and orthopedic residents. Only seven surgeons out of the 34 were recruited in a side-by-side cadaver trial, where participants completed an OLLIF surgery first on a cadaver and then immediately on the simulator. Participants were separated a priori into three groups: post-, senior-, and junior-residents. Post-residents rated validity, median > 3, for 13/20 face-validity and 9/25 content-validity statements. Seven face-validity and 12 content-validity statements were rated neutral. Chi-square test indicated agreeability between group responses. Construct validity found eight metrics with significant differences (p < 0.05) between the three groups. Validity was established. Most face-validity statements were positively rated, with few neutrally rated pertaining to the simulation's graphics. Although fewer content-validity statements were validated, most were rated neutral (only four were negatively rated). The findings underscored the importance of using realistic physics-based forces in surgical simulations. Construct validity demonstrated the simulator's capacity to differentiate surgical expertise.

An update on simulation training in rhinology: a systematic review of evidence

BACKGROUND

Rhinological procedures demand a high degree of technical expertise and anatomical knowledge. Because of limited surgical opportunities, ethical considerations and the complexity of these procedures, simulation-based training has become increasingly important. This review aimed to evaluate the effectiveness of simulation models used in rhinology training.

METHODS

Searches were conducted on PubMed, Embase, Cochrane and Google Scholar for studies conducted between July 2012 and July 2022. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis ('PRISMA') protocol defined a final list of articles. Each validated study was assigned a level of evidence and a level of recommendation based on the Oxford Centre of Evidence-Based Medicine classification.

RESULTS

Following exclusions, 42 articles were identified which encompassed six types of simulation models and 26 studies evaluated validity. The rhinological skills assessed included endoscopic sinus surgery (n = 28), skull base/cerebrospinal fluid leak repair (n = 14), management of epistaxis and/or sphenopalatine artery ligation (n = 8), and septoplasty and septorhinoplasty (n = 6). All studies reported the beneficial impact of their simulation models on trainee development.

CONCLUSION

Simulation training in rhinology is a valuable adjunct to traditional surgical education. Although evidence is of moderate quality, the findings highlight the importance of simulation-based training in rhinology training.

Transferring laparoscopic skills to robotic-assisted surgery: a systematic review

Robotic-Assisted Surgery (RAS) is experiencing rapid expansion, prompting the integration of robotic technical skills training into surgical education programs. As access to robotic training platforms remains limited, it is important to investigate the transferability of laparoscopic skills to RAS. This could potentially support the inclusion of early years laparoscopic training to mitigate the learning curve associated with robotic surgery. This study aims to assess the transferability of laparoscopic skills to robotic surgery. A systematic search was conducted using the PRISMA checklist to identify relevant articles. PubMed, MEDLINE, Embase, and Cochrane databases were searched, and inclusion and exclusion criteria were applied to collate eligible articles. Included were original articles comparing the performance of comparable tasks on both laparoscopic and robotic platforms written in English. Non-peer reviewed papers, conference abstracts, reviews, and case series were excluded. Seventeen articles met the inclusion criteria. Among these, 10 studies (59%) demonstrated skill transferability from laparoscopic surgery (LS) to robotic surgery (RS); while one study (5.8%) showed no significant transferability. Four studies highlighted the positive impact of prior laparoscopic training on robotic skill, whereas six papers suggested no significant difference between laparoscopic novices and experienced laparoscopists when utilizing a robotic simulator. Five studies evaluated advanced surgical skills such as intracorporeal knot tying and suturing, revealing superior robotic performance among experienced laparoscopists compared to novice learners. Laparoscopic skills appear to be transferrable to robotic surgery, particularly in complex surgical techniques. Robotic simulators demonstrate a significant reduction in the learning curve for surgical novices, albeit to a lesser extent for experienced laparoscopists.

Evaluation of objective tools and artificial intelligence in robotic surgery technical skills assessment: a systematic review

BACKGROUND

There is a need to standardize training in robotic surgery, including objective assessment for accreditation. This systematic review aimed to identify objective tools for technical skills assessment, providing evaluation statuses to guide research and inform implementation into training curricula.

METHODS

A systematic literature search was conducted in accordance with the PRISMA guidelines. Ovid Embase/Medline, PubMed and Web of Science were searched. Inclusion criterion: robotic surgery technical skills tools. Exclusion criteria: non-technical, laparoscopy or open skills only. Manual tools and automated performance metrics (APMs) were analysed using Messick's concept of validity and the Oxford Centre of Evidence-Based Medicine (OCEBM) Levels of Evidence and Recommendation (LoR). A bespoke tool analysed artificial intelligence (AI) studies. The Modified Downs-Black checklist was used to assess risk of bias.

RESULTS

Two hundred and forty-seven studies were analysed, identifying: 8 global rating scales, 26 procedure-/task-specific tools, 3 main error-based methods, 10 simulators, 28 studies analysing APMs and 53 AI studies. Global Evaluative Assessment of Robotic Skills and the da Vinci Skills Simulator were the most evaluated tools at LoR 1 (OCEBM). Three procedure-specific tools, 3 error-based methods and 1 non-simulator APMs reached LoR 2. AI models estimated outcomes (skill or clinical), demonstrating superior accuracy rates in the laboratory with 60 per cent of methods reporting accuracies over 90 per cent, compared to real surgery ranging from 67 to 100 per cent.

CONCLUSIONS

Manual and automated assessment tools for robotic surgery are not well validated and require further evaluation before use in accreditation processes.PROSPERO: registration ID CRD42022304901.

Assessment of a Novel, Adjustable Task Trainer for Cardiac Surgical Skills

INTRODUCTION

A recent needs assessment in Canadian cardiac surgery programs identified the desire for a coronary artery bypass (CABG) and aortic valve replacement (AVR) simulation model for home practice. We aimed to develop and assess a portable, adjustable task trainer for cardiac surgical skills with high functional task alignment.

METHODS

Intraoperative measurements were taken from patients undergoing elective CABG and AVR (N = 30). Measurements were taken in 3 axes and used to create a chest cavity that resembles the mediastinal constraints of a patient undergoing CABG and AVR. The task trainer is adjustable on the following 3 levels: (1) size of the incision, (2) depth of the chest, and (3) relative position of coronary artery or aortic valve model within the chest. Three groups (novices, intermediates, and experts) of cardiac surgery members evaluated the task trainer for functional task alignment and construct validity.

RESULTS

The CABG and AVR model had high functional task alignment. There was a high satisfaction for both models and all participants would recommend the AVR and CABG model as an educational tool. Performance time significantly differed between the groups for both models (CABG: P = 0.032 and AVR: P = 0.001), as well as number of errors (CABG: P = 0.04 and AVR: P = 0.043).

CONCLUSIONS

Using real patient data, we were able to develop an adjustable task trainer for training principles of CABG and AVR. Our pilot study provides preliminary sources of evidence for validity and future study will look to assess transferability of skill to the operating room.

Construct Validity and Experience of Using a Low-cost Arthroscopic Shoulder Surgery Simulator

Objective  To validate the low-cost model for arthroscopy training and analyze the acceptance and usefulness of the developed simulator in medical teaching and training. Method  Ten medical students, ten third-year orthopedic residents, and ten shoulder surgeons performed predetermined tasks on a shoulder simulator twice. The parameters used were time to complete the tasks, number of looks at the hands, GOALS score (Global Operative Assessment of Laparoscopic Skills) and comparison between groups and within groups. An adapted Likert scale was applied addressing the individuals' impressions about the simulator and its applicability. Results  In the intergroup comparison, the shoulder surgeons had better scores and times than the other groups. When the tasks were repeated, the group of surgeons had a 59% improvement in time (p < 0.05), as did the group of medical students. In the GOALS score, shoulder surgeons had consistently better scores than the other groups. And when we evaluated the evolution from the first to the second test, the group of surgeons and the group of academics had a statistically significant improvement (p < 0.05). In terms of lookdowns, there was a decrease in all groups. There was consensus that the simulator is useful in training. Conclusion  The simulator developed allowed the differentiation between individuals with different levels of training in arthroscopic surgery. It was accepted by 100% of the participants as a useful tool in arthroscopic shoulder surgical training.

Virtual reality simulation as a training tool for perfusionists in extracorporeal circulation: Establishing face and content validity

Objective

We conducted a prospective study to assess the face and content validity of a new virtual reality (VR) extracorporeal circulation simulator (ECC) developed for perfusionists to facilitate training and practice. We evaluated the opinions of students and staff members about the feasibility of the simulation. The 2 groups consisted of experts (qualified perfusionists) and novices (trainee perfusionists).

Methods

Perfusionists (n = 12 experts and n = 11 trainees) received instructions on how to use the VR simulator and then proceeded to perform the start of cardiopulmonary bypass in the VR environment. Participants then completed a Usefulness, Satisfaction, and Ease of Use Questionnaire. The questions were rated on a 5-point Likert scale, ranging from 1 (fully disagree) to 5 (fully agree), to assess the face validity and content validity of this simulator.

Results

Participants reported a predominantly positive experience with the VR-ECC simulator, with 96% (n = 22) agreeing that the simulator was a useful way of training ECC scenarios. All participants found it easy to interact with the software (100%, n = 23), and 82% of students (n = 9) believed it helped them remember the steps involved with initiating ECC. Finally, (87% [n = 20]) of participants believed the image quality and depth perception were good.

Conclusions

Our next-generation simulator was valid for face and content constructs, and almost all participants found it to be a useful way of training for ECC scenarios. This simulator represents a first step toward truly blended digital learning and a new interactive, flexible, and innovative modality for perfusion training.

Advancing Surgical Education: A Comprehensive Systematic Review with Meta-Analysis and Novel Approach to Training Models for Local Skin Advancement Flaps

Performing local skin flaps is a challenging task that requires cognitive and technical skills to design flaps with proper orientation to avoid distorting normal anatomy. Junior trainees need adequate exposure to gain confidence and expertise in such procedures. This article systematically reviews the literature's different local skin advancement flap training models and describes a new, easy-to-use training model. A systematic review was performed following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. PubMed, Cochrane, Web of Science, and Google Scholar databases were searched from their inception until August 2022 for articles about local skin advancement flap training models. The meta-analysis results were pooled across the studies using a random-effects model and presented as a weighted mean difference with a 95% confidence interval (95% CI). Out of 773 reviewed articles, 18 were included in the systematic review, and four reported enough data to be included in the meta-analysis. Rhomboid and Z-plasty flaps were the most commonly taught flaps by training models. The most commonly used training models were synthetic-based, followed by animal-based models. The training models significantly increased the trainees' confidence and expertise regarding local skin flap procedures (p<0.00001) for both domains. Training models, per our reported data, significantly improve the trainees' confidence and expertise in performing local skin advancement flap procedures; continuous efforts in developing and establishing new, simple-to-use, and effective training models are strongly encouraged to further improvement of surgical education and enhance the trainees' surgical skills.

Can Non-Virtual Reality Simulation Improve Surgical Training in Endoscopic Sinus Surgery? A Literature Review

Simulation in endoscopic sinus surgery allows residents to learn anatomy, to achieve the correct handling of various rhinological instruments, and to practice different surgical procedures. Physically or non-virtual reality models are the main items in endoscopic sinus surgery simulation. The objective of this review is to identify and make a descriptive analysis of non-virtual endoscopic sinus surgery simulators which have been proposed for training. As a new state of the art, surgical simulators are developed continuously, so they can be used to learn basic endoscopic surgery skills by repetitive maneuvers, permitting detection of surgical error and incidents without risk for the patient. Of all training physical models, the ovine model stands out because of the similarities of the sinonasal pathways, the wide availability, and the low costs. Considering the similar nature of the tissues involved, the techniques and surgical instruments can be used almost interchangeably with minimal differences. Every surgical technique studied until now has a degree of risk and the only aspects that consistently reduced the number of complications are training, repetition, and hands-on experience.

Application of Virtual Reality Systems in Bone Trauma Procedures

Background and Objectives: Bone fractures contribute significantly to the global disease and disability burden and are associated with a high and escalating incidence and tremendous economic consequences. The increasingly challenging climate of orthopaedic training and practice re-echoes the established potential of leveraging computer-based reality technologies to support patient-specific simulations for procedural teaching and surgical precision. Unfortunately, despite the recognised potential of virtual reality technologies in orthopaedic surgery, its adoption and integration, particularly in fracture procedures, have lagged behind other surgical specialities. We aimed to review the available virtual reality systems adapted for orthopaedic trauma procedures. Materials and Methods: We performed an extensive literature search in Medline (PubMed), Science Direct, SpringerLink, and Google Scholar and presented a narrative synthesis of the state of the art on virtual reality systems for bone trauma procedures. Results: We categorised existing simulation modalities into those for fracture fixation techniques, drilling procedures, and prosthetic design and implantation and described the important technical features, as well as their clinical validity and applications. Conclusions: Over the past decade, an increasing number of high- and low-fidelity virtual reality systems for bone trauma procedures have been introduced, demonstrating important benefits with regard to improving procedural teaching and learning, preoperative planning and rehearsal, intraoperative precision and efficiency, and postoperative outcomes. However, further technical developments in line with industry benchmarks and metrics are needed in addition to more standardised and rigorous clinical validation.

Training and assessment using the LapSim laparoscopic simulator: a scoping review of validity evidence

INTRODUCTION

The LapSim (Surgical Science, Sweden) laparoscopic simulator is a high-fidelity virtual reality simulator for use in endoscopic surgical training. This review critiques the current validity evidence for the LapSim laparoscopic simulator, specifically with respect to its potential use as a tool and method of training and assessment in surgery.

METHODS

A scoping review of the MEDLINE (PubMed), EMBASE, Cochrane and Web of Science databases was conducted in accordance with PRISMA guidelines (2020)-scoping review extension. Articles were included if they presented validity evidence for the use of the LapSim in operative skill training or assessment, in accordance with Messick's validity framework. European Association of Endoscopic Surgeons (EAES) guidelines (2005) were used to provide recommendations for the use of the LapSim in operative performance training and assessments.

RESULTS

Forty-nine articles were included. An EAES level 2 recommendation was provided with regard to the internal consistency reliability of automated performance metrics in assessing performance. An EAES recommendation of 2 was awarded with respect to the ability of the LapSim to discriminate based on case volume and overall laparoscopic experience (relationships with other variables). Performance assessment metrics on the LapSim correlate with improved performance in the operating room (EAES level of recommendation 1-consequential validity).

CONCLUSION

The LapSim has accumulated substantial evidence supporting the validity of its use in surgical training and assessment. Future studies should explore the relationship between the achievement of performance benchmarks on the LapSim and subsequent patient outcomes, and interrogate the benefits of implementing virtual reality simulation training and assessment curricula in post-graduate surgical training programmes.

Validated simulation models in pediatric surgery: A review

INTRODUCTION

This review evaluates the validation and availability of simulation models in the field of pediatric surgery that can be used for training purposes.

METHODS

MEDLINE and EMBASE were searched for studies describing a simulation models in pediatric surgery. Articles were included if face, content and/or construct validity was described. Additionally, the costs and availability were assessed. Validation scores for each model were depicted as percentage (0-100), based on the reported data, to compare the outcomes. A score of >70% was considered adequate.

RESULTS

Forty-three studies were identified, describing the validation process of 38 simulation models. Face validity was evaluated in 33 articles, content in 36 and construct in 19. Twenty-two models received adequate validation scores (>70%). The majority (27/38, 70%) was strictly inanimate. Five models were available for purchase and eleven models were replicable based on the article.

CONCLUSION

The number of validated inanimate simulation models for pediatric surgery procedures is growing, however, few are replicable or available for widespread training purposes.

LEVEL OF EVIDENCE

Level II.

The Validation of Surgical Simulators: A Technical Report on Current Validation Terminology as a Reference for Future Research

In recent years, surgical trainees have been exposed to a lower volume of operative procedures. In part, this is due to the reduction in working hours and further disruption by the coronavirus disease 2019 pandemic. Much has been done to develop the techniques of surgical skill training outside of the operating theatre. Simulation-based interventions must undergo a process of validation to assess their appropriateness and effectiveness for use in training. The terminology of validation within current literature, however, has not evolved in line with the education community, resulting in varying definitions for the same phrase across domains. This can result in confusion and misinterpretation among researchers and surgeons working within this domain. This technical report describes the "types of validity" definitions used in the traditional framework of surgical simulation literature and the contemporary, unitary framework of validity adopted by educationalist theorists. There is a clear overlap between the traditional "types of validity" and the contemporary, unitary framework. The divergence in the use of those definitions seems, at least partly, influenced by the context of the investigations being conducted. By utilising the contemporary definitions, authors may have struggled to provide the evidence required to justify the use of the multitude of surgical skill simulators developed in the recent past. This report has provided an overview of the current terminology within the validation frameworks and can be used as a reference for future surgical simulation research.

[Impact of simulation-based training in endourology: A systematic review of the literature]

INTRODUCTION

The impact of simulation on the acquisition of surgical skills as well as their transfer to the operating room is still debated. The objective was to assess these two specific points, focusing on the field of endourology.

METHODS

A systematic review of the literature, following the PRIMA statement, was performed using Medline database through September 2021 without time limit. Studies focusing on the impact of simulators on the acquisition of surgical technical skills as well as their transfer to the operating room in the field of endourology were included. The endourological procedures were: cystoscopy, ureteroscopy, percutaneous nephrolithotomy, endoscopic treatment of benign prostatic hyperplasia, endoscopic bladder resection.

RESULTS

Among the 11,442 publications identified, fifty-two studies were included in the analysis. The majority reported an improvement in procedure time of the requested tasks and dexterity of participants, regardless of the type of simulator and procedure. The level of evidence of included studies was often low. Few studies evaluated the transfer of acquired skills from the simulator to the operating room.

CONCLUSION

This review showed the positive impact of simulation on the acquisition of technical skills in endourology. However, in order to include proficiency-based progression in the curriculum of trainees, some parameters such as the choice of reference simulators, choice of tasks, and method of validation of acquired skills must be validated in a consensual manner to offer a quality training.

Simulation models for learning local skin flap design and execution: A systematic review of the literature

Introduction

Early exposure to practical skills in surgical training is essential in order to master technically demanding procedures such as the design and execution of local skin flaps. Changes in working patterns, increasing subspecializations, centralization of surgical services, and the publication of surgeon-specific outcomes have all made hands-on-training in a clinical environment increasingly difficult to achieve for the junior surgeon. This has been further compounded by the COVID-19 pandemic. This necessitates alternative methods of surgical skills training. To date, there are no standardized or ideal simulation models for local skin flap teaching.

Aim

This systematic review aims to summarize and evaluate local skin flap simulation and teaching models published in the literature.

Materials and Methods

A systematic review protocol was developed and undertaken in accordance with PRISMA guidelines. Key search terms encompassed both “local skin flaps” and “models” or “surgical simulation”. These were combined using Boolean logic and used to search Embase, Medline, and the Cochrane Library. Studies were collected and screened according to the inclusion criteria. The final included articles were graded for their level of evidence and recommendation based on a modified educational Oxford Center for evidence-based medicine classification system and assessed according to the CRe-DEPTH tool for articles describing training interventions in healthcare professionals.

Results

A total of 549 articles were identified, resulting in the inclusion of 16 full-text papers. Four articles used 3D simulators for local flap teaching and training, while two articles described computer simulation as an alternative method for local flap practicing. Four models were silicone based, while gelatin, Allevyn dressings, foam rubber, and ethylene-vinyl acetate-based local flap simulators were also described. Animal models such as pigs head, porcine skin, chicken leg, and rat, as well as a training model based on fresh human skin excised from body-contouring procedures, were described. Each simulation and teaching method was assessed by a group of candidates via a questionnaire or evaluation survey grading system. Most of the studies were graded as level of evidence 3 or 4.

Conclusion

Many methods of simulation for the design and execution of local skin flaps have been described. However, most of these have been assessed only in small cohort numbers, and, therefore, larger candidate sizes and a standardized method for assessment are required. Moreover, some proposed simulators, although promising, are in a very preliminary stage of development. Further development and evaluation of promising high-fidelity models is required in order to improve training in such a complex area of surgery.

Virtual Reality Simulation Training for Cardiopulmonary Resuscitation After Cardiac Surgery: Face and Content Validity Study

BACKGROUND

Cardiac arrest after cardiac surgery commonly has a reversible cause, where emergency resternotomy is often required for treatment, as recommended by international guidelines. We have developed a virtual reality (VR) simulation for training of cardiopulmonary resuscitation (CPR) and emergency resternotomy procedures after cardiac surgery, the Cardiopulmonary Resuscitation Virtual Reality Simulator (CPVR-sim). Two fictive clinical scenarios were used: one case of pulseless electrical activity (PEA) and a combined case of PEA and ventricular fibrillation. In this prospective study, we researched the face validity and content validity of the CPVR-sim.

OBJECTIVE

We designed a prospective study to assess the feasibility and to establish the face and content validity of two clinical scenarios (shockable and nonshockable cardiac arrest) of the CPVR-sim partly divided into a group of novices and experts in performing CPR and emergency resternotomies in patients after cardiac surgery.

METHODS

Clinicians (staff cardiothoracic surgeons, physicians, surgical residents, nurse practitioners, and medical students) participated in this study and performed two different scenarios, either PEA or combined PEA and ventricular fibrillation. All participants (N=41) performed a simulation and completed the questionnaire rating the simulator's usefulness, satisfaction, ease of use, effectiveness, and immersiveness to assess face validity and content validity.

RESULTS

Responses toward face validity and content validity were predominantly positive in both groups. Most participants in the PEA scenario (n=26, 87%) felt actively involved in the simulation, and 23 (77%) participants felt in charge of the situation. The participants thought it was easy to learn how to interact with the software (n=24, 80%) and thought that the software responded adequately (n=21, 70%). All 15 (100%) expert participants preferred VR training as an addition to conventional training. Moreover, 13 (87%) of the expert participants would recommend VR training to other colleagues, and 14 (93%) of the expert participants thought the CPVR-sim was a useful method to train for infrequent post-cardiac surgery emergencies requiring CPR. Additionally, 10 (91%) of the participants thought it was easy to move in the VR environment, and that the CPVR-sim responded adequately in this scenario.

CONCLUSIONS

We developed a proof-of-concept VR simulation for CPR training with two scenarios of a patient after cardiac surgery, which participants found was immersive and useful. By proving the face validity and content validity of the CPVR-sim, we present the first step toward a cardiothoracic surgery VR training platform.

Effect of Short-term-focused Training on a Phantom Model in Improving Operative Room Performance Among Surgical Residents: A Randomized Trial

BACKGROUND

Meta-analysis has shown the effectiveness of various training methods for the acquisition of laparoscopic skills in surgical training. However, there is very limited literature focusing on the translation of skill acquisition on training models into improved operating room (OR) performance. This study was conducted to evaluate the effectiveness of the Tuebingen trainer with integrated Porcine tissue in improving OR the performance of surgical trainees using standard assessment tools.

MATERIALS AND METHODS

The study was a single-blinded double-armed randomized control study conducted between July 2016 and March 2018. Eighteen, fourth, and fifth semesters of surgery residents were included in the study. The baseline performance was assessed in OR by performing laparoscopic cholecystectomy using validated scores, that is, Global Operative Assessment of Laparoscopic Skills (GOALS), Additional Five Criteria, Task-specific Checklist, Error Checklist, Visual Analogue Scale. The residents were then randomized into trainee and nontrainee groups. The training group received 5 days of short-term-focused training on the Tuebingen trainer, and the improvement was reassessed in OR.

RESULTS

The demographic profile of residents was similar. The baseline scores were comparable. The training group showed statistically significant improvement in GOALS (9.88±1.76 to 12±0.66, P=0.05 vs. 10.33±1.5 to 11.4±2.24, P=0.28), task-specific checklist (42.22±10.92 to 53.33±14.14, P=0.027 vs. 45.55±10.13 to 50±17.32, P=0.51), and error checklist. The operating time significantly reduced (36.0±4.03 vs. 50.44±11.39, P=0.0025) following training.

CONCLUSIONS

Our study concludes that the training on the Tuebingen trainer with integrated porcine organs results in a statistically significant improvement in the OR performance of surgical residents as compared with the nontrained residents, thereby indicating a transfer of skills from training to OR.

Validity of scoring systems for the assessment of technical and non-technical skills in ophthalmic surgery-a systematic review

Evaluation and recommendation of the scoring systems for technical skills (TS) and non-technical skills (NTS) assessments in ophthalmic surgery. A literature search was performed between December 2019 and May 2020. Studies describing the development or validation of TS or NTS scoring systems in ophthalmic surgery were included. Only scoring systems for completion by hand were included. The primary outcome was the validity and reliability status for each scoring system. The secondary outcome was recommendation based on modified Oxford Centre for Evidence-Based Medicine guidelines. Nineteen and five scoring systems were identified for TS and NTS respectively. TS scoring systems exist for cataract surgery (including the steps of phacoemulsification and paediatric cataract surgery) ptosis, strabismus, lateral tarsal strip, vitrectomy, and intraocular surgery in general. NTS scoring systems apply to cataract surgery or ophthalmic surgery in general. No single scoring system satisfied all validity and reliability measures. The recommended TS scoring systems are 'International Council of Ophthalmology's Ophthalmology Surgical Competency Assessment Rubrics' (ICO-OSCAR) for phacoemulsification, strabismus and paediatric cataract surgery, and 'Objective Structured Assessment of Cataract Surgical Skill' (OSACSS). Non-Technical Skills for Surgeons (NOTSS), Observational Teamwork Assessment for Surgery (OTAS) and Anaesthetists Non-Technical Skills (ANTS) are recommended for NTS. There is a paucity of NTS scoring systems. Further research is required to validate all scoring systems to consistent standards. Limitations of the assessment tools included infrequent quantification of face and content validity, and inconsistency in terminology and statistical methods between studies.

A Technique for Mimicking Soft Tissue Manipulation from Experimental Data to a Wave Equation Model for a New Laparoscopic Virtual Reality Training System

Background:

The difficulty of laparoscopic procedures and the specific psychomotor skills required support the need for a training system for intensive and repetitive practice to acquire the specific skills. The present VR training systems have some limitations with respect to the soft tissue models in the training system. This is associated with the need for a real-time simulation, which requires a balance between computational cost and accuracy.

Objective:

The primary objective of the study is to develop a two dimensional wave equation model that closely mimics the soft tissue manipulation in a laparoscopic procedure for a VR training system.

Methods:

A novel mathematical model based on the wave equation is prepared to represent the interaction between the laparoscopic tool and the soft tissue. The parameters within the model are determined through experimental analysis of a soft tissue phantom. The experimental setup involves a linear actuator applying force to the soft tissue phantom to generate deformation. Data acquisition is conducted through a camera and a robotic force acquisition system which measures force, displacement of the linear actuator and records a video. Through image processing, the displacements of the markers on the phantom’s x-y plane during its deformation are determined and these parameters are used to develop the model, which finally is validated through a comparative analysis.

Results:

The results from the developed model are observed and compared statistically as well as graphically with the finite element model based on deformation data. The results show that the deformation data between the developed model and the available model is significantly similar.

Conclusion:

This study demonstrates the adaptability of the wave equation to meet the needs of the specific surgical procedure through modification of the model based on the experimental data. Moreover, the comparative analysis further corroborates the relevance and validity of the model for the surgical training system.

A review of participant recruitment transparency for sound validation of hip surgery simulators: a novel umbrella approach

Malposition of implants is associated with complications, higher wear and increased revision rates in total hip replacement (THR) along with surgeon inexperience. Training THR residents to reach expert proficiency is affected by the high cost and resource limitations of traditional training techniques. Research in extended reality (XR) technologies can overcome such barriers. These offer a platform for learning, objective skill-monitoring and, potentially, for automated certification. Prior to their incorporation into curricula however, thorough validation must be undertaken. As validity is heavily dependent on the participants recruited, there is a need to review, scrutinise and define recruitment criteria in the absence of pre-defined standards, for sound simulator validation. A systematic review on PubMed and IEEE databases was conducted. Training simulator validation research in fracture, arthroscopy and arthroplasty relating to the hip was included. 46 validation studies were reviewed. It was observed that there was no uniformity in reporting or recruitment criteria, rendering cross-comparison challenging. This work developed Umbrella categories to help prioritise recruitment, and has formulated a detailed template of fields and guidelines for reporting criteria so that, in future, research may come to a consensus as to recruitment criteria for a hip "expert" or "novice".

A haptic laparoscopic trainer based on affine velocity analysis: engineering and preliminary results

BACKGROUND

There is a general agreement upon the importance of acquiring laparoscopic skills outside the operation room through simulation-based training. However, high-fidelity simulators are cost-prohibitive and elicit a high cognitive load, while low-fidelity simulators lack effective feedback. This paper describes a low-fidelity simulator bridging the existing gaps with affine velocity as a new assessment variable. Primary validation results are also presented.

METHODS

Psycho-motor skills and engineering key features have been considered e.g. haptic feedback and complementary assessment variables. Seventy-seven participants tested the simulator (17 expert surgeons, 12 intermediates, 28 inexperienced interns, and 20 novices). The content validity was tested with a 10-point Likert scale and the discriminative power by comparing the four groups' performance over two sessions.

RESULTS

Participants rated the simulator positively, from 7.25 to 7.72 out of 10 (mean, 7.57). Experts and intermediates performed faster with fewer errors (collisions) than inexperienced interns and novices. The affine velocity brought additional differentiations, especially between interns and novices.

CONCLUSION

This affordable haptic simulator makes it possible to learn and train laparoscopic techniques. Self-assessment of basic skills was easily performed with slight additional cost compared to low-fidelity simulators. It could be a good trade-off among the products currently used for surgeons' training.

Validation of a high-fidelity training model for fetoscopic spina bifida surgery

Open fetal surgery for spina bifida (SB) is safe and effective yet invasive. The growing interest in fetoscopic SB repair (fSB-repair) prompts the need for appropriate training. We aimed to develop and validate a high-fidelity training model for fSB-repair. fSB-repair was simulated in the abdominal cavity and on the stomach of adult rabbits. Laparoscopic fetal surgeons served either as novices (n = 2) or experts (n = 3) based on their experience. Technical performance was evaluated using competency Cumulative Sum (CUSUM) analysis and the group splitting method. Main outcome measure for CUSUM competency was a composite binary outcome for surgical success, i.e. watertight repair, operation time ≤ 180 min and Objective-Structured-Assessment-of-Technical-Skills (OSATS) score ≥ 18/25. Construct validity was first confirmed since competency levels of novices and experts during their six first cases using both methods were significantly different. Criterion validity was also established as 33 consecutive procedures were needed for novices to reach competency using learning curve CUSUM, which is a number comparable to that of clinical fSB-repair. Finally, we surveyed expert fetal surgeons worldwide to assess face and content validity. Respondents (26/49; 53%) confirmed it with ≥ 71% of scores for overall realism ≥ 4/7 and usefulness ≥ 3/5. We propose to use our high-fidelity model to determine and shorten the learning curve of laparoscopic fetal surgeons and retain operative skills.

Validated specialty-specific models for multi-disciplinary microsurgery training laboratories: a systematic review

BACKGROUND

Laboratory simulation is increasingly important for teaching microsurgical skills. Training microsurgeons of different specialties within the same simulation laboratory increases efficiency of resource use. For maximal benefit, simulations should be available for trainees to practice specialty-specific, higher-order skills. Selection of appropriate simulations requires knowledge of the efficacy and validity of the numerous described laboratory models. Here we present a systematic review of validated training models that may serve as useful adjuncts to achieving competency in specialty elements of microsurgery, and appraise the evidence behind them.

METHODS

In setting up a multi-disciplinary microsurgery training course, we performed a systematic review according to preferred reporting items for systematic reviews and meta-analyses guidelines. EMBASE, MEDLINE, Cochrane and PubMed databases were searched for studies describing validated, microscope-based, specialty-specific simulations, and awarded a level of evidence and level of recommendation based on a modified Oxford Centre for Evidence-Based Medicine classification.

RESULTS

A total of 141 papers describing specialty-specific microsimulation models were identified, 49 of which included evidence of validation. Eleven were in the field of neurosurgery, 21 in otolaryngology/head and neck surgery, two in urology/gynaecology and 15 plastic and reconstructive surgery. These papers described synthetic models in 19 cases, cadaveric animals in 10 cases, live animals in 12 cases and human cadaveric material in 10 cases.

CONCLUSION

Numerous specialty-specific models for use in the microscope laboratory are available, but the quality of evidence for them is poor. Provision of models that span numerous specialties may encourage use of a microscope lab whilst still enabling more specific skills training over a 'one-size-fits-all' approach.

Re-defining the virtual reality dental simulator: Demonstrating concurrent validity of clinically relevant assessment and feedback

INTRODUCTION

Virtual reality (VR) dental simulators are gaining momentum as a useful tool to educate dental students. To date, no VR dental simulator exercise has been designed which is capable of reliably providing validated, meaningful clinical feedback to dental students. This study aims to measure the concurrent validity of the assessment and the provision of qualitative feedback, pertaining to cavity preparations by VR dental simulators.

METHODS

A cavity preparation exercise was created on a VR dental simulator, and assessment criteria for cavity preparations were developed. The exercise was performed 10 times in order to demonstrate a range of performances, and for each, the simulator feedback was recorded. The exercises were subsequently three-dimensionally printed, and 12 clinical teachers were asked to assess the preparations according to the same criteria. Inter-rater reliability (IRR) between clinical teachers was measured using a free-marginal multirater kappa value. Clinical teacher assessment responses were compared with the VR simulator responses and percentage agreements calculated.

RESULTS

IRR values for each exercise ranged from 0.39 to 0.77 (69.39%-88.48%). The assessment of smoothness (κfree 0.58, 78.79%) and ability to follow the outline (κfree 0.56, 77.88%) demonstrated highest agreement between clinical teachers, whilst the assessment of undercut (κfree 0.15, 57.58%) and depth (κfree 0.28, 64.09%) had the lowest agreement. The modal percentage agreement between clinical teachers and the VR simulator was, on average, 78% across all exercises.

CONCLUSION

The results of this study demonstrate that it is possible to provide reliable and clinically relevant qualitative feedback via a VR dental simulator. Further research should look to employ this technique across a broader range of exercises that help to develop other complex operative dental skills.

Skill-Oriented and Performance-Driven Adaptive Curricula for Training in Robot-Assisted Surgery Using Simulators: A Feasibility Study

OBJECTIVE

Virtual Reality (VR) simulators represent a remarkable educational opportunity in order to acquire and refine surgical practical skills. Nevertheless, there exists no consensus regarding a standard curriculum of simulation-based training. This study introduces an automatic, adaptive curriculum where the training session is real-time scheduled on the basis of the trainee's performances.

METHODS

An experimental study using the master console of the da Vinci Research Kit (Intuitive Surgical Inc., Sunnyvale, US) was carried out to test this approach. Tasks involving fundamental skills of robotic surgery were designed and simulated in VR. Twelve participants without medical background along with twelve medical residents were randomly and equally divided into two groups: a control group, self-managing the training session, and an experimental group, undergoing the proposed adaptive training.

RESULTS

The performances of the experimental users were significantly better with respect to the ones of the control group after training (non-medical: p < 0.01; medical: p = 0.02). This trend was analogous in the non-medical and medical populations and no significant difference was identified between these two classes (even in the baseline assessment).

CONCLUSION

The analysis of the learning of the involved surgical skills highlighted how the proposed adaptive training managed to better identify and compensate for the trainee's gaps. The absence of initial difference between the non-medical and medical users underlines that robotic surgical devices require specific training before clinical practice.

SIGNIFICANCE

This feasibility study could pave the way towards the improvement of simulation-based training curricula.

Assessment of validity evidence for the RobotiX robot assisted surgery simulator on advanced suturing tasks

Background

Robot assisted surgery has expanded considerably in the past years. Compared to conventional open or laparoscopic surgery, virtual reality (VR) training is an essential component in learning robot assisted surgery. However, for tasks to be implemented in a curriculum, the levels of validity should be studied for proficiency-based training. Therefore, this study was aimed to assess the validity evidence of advanced suturing tasks on a robot assisted VR simulator.

Method

Participants were voluntary recruited and divided in the robotic experienced, laparoscopic experienced or novice group, based on self-reported surgical experience. Subsequently, a questionnaire on a five-point Likert scale was completed to assess the content validity. Three component tasks of complex suturing were performed on the RobotiX simulator (Task1: tilted plane needle transfer, Task: 2 intracorporal suturing, Task 3: anastomosis needle transfer). Accordingly, the outcome of the parameters was used to assess construct validity between robotic experienced and novice participants. Composite scores (0–100) were calculated from the construct parameters and corresponding pass/fail scores with false positive (FP) and false negative (FN) percentages.

Results

Fifteen robotic experienced, 26 laparoscopic experienced and 29 novices were recruited. Overall content validity outcomes were scored positively on the realism (mean 3.7), didactic value (mean 4.0) and usability (mean 4.2). Robotic experienced participants significantly outperformed novices and laparoscopic experienced participants on multiple parameters on all three tasks of complex suturing. Parameters showing construct validity mainly consisted of movement parameters, needle precision and task completion time. Calculated composite pass/fail scores between robotic experienced and novice participants resulted for Task 1 in 73/100 (FP 21%, FN 5%), Task 2 in 85/100 (FP 28%, FN 4%) and Task 3 in 64/100 (FP 49%, FN 22%).

Conclusion

This study assessed the validity evidence on multiple levels of the three studied tasks. The participants score the RobotiX good on the content validity level. The composite pass/fail scores of Tasks 1 and 2 allow for proficiency-based training and could be implemented in a robot assisted surgery training curriculum.

Use of a Low-Cost Portable 3D Virtual Reality Gesture-Mediated Simulator for Training and Learning Basic Psychomotor Skills in Minimally Invasive Surgery: Development and Content Validity Study

Background

Simulation in virtual environments has become a new paradigm for surgeon training in minimally invasive surgery (MIS). However, this technology is expensive and difficult to access.

Objective

This study aims first to describe the development of a new gesture-based simulator for learning skills in MIS and, second, to establish its fidelity to the criterion and sources of content-related validity evidence.

Methods

For the development of the gesture-mediated simulator for MIS using virtual reality (SIMISGEST-VR), a design-based research (DBR) paradigm was adopted. For the second objective, 30 participants completed a questionnaire, with responses scored on a 5-point Likert scale. A literature review on the validity of the MIS training-VR (MIST-VR) was conducted. The study of fidelity to the criterion was rated using a 10-item questionnaire, while the sources of content-related validity evidence were assessed using 10 questions about the simulator training capacity and 6 questions about MIS tasks, and an iterative process of instrument pilot testing was performed.

Results

A good enough prototype of a gesture-based simulator was developed with metrics and feedback for learning psychomotor skills in MIS. As per the survey conducted to assess the fidelity to the criterion, all 30 participants felt that most aspects of the simulator were adequately realistic and that it could be used as a tool for teaching basic psychomotor skills in laparoscopic surgery (Likert score: 4.07-4.73). The sources of content-related validity evidence showed that this study’s simulator is a reliable training tool and that the exercises enable learning of the basic psychomotor skills required in MIS (Likert score: 4.28-4.67).

Conclusions

The development of gesture-based 3D virtual environments for training and learning basic psychomotor skills in MIS opens up a new approach to low-cost, portable simulation that allows ubiquitous learning and preoperative warm-up. Fidelity to the criterion was duly evaluated, which allowed a good enough prototype to be achieved. Content-related validity evidence for SIMISGEST-VR was also obtained.

Training benchmarks based on validated composite scores for the RobotiX robot-assisted surgery simulator on basic tasks

The RobotiX robot-assisted virtual reality simulator aims to aid in the training of novice surgeons outside of the operating room. This study aimed to determine the validity evidence on multiple levels of the RobotiX simulator for basic skills. Participants were divided in either the novice, laparoscopic or robotic experienced group based on their minimally invasive surgical experience. Two basic tasks were performed: wristed manipulation (Task 1) and vessel energy dissection (Task 2). The performance scores and a questionnaire regarding the realism, didactic value, and usability were gathered (content). Composite scores (0–100), pass/fail values, and alternative benchmark scores were calculated. Twenty-seven novices, 21 laparoscopic, and 13 robotic experienced participants were recruited. Content validity evidence was scored positively overall. Statistically significant differences between novices and robotic experienced participants (construct) was found for movements left (Task 1 p = 0.009), movements right (Task 1 p = 0.009, Task 2 p = 0.021), path length left (Task 1 p = 0.020), and time (Task 1 p = 0.040, Task 2 p < 0.001). Composite scores were statistically significantly different between robotic experienced and novice participants for Task 1 (85.5 versus 77.1, p = 0.044) and Task 2 (80.6 versus 64.9, p = 0.001). The pass/fail score with false-positive/false-negative percentage resulted in a value of 75/100, 46/9.1% (Task 1) and 71/100, 39/7.0% (Task 2). Calculated benchmark scores resulted in a minority of novices passing multiple parameters. Validity evidence on multiple levels was assessed for two basic robot-assisted surgical simulation tasks. The calculated benchmark scores can be used for future surgical simulation training.

Current status of wet lab and cadaveric simulation in urological training: A systematic review

INTRODUCTION

We undertook a systematic review of the use of wet lab (animal and cadaveric) simulation models in urological training, with an aim to establishing a level of evidence (LoE) for studies and level of recommendation (LoR) for models, as well as evaluating types of validation.

METHODS

Medline, EMBASE, and Cochrane databases were searched for English-language studies using search terms including a combination of "surgery," "surgical training," and "medical education." These results were combined with "wet lab," "animal model," "cadaveric," and "in-vivo." Studies were then assigned a LoE and LoR if appropriate as per the education-modified Oxford Centre for Evidence-Based Medicine classification.

RESULTS

A total of 43 articles met the inclusion criteria. There was a mean of 23.1 (±19.2) participants per study with a median of 20. Overall, the studies were largely of low quality, with 90.7% of studies being lower than LoE 2a (n=26 for LoE 2b and n=13 for LoE 3). The majority (72.1%, n=31) of studies were in animal models and 27.9% (n=12) were in cadaveric models.

CONCLUSIONS

Simulation in urological education is becoming more prevalent in the literature, however, there is a focus on animal rather than cadaveric simulation, possibly due to cost and ethical considerations. Studies are also predominately of a low LoE; higher LoEs, especially randomized controlled studies, are needed.

Assessment of a virtual reality temporal bone surgical simulator: a national face and content validity study

BACKGROUND

Trainees in Otolaryngology-Head and Neck Surgery must gain proficiency in a variety of challenging temporal bone surgical techniques. Traditional teaching has relied on the use of cadavers; however, this method is resource-intensive and does not allow for repeated practice. Virtual reality surgical training is a growing field that is increasingly being adopted in Otolaryngology. CardinalSim is a virtual reality temporal bone surgical simulator that offers a high-quality, inexpensive adjunct to traditional teaching methods. The objective of this study was to establish the face and content validity of CardinalSim through a national study.

METHODS

Otolaryngologists and resident trainees from across Canada were recruited to evaluate CardinalSim. Ethics approval and informed consent was obtained. A face and content validity questionnaire with questions categorized into 13 domains was distributed to participants following simulator use. Descriptive statistics were used to describe questionnaire results, and either Chi-square or Fishers exact tests were used to compare responses between junior residents, senior residents, and practising surgeons.

RESULTS

Sixty-two participants from thirteen different Otolaryngology-Head and Neck Surgery programs were included in the study (32 practicing surgeons; 30 resident trainees). Face validity was achieved for 5 out of 7 domains, while content validity was achieved for 5 out of 6 domains. Significant differences between groups (p-value of < 0.05) were found for one face validity domain (realistic ergonomics, p = 0.002) and two content validity domains (teaching drilling technique, p = 0.011 and overall teaching utility, p = 0.006). The assessment scores, global rating scores, and overall attitudes towards CardinalSim, were universally positive. Open-ended questions identified limitations of the simulator.

CONCLUSION

CardinalSim met acceptable criteria for face and content validity. This temporal bone virtual reality surgical simulation platform may enhance surgical training and be suitable for patient-specific surgical rehearsal for practicing Otolaryngologists.

Creating a Validated Simulation Training Curriculum in Otolaryngology

Purpose of Review

Simulation-based training is an integral component of surgical training. It allows practice of technical skills within a safe environment without compromising patient safety. This article seeks to review current virtual and non-virtual reality simulation models within the literature and review their validation status.

Recent Findings

Many simulation models exist within otolaryngology and are currently being used for education. New models are also continuously being developed; however, validity should be proven for the models before incorporating their use for educational purposes. Validity should be determined by experts and trainees themselves.

Summary

A validated simulation curriculum should be incorporated within the otolaryngology training programme. A curriculum based on the current training programme at our institution serves as an exemplar for local adoption.

Evaluation of a Virtual Reality Percutaneous Nephrolithotomy (PCNL) Surgical Simulator

Percutaneous Nephrolithotomy is the standard surgical procedure used to remove large kidney stones. PCNL procedures have a steep learning curve; a physician needs to complete between 36 and 60 procedures, to achieve clinical proficiency. Marion Surgical K181 is a virtual reality surgical simulator, which emulates the PCNL procedures without compromising the well-being of patients. The simulator uses a VR headset to place a user in a realistic and immersive operating theater, and haptic force-feedback robots to render physical interactions between surgical tools and the virtual patient. The simulator has two modules for two different aspects of PCNL kidney stone removal procedure: kidney access module where the user must insert a needle into the kidney of the patient, and a kidney stone removal module where the user removes the individual stones from the organ. In this paper, we present user trials to validate the face and construct validity of the simulator. The results, based on the data gathered from 4 groups of users independently, indicate that Marion's surgical simulator is a useful tool for teaching and practicing PCNL procedures. The kidney stone removal module of the simulator has proven construct validity by identifying the skill level of different users based on their tool path. We plan to continue evaluating the simulator with a larger sample of users to reinforce our findings.

Benefits and Limitations of Transurethral Resection of the Prostate Training With a Novel Virtual Reality Simulator

PURPOSE

Profound endourological skills are required for optimal postoperative outcome parameters after transurethral resection of the prostate (TURP). We investigated the Karl Storz (Tuttlingen, Germany) UroTrainer for virtual simulation training of the TURP.

MATERIALS AND METHODS

Twenty urologists underwent a virtual reality (VR) TURP training. After a needs analysis, performance scores and self-rated surgical skills were compared before and after the curriculum, the realism of the simulator was assessed, and the optimal level of experience for VR training was evaluated. Statistical testing was done with SPSS 25.

RESULTS

Forty percent of participants indicated frequent intraoperative overload during real-life TURP and 80% indicated that VR training might be beneficial for endourological skills development, underlining the need to advance classical endourological training. For the complete cohort, overall VR performance scores (P = 0.022) and completeness of resection (P < 0.001) significantly improved. Self-rated parameters including identification of anatomical structures (P = 0.046), sparing the sphincter (P = 0.002), and handling of the resectoscope (P = 0.033) became significantly better during the VR curriculum. Participants indicated progress regarding handling of the resectoscope (70%), bleeding control (55%), and finding the correct resection depth (50%). Although overall realism and handling of the resectoscope was good, virtual bleeding control and correct tissue feedback should be improved in future VR simulators. Seventy percent of participants indicated 10 to 50 virtual TURP cases to be optimal and 80% junior residents to be the key target group for VR TURP training.

CONCLUSIONS

There is a need to improve training the TURP and VR simulators might be a valuable supplement, especially for urologists beginning with the endourological desobstruction of the prostate.

Systematic review of the current status of cadaveric simulation for surgical training

Background

There is growing interest in and provision of cadaveric simulation courses for surgical trainees. This is being driven by the need to modernize and improve the efficiency of surgical training within the current challenging training climate. The objective of this systematic review is to describe and evaluate the evidence for cadaveric simulation in postgraduate surgical training.

Methods

A PRISMA‐compliant systematic literature review of studies that prospectively evaluated a cadaveric simulation training intervention for surgical trainees was undertaken. All relevant databases and trial registries were searched to January 2019. Methodological rigour was assessed using the widely validated Medical Education Research Quality Index (MERSQI) tool.

Results

A total of 51 studies were included, involving 2002 surgical trainees across 69 cadaveric training interventions. Of these, 22 assessed the impact of the cadaveric training intervention using only subjective measures, five measured impact by change in learner knowledge, and 23 used objective tools to assess change in learner behaviour after training. Only one study assessed patient outcome and demonstrated transfer of skill from the simulated environment to the workplace. Of the included studies, 67 per cent had weak methodology (MERSQI score less than 10·7).

Conclusion

There is an abundance of relatively low‐quality evidence showing that cadaveric simulation induces short‐term skill acquisition as measured by objective means. There is currently a lack of evidence of skill retention, and of transfer of skills following training into the live operating theatre.

Current status of simulation-based training in pediatric surgery: A systematic review

BACKGROUND

Simulation based training enables pediatric surgical trainees to attain proficiency in surgical skills. This study aims to identify the currently available simulators for pediatric surgery, assess their validation and strength of evidence supporting each model.

METHODS

Both Medline and EMBASE were searched for English language articles either describing or validating simulation models for pediatric surgery. A level of evidence (LoE) followed by a level of recommendation (LoR) was assigned to each validation study and simulator, based on a modified Oxford Centre for Evidence-Based Medicine classification for educational studies.

RESULTS

Forty-nine articles were identified describing 44 training models and courses. Of these articles, 44 were validation studies. Face validity was evaluated by 20 studies, 28 for content, 24 demonstrated construct validity and 1 showed predictive validity. Of the validated models, 3 were given an LoR of 2, 21 an LoR of 3 and 12 an LoR of 4. None reached the highest LoR.

CONCLUSIONS

There are a growing number of simulators specific to pediatric surgery. However, these simulators have limited LoE and LoR in current studies. The lack of NoTSS training is also apparent. We advocate more randomized trials to validate these models, and attempts to determine predictive validity.

TYPE OF STUDY

Original / systematic review.

LEVEL OF EVIDENCE

1.

Construct, content and face validity of the eoSim laparoscopic simulator on advanced suturing tasks

Background

The purpose of this study was to validate the eoSim, an affordable and mobile inanimate laparoscopic simulator with instrument tracking capabilities, regarding face, content and construct validity on complex suturing tasks.

Methods

Participants recruited for this study were novices (no laparoscopic experience), target group for this training (surgical/gynaecologic/urologic residents, > 10 basic and < 20 advanced laparoscopic procedures) and experts (> 20 advanced laparoscopic procedures). Each participant performed the intracorporeal suturing exercise (Task 1), an upside down needle transfer (Task 2, developed for this study) and an anastomosis needle transfer (Task 3). Following, the participants completed a questionnaire regarding their demographics and opinion on the eoSim in terms of realism, didactic value and usability. Measured outcome parameters were time, distance, percentage of instrument tip off-screen, working area, speed, acceleration and smoothness.

Results

In total, 104 participants completed the study, of which 60 novices, 31 residents and 13 experts. Face and content validity results showed a mean positive opinion on realism (3.9 Task 1, 3.6 Task 2 and 3.7 Task 3), didactic value (4.0, 3.4 and 3.7, respectively) and usability (4.2. 3.7 and 4.0, respectively). There were no significant differences in these outcomes between the specified expertise groups. Construct validity results showed significant differences between experts, target group or novices for Task 1 in terms of time (means 339, 607 and 1224 s, respectively, p < 0.001) and distance (means 8.1, 15.6 and 21.7 m, respectively, p < 0.001). Task 2 showed significant differences between groups regarding time (p < 0.001), distance (p 0.003), off-screen (p < 0.001) and working area (p < 0.001). Task 3 showed significant differences between groups, after subanalyses, on total number of stitches (p < 0.001), time per stitch (p < 0.001) and distance per stitch (p < 0.001).

Conclusions

The results of this study indicate that the eoSim is a potential meaningful and valuable simulator in the training of suturing tasks.

Nontechnical Skills in Surgery: A Systematic Review of Current Training Modalities

BACKGROUND

The complexity of the operating room requires a surgeon to have both technical ability and an array of nontechnical skills. The emphasis on technical skills during surgical training is well established, however it is deficiencies in nontechnical skills that have been identified as the main cause of errors in the operating room.

OBJECTIVE

This systematic review aims to identify current methods used to teach nontechnical skills and how these methods are assessed to determine their validity, evidence, and role in training.

METHODS

MEDLINE and Embase databases were searched for English language articles between 2000 and 2017 for nontechnical surgical skills training. Original research articles were included if they described non-technical surgical skills training modalities and their assessment. Results were assessed for the level of evidence and each modality was awarded a level of recommendation, using a modified educational Oxford Centre for Evidence-Based Medicine classification, as adapted by the European Association of Endoscopic Surgery.

RESULTS

A total of 19 studies were identified pertaining to high fidelity simulation (n = 8), low fidelity simulation (n = 6), didactic teaching (n = 2) and crisis resource management (n = 3). Of the included studies 1 was classified Level 1b, 1 level 2b, 7 level 2b, 2 level 2c, and 8 level 3.

CONCLUSION

With the importance of nontechnical skills being increasingly recognized, it is essential for surgeons to receive adequate training in nontechnical skills. Therefore the most valuable teaching modalities such as high and low fidelity simulation needs to be implemented into surgical training curricula.

Exploring Validation and Verification: How they Different and What They Mean to Healthcare Simulation

STATEMENT

The healthcare simulation (HCS) community recognizes the importance of quality management because many novel simulation devices and techniques include some sort of description of how they tested and assured their simulation's quality. Verification and validation play a key role in quality management; however, literature published on HCS has many different interpretations of what these terms mean and how to accomplish them. The varied use of these terms leads to varied interpretations of how verification process is different from validation process. We set out to explore the concepts of verification and validation in this article by reviewing current psychometric science description of the concepts and exploring how other communities relevant to HCS, such as medical device manufacturing, aviation simulation, and the fields of software and engineering, which are building blocks of technology-enhanced HCS, use the terms, with the focus of trying to clarify the process of verification. We also review current literature available on verification, as compared with validation in HCS and, finally, offer a working definition and concept for each of these terms with hopes to facilitate improved communication within, and with colleagues outside, the HCS community.

Predictability of Clinical Knowledge Through Mobile App-based Simulation for the Treatment of Pediatric Septic Arthritis: A Pilot Study

BACKGROUND

Recently the American Board of Orthopaedic Surgery and the Accreditation Council of Graduate Medical Education have identified the treatment of septic arthritis of the hip in children as a milestone skill for all US orthopaedic residents. The purpose of this study was to test correlation between clinical knowledge and examination score on a mobile app-based training module for the treatment of pediatric septic hip arthritis.

METHODS

A 4-part simulation model on surgical decision-making associated with the treatment of pediatric septic arthritis was developed through expert consensus. Orthopaedic trainees participating in the "Top Gun" program of the 2015 and 2016 International Pediatric Orthopaedic Symposiums were recruited to participate in this pilot study. Trainees completed a presimulation quiz on their knowledge of diagnosis, arthrocentesis, and surgical irrigation and debridement on a pediatric patient presenting with septic arthritis of the hip. Trainees then completed the 4-part simulation on the mobile app. Pearson correlation analysis was used to assess the relationship between the quiz and the simulation.

RESULTS

A total of 53 orthopaedic residents and fellows participated in the simulation. Median quiz score was 87 points [interquartile range (IQR), 81 to 94] before the intervention and 100 points (IQR, 94 to 100) postintervention. The median simulation test score was 89 (IQR, 81 to 92) which demonstrated a positive correlation with the postintervention quiz (r=0.44, P<0.001). The preintervention metrics demonstrated a positive correlation with postintervention metrics (r=0.53, P<0.001).

CONCLUSIONS

This study revealed a statistically significant positive correlation between the mobile app simulation and the clinical knowledge of the participants, as well as the ability to improve knowledge about a procedure during the testing period. These findings support the ability for the mobile app to test clinical knowledge. In the current environment of decreased work hours and patient exposure for orthopaedic trainees, mobile app-based simulation has the potential to safely aid in assessment of orthopaedic residents and fellows.

Development of a vibration haptic simulator for shoulder arthroplasty

PURPOSE

Glenoid reaming is a technically challenging step during shoulder arthroplasty that could possibly be learned during simulation training. Creation of a realistic simulation using vibration feedback in this context is innovative. Our study focused on the development and internal validation of a novel glenoid reaming simulator for potential use as a training tool.

METHODS

Vibration and force profiles associated with glenoid reaming were quantified during a cadaveric experiment. Subsequently, a simulator was fabricated utilizing a haptic vibration transducer with high- and low-fidelity amplifiers; system calibration was performed matching vibration peak-peak values for both amplifiers. Eight experts performed simulated reaming trials. The experts were asked to identify isolated layer profiles produced by the simulator. Additionally, experts' efficiency to successfully perform a simulated glenoid ream based solely on vibration feedback was recorded.

RESULTS

Cadaveric experimental cartilage reaming produced lower vibrations compared to subchondral and cancellous bones ([Formula: see text]). Gain calibration of a lower-fidelity (3.5 [Formula: see text] and higher-fidelity (3.4 [Formula: see text] amplifier resulted in values similar to the cadaveric experimental benchmark (3.5 [Formula: see text]. When identifying random tissue layer samples, experts were correct [Formula: see text] of the time and success rate varied with tissue type ([Formula: see text]). During simulated reaming, the experts stopped at the targeted subchondral bone with a success rate of [Formula: see text]. The fidelity of the simulation did not have an effect on accuracy, applied force, or reaming time ([Formula: see text]). However, the applied force tended to increase with trial number ([Formula: see text]).

CONCLUSIONS

Development of the glenoid reaming simulator, coupled with expert evaluation furthered our understanding of the role of haptic vibration feedback during glenoid reaming. This study was the first to (1) propose, develop and examine simulated glenoid reaming, and (2) explore the use of haptic vibration feedback in the realm of shoulder arthroplasty.

Virtual Reality and Simulation in Neurosurgical Training

Recent biotechnological advances, including three-dimensional microscopy and endoscopy, virtual reality, surgical simulation, surgical robotics, and advanced neuroimaging, have continued to mold the surgeon-computer relationship. For developing neurosurgeons, such tools can reduce the learning curve, improve conceptual understanding of complex anatomy, and enhance visuospatial skills. We explore the current and future roles and application of virtual reality and simulation in neurosurgical training.

VCSim3: a VR simulator for cardiovascular interventions

PURPOSE

Effective and safe performance of cardiovascular interventions requires excellent catheter/guidewire manipulation skills. These skills are currently mainly gained through an apprenticeship on real patients, which may not be safe or cost-effective. Computer simulation offers an alternative for core skills training. However, replicating the physical behaviour of real instruments navigated through blood vessels is a challenging task.

METHODS

We have developed VCSim3-a virtual reality simulator for cardiovascular interventions. The simulator leverages an inextensible Cosserat rod to model virtual catheters and guidewires. Their mechanical properties were optimized with respect to their real counterparts scanned in a silicone phantom using X-ray CT imaging. The instruments are manipulated via a VSP haptic device. Supporting solutions such as fluoroscopic visualization, contrast flow propagation, cardiac motion, balloon inflation, and stent deployment, enable performing a complete angioplasty procedure.

RESULTS

We present detailed results of simulation accuracy of the virtual instruments, along with their computational performance. In addition, the results of a preliminary face and content validation study conveyed on a group of 17 interventional radiologists are given.

CONCLUSIONS

VR simulation of cardiovascular procedure can contribute to surgical training and improve the educational experience without putting patients at risk, raising ethical issues or requiring expensive animal or cadaver facilities. VCSim3 is still a prototype, yet the initial results indicate that it provides promising foundations for further development.

Navigation of guidewires and catheters in the body during intervention procedures: a review of computer-based models

Guidewires and catheters are used during minimally invasive interventional procedures to traverse in vascular system and access the desired position. Computer models are increasingly being used to predict the behavior of these instruments. This information can be used to choose the right instrument for each case and increase the success rate of the procedure. Moreover, a designer can test the performance of instruments before the manufacturing phase. A precise model of the instrument is also useful for a training simulator. Therefore, to identify the strengths and weaknesses of different approaches used to model guidewires and catheters, a literature review of the existing techniques has been performed. The literature search was carried out in Google Scholar and Web of Science and limited to English for the period 1960 to 2017. For a computer model to be used in practice, it should be sufficiently realistic and, for some applications, real time. Therefore, we compared different modeling techniques with regard to these requirements, and the purposes of these models are reviewed. Important factors that influence the interaction between the instruments and the vascular wall are discussed. Finally, different ways used to evaluate and validate the models are described. We classified the developed models based on their formulation into finite-element method (FEM), mass-spring model (MSM), and rigid multibody links. Despite its numerical stability, FEM requires a very high computational effort. On the other hand, MSM is faster but there is a risk of numerical instability. The rigid multibody links method has a simple structure and is easy to implement. However, as the length of the instrument is increased, the model becomes slower. For the level of realism of the simulation, friction and collision were incorporated as the most influential forces applied to the instrument during the propagation within a vascular system. To evaluate the accuracy, most of the studies compared the simulation results with the outcome of physical experiments on a variety of phantom models, and only a limited number of studies have done face validity. Although a subset of the validated models is considered to be sufficiently accurate for the specific task for which they were developed and, therefore, are already being used in practice, these models are still under an ongoing development for improvement. Realism and computation time are two important requirements in catheter and guidewire modeling; however, the reviewed studies made a trade-off depending on the purpose of their model. Moreover, due to the complexity of the interaction with the vascular system, some assumptions have been made regarding the properties of both instruments and vascular system. Some validation studies have been reported but without a consistent experimental methodology.