Beyond the simulation laboratory: a realist synthesis review of clinical outcomes of simulation-based mastery learning

Using Simulation-Based Mastery Learning to Achieve Excellent Learning Outcomes in Physical Therapist Education

INTRODUCTION

The 2 aims of this observational study are (a) to describe the implementation and feasibility of a bed mobility skills simulation-based mastery learning (SBML) curricular module for physical therapist students and (b) to measure learning outcomes and student perceptions of this module.

REVIEW OF LITERATURE

Simulation-based mastery learning is an outcome-based educational approach that has been successful in other health professions but has not been explored in physical therapy education.

SUBJECTS

Eighty-seven students in a single cohort of a Doctor of Physical Therapy program.

METHODS

The SBML module in this pretest-posttest study included a pretest, instruction, initial posttest, and additional rounds of instruction and assessment as needed for all learners to achieve the minimum passing standard (MPS) set using the Mastery Angoff and Patient Safety methods. Outcome measures were bed mobility assessment pass rates and scores, additional student and faculty time compared with a traditional approach, and student perceptions of their self-confidence and the module.

RESULTS

All students achieved the MPS after 3 rounds of training and assessment beyond the initial posttest. Mean Total Scores improved from 67.6% (12.9%) at pretest to 91.4% (4.8%) at mastery posttest ( P < .001, Cohen's d = 1.8, 95% CI [1.4-2.1]); mean Safety Scores improved from 75.2% (16.0%) at pretest to 100.0% (0.0%) at mastery posttest ( P < .001, Cohen's d = 1.5, 95% CI [1.2-1.9]). Students who did not achieve the MPS at the initial posttest ( n = 30) required a mean of 1.2 hours for additional instruction and assessment. Survey results revealed an increase in student confidence ( P < .001) and positive student perceptions of the module.

DISCUSSION AND CONCLUSION

Implementation of this SBML module was feasible and resulted in uniformly high levels of bed mobility skill acquisition. Based on rigorous learning outcomes, feasible requirements for implementation, and increased student confidence, SBML offers a promising approach for wider implementation in physical therapy education.

A structured simulation-based mastery learning curriculum in chest tube insertion results in superior skills compared to traditional training programs

BACKGROUND

Chest tube insertions (CTI) have a high complication rate, warranting a dedicated Simulation-Based Mastery Learning (SBML) curriculum to acquire technical skills. This randomized controlled trial compares residents' skills in CTI after completing a SBML curriculum with those enrolled in a traditional residency program.

METHODS

Junior residents were baseline tested on cognitive and technical skills (Thiel bodies) before randomization into an intervention and control group. The former deliberately trained CTI on a porcine rib model until passing a predefined pass/fail score and were then summatively tested on Thiel bodies. The latter had no additional training opportunities and was evaluated 3 months later.

RESULTS

Seventeen residents were recruited and randomized. Following the per-protocol principle, a significant interaction effect for Group × Procedure (F(1,14) = 6.2, p = 0.026) was observed. Between baseline and summative assessment, both the control group (28.0 ± 8.2 vs. 43.6 ± 8.1, p < 0.001) and the intervention group (33.2 ± 7.7 vs. 57.6 ± 5.7, p < 0.001) significantly increased their scores. The intervention group outperformed the control group at summative assessment (43.6 ± 8.1 vs. 57.6 ± 5.7, p < 0.001). All participants in the intervention group and one resident in the control group achieved the pass/fail score.

CONCLUSION

This SBML curriculum enabled quicker and superior skill acquisition. Skills trained on a porcine model are transferred to the highly realistic Thiel bodies and reach expert level, potentially increasing resident skill in clinical practice.

Bridging the procedures skill gap from medical school to residency: a simulation-based mastery learning curriculum

BACKGROUND

The transition from medical student to intern is a recognized educational gap. To help address this, the Association of American Medical Colleges developed the Core Entrustable Professional Activities for entering residency. As these metrics outline expectations for all graduating students regardless of specialty, the described procedural expectations are appropriately basic. However, in procedure-heavy specialties such as emergency medicine, the ability to perform advanced procedures continues to contribute to the disconnect between undergraduate and graduate medical education. To prepare our graduating students for their internship in emergency medicine, we developed a simulation-based mastery learning curriculum housed within a specialty-specific program. Our overall goal was to develop the students' procedural competency for central venous catheter placement and endotracheal intubation before graduation from medical school.

METHODS

Twenty-five students participated in a simulation-based mastery learning procedures curriculum for ultrasound-guided internal jugular central venous catheter placement and endotracheal intubation. Students underwent baseline assessment, deliberate practice, and post-test assessments. Both the baseline and post-test assessments used the same internally developed checklists with pre-established minimum passing scores.

RESULTS

Despite completing an emergency medicine rotation and a critical care rotation, none of the students met the competency standard during their baseline assessments. All twenty-five students demonstrated competency on both procedures by the end of the curriculum. A second post-test was required to demonstrate achievement of the central venous catheter and endotracheal intubation minimum passing scores by 16% and 28% of students, respectively.

CONCLUSIONS

Students demonstrated procedural competency for central venous catheter placement and endotracheal intubation by engaging in simulation-based mastery learning procedures curriculum as they completed their medical school training. With three instructional hours, students were able to achieve basic procedural competence for two common, high-risk procedures they will need to perform during emergency medicine residency training.

Evaluation of high-fidelity and virtual reality simulation platforms for assessing fourth-year medical students' encounters with patients in need of urgent or emergent care

BACKGROUND

Medical students in the U.S. must demonstrate urgent and emergent care competence before graduation. Urgent and emergent care competence involves recognizing, evaluating and initiating management of an unstable patient. High-fidelity (HF) simulation can improve urgent and emergent care skills, but because it is resource intense, alternative methods are needed.

STUDY OBJECTIVE

Our primary purpose was to use program evaluations to compare medical student experiences with HF and virtual reality (VR) simulations as assessment platforms for urgent and emergent care skills.

METHODS

During their emergency medicine clerkship, students at The Ohio State University College of Medicine must demonstrate on HF manikins, competence in recognizing and initiating care of a patient requiring urgent or emergent care. Students evaluated these simulations on a five-point quality scale and answered open-ended questions about simulation strengths and weaknesses. Faculty provided feedback on student competence in delivering urgent or emergent care. In 2022, we introduced VR as an alternative assessment platform. We used Wilcoxon Signed Ranks and Boxplots to compare ratings of HF to VR and McNemar Test to compare competence ratings. Comments were analyzed with summative content analysis or thematic coding.

RESULTS

We received at least one evaluation survey from 160 of 216 (74.1%) emergency medicine clerkship students. We were able to match 125 of 216 (57.9%) evaluation surveys for students who completed both. Average ratings of HF simulations were 4.6 of 5, while ratings of VR simulations were slightly lower at 4.4. Comments suggested that feedback from both simulation platforms was valued. Students described VR as novel, immersive, and good preparation for clinical practice. Constructive criticism identified the need for additional practice in the VR environment. Student performance between platforms was significantly different with 91.7% of students achieving competence in HF, but only 65.5% in VR (p≤.001, odds-ratio = 5.75).

CONCLUSION

VR simulation functions similarly to HF for formative assessment of urgent and emergent care competence. However, using VR simulation for summative assessment of urgent and emergent care competence must be considered with caution because students require considerable practice and acclimation to the virtual environment.

A Pilot Study of a Simulation-Based Mastery Learning Procedural Curriculum for Pediatric Emergency Medicine Fellows

OBJECTIVES

Pediatric emergency medicine (PEM) fellows are expected to perform high-risk procedures across a wide range of patient age, size, and physiology including procedures that are no longer required during pediatric residency training. Examples include central venous catheter (CVC) placement, endotracheal intubation of children and adolescents, and tube thoracostomy placement. Simulation-based mastery learning has demonstrated decreased patient morbidity. In this study, we describe implementation of a simulation-based mastery learning (SBML) procedural curriculum for PEM fellows.

METHODS

Our PEM fellows underwent an SBML procedural curriculum for lumbar puncture (LP), CVC placement, endotracheal intubation, and tube thoracostomy placement. These procedures are mandatory for fellows to learn, have known association with potential iatrogenic injury, and are widely available commercial task trainers. Fellows underwent baseline assessments, group demonstration, deliberate practice, and then postassessments. For both pre- and postassessments, we used internally developed checklists with minimum passing scores (MPSs) calculated by the Mastery-Angoff technique.

RESULTS

Nineteen pediatrics residency-trained PEM fellows underwent this curriculum over a 2-year period. Six fellows (31.58%) achieved the MPS on all four procedures during their first posttest attempt. All fellows achieved the MPS on all four procedures by the second posttest attempt. Most fellows (17/19 or 89.47%) did not achieve the MPS on baseline LP assessments, despite inclusion of this procedure as an Accreditation Council of Graduate Medical Education (ACGME) requirement during their pediatric residency training.

CONCLUSIONS

All participating PEM fellows demonstrated competency after training on four procedures associated with iatrogenic injuries. Using an SBML framework is a feasible method to teach procedural skills to PEM fellows, allowing them to demonstrate objective measures of competency in the simulation laboratory.

Understanding ACGME Standards for Simulation: A Document Analysis of Institutional and Program Requirements

Background Our institution has established priorities for graduate medical education (GME) simulation which include increasing adoption of, garnering additional financial support for, and creating a core simulation curriculum. Better understanding of the Accreditation Council for Graduate Medical Education (ACGME) simulation requirements will inform our efforts and serve as a guide for other institutions. Objective The purpose of this study was to perform a structured review of ACGME simulation standards using a document analysis to guide GME simulation activities at an institutional level. Methods A document analysis was performed from May 2023 to June 2024 to select and search ACGME Institutional and Program Requirements corresponding to the primary specialties for 21 clinical departments that financially support our simulation center. Content relevant to simulation was identified, and iterative coding with investigator team consensus was performed to assign categories, characterize the requirements, and interpret the findings. Results Twenty-four documents included 120 simulation requirements that were assigned to 12 categories; 70 (58%) requirements were mandatory whereas 50 (42%) were not, and 48 (40%) were simulation-specific, whereas 72 (60%) were simulation-optional. All reviewed specialties had simulation requirements (average 5.4, range 2-12), but the ACGME Institutional Requirements did not. Moderate to strong evidence supported (1) simulation usage by all 21 departments; (2) the need for institutional resource support; and (3) institutional-level patient safety simulation curricula. Conclusions This study identified a large number of simulation requirements, including mandatory patient safety curricula requirements, for all specialties analyzed.

Barriers and Facilitators to Implementing Simulation-Based Translational Research: A Qualitative Study

INTRODUCTION

Translational research has been identified as a research priority for the National Institutes of Health (NIH) and the Society for Simulation in Healthcare (SSH). Despite a larger focus on translational research in recent years, the overall amount of simulation-based translational research remains low. Greater understanding of how to approach translational simulation is required to inform novice simulation and translational researchers. This study sought to answer the following research questions: How do simulation experts describe the barriers and facilitators to implementing translational simulation programs? How do simulation experts describe their various approaches to implementing translational simulation programs? What recommendations do simulation experts describe for overcoming barriers to implementing translational simulation programs?

METHODS

A qualitative instrumental case study was used to elicit multiple instances of translational simulation research to gather an in-depth description from study participants. Three data sources were used: documents, semistructured interviews, and a focus group.

RESULTS

Data analyses revealed 5 major themes: clarifying goals and definitions, special considerations, social networking, research, and factors external to the simulation program.

CONCLUSIONS

Key findings include a lack of a standardized definitions for translational simulation and simulation-based translational research, the challenge of demonstrating the value of translational simulation, and the need for translational simulation programs to be integrated into departmental quality, patient safety, and risk management work. The findings and advice from the experts in this research can assist new researchers or those encountering challenges in implementing translational simulations.

Procedural simulation in venipuncture for medical undergraduates and its transfer to the bedside: a cluster randomized study

Simulation is accepted as an effective method of learning procedural skills. However, the translational outcomes of skills acquired through simulation still warrants investigation. We designed this study to assess if skills laboratory training in addition to bedside learning (intervention group [IG]) would provide better learning results than bedside learning alone (control group [CG]) in the context of venipuncture training. This prospective, cluster-randomized, single-blind study took place at the Faculty of Medicine, University of Kelaniya, Sri Lanka. Seventeen clusters of second-year medical students were randomly assigned to either IG or CG. The IG trained on venipuncture in the skills laboratory, receiving instruction after modified Payton's Four Step Method. Following the training, students of both IG and CG underwent bedside learning for one month. Afterward, students of both groups performed venipuncture on actual patients in a clinical setting. An independent, blinded assessor scored students' performance using the Integrated Procedural Protocol Instrument (IPPI) and a checklist. Patients assessed students' performance with the Communication Assessment Tool (CAT). Eight and nine clusters were randomized to the intervention and control groups, respectively. IG completed significantly more single steps of the procedure correctly (IG: 19.36 ± 3.87 for checklist items; CG: 15.57 ± 4.95; p < 0.001). IG also scored significantly better on IPPI ratings (median: IG: 27 (12) vs. CG: 21 (8); p < 0.001). Rated by patients, students' communication skills did not significantly differ between the two groups. Simulation-based venipuncture training enabled students to perform the procedure on actual patients with a higher technical accuracy than students who learned venipuncture at the bedside. Students were able to transfer the skills acquired through venipuncture training at the skill laboratory to the bedside.

Simulation-based mastery learning for intrathecal baclofen pump management in physical medicine and rehabilitation: A pilot study

BACKGROUND

Simulation-based mastery learning (SBML) has demonstrated superiority as a procedural learning method. Implementation of SBML in Physical Medicine and Rehabilitation (PM&R) education has not been described and there is a paucity of literature regarding standardized procedural learning in this specialty.

OBJECTIVE

To evaluate if implementation of intrathecal baclofen pump management SBML affects knowledge, attitudes, and skills in PM&R residents.

DESIGN

Pretest-posttest design.

SETTING

Academic rehabilitation hospital.

PARTICIPANTS

Twenty-two PM&R residents.

INTERVENTIONS

Participants engaged in SBML steps: pre-test, demonstration of proper technique, deliberate practice, and a post-test.

MAIN OUTCOME MEASURES

Ability to achieve minimum passing score, quiz scores, and survey scores. Pre- and post-test results were compared using Wilcoxon signed-rank tests. Post graduate year (PGY)-3 and PGY-4 resident results were compared using Mann-Whitney U tests.

RESULTS

Based on institutional expert consensus, a simulation procedural checklist was created with a minimum passing score set at correctly performing 30 of 31 items. On pre-test, no learners achieved the minimum passing score (22.5, interquartile range [IQR] 17.0-23.0). Initial post-test score for all learners was 30 (IQR 29.8-31.0). Six learners required additional attempts to reach mastery as defined by reaching the minimum passing score. Quiz scores significantly improved from 9 to 11 (pre-test IQR 9-10; post-test IQR 10-11; p < .05). Confidence in procedure significantly improved (pre-test 2.7/5.9, IQR 2.2-3.8; post-test 4.1/5.0, IQR 3.7-4.5; p < .01). Likelihood to perform procedure in future practice did not change (pre-test 2.2, IQR 1.0-3.8; post-test 3.0, IQR 1.0-4.0; p = .89).

CONCLUSION

SBML is an effective tool to improve multiple domains of learning in PM&R residents.

Preparing interns for clinical practice through an institution-wide simulation-based mastery learning program for teaching central venous catheter placement

Central venous catheter (CVC) placement is a challenging procedure with known iatrogenic risks. However, there are no residency program requirements to demonstrate baseline CVC procedural competency. Competency-based procedural education has been shown to decrease CVC-associated morbidity, but there has been limited literature about institution-wide efforts to ensure initial trainee competency for CVC placement. This study describes the implementation of a competency-based CVC curriculum for first-year interns across an institution before supervised clinical care. An institution-wide, simulation-based mastery training curriculum was designed to assess initial competency in CVC placement in first-year residents during 2021 and 2022. A checklist was internally developed with a multidisciplinary team. Using the Mastery-Angoff technique, minimum passing standards were derived to define competency levels considered appropriate for intern participation in supervised clinical care. Interns were trained through the competency-based program with faculty assessing intern performance using the CVC checklist to verify procedural competency. Over 2 academic cycles, 229 interns from 20 specialties/subspecialties participated. Overall, 83% of interns met performance standards on their first posttest attempt, 14% on the second attempt, and 3% on the third attempt. Interns from both cycles demonstrated significant improvement from baseline to posttest scores (P < .001). Overall, 10.5% of interns performed dangerous actions during assessment (malpositioning, retained guidewire, or carotid dilation). All interns ultimately achieved the passing standard to demonstrate initial competency in the simulation assessment. All participating interns demonstrated simulation-based competency allowing them to place CVCs under supervised clinical care. Dangerous actions, however, were not uncommon. Simulation-based teaching and learning frameworks were a feasible method to promote patient safety through an institutional-wide verification of preliminary procedural competency.

The Rapid and International Expansion of Veterinary Clinical Skills Laboratories: A Survey to Establish Recent Developments

Veterinary clinical skills laboratories are used for teaching a wide range of practical, clinical, and surgical skills on models and simulators. A survey conducted in 2015 identified the role of such facilities in veterinary education in North America and Europe. The current study aimed to capture recent changes using a similar survey with three sections to collect data about the structure of the facility, its uses in teaching and assessment, and the staffing. The survey consisted of multiple choice and free text questions, was administered online using Qualtrics and was disseminated in 2021 via clinical skills networks and Associate Deans. Responses were received from 91 veterinary colleges in 34 countries; 68 had an existing clinical skills laboratory and 23 were planning to open one within 1-2 years. Collated information from the quantitative data described the facility, teaching, assessment, and staffing. Major themes emerged from the qualitative data relating to aspects of the layout, location, integration in the curriculum, contributions to student learning, and the team managing and supporting the facility. Challenges were associated with budgeting, the ongoing need for expansion and leadership of the program. In summary, veterinary clinical skills laboratories are increasingly common around the world and the contributions to student learning and animal welfare were well recognized. The information about existing and planned laboratories and the tips from those managing the facilities provides valuable guidance for anyone intending to open or expand an existing clinical skills laboratory.

A view from the top: A qualitative exploration of top-level health care leaders' perceptions on the implementation of simulation-based education in postgraduate medical education

INTRODUCTION

Although there is substantial evidence supporting the benefits of simulation-based education (SBE), its widespread and effective implementation remains challenging. The aim of this study was to explore the perceptions of top-level health care leaders regarding SBE and the barriers and facilitators that influence its wide implementation as part of the postgraduate surgical curricula in Denmark.

METHODS

We conducted semi-structured interviews with top-level health care leaders who were chosen based on their roles in ensuring high-quality patient care delivery and developing strategies to achieve the goals of the entire health care system. The interview transcripts were translated into English, and a thematic approach was used to code and inductively analyse the data. We used the Consolidated Framework for Implementation Research to identify and understand the determinants to a successful implementation of SBE.

RESULTS

We interviewed 13 participants from different political and administrative levels. We found that the participants had limited knowledge about SBE, which highlighted a disconnection between these leaders and the educational environment. This was further compounded by a lack of effective communication and inadequate information dissemination between simulation centres and higher-level organisations. While participants recognised the benefits of SBE for doctors in training, they expressed concerns about the implementation given the already strained health care system and limited resources. The need for evidence, particularly in the context of patient safety, was emphasised to facilitate SBE implementation. Although participants supported the implementation of SBE, it was unclear who should initiate action.

DISCUSSION

This study highlighted the perspectives of top-level health care leaders regarding SBE and identified the determinant factors for a successful implementation. Effective communication channels are crucial to enhance collaborations and reduce the disconnection between the different health care organisational levels. Strategic implementation processes, including the roles and responsibilities, should be defined and established. These will inform decisions regarding the implementation strategies to effectively integrate SBE into the residency training curricula.

Powerful medical education improves health care quality and return on investment

INTRODUCTION

Powerful medical education (PME) involves the use of new technologies informed by the science of expertise that are embedded in laboratories and organizations that value evidence-based education and support innovation. This contrasts with traditional medical education that relies on a dated apprenticeship model that yields uneven results. PME involves an amalgam of features, conditions and assumptions, and contextual variables that comprise an approach to developing clinical competence grounded in education impact metrics including efficiency and cost-effectiveness.

METHODS

This article is a narrative review based on SANRA criteria and informed by realist review principles. The review addresses the PME model with an emphasis on mastery learning and deliberate practice principles drawn from the new science of expertise. Pub Med, Scopus, and Web of Science search terms include medical education, the science of expertise, mastery learning, translational outcomes, cost effectiveness, and return on investment. Literature coverage is comprehensive with selective citations.

RESULTS

PME is described as an integrated set of twelve features embedded in a group of seven conditions and assumptions and four context variables. PME is illustrated via case examples that demonstrate improved ventilator patient management learning outcomes compared to traditional clinical education and mastery learning of breaking bad news communication skills. Evidence also shows that PME of physicians and other health care providers can have translational, downstream effects on patient care practices, patient outcomes, and return on investment. Several translational health care quality improvements that derive from PME include reduced infections; better communication among physicians, patients, and families; exceptional birth outcomes; more effective patient education; and return on investment.

CONCLUSIONS

The article concludes with challenges to hospitals, health systems, and medical education organizations that are responsible for producing physicians who are expected to deliver safe, effective, and cost-conscious health care.

Simulation-Based Education in US Undergraduate Medical Education: A Descriptive Study

PURPOSE

Simulation-based education (SBE) provides experiential learning, improvement in quality of care, and reduction in errors. In 2011, the Association of American Medical Colleges described adoption of SBE in 68.0% of medical schools and 25.0% of teaching hospitals. We sought to examine current trends of SBE integration in American undergraduate medical education since previous publications.

METHODS

From 2016 to 2019, University of Texas Southwestern Medical Center postgraduate year 1 residents were invited to participate in a survey assessing medical school simulation experience with 26 clinical tasks from three categories: procedural, communication, and other. Deidentified results were analyzed to assess demographics including sex, specialty, residency program type, allopathic versus osteopathic medical school, and medical school region.

RESULTS

Nine hundred sixty-seven of 1047 (92.3%) responses were obtained, representing 139 US medical schools, 91% from allopathic training. Of procedural tasks, most simulated was suturing (n = 848, 89.6%) and least simulated was thoracentesis (n = 737, 80.9%). Of communication tasks, most simulated was taking a history (n = 475, 51.1% reporting simulation >30) and least simulated (never or ≤1) were obtaining a consent (n = 669, 73.2%) and disclosing a medical error (n = 666, 72.4%). Of other tasks, most simulated was chest compressions (n = 898, 96.0%) and least simulated was operating a defibrillator (n = 206, 22.1%). Results were similar regardless of procedural or nonprocedural program. There was no significant difference in SBE exposure between allopathic and osteopathic students ( P = 0.89). Two participants (0.002%) reported no simulation exposure.

CONCLUSIONS

Our study is the first to describe a high prevalence of SBE adoption in medical schools nationwide since the Association of American Medical Colleges' 2011 publication, with overall equal exposure for students regardless of residency type and allopathic or osteopathic medical school. Despite widespread adoption of simulation, opportunities remain to expand SBE use to teach critically important communication skills.

Using a novel virtual-reality simulator to assess performance in lumbar puncture: a validation study

BACKGROUND

A lumbar puncture procedure's success depends on a competent physician minimizing the risk of failing to get a sample and avoiding complications such as post-dural headache. A new virtual-reality simulator might be helpful in deciding when a physician is competent to perform lumbar puncture. We aimed to investigate validity evidence for a simulator-based test in lumbar puncture and establish a pass/fail standard to allow a mastery learning training program.

METHODS

Validity evidence was investigated using Messick's framework by including participants who were novices, intermediates, or experienced in lumbar puncture. Each participant performed two lumbar puncture procedures on the simulator, and fifty-nine predefined simulator metrics were automatically recorded. Cronbach's alpha was used to explore internal consistency reliability. Intergroup comparisons were made using independent sample t-tests with Tukey's correction for multiple comparisons. The learning effect was explored using paired sample t-test analysis, and a pass/fail standard was established using the contrasting groups' method.

RESULTS

73 novices, 18 intermediates, and 19 physicians performed the test resulting in a total of 220 procedures. 25 metrics (42.4%) had good discriminatory ability, and the reliability of these metrics was good, Cronbach's α = 0.81. The experienced physicians were significantly better than the novices (18.3 vs. 13.3, p < 0.001), and the pass/fail standard was established at 16 points. This standard resulted in 22 (30.1%) novices passing (i.e., false positives) and 5 (26.3%) physicians failing (i.e., false negatives).

CONCLUSION

This study provides validity evidence for a simulator-based test of lumbar puncture competence. The test can help ensure basic competence at the end of a simulation-based training program for trainees, i.e., a mastery learning training program.

Necessity of Pretests in Central Venous Catheter Insertion Simulation-Based Mastery Learning: A Randomized Controlled Trial

PURPOSE

Simulation-based mastery learning (SBML) is a rigorous form of competency-based learning. Components of SBML include a pretest, deliberate practice, and a posttest; all learners must meet or exceed a minimum passing standard (MPS) on the posttest before completing training. The authors aimed to explore whether a modified SBML curriculum (without a pretest assessment) was as effective as the standard SBML curriculum (with a pretest assessment).

METHOD

The authors performed a randomized controlled trial of internal medicine residents who participated in an internal jugular central venous catheter insertion SBML curriculum at a tertiary care academic medical center in Chicago, Illinois, from December 2018 through December 2021. Residents were randomly assigned to complete the usual SBML intervention (pretest group) or to complete a modified SBML intervention without a pretest (no pretest group). The authors compared initial posttest performance and training time between groups.

RESULTS

Eighty-nine of 120 eligible residents (74.1%) completed the study: 43 in the pretest group and 46 in the no pretest group. Median (IQR) initial posttest scores were not statistically different between the pretest group (96.6 [93.1-100]) and the no pretest group (96.6 [92.4-100]). However, all 43 residents (100%) in the pretest group reached the MPS at the initial posttest compared with 41 of the 46 (89%) in the no pretest group ( P = .06). Residents in the pretest group required 16.5 hours more faculty and learning time than the no pretest group.

CONCLUSIONS

More residents who completed a pretest reached the MPS at initial posttest. However, incorporating a pretest during the internal jugular central venous catheter SBML curriculum required substantially more learner and faculty time without clear performance benefits.

Development, implementation, and results of a simulation-based hands-on brachytherapy workshop for medical students

PURPOSE

The new Medical Licensing Regulations 2025 (Ärztliche Approbationsordnung, ÄApprO) require the development of competence-oriented teaching formats. In addition, there is a great need for high-quality teaching in the field of radiation oncology, which manifests itself already during medical school. For this reason, we developed a simulation-based, hands-on medical education format to teach competency in performing accelerated partial breast irradiation (APBI) with interstitial multicatheter brachytherapy for early breast cancer. In addition, we designed realistic breast models suitable for teaching both palpation of the female breast and implantation of brachytherapy catheters.

METHODS

From June 2021 to July 2022, 70 medical students took part in the hands-on brachytherapy workshop. After a propaedeutic introduction, the participants simulated the implantation of single-lead catheters under supervision using the silicone-based breast models. Correct catheter placement was subsequently assessed by CT scans. Participants rated their skills before and after the workshop on a six-point Likert scale in a standardized questionnaire.

RESULTS

Participants significantly improved their knowledge-based and practical skills on APBI in all items as assessed by a standardized questionnaire (mean sum score 42.4 before and 16.0 after the course, p < 0.001). The majority of respondents fully agreed that the workshop increased their interest in brachytherapy (mean 1.15, standard deviation [SD] 0.40 on the six-point Likert scale). The silicone-based breast model was found to be suitable for achieving the previously defined learning objectives (1.19, SD 0.47). The learning atmosphere and didactic quality were rated particularly well (mean 1.07, SD 0.26 and 1.13, SD 0.3 on the six-point Likert scale).

CONCLUSION

The simulation-based medical education course for multicatheter brachytherapy can improve self-assessed technical competence. Residency programs should provide resources for this essential component of radiation oncology. This course is exemplary for the development of innovative practical and competence-based teaching formats to meet the current reforms in medical education.

Experiences of simulation-based learning among undergraduate nursing students: A systematic review and meta-synthesis

OBJECTIVES

We aimed to examine the experiences of simulation-based learning (SBL) among undergraduate nursing students.

DESIGN

We conducted a systematic review and meta-synthesis on the experiences among undergraduate nursing studies who received SBL.

DATA SOURCES

We searched through six databases namely PubMed, CINAHL, Scopus, Web of Science, Embase, ProQuest, for qualitative studies published from January 2011 to January 2022.

REVIEW METHODS

A meta-synthesis was conducted according to the three-stage framework outlined by Thomas and Harden (2008). Critical appraisal was performed using the Critical Appraisal Skills Program (CASP) checklist. A standardised data extraction form was developed with reference from JBI Qualitative Assessment and Review Instrument Data Extraction Tools for Qualitative Research (JBI-QARI) checklist for data extraction.

RESULTS

Fifteen studies were included, and four themes emerged from the synthesis: (1) acquiring knowledge and skills through SBL; (2) positive experiences of using high-fidelity simulation (HFS) and virtual reality simulation (VRS) methods; (3) challenges encountered while using SBL methods and (4) drawing parallels between simulation and real clinical settings.

CONCLUSION

SBL allowed undergraduate nursing students to gain knowledge, acquire skills and have a positive SBL experiences. However, the provision of innovative strategies and resources for nursing students to overcome SBL-based challenges are still needed.

Procedural Curriculum to Verify Intern Competence Prior to Patient Care

INTRODUCTION

Emergency medicine (EM) programs train residents to perform clinical procedures with known iatrogenic risks. Currently, there is no established framework for graduating medical students to demonstrate procedural competency prior to matriculating into residency. Mastery-based learning has demonstrated improved patient-safety outcomes. Incorporation of this framework allows learners to demonstrate procedural competency to a predetermined standard in the simulation laboratory prior to performing invasive procedures on patients in the clinical setting. This study describes the creation and implementation of a competency-based procedural curriculum for first-year EM residents using simulation to prepare learners for supervised participation in procedures during patient care.

METHODS

Checklists were developed internally for five high-risk procedures (central venous line placement, endotracheal intubation, lumbar puncture, paracentesis, chest tube placement). Performance standards were developed using Mastery-Angoff methods. Minimum passing scores were determined for each procedure. Over a two-year period, 38 residents underwent baseline assessment, deliberate practice, and post-testing against the passing standard score to demonstrate procedural competency in the simulation laboratory during intern orientation.

RESULTS

We found that 37% of residents required more than one attempt to achieve the minimum passing score on some procedures, however, all residents ultimately met the competency standard on all five high-risk procedures in simulation. One critical incident of central venous catheter guideline retention was identified in the simulation laboratory during the second year of implementation.

CONCLUSION

All incoming first-year EM residents demonstrated procedural competence on five different procedures using a mastery-based educational framework. A competency-based EM curriculum allowed for demonstration of procedural competence prior to resident participation in supervised clinical patient care.

Robotic training for medical students: feasibility of a pilot simulation curriculum

While robotic procedures are growing rapidly, medical students have a limited role in robotic surgeries. Curricula are needed to enhance engagement. We examined feasibility of augmenting Intuitive Surgical (IS) robotic training for medical students. As a pilot, 18 senior students accepted an invitation to a simulation course with a daVinci robot trainer. Course teaching objectives included introducing robotic features, functionalities, and roles. A 1-h online module from the IS learning platform and a 4-h in-person session comprised the course. The in-person session included an overview of the robot by an IS trainer (1.5 h), skills practice at console (1.5 h), and a simulation exercise focused on the bedside assist role (1 h). Feasibility included assessing implementation and acceptability using a post-session survey and focus group (FG). Survey responses were compiled. FG transcripts were analyzed using inductive thematic analysis techniques. Fourteen students participated. Implementation was successful as interested students signed up and completed each of the course components. Regarding acceptability, students reported the training valuable and recommended it as preparation for robotic cases during core clerkships and sub-internships. In addition, FGs revealed 4 themes: (1) perceived expectations of students in the OR; (2) OR vs. outside-OR learning; (3) simulation of stress; and (4) opportunities to improve the simulation component. To increase preparation for the robotic OR and shift robotic training earlier in the surgical education continuum, educators should consider hands-on simulation for medical students. We demonstrate feasibility although logistics may limit scalability for large numbers of students.

Teaching, coaching, or debriefing With Good Judgment: a roadmap for implementing "With Good Judgment" across the SimZones

Simulation-based learning occurs in multiple contexts, and one teaching style cannot adequately cover the needs at each learning level. For example, reflective debriefing, often used following a complex simulation case, is not what is needed when learning new skills. When to use which facilitation style is a question that educators often overlook or struggle to determine. SimZones is a framework used to clarify the multiple contexts in simulation. This framework, combined with elements of Debriefing With Good Judgment, can help educators match the appropriate facilitation style with learner needs and learning context. We have distilled the core elements of the "with good judgment" approach to debriefing and applied them to the SimZones framework to guide educators with (1) what type of learning can be expected with each learning context, (2) what behaviors and activities can be expected of the learners in each learning context, (3) what instructional strategies are most effectively used at each stage, and (4) what are the implications for the teacher-learner relationship.

Improving performance and self‐efficacy of novice nurses using hybrid simulation‐based mastery learning

AIM

Acute chest pain is a commonly encountered symptom in hospital medical/surgical units; however, almost half of nurses in their second year of clinical experience in our facility have reported struggling to care for acute chest pain patients. We developed, implemented, and examined the effectiveness of a simulation-based, mastery learning clinical nursing educational program to improve self-efficacy and performance in caring for patients with acute chest pain.

METHODS

The study adopted a single-site, single-cohort design using simulation-based performance assessment and self-efficacy surveys on a convenience sample of 37 second-year clinical nurse participants in multi-stage hybrid mastery learning educational intervention using asynchronous e-learning, and hands-on simulation training and assessment with feedback on caring for chest pain patients. Performance assessments and self-efficacy surveys were administered pre-, post-, and 5 months post-intervention.

RESULTS

Clinical performance on the post- and 5 months follow-up assessments were significantly higher than those for the pre-test (P < .0001). The self-efficacy scores for the post- and the 5 months follow-up assessments were significantly higher than the pre-course scores (P < .0001). Participants' self-efficacy perceptions were positively correlated with their performances at 5 months post-intervention.

CONCLUSION

Performance and self-efficacy of novice nurses in caring for acute chest pain patients improved significantly with the multi-stage hybrid mastery learning educational intervention, with improvements retained 5 months post-intervention. The results suggest the applicability of simulation-based mastery learning in a clinical setting for novice nurses to attain specific skills, and raise their self-perception of competence to care for patients in acute settings.

Hands-On Training in a Digital World: A Novel Simulation-Based Virtual Training Program for Placement and Removal of the Subdermal Contraceptive Implant

OBJECTIVE

The COVID-19 pandemic necessitated a shift from traditional in-person instruction for learning new technical skills to virtual delivery of medical education training. The objectives of this study were to develop and evaluate a virtual simulation-based training program for Canadian health care professionals (HCPs) on the insertion, localization, and removal of the etonogestrel subdermal contraceptive implant.

METHODS

A scientific committee of Canadian family planning experts developed a 2-part virtual training program during the COVID-19 pandemic. Core educational content (part 1) was provided in an asynchronous, self-directed, online format. Part 2 consisted of synchronous, simulation-based training using web conferencing. The HCPs were provided with model arms and placebo applicators; the trainers demonstrated implant insertion and removal techniques, and trainees received individual feedback. All trainees were asked to complete an online evaluation upon completion of the program.

RESULTS

Between September 22, 2020, and December 31, 2021, 83 trainers conducted 565 virtual training sessions. A total of 3162 HCPs completed part 1 of the training program, of whom 2740 had completed part 2 by December 31, 2021. Participants reported high levels of satisfaction with virtual simulation-based training; 96.5% of respondents (1570/1627) agreed that the virtual format was effective. Additional training prior to inserting the implant in clinical practice was requested by 4.5% of respondents (75/1671).

CONCLUSION

Virtual simulation-based learning provides effective education and technique training for etonogestrel implant insertion and removal. Online training for implant use can be scaled, as needed, to reach professionals in remote or underserved locations. This virtual training approach may be appropriate for other technical or minor surgical procedures.

Can virtual-reality simulation ensure transthoracic echocardiography skills before trainees examine patients?

OBJECTIVES

This study aimed to develop and gather the validity evidence for a standardised simulation-based skills test in transthoracic echocardiography and to establish a credible pass/fail score.

METHODS

Experts developed a virtual-reality simulator test in cardiology, medical education and simulation-based education. Thirty-six physicians with different experiences in transthoracic echocardiography completed the test at Odense University Hospital, Denmark. The performances of novice, intermediate and experienced participants were compared using the Bonferroni post hoc test. Cronbach's alpha was used to determine the internal consistency reliability of the test. The consistency of performance was analysed using the intraclass correlation coefficient. A pass/fail score was established using the contrasting groups' standard-setting method.

RESULTS

We developed a test with high consistent reliability (Alpha = .81), 95% CI [.69, .89]. In both cases, the performers' level was consistent, fitting others at the same level of experience (intraclass correlation r₍₃₅₎=.81, p<.001). A pass/fail score of 48/50 points was established based on the mean test score of novice and experienced physicians.

CONCLUSIONS

We developed a standardised virtual-reality simulation-based test of echocardiography skills with the ability to distinguish between participants with different levels of transthoracic echocardiography experience. This test could direct a mastery learning training program where trainees practise until they reach the pre-defined level and secure a higher level of competency to ensure quality and safety for patients.

ComPsych communication skills training: Applicability of simulated patients in psychiatry communication skills training

OBJECTIVE

Communication skills training (CST) programs within postgraduate psychiatry training are rare. ComPsych CST utilises simulated patients (SPs) for trainees to practice communication skills for discussing severe mental illness with patients and their families/carers. This study examined the applicability of using SPs in a psychiatry-specific CST.

METHODS

A total of 41 postgraduate psychiatry trainees attended at least one of four modules of training in their cohort year and completed a questionnaire after each module presenting eight questions rating the use of SPs and ratings of course deliverables.

RESULTS

Overall, trainees rated contact with SPs very highly across all modules, with a mean rating of 9.11 out of 10 (SD = 0.97). Trainees agreed that SPs appeared authentic, that their reactions showed they listened to the trainee.

CONCLUSIONS

Trainees valued training with SPs, providing evidence that using SPs for psychiatry-specific CST is feasible. Despite subjectivity, this is valuable to course providers as it highlights benefits perceived by trainees to be useful and provides further evidence for the program's feasibility and utility.

The effectiveness of improving healthcare teams' human factor skills using simulation-based training: a systematic review

Background

Simulation-based training used to train healthcare teams’ skills and improve clinical practice has evolved in recent decades. While it is evident that technical skills training is beneficial, the potential of human factor training has not been described to the same extent. Research on human factor training has been limited to marginal and acute care scenarios and often to validate instruments. This systematic review aimed to investigate the effectiveness of simulation-based training in improving in-hospital qualified healthcare teams’ human factor skills.

Method

A review protocol outlining the study was registered in PROSPERO. Using the PRISMA guidelines, the systematic search was conducted on September 28th, 2021, in eight major scientific databases. Three independent reviewers assessed title and abstract screening; full texts were evaluated by one reviewer. Content analysis was used to evaluate the evidence from the included studies.

Results

The search yielded 19,767 studies, of which 72 were included. The included studies were published between 2004 and 2021 and covered research from seven different in-hospital medical specialisms. Studies applied a wide range of assessment tools, which made it challenging to compare the effectiveness of human factor skills training across studies. The content analysis identified evidence for the effectiveness. Four recurring themes were identified: (1) Training human factor skills in qualified healthcare teams; (2) assessment of human factor skills; (3) combined teaching methods, and (4) retention and transfer of human factor skills. Unfortunately, the human factor skills assessments are variable in the literature, affecting the power of the result.

Conclusion

Simulation-based training is a successful learning tool to improve qualified healthcare teams’ human factor skills. Human factor skills are not innate and appear to be trainable similar to technical skills, based on the findings of this review. Moreover, research on retention and transfer is insufficient. Further, research on the retention and transfer of human factor skills from simulation-based training to clinical practice is essential to gain knowledge of the effect on patient safety.

Supplementary Information

The online version contains supplementary material available at 10.1186/s41077-022-00207-2.

Accreditation of Simulation Fellowships and Training Programs: More Checkboxes or Elevating the Field?

SUMMARY STATEMENT

As the field of healthcare simulation matures, formal accreditation for simulation fellowships and training programs has become increasingly available and touted as a solution to standardize the education of those specializing in healthcare simulation. Some simulation experts hold opposing views regarding the potential value of simulation fellowship program accreditation. We report on the proceedings of a spirited debate at the 20th International Meeting on Simulation in Healthcare in January 2020. Pro arguments view accreditation as the logical evolution of a maturing profession: improving training quality through standard setting, providing external validation for individual programs, and enhancing the program's return on investment. Con arguments view accreditation as an incompletely formulated construct; burdensome to the "financially strapped" fellowship director, misaligned with simulation fellows' needs and expectations, and confusing to administrators mistakenly equating accreditation with credentialing. In addition, opponents of accreditation postulate that incorporating curricular standards, practice guidelines, and strategies derived and implemented without rigor, supporting evidence and universal consensus is premature. This narrative review of our debate compares and contrasts contemporary perspectives on simulation fellowship program accreditation, concluding with formal recommendations for learners, administrators, sponsors, and accrediting bodies.

Medical Student Ultrasound-Guided Intravenous Catheter Education: A Randomized Controlled Trial of Overtraining in a Simulation-Based Mastery Learning Setting

INTRODUCTION

Simulation-based mastery learning (SBML) improves skill transfer to humans. However, limited data exist to support the practice of performing additional training attempts once mastery has been achieved. We explored whether implementing this concept in an SBML model improves skill transfer in ultrasound-guided peripheral intravenous (USGPIV) catheter placement from simulator to humans.

METHODS

Forty-eight first- and second-year medical students underwent SBML for USGPIV placement to mastery standards on a simulator once to achieve minimum passing standard based on a 19-item checklist. Next, they completed either 0, 4, or 8 additional mastery level simulated performances in nonconsecutive order before attempting USGPIV placement on a human. An unplanned post hoc retention phase occurred 6 months later where participants' USGPIV skills were reassessed on a second human volunteer using the same checklist.

RESULTS

In this sample, the success rate among the 3 training cohorts did not improve as the number of additional training sessions increased (exact P = 0.60) and were comparable among the 3 cohorts (exact P = 0.82). The overall checklist performance was also comparable among the 3 cohorts (exact P = 0.57). In an unplanned (small) sample of returning participants, the USGPIV retention rate exceeded 80% for those originally assigned to 4 or 8 sessions, whereas the retention rate fell less than 50% for those originally assigned to control. Among these returning participants, the overall success rate was 58% on the initial assessment and was 68% on retention.

CONCLUSIONS

In this sample of novices, we were unable to show that up to 8 additional mastery level performances of USGPIV placement on a simulator following SBML training improve skill transfer for both USGPIV placement success and total checklist performance.

Rethinking Our Annual Congress-Meeting the Needs of Specialist Physicians by Partnering With Provincial Simulation Centers

Canada's maintenance of certification programs for physicians has evolved to emphasize assessment activities. Our organization recognized the importance of offering more practice assessment opportunities to our members to enhance their practice and help them comply with a regulation from our provincial professional body related to ongoing continuing education. This led us to rethink our annual congress and enrich the program with a curriculum of interdisciplinary simulation sessions tailored to meet the needs of a broad audience of specialists. Our challenges are similar to those of many national specialty societies having limited access to simulation facilities, instructors, and simulation teams that can cover the breadth and scope of perceived and unperceived simulation needs for their specialty. Our innovative solution was to partner with local experts to develop 22 simulation sessions over the past three years. The response was very positive, drawing 867 participants. Over 95% of participants either agreed or strongly agreed that their simulation session (1) met their learning objectives, (2) was relevant for their practice, and (3) encouraged them to modify their practice. Narrative comments from a survey sent to the 2018 participants four months after their activity indicated several self-reported changes in their practice or patient outcomes. We were able to centralize offers from organizations that had previously worked in silo to develop simulation sessions meeting the needs of our members. Proposing simulation sessions allowed our organization to establish long-term partnerships and to expend our "educational toolbox" to address skill gaps not usually addressed during annual meetings.

Role of 3D printing technology in paediatric teaching and training: a systematic review

BACKGROUND

In the UK, undergraduate paediatric training is brief, resulting in trainees with a lower paediatric knowledge base compared with other aspects of medicine. With congenital conditions being successfully treated at childhood, adult clinicians encounter and will need to understand these complex pathologies. Patient-specific 3D printed (3DP) models have been used in clinical training, especially for rarer, complex conditions. We perform a systematic review to evaluate the evidence base in using 3DP models to train paediatricians, surgeons, medical students and nurses.

METHODS

Online databases PubMed, Web of Science and Embase were searched between January 2010 and April 2020 using search terms relevant to "paediatrics", "education", "training" and "3D printing". Participants were medical students, postgraduate trainees or clinical staff. Comparative studies (patient-specific 3DP models vs traditional teaching methods) and non-comparative studies were included. Outcomes gauged objective and subjective measures: test scores, time taken to complete tasks, self-reported confidence and personal preferences on 3DP models. If reported, the cost of and time taken to produce the models were noted.

RESULTS

From 587 results, 15 studies fit the criteria of the review protocol, with 5/15 being randomised controlled studies and 10/15 focussing on cardiovascular conditions. Participants using 3DP models demonstrated improved test scores and faster times to complete procedures and identify anatomical landmarks compared with traditional teaching methods (2D diagrams, lectures, videos and supervised clinical events). User feedback was positive, reporting greater user self-confidence in understanding concepts with users wishing for integrated use of 3DP in regular teaching. Four studies reported the costs and times of production, which varied depending on model complexity and printer. 3DP models were cheaper than 'off-the-shelf' models available on the market and had the benefit of using real-world pathologies. These mostly non-randomised and single-centred studies did not address bias or report long-term or clinically translatable outcomes.

CONCLUSIONS

3DP models were associated with greater user satisfaction and good short-term educational outcomes, with low-quality evidence. Multicentred, randomised studies with long-term follow-up and clinically assessed outcomes are needed to fully assess their benefits in this setting.

PROSPERO REGISTRATION NUMBER

CRD42020179656.

The Equivalency of the Binocular Indirect Ophthalmoscope Simulator to Peer Practice: A Pilot Study

SIGNIFICANCE

Optometric educators are constantly looking for learning and teaching approaches to improve clinical skills training. In addition, the COVID-19 pandemic has made educators scrutinize the time allocated to face-to-face teaching and practice. Simulation learning is an option, but its use must first be evaluated against traditional learning methods.

PURPOSE

The purpose of this study was to compare the training of binocular indirect ophthalmoscopy skills achieved by students and optometrists through deliberate practice on the Eyesi Indirect Ophthalmoscope simulator with deliberate practice using a peer.

METHODS

Students and optometrists were randomly allocated to practice on either the simulator or a peer. Binocular indirect ophthalmoscopy performance was assessed using a peer and the simulator as the patients at different time points. Knowledge and confidence were examined before and following all practice sessions using a quiz and survey.

RESULTS

Significant improvements in binocular indirect ophthalmoscopy performance using either a peer or the simulator as the patient for assessment were seen after 8 hours of student practice (P < .001) and after a half-hour practice time for optometrists (P < .001). There was no significant difference in performance overall between those practicing on a simulator and those practicing on a peer (P > .05). Confidence in ability to perform the technique was lower for students who had practiced on the simulator.

CONCLUSIONS

The simulator has similar efficacy to peer practice for teaching binocular indirect ophthalmoscopy to students and maintenance of this clinical skill for optometrists. Simulation does not replace the need for peer practice but may be a useful adjunct reducing the face-to-face hours required. These findings present a need for further research regarding diverse applications of the Eyesi Indirect Ophthalmoscope simulator in the curriculum for training optometry students and as a continuing professional development offering for optometrists, given the short exposure duration required to observe a significant improvement in skill.

Self-directed video versus instructor-based neonatal resuscitation training: a randomized controlled blinded non-inferiority multicenter international study

OBJECTIVE

To compare the efficacy of video-assisted self-directed neonatal resuscitation skills course with video-assisted facilitator-led course.

METHODS

This multicenter, randomized, blinded, non-inferiority-controlled trial compared two methods of teaching basic neonatal resuscitation skills using mask ventilation. Groups of novice providers watched an instructional video. One group received instructor facilitation (Ins-Video). The other group did not (Self-Video). An Objective Structured Clinical Exam (OSCE) measured skills performance, and a written test gauged knowledge.

RESULTS

One hundred and thirty-four students completed the study. Sixty-three of 68 in the Self-Video Group (92.6%) and 59 of 66 in the Ins-Video Group (89.4%) achieved post-training competency in positive pressure ventilation (primary outcome). OSCE passing rates were low in both groups. Knowledge survey scores were comparable between groups and non-inferior.

CONCLUSIONS

Video self-instruction taught novice providers positive pressure ventilation skills and theoretical knowledge, but it was insufficient for mastery of basic neonatal resuscitation in simulation environment.

Collaborative clinical simulation in cardiologic emergency scenarios for medical students. An exploratory study on model applicability and assessment instruments

Aims: This paper evaluates the feasibility of piloting the collaborative clinical simulation (CCS) model and its assessment instruments applicability for measuring interpersonal, collaborative, and clinical competencies in cardiologic emergency scenarios for medical students. The CCS model is a structured learning model for the acquisition and assessment of clinical competencies through small groups working collaboratively to design and perform in simulated environments supported by technology. Methods: Fifty-five students were allocated in five sessions (one weekly session) conducted with the CCS model within the course Cardiovascular Diseases. The applied practice aimed at the diagnosis and treatment of tachyarrhythmias in a simulated emergency department. In addition to the theoretical classes four weeks before the simulation sessions, students were sent a study guide that summarized the Guide to the European Society of Cardiology. For each simulation session, one clinical simulation instructor, one cardiologist teacher, and the principal investigator participated. Students were divided into three groups (3-5 students) for each-session. They designed, performed, role-played, and debriefed three different diagnoses. Three instruments to assess each group's performance were applied: peer assessment used by groups, performance assessment, created and applied by the cardiologist teacher, and individual satisfaction questionnaire for students. Results: The applicability of the CCS model was satisfactory for both students and teachers. The assessment instruments' internal reliability was good, as was internal consistency with a Cronbach Alpha of 0.7, 0.4, and 0.8 for each section (Interpersonal, Clinical, and Collaborative competencies, respectively). The performance group's evaluation was 0.8 for the two competencies assessed (Tachyarrhythmia and Electrical Cardioversion) and 0.8 for the satisfaction questionnaire's reliability. Conclusions: The CCS model for teaching emergency tachyarrhythmias to medical students was applicable and well accepted. The internal reliability of the assessment instruments was considered satisfactory by measuring satisfaction and performance in the exploratory study.

Educating Anesthesiologists During the Coronavirus Disease 2019 Pandemic and Beyond

The coronavirus disease 2019 (COVID-19) pandemic has altered approaches to anesthesiology education by shifting educational paradigms. This vision article discusses pre-COVID-19 educational methodologies and best evidence, adaptations required under COVID-19, and evidence for these modifications, and suggests future directions for anesthesiology education. Learning management systems provide structure to online learning. They have been increasingly utilized to improve access to didactic materials asynchronously. Despite some historic reservations, the pandemic has necessitated a rapid uptake across programs. Commercially available systems offer a wide range of peer-reviewed curricular options. The flipped classroom promotes learning foundational knowledge before teaching sessions with a focus on application during structured didactics. There is growing evidence that this approach is preferred by learners and may increase knowledge gain. The flipped classroom works well with learning management systems to disseminate focused preclass work. Care must be taken to keep virtual sessions interactive. Simulation, already used in anesthesiology, has been critical in preparation for the care of COVID-19 patients. Multidisciplinary, in situ simulations allow for rapid dissemination of new team workflows. Physical distancing and reduced availability of providers have required more sessions. Early pandemic decreases in operating volumes have allowed for this; future planning will have to incorporate smaller groups, sanitizing of equipment, and attention to use of personal protective equipment. Effective technical skills training requires instruction to mastery levels, use of deliberate practice, and high-quality feedback. Reduced sizes of skill-training workshops and approaches for feedback that are not in-person will be required. Mock oral and objective structured clinical examination (OSCE) allow for training and assessment of competencies often not addressed otherwise. They provide formative and summative data and objective measurements of Accreditation Council for Graduate Medical Education (ACGME) milestones. They also allow for preparation for the American Board of Anesthesiology (ABA) APPLIED examination. Adaptations to teleconferencing or videoconferencing can allow for continued use. Benefits of teaching in this new era include enhanced availability of asynchronous learning and opportunities to apply universal, expert-driven curricula. Burdens include decreased social interactions and potential need for an increased amount of smaller, live sessions. Acquiring learning management systems and holding more frequent simulation and skills sessions with fewer learners may increase cost. With the increasing dependency on multimedia and technology support for teaching and learning, one important focus of educational research is on the development and evaluation of strategies that reduce extraneous processing and manage essential and generative processing in virtual learning environments. Collaboration to identify and implement best practices has the potential to improve education for all learners.

TeamSTEPPS online simulation: expanding access to teamwork training for medical students

Background

The Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) programme is an evidence-based approach to teamwork training. In-person education is not always feasible for medical student education. The aim of this study was to evaluate the impact of online, interactive TeamSTEPPS simulation versus an in-person simulation on medical students’ TeamSTEPPS knowledge and attitudes.

Methods

Fourth-year medical students self-selected into an in-person or online training designed to teach and evaluate teamwork skills. In-person participants received didactic sessions, team-based medical simulations and facilitated debriefing sessions. The online group received an equivalent online didactic session and participated in an interactive software-based simulation with immediate, personalised performance-based feedback and scripted debriefing. Both trainings used three iterations of a case of septic shock, each with increasing medical complexity. Participants completed a demographic survey, a preintervention/postintervention TeamSTEPPS Benchmarks test and a retrospective preintervention/postintervention TeamSTEPPS teamwork attitudes questionnaire. Data were analysed using descriptive statistics and repeated measures analysis of variance.

Results

Thirty-one students (18 in-person, 13 online) completed preintervention/postintervention surveys, tests and questionnaires. Gender, age and exposure to interprofessional education, teamwork training and games were similar between groups. There were no statistical differences in preintervention knowledge or teamwork attitude scores between in-person and online groups. Postintervention knowledge scores increased significantly from baseline (+2.0% p=0.047), and these gains did not differ significantly based on whether participants received in-person versus online training (+1.5% vs +2.9%; p=0.49). Teamwork attitudes scores also showed a statistically significant increase with training (+0.9, p<0.01) with no difference in the effect of training by group (+0.8 vs +1.0; p=0.64).

Conclusions

Graduating medical students who received in-person and online teamwork training showed similar increases in TeamSTEPPS knowledge and attitudes. Online simulations may be used to teach and reinforce team communication skills when in-person, interprofessional simulations are not feasible.

Simulation-based curriculum development: lessons learnt in Global Health education

BACKGROUND

Simulation based medical education (SBME) allows learners to acquire clinical skills without exposing patients to unnecessary risk. This is especially applicable to Emergency Medicine training programs where residents are expected to demonstrate proficiency in the management of time critical, low frequency, and highly-morbidity conditions. This study aims to describe the process through which a SBME curriculum was created, in a limited simulation resource setting at a 4-year Emergency Medicine (EM) residency program at the American University of Beirut Medical Center.

METHODS

A case-based pilot simulation curriculum was developed following Kern's 6 step approach to curriculum design. The curricular objectives were identified through an anonymous survey of the program's residents and faculty. Curriculum outcomes were assessed, and the curriculum was revised to address curricular barriers. Evaluations of the revised curriculum were collected during the simulation sessions and through a whole revised curriculum evaluation at the end of the first year of its implementation.

RESULTS

14/20 residents (70%) and 8/8 faculty (100%) completed the needs assessment from which objectives for the pilot curriculum were developed and implemented through 6 2-h sessions over a 1-year period. Objectives were not met and identified barriers included cost, scheduling, resources, and limited faculty time. The revised curriculum addressed these barriers and 24 40-min sessions were successfully conducted during the following year. The sessions took place 3 at a time, in 2-h slots, using the same scenario to meet the objectives of the different learners' levels. 91/91 evaluations were collected from participants with overall positive results. The main differences between the pilot and the revised curricula included: a better understanding of the simulation center resources and faculty's capabilities.

CONCLUSION

Simulation-based education is feasible even with limited-resources. However, understanding the resources available, and advocating for protected educator time are essential to implementing a successful EM simulation curriculum.

Simulation-based mastery improves nursing skills in BSc nursing students: a quasi-experimental study

BACKGROUND

Clinical education is an essential part of nursing education. Selected clinical teaching methods influence the quality of education. Simulation-based mastery learning has been used to improve clinical skills among nursing students and may provide a novel way to enhance nursing skills. This study aimed to assess the effect of simulation-based mastery learning on the clinical skills of undergraduate nursing students from 2017 to 2019.

METHODS

This quasi-experimental study was conducted with two groups (the control and intervention). A hundred and five students were selected by random convenience sampling, and written consent was obtained. The intervention group participated in a simulation-based mastery learning intervention, and the control group received no intervention except for traditional training. The students of both groups completed the demographic information questionnaire and the checklist before and after the intervention. The results were analyzed by SPSS version 21 and descriptive and inferential statistics.

RESULTS

The results showed no significant differences between the two groups before the intervention (p> 0.05). In addition, students' performance in the intervention and control groups improved significantly at the post-test compared with the baseline (p< 0.05). Furthermore, the Cohen test implied that the simulation-based mastery model used by the intervention group was significantly more effective than the traditional training used by the control.

CONCLUSION

These findings showed that mastery learning was more effective in improving clinical skills in undergraduate nursing students. The results suggest that other nursing and health programs can be developed by implementing a mastery-based learning model.

Operating Room Times For Teaching and Nonteaching Cases are Converging: Less Time for Learning?

OBJECTIVE

To compare differences in operating room (OR) times between teaching and nonteaching cases across calendar years. We hypothesize that time devoted to intraoperative resident education is decreasing, therefore, OR times for teaching and nonteaching cases will be converging.

BACKGROUND

Teaching cases take longer than similar nonteaching cases, in part due to intraoperative resident education. Pressures to improve OR efficiency and patient safety may threaten resident education and leave less time for intraoperative learning; however, the magnitude of impact is unknown.

SETTING/PARTICIPANTS

National Surgical Quality Improvement Program (NSQIP) deidentified national databases from 2006 to 2012, queried for 30 most common General surgery procedures and case teaching status (i.e., teaching vs. nonteaching cases).

DESIGN

The NSQIP database was retrospectively reviewed to identify the 30 most common General Surgery procedures. Teaching cases included all operations in which a resident participated. Multivariable regression analyses were constructed to determine the impact of resident involvement on OR times, controlling for year, resident participation, procedure, and patient demographics and comorbidities. Difference-in-difference analysis was performed to assess OR time differences between teaching and nonteaching cases across calendar years and within subpopulations.

RESULTS

A total of 693,223 cases met inclusion criteria. Average overall OR times were 98.89 minutes (teaching) vs. 74.22 minutes (nonteaching), with a difference of 24.67 minutes (95% confidence interval [CI] 24.34-24.99 minutes, p < 0.001). In multivariable analyses, the difference between teaching and nonteaching cases was 21.94 minutes (95% CI = 21.11-22.76) in 2006 and 13.95 minutes (95% CI = 10.62-17.28) in 2012, with a difference-in-difference of 7.99 minutes per case. A similar trend was observed across individual PGYs and several individual procedures.

CONCLUSIONS

OR times for teaching and nonteaching cases converged by approximately 8 minutes per general surgery procedure during the 7-year study period, representing a 36% reduction in the difference between groups. We must seek to better understand the source of this convergence, and in doing so ensure to preserve and enhance the intraoperative learning experience of surgical trainees.

Simulation Education for Preterm Infant Delivery Room Management at Community Hospitals

BACKGROUND

Preterm infants are at risk for hypothermia in the delivery room (DR). Hypothermia and community hospital delivery are factors associated with increased morbidities and mortality in preterm infants. Community hospital providers have less experience with preterm deliveries and thermoregulation methods in the DR.

METHODS

Community hospital DR providers in Indiana completed a cognitive test evaluating preterm infant DR management. A simulation-based team assessment was evaluated by using a scoring tool. After debriefing, the simulation performance was repeated and rescored. Afterward, providers completed a cognitive knowledge posttest. Eleven to eighteen months later, sites were revisited with cognitive knowledge and team simulated scenarios reassessed.

RESULTS

Twenty-five community hospitals with >400 multidisciplinary providers participated in the initial study visit. Average pre- and posttest scores were 48.8% and 94.1% respectively (P value <.001). Participants performed an average of 22.8 of 36 scoring items during a simulated preterm delivery including 4.4 of 8 thermoregulation related actions. Performance immediately improved in a statistically significant manner during the repeat scenario. When sites were revisited, participants performed an average of 26.7 of 36 scoring items including 6.1 of 8 thermoregulation actions during a simulated preterm delivery a statistically significant (P <.001) improvement from the initial visit.

CONCLUSIONS

Simulation education regarding preterm infant DR management improved community providers immediate knowledge and skills and also follow-up performance at ∼1 year. In simulation, providers performed thermoregulation maneuvers more often and efficiently, critical to resuscitation because delays in thermoregulation can significantly adversely affect outcomes.

Recent Advances in Simulation for Pediatric Critical Care Medicine

PURPOSE OF REVIEW

This review highlights the emerging fields of simulation research by tying innovation into principles of learning and process improvement.

RECENT FINDINGS

Advances have been made in both educational simulation and simulation for quality improvement, allowing this versatile modality to be more broadly applied to healthcare and systems.

SUMMARY

Simulation in pediatric critical care medicine continues to evolve. Although the majority of simulation is focused on learner education, emerging research has broadened to focus on patient- and system-centered outcomes, leading to improvement in the quality of care delivered in the ICU.

Creation and Implementation of a Mastery Learning Curriculum for Emergency Department Thoracotomy

Introduction

Emergency department thoracotomy (EDT) is a lifesaving procedure within the scope of practice of emergency physicians. Because EDT is infrequently performed, emergency medicine (EM) residents lack opportunities to develop procedural competency. There is no current mastery learning curriculum for residents to learn EDT. The purpose of this study was to develop and implement a simulation-based mastery learning curriculum to teach and assess EM residents’ performance of the EDT.

Methods

We developed an EDT curriculum using a mastery learning framework. The minimum passing standard (MPS) for a previously developed 22-item checklist was determined using the Mastery Angoff approach. EM residents at a four-year academic EM residency program underwent baseline testing in performing an EDT on a simulation trainer. Performance was scored by two raters using the checklist. Learners then participated in a novel mastery learning EDT curriculum that included an educational video, hands-on instruction, and deliberate practice. After a three-month period, residents then completed initial post testing. Residents who did not meet the minimum passing standard after post testing participated in additional deliberate practice until mastery was obtained. Baseline and post-test scores, and time to completion of the procedure were compared with paired t-tests.

Results

Of 56 eligible EM residents, 54 completed baseline testing. Fifty-two residents completed post-testing until mastery was reached. The minimum passing standard was 91.1%, (21/22 items correct on the checklist). No participants met the MPS at the baseline assessment. After completion of the curriculum, all residents subsequently reached the MPS, with deliberate practice sessions not exceeding 40 minutes. Scores from baseline testing to post-testing significantly improved across all postgraduate years from a mean score of 10.2/22 to 21.4/22 (p <0.001). Mean time to complete the procedure improved from baseline testing (6 minutes [min] and 21 seconds [sec], interquartile range [IQR] = 4 min 54 sec - 7 min 51 sec) to post-testing (5 min 19 seconds, interquartile range 4 min 17sec - 6 min 15 sec; p = 0.001).

Conclusion

This simulation-based mastery learning curriculum resulted in all residents performing an EDT at a level that met or exceeded the MPS with an overall decrease in time needed to perform the procedure.

[Overview on research in the field of medical education]

Medical education is a relatively young field of research that has been characterized by many innovations over the last 20 years. The main topics were the preparation of students for their medical work and imparting of profound knowledge about scientific contexts in medicine. For a long time, the main focus was on learning the necessary knowledge and skills; however, this is no longer considered sufficient. In order to carry out the work as a physician well, other qualities are also necessary, such as communication, collaboration, professionalism, science and reflectivity. Worldwide these aspects are being increasingly integrated into the learning objectives of medical schools and residency programs. The structure of medical studies at many universities has also changed. So-called vertical integration strengthens the integration of theoretical preclinical training and the clinical phase. This means that in the preclinical phase training is more practice-oriented and in the clinical phase a more structured form of training takes place. In the first years of study, students are prepared for the clinical phase by discussing patient cases and practicing skills in simulation centers. In addition, the clinical working environment is increasingly used as a learning environment. Developments have also been made regarding examinations: in addition to knowledge and skills, students and residents are now also assessed regarding performance in practice. Using more realistic assessments, e.g., Objective Structured Clinical Examinations and Workplace-based Assessments, students are evaluated more on their actual performance in practice. By means of the Entrustable Professional Activities method, students are gradually given more responsibility in order to prepare them for their future tasks.

Procedure Training Workshop for Internal Medicine Residents that Emphasizes Procedural Ultrasound: Logistics and Teaching Materials

INTRODUCTION

Early formal instruction in procedural skills may increase the frequency with which residents perform procedures in the clinical setting. This workshop trained internal medicine residents in ultrasound skills and manual skills required to perform procedures common on medicine wards and required for board eligibility.

METHODS

Since 2016, our internal medicine residency program has executed three annual half-day workshops for interns during orientation, before clinical duties began. Prior to the workshop, we directed interns to relevant educational resources in the form of online modules and videos. At the workshop, trainees rotated in small groups through facilitated stations to learn basics of procedural ultrasound and to practice manual tasks performed during paracentesis, thoracentesis, lumbar puncture, and peripheral intravenous catheter placement. We administered questionnaires before and immediately after the workshop and used Wilcoxon signed rank tests to compare self-assessed independence and confidence.

RESULTS

Two hundred four interns with little prior procedural training participated in the workshop. Most participants (85%) indicated that orientation was the best timing for this training experience when compared to later options. Confidence and independence increased for ultrasound-marked thoracentesis, paracentesis, and peripheral intravenous catheters and for lumbar puncture without ultrasound.

DISCUSSION

This internal medicine intern orientation workshop on procedures and procedural ultrasound was well received and increased participants' confidence and sense of independence. This publication contains materials needed to reproduce the training experience.

The Robotic Surgery Learning Experience Through the Eyes of the Medical Student: What Do They See?

INTRODUCTION

Robotic technology has an increasing prevalence in the operating room across various specialties. How this technology influences the learning of medical students has yet to be identified. The purpose of this study is to identify the impact of robotic surgery on medical student learning in the operating room.

METHODS

Medical students at the Medical College of Wisconsin who had completed their core clinical clerkships and were exposed to robotic surgery qualified for the study. An email survey was sent to all fourth year medical students requesting volunteers to be interviewed. During the 30-minute interviews, students responded to open-ended questions about their experiences with robotic surgery. The interviews were transcribed, codes were identified using an inductive approach, and themes were generated.

RESULTS

Overall 10 interviews were conducted: 4 females and 6 males. No new themes were identified after the seventh interview. Through the medical student interviews, 4 themes emerged regarding the influence of robotic surgery on medical student learning: the robot itself, enhanced learning through the robotic technology, the medical student learning experience, and how others influence the medical student learning environment.

CONCLUSIONS

The robotic operating room is not currently a motivating learning environment for medical students. However, by providing a more effective curriculum for students, incorporating elements of orientation, legitimate roles of participation, and simulation, students may gain a learning benefit from this technology versus avoiding the technology altogether. Additionally, it is important to educate faculty and residents on better ways to incorporate students into the robotic operating room environment to ensure an optimal learning experience.