A critical review of simulation-based mastery learning with translational outcomes

Reliability and validity of simulation-based Electrocardiogram assessment rubrics for cardiac life support skills among medical students using generalizability theory

Simulation-based learning (SBL) is effective for EKG interpretation training in the advanced cardiac life support (ACLS) context, enhancing motivation, confidence, and learning outcomes. However, research on the psychometrics of assessment rubrics for ACLS skills among pre-clinical students is limited. This study investigates the validity and reliability of assessment rubrics for ACLS skills, including EKG interpretation, scenario and pharmacological management, and teamwork. An SBL course that integrates basic EKG interpretation into ACLS Stations was conducted at Phramongkutklao College of Medicine, utilizing high-fidelity mannequins to simulate realistic scenarios, enrolling 96 medical students. The course consisted of five independent stations, and each student was assessed once by two raters using ten-item assessment rubrics. The rubrics included three domains: (1) EKG and ACLS algorithm skills, (2) management and mechanisms of action, and (3) affective domains. Validity evidence on the content was gathered, and construct validity was confirmed with confirmatory factor analysis (CFA). Inter-rater and internal consistency reliability were calculated. Generalizability theory was utilized to analyse the data. Three expert reviews yielded an item-objective congruence index of 0.67-1.00, with iterative validation through alpha and beta tests. The CFA demonstrated a good fit, but two questions with loading factors below 0.30 were removed, resulting in an eight-item assessment form. An inter-rater correlation of 0.70 (p < 0.001) and a Cronbach's alpha of 0.76 was demonstrated. To achieve a Phi-coefficient ≥0.80, three raters and at least 10 items are required in a p×i×r crossed design. With eight items, r:(p×i) nested design reliability was 0.69, 0.79, and 0.83 for one, two, and three raters, respectively. While a single rater with 10 items achieved a Phi-coefficient of 0.74. The rubrics for assessing ACLS skills among pre-clinical students demonstrated acceptable validity and reliability. A condensed eight-item rubric with acceptable reliability is proposed as a practical tool for optimizing assessment in future evaluations relevant to the pre-clinical context.

Development and Validation of a Bovine Left Displaced Abomasum Reduction Model and Rubric

Left displaced abomasum (LDA) is a common condition in dairy cattle where the abomasum dilates and migrates to the left side of the abdomen. This condition causes significant economic losses for farmers and can result in life-threatening complications, so it is critical that veterinary students be taught to surgically correct a LDA before graduating and entering food animal practice. Models have been successfully used to teach students to perform other surgical procedures, but limited models exist to teach surgical skills to prospective dairy veterinary students. This study sought to develop and validate a bovine LDA reduction model and scoring rubric using a validity framework consisting of content evidence (expert opinion), internal structure evidence (reliability of rubric scores), and evidence showing the relationship with other variables (comparing expert to novice performance). Experienced veterinarians (n = 12) and novice veterinary students (n = 30) surgically deflated and reduced the model's LDA while being recorded. Videos were scored by a blinded expert. Participants completed a survey afterward. All veterinarians reported that the model was suitable for use in teaching and assessing students, offering content evidence for validation. Scores produced by the checklist had good reliability (α = .886), offering internal structure evidence. Veterinarians achieved higher checklist (p = .025) and global rating scores (p = .005) than students, offering relationship with other variables evidence. The development and use of food animal models promotes students' development of competence in performing food animal procedures, leading to better qualified new graduates entering food animal practice. The use of models also protects animal welfare during students' training.

Evaluating Learning Curves in Virtual Reality Cortical Mastoidectomy Training Across Expertise Levels

HYPOTHESIS

Virtual reality (VR) simulation has been established as an effective method of supplementing traditional surgical training. Learning curves can analyze skill acquisition over time in VR settings. Although previous studies explored learning curves of mastoidectomy performances on single specimen, this study analyzed learning curves on anatomically different virtual temporal bones across three expertise levels (novice, intermediate, and expert).

METHODS

Thirty participants were divided into three groups: 10 medical students (novice), 10 ear, nose and throat (ENT) registrars (intermediate), and 10 senior ear surgeons (expert). They performed mastoidectomy on eight anatomically different temporal bones on the University of Melbourne Temporal Bone Surgery Simulator. A blinded senior ENT surgeon assessed the final products of the dissections using the Melbourne Mastoidectomy Scale (MMS). Learning curves of MMS scores, number of strokes, average force, drilling time, and total time were compared between groups using Friedman tests. Within-group analyses were conducted with Wilcoxon signed-rank tests.

RESULTS

All performance metrics showed significant differences across all groups. Only comparison between intermediate and expert groups for average force was not significant. Within-group analyses showed significant differences in expert group for total and drilling time, intermediate group for average force, and novice group for total time. Individual learning curves of intermediate group demonstrated varied learning behavior.

CONCLUSIONS

Trainee performance was not seen to reach the level of experts after eight repetitions on anatomically different specimens. Enhancing training with individualized feedback and increased repetitions may optimize skills acquisition.

Artificial Intelligence Guided Bronchoscopy is Superior to Human Expert Instruction for the Performance of Critical-Care Physicians: A Randomized Controlled Trial

OBJECTIVES

Bronchoscopy in the mechanically ventilated patient is an important skill for critical-care physicians. However, training opportunity is heterogenous and limited by infrequent caseload or inadequate instructor feedback for satisfactory competencies. A new artificial intelligence (AI) navigational system using augmented reality - the Ambu Broncho Simulator - can guide bronchoscopy training. Is training with the AI system comparable to bedside, expert tutor instruction in improving bronchoscopy performance?

DESIGN

A nonblinded, parallel group randomized controlled trial was conducted.

SETTING

The study was conducted in a simulated setting at an academic university hospital.

SUBJECTS

Critical-care physicians were invited to take part in the study.

INTERVENTIONS

Forty participants received 30 minutes of bronchoscopy training, either guided by AI only (artificial intelligence group [AIG]) or by expert tutor feedback (expert tutor group [ETG]). All participants performed a final full navigation bronchoscopy performance test and completed a cognitive load questionnaire, the NASA Task Load Index .

MEASUREMENTS AND MAIN RESULTS

Mean intersegmental time (MIT = PT/DC), diagnostic completeness (DC), procedure time (PT), structured progress (SP), and number of segments revisited (SR) were measured. The primary outcome measure assessed was MIT, a measure of bronchoscopic performance efficiency. The secondary outcome measures were DC, PT, SP, and SR. Nineteen participants were randomized to the AIG and 21 participants to the ETG. MIT, PT, and SR were significantly better in the AIG compared to the ETG (median difference, p): MIT (-7.9 s, 0.027), PT (-77 s, 0.022), SR (-7 segments, 0.019); all showing moderate effect sizes (0.35, 0.36, and 0.37, respectively) as per Cohen's classification.There was no significant difference between the groups for all other final test measures.

CONCLUSIONS

Training using an AI system resulted in faster and more efficient bronchoscopy performance by critical-care physicians when compared to expert human tutor instruction. This could change the future of bronchoscopy training in critical care and warrants validation in patients through clinical studies.

Insights from 40 years of educational research: honoring Jeroen van Merriënboer

Simulation-based education in healthcare has advanced significantly, yet a persistent gap remains between educational science and healthcare simulation research. The late Jeroen van Merriënboer's extensive work in educational science provides valuable guidance for bridging this gap. Four key insights from his research can serve as a strong theoretical bedrock for educators and researchers aiming to design more effective and cohesive simulation-based learning experiences: (1) integrating learning in both simulated and real environments to improve transfer, (2) offering targeted learner support that evolves with expertise, (3) embracing the complexity of educational practice and avoiding one-size-fits-all solutions, and (4) embedding domain-general skills within specific disciplines. Championing these insights may catalyze more theory-informed practice and research in healthcare simulation. Nevertheless, applying these principles in practice remains a challenge, highlighting the need for further research into the "how"-specifically how to interconnect learning environments, adapt instruction to diverse needs, integrate theory with practice, and combine the teaching of domain-general and domain-specific skills.

Using Simulation-Based Mastery Learning to Achieve Excellent Learning Outcomes in Physical Therapist Education

INTRODUCTION

The 2 aims of this observational study are (a) to describe the implementation and feasibility of a bed mobility skills simulation-based mastery learning (SBML) curricular module for physical therapist students and (b) to measure learning outcomes and student perceptions of this module.

REVIEW OF LITERATURE

Simulation-based mastery learning is an outcome-based educational approach that has been successful in other health professions but has not been explored in physical therapy education.

SUBJECTS

Eighty-seven students in a single cohort of a Doctor of Physical Therapy program.

METHODS

The SBML module in this pretest-posttest study included a pretest, instruction, initial posttest, and additional rounds of instruction and assessment as needed for all learners to achieve the minimum passing standard (MPS) set using the Mastery Angoff and Patient Safety methods. Outcome measures were bed mobility assessment pass rates and scores, additional student and faculty time compared with a traditional approach, and student perceptions of their self-confidence and the module.

RESULTS

All students achieved the MPS after 3 rounds of training and assessment beyond the initial posttest. Mean Total Scores improved from 67.6% (12.9%) at pretest to 91.4% (4.8%) at mastery posttest ( P < .001, Cohen's d = 1.8, 95% CI [1.4-2.1]); mean Safety Scores improved from 75.2% (16.0%) at pretest to 100.0% (0.0%) at mastery posttest ( P < .001, Cohen's d = 1.5, 95% CI [1.2-1.9]). Students who did not achieve the MPS at the initial posttest ( n = 30) required a mean of 1.2 hours for additional instruction and assessment. Survey results revealed an increase in student confidence ( P < .001) and positive student perceptions of the module.

DISCUSSION AND CONCLUSION

Implementation of this SBML module was feasible and resulted in uniformly high levels of bed mobility skill acquisition. Based on rigorous learning outcomes, feasible requirements for implementation, and increased student confidence, SBML offers a promising approach for wider implementation in physical therapy education.

Procedural sedation competencies: a review and multidisciplinary international consensus statement on knowledge, skills, training, and credentialing

Procedural sedation is practised by a heterogeneous group of practitioners working in a wide array of settings. However, there are currently no accepted standards for the competencies a sedation practitioner should have, the content of sedation training programmes, and guidelines for credentialing. The multidisciplinary International Committee for the Advancement of Procedural Sedation sought to develop a consensus statement on the following: which competencies should medical or dental practitioners have for procedural sedation and how are they obtained, assessed, maintained, and privileged. Using the framework of Competency-Based Medical Education, the practice of procedural sedation was defined as a complex professional task requiring demonstrable integration of different competencies. For each question, the results of a literature review were synthetised into preliminary statements. Following an iterative Delphi review method, final consensus was reached. Using multispeciality consensus, we defined procedural sedation competence by identifying a set of core competencies in the domains of knowledge, skills, and attitudes across physical safety, effectiveness, psychological safety, and deliberate practice. In addition, we present a standardised framework for competency-based training and credentialing of procedural sedation practitioners.

Abdominal POCUS Education for Clinicians: A Systematic Review of Teaching Methods for Point-of-Care Abdominal Ultrasonography

This systematic review examines educational strategies in clinician-performed abdominal point-of-care ultrasound (POCUS), a critical skill with increasing relevance in medical care. Analyzing 28 studies, we highlight the strategies as well as advantages and disadvantages of various theoretical and practical components, including, for example, e-learning and simulation in training programs. The findings emphasize the necessity of blending various educational methods to enhance effectiveness and adaptability in training environments. Ultimately, robust training frameworks are essential to maximize diagnostic accuracy and improve patient outcomes in abdominal POCUS.

Bringing competency-based communication training to scale: A multi-institutional virtual simulation-based mastery learning curriculum for Emergency Medicine residents

PURPOSE

Serious illness communication skills are essential for physicians, yet competency-based training is lacking. We address scalability barriers to competency-based communication skills training by assessing the feasibility of a multi-center, virtual simulation-based mastery learning (vSBML) curriculum on breaking bad news (BBN).

METHODS

First-year emergency medicine residents at three academic medical centers participated in the virtual curriculum. Participants completed a pretest with a standardized patient (SP), a workshop with didactics and small group roleplay with SPs, a posttest with an SP, and additional deliberate practice sessions if needed to achieve the minimum passing standard (MPS). Participants were assessed using a previously published BBN assessment tool that included a checklist and scaled items. Authors compared pre- and posttests to evaluate the impact of the curriculum.

RESULTS

Twenty-eight (90%) of 31 eligible residents completed the curriculum. Eighty-nine percent of participants did not meet the MPS at pretest. Post-intervention, there was a statistically significant improvement in checklist performance (Median= 93% vs. 53%, p < 0.001) and on all scaled items assessing quality of communication. All participants ultimately achieved the MPS.

CONCLUSIONS

A multi-site vSBML curriculum brought all participants to mastery in the core communication skill of BBN and represents a feasible, scalable model to incorporate competency-based communication skills education in a widespread manner.

The Effect of "Online-Simulation-Bedside" Three-Step Teaching Method in Team Cardiopulmonary Resuscitation Skills Training of Emergency Department and Critical Care Nursing Interns-An Analysis Based on Kirkpatrick Model

Aim: This study designed a three-step teaching method of "online-simulation-bedside," which was applied to the teaching of cardiopulmonary resuscitation skills in the team of emergency and critical care nursing interns, and the effectiveness of this teaching method was evaluated using the Kirkpatrick model. Background: Mastering knowledge of cardiopulmonary resuscitation and emergency techniques is necessary for nursing interns to fulfill their roles. At present, the first aid awareness and rescue skills of nursing interns are not optimistic. Training programs can improve the cardiopulmonary resuscitation ability of nursing students, but most of them only evaluate the changes in cardiopulmonary resuscitation ability before and after training and rarely use educational evaluation theories and models to evaluate the effectiveness of cardiopulmonary resuscitation training research. Methods: This is a quasi-experimental research design: pretest and posttest design. Our research focuses on nursing interns who interned in the emergency and critical care departments of a hospital from September 2023 to January 2024. Adopting the three-step teaching method of "online-simulation-bedside", provide team cardiopulmonary resuscitation skills training for emergency and critical care nursing interns. The four levels of Kirkpatrick's model, that is, reaction, learning, behavior, and result, were applied for the evaluation together with questionnaires. Conduct a baseline survey before online learning. Evaluate team working ability, clinical thinking ability, confidence in skills, and active participation on the first day after simulated training and the 30th day of clinical practice. On the 30th day of clinical practice, satisfaction and skill level assessments will also be conducted. Results: At the reaction level, the satisfaction rate of nursing interns is 98.80%. At the learning level, there was a statistically significant difference (p < 0.001) in the team work ability and clinical thinking ability of nursing interns before training, on the first day after training, and on the 30th day of clinical practice. At the behavioral level, there was a statistically significant difference (p < 0.001) in the skill confidence and participation initiative of nursing interns before training, on the first day after training, and on the 30th day of clinical practice. At the result level, on the 30th day of clinical practice, the nursing intern team's cardiopulmonary resuscitation skills assessment scores were all good or above, with an average score of 90.09 ± 1.58. Conclusion: The Kirkpatrick model can be used to evaluate the effectiveness of cardiopulmonary resuscitation skills training for nursing intern teams. Nursing interns are highly satisfied with the "online-simulation-bedside" three-step teaching method, which can improve their teamwork ability, clinical thinking ability, active participation, and skill confidence. In addition, through training, students can effectively apply the learned content to clinical practice and possess a good level of skills. Implications for Nursing Management: Managers can develop a distinctive team cardiopulmonary resuscitation training model based on this, improve the team cardiopulmonary resuscitation skills of nursing interns, and cultivate more nursing talents with high-level emergency capabilities for hospitals.

Effectiveness of a Web-Based Training Intervention in Teaching Emergency Physicians First-Trimester Point-of-Care Ultrasound Image Interpretation

STUDY OBJECTIVE

To examine the effectiveness of an education intervention on emergency physician accuracy in identifying pregnancy-related findings from first-trimester point-of-care ultrasound. Case features associated with the odds of a correct response were also determined.

METHODS

This was a multicenter prospective cross-sectional study in a convenience sample of emergency physicians in the United States and Canada. The unsupervised web-based education intervention included first-trimester point-of-care ultrasound cases acquired through the transabdominal (n=200 cases) or transvaginal (n=200 cases) approach. Physicians deliberately practiced identifying pregnancy-related imaging findings until they achieved a mastery standard.

RESULTS

In 204 participants, there were learning gains in accuracy (15.2%; 95% confidence interval [CI] 14.6 to 15.8), sensitivity (15.1%; 95% CI 14.3 to 15.9), and specificity (14.3%; 95% CI 13.7 to 15.0). Of these, 132 (64.7%) achieved the mastery standard in a median of 60 cases (interquartile range 58 to 83). Case features associated with an increased odds of a correct intrauterine pregnancy "present" diagnosis were transvaginal versus transabdominal-acquired images (odds ratio [OR]=1.5; 95% CI 1.3 to 1.8) and fetal heartbeat (OR=4.3; 95% CI 3.4 to 5.5). A decreased odds was associated with an eccentrically located intrauterine pregnancy (OR=0.2; 95% CI 0.1 to 0.2), subchorionic hemorrhage (OR=0.5; 95% CI 0.4 to 0.6), adnexal mass (OR=0.7; 95% CI 0.6 to 0.9), and endometrial collection (OR=0.1; 95% CI 0.09 to 0.2).

CONCLUSIONS

This study's intervention was effective in teaching first-trimester point-of-care ultrasound image interpretation and identified the specific variables that posed the greatest diagnostic challenges. The methods and results from this work can serve to expand learning opportunities for this critical skill in emergency medicine.

Twelve tips for developing simulation-based mastery learning clinical skills checklists

Simulation-based mastery learning is a powerful educational paradigm that leads to high levels of performance through a combination of strict standards, deliberate practice, formative feedback, and rigorous assessment. Successful mastery learning curricula often require well-designed checklists that produce reliable data that contribute to valid decisions. The following twelve tips are intended to help educators create defensible and effective clinical skills checklists for use in mastery learning curricula. These tips focus on defining the scope of a checklist using established principles of curriculum development, crafting the checklist based on a literature review and expert input, revising and testing the checklist, and recruiting judges to set a minimum passing standard. While this article has a particular focus on mastery learning, with the exception of the tips related to standard setting, the general principles discussed apply to the development of any clinical skills checklist.

Educational outcomes of simulation-based training in regional anaesthesia: a scoping review

BACKGROUND

Structured training in regional anaesthesia includes pretraining on simulation-based educational platforms to establish a safe and controlled learning environment before learners are provided clinical exposure in an apprenticeship model. This scoping review was designed to appraise the educational outcomes of current simulation-based educational modalities in regional anaesthesia.

METHODS

This review conformed to PRISMA-ScR guidelines. Relevant articles were searched in PubMed, Scopus, Google Scholar, Web of Science, and EMBASE with no date restrictions, until November 2023. Studies included randomised controlled trials, pre-post intervention, time series, case control, case series, and longitudinal studies, with no restrictions to settings, language or ethnic groups. The Kirkpatrick framework was applied for extraction of educational outcomes.

RESULTS

We included 28 studies, ranging from 2009 to 2023, of which 46.4% were randomised controlled trials. The majority of the target population was identified as trainees or residents (46.4%). Higher order educational outcomes that appraised translation to real clinical contexts (Kirkpatrick 3 and above) were reported in 12 studies (42.9%). Two studies demonstrated translational patient outcomes (Level 4) with reduced incidence of paraesthesia and clinical complications. The majority of studies appraised Level 3 outcomes of performance improvements in either laboratory simulation contexts (42.9%) or demonstration of clinical performance improvements in regional anaesthesia (39.3%).

CONCLUSIONS

There was significant heterogeneity in the types of simulation modalities used, teaching interventions applied, study methodologies, assessment tools, and outcome measures studied. When improvisations were made to regional anaesthesia simulation platforms (hybrid simulation), there were sustained educational improvements beyond 6 months. Newer technology-enhanced innovations such as virtual, augmented, and mixed reality simulations are evolving, with early reports of educational effectiveness.

A structured simulation-based mastery learning curriculum in chest tube insertion results in superior skills compared to traditional training programs

BACKGROUND

Chest tube insertions (CTI) have a high complication rate, warranting a dedicated Simulation-Based Mastery Learning (SBML) curriculum to acquire technical skills. This randomized controlled trial compares residents' skills in CTI after completing a SBML curriculum with those enrolled in a traditional residency program.

METHODS

Junior residents were baseline tested on cognitive and technical skills (Thiel bodies) before randomization into an intervention and control group. The former deliberately trained CTI on a porcine rib model until passing a predefined pass/fail score and were then summatively tested on Thiel bodies. The latter had no additional training opportunities and was evaluated 3 months later.

RESULTS

Seventeen residents were recruited and randomized. Following the per-protocol principle, a significant interaction effect for Group × Procedure (F(1,14) = 6.2, p = 0.026) was observed. Between baseline and summative assessment, both the control group (28.0 ± 8.2 vs. 43.6 ± 8.1, p < 0.001) and the intervention group (33.2 ± 7.7 vs. 57.6 ± 5.7, p < 0.001) significantly increased their scores. The intervention group outperformed the control group at summative assessment (43.6 ± 8.1 vs. 57.6 ± 5.7, p < 0.001). All participants in the intervention group and one resident in the control group achieved the pass/fail score.

CONCLUSION

This SBML curriculum enabled quicker and superior skill acquisition. Skills trained on a porcine model are transferred to the highly realistic Thiel bodies and reach expert level, potentially increasing resident skill in clinical practice.

A Systematic Review of Simulation-Based Training Tools in Plastic Surgery

OBJECTIVES

The recent shift from traditional surgical teaching to the incorporation of simulation training in plastic surgery has resulted in the development of a variety of simulation models and tools. We aimed to assess the validity and establish the effectiveness of all currently available simulators and tools for plastic surgery.

DESIGN

Systematic review.

METHODS

The PRISMA 2020 checklist was followed. The review protocol was prospectively registered in PROSPERO (CRD42021231546). Published and unpublished literature databases were searched to the 29th of October 2023. Each model was appraised in accordance with the Messick validity framework, and a rating was given for each section. To determine the effectiveness of each model, the McGaghie model of translational outcomes was used.

RESULTS

On screening 1794 articles, 116 were identified to discuss validity and effectiveness of simulation models in plastic surgery. These were hand surgery (6 studies), breast surgery (12 studies), facial surgery (25 studies), cleft lip and palate surgery (29 studies), rhinoplasty (4 studies), hair transplant surgery (1 study), surgery for burns (10 studies), and general skills in plastic surgery (29 studies). Only 1 model achieved an effectiveness level > 3, and no model had a rating > 2 in all aspects of the Messick validity framework.

CONCLUSION

There are limited models enabling the transfer of skills to clinical practice. No models achieved reductions in surgical complications or costs. There must be more validity studies conducted using updated validity frameworks, with an increased emphasis on the applicability of these simulators to improve patient outcomes and surgical technique. More training tools evaluating both technical and non-technical surgical skills are recommended.

Measuring Residents' Competence in Chest Tube Insertion on Thiel-Embalmed Bodies: A Validity Study

INTRODUCTION

Chest tube insertions (CTIs) have a high complication rate, prompting the training of technical skills in simulated settings. However, assessment tools require validity evidence prior to their implementation. This study aimed to collect validity evidence for assessment of technical skills in CTI on Thiel-embalmed human bodies.

METHODS

Invitations were sent to residents and staff from the departments of surgery, pulmonology, and emergency medicine. Participants were familiarized with the Thiel body and the supplied equipment. Standardized clinical context and instructions were provided. All participants performed 2 CTIs and were assessed with the Assessment for Competence in Chest Tube InsertiON (ACTION) tool, consisting of a 17-item rating scale and a 16-item error checklist. Live and post hoc video-based assessments by 2 raters were performed. Generalizability analysis was performed to evaluate reliability. Mean scores and errors were compared using a mixed-model repeated measures analysis of variance (ANOVA). A pass/fail score was determined using the contrasting groups' method.

RESULTS

Ten novices and 8 experienced participants completed the study. The Generalizability coefficients were moderate for the rating scale (0.75), and low for the error checklist (0.4). Novices scored lower on the rating scale?? (44±6.7/68 vs 50.8 ± 5.7/68, P = 0.024), but did not commit significantly more errors (1.6 ± 1.1/16 vs 1.0 ± 0.6/16, P = 0.066). A pass/fail score of 47/68 was established.

CONCLUSION

The rating scale in the Assessment for Competence in Chest Tube InsertiON tool has a robust validity argument for use on Thiel-embalmed bodies, allowing it to be used in simulation-based mastery learning curricula. In contrast, its error checklist has insufficient reliability and validity to be used for summative assessment.

The Scottish Simulation 'KSDP' Design Framework: a sense-making and ordered approach for building aligned simulation programmes

Impactful learning through simulation-based education involves effective planning and design. This can be a complex process requiring educators to master a varied toolkit of analysis tools, learning methodologies, and evaluative strategies; all to ensure engagement of learners in a meaningful and impactful way. Where there is a lack of thoughtful design, simulation-based education programmes may be inefficiently deployed at best, and completely ineffective or even harmful to learning and learners at worst. This paper presents a useful sense-making framework, designed to support simulation educators in designing their learning activities in a systematic, stepwise, and learner centred way.

Development and Initial Validity Evidence for a Pregnancy Disclosure and Options Counseling Checklist in Emergency Medicine

Background

Many patients first learn of a pregnancy in the emergency department (ED). However, limited ED physician knowledge in pregnancy disclosure and options counseling may contribute to challenges in patient care and potential missed opportunities for timely access to reproductive care. No standardized instruments exist to teach or assess this important communication skill for the ED physicians. This study aimed to develop and collect validity evidence for a checklist for effective pregnancy disclosure and options counseling by ED physicians in an environment with unrestricted access to comprehensive reproductive care.

Methods

A prospective checklist creation and validity evidence collection study was conducted, involving literature review, expert input through a modified Delphi process, and pilot testing with ED faculty and residents at an urban academic medical center. We structured the validity evidence collection process using Messick's criteria for construct validity, addressing content, response process, internal structure, and relations to other variables. Data analysis focused on collecting validity evidence, including inter-rater reliability and participant performance assessment based on faculty or resident status.

Results

The study resulted in a final 17-item checklist for pregnancy disclosure and options counseling in the ED. Pilot testing with 20 participants (eight faculty members and 12 residents) revealed high overall inter-rater reliability with almost perfect agreement (kappa = 0.81) and acceptable internal consistency (Cronbach's alpha = 0.88). Checklist scores showed no significant difference across standardized patients, indicating consistency. Faculty members outperformed residents, suggesting concurrent validity based on levels of clinical experience.

Conclusions

Preliminary validity evidence supports the use of this novel checklist to assess physician competency in pregnancy disclosure and options counseling in the ED. Given the frequency of new pregnancy encounters in the ED and the dynamic changes to pregnant patients' reproductive rights, enhancing physician education in these areas is critical for optimizing patient care and autonomy.

Embedding an Education Intervention about Shared Decision Making into an RCT: Ensuring competency and fidelity

Objective

To describe the outcomes of training nephrology clinicians and clinical research participants, to use the Best Case/Worst Case Communication intervention, for discussions about dialysis initiation for patients with life-limiting illness, during a randomized clinical trial to ensure competency, fidelity to the intervention, and adherence to study protocols and the intervention throughout the trial.

Methods

We enrolled 68 nephrologists at ten study sites and randomized them to receive training or wait-list control. We collected copies of completed graphic aids (component of the intervention), used with study-enrolled patients, to measure fidelity and adherence.

Results

We trained 34 of 36 nephrologists to competence and 27 completed the entire program. We received 60 graphic aids for study-enrolled patients for a 73% return rate in the intervention arm. The intervention fidelity score for the graphic aid reflected completion of all elements throughout the study.

Conclusion

We successfully taught the Best Case/Worst Case Communication intervention to clinicians as research participants within a randomized clinical trial.

Innovation

Decisions about dialysis are an opportunity to discuss prognosis and uncertainty in relation to consideration of prolonged life supporting therapy. Our study reveals a strategy to evaluate adherence to a communication intervention in real time during a clinical study.

Using Mixed Reality to Teach Medical Students Lumbar Punctures

BACKGROUND

Gaining familiarity with bedside procedures during medical school is essential for preparing medical students for intern year. However, this is often not a core component of undergraduate medical education, and students often feel unprepared to perform bedside procedures upon graduation from medical school. Extended reality technologies have rapidly developed in the past few years, become more accessible, and show great promise in education, most recently in healthcare. We aim to show the feasibility and usability of a mixed-reality lumbar puncture simulated experience.

METHODS

We created the mixed-reality simulation by filming a procedure conducted by an expert on a lumbar puncture simulator from the first-person view. We performed a study to assess the feasibility and usability of mixed-reality lumbar puncture training compared with traditional faculty-led high-fidelity mannikin-based simulation training. Fourth-year medical students were randomly selected to participate in the mixed-reality lumbar puncture training. Students watched the virtual first-person teaching via a virtual reality headset while mimicking hand movements on a lumbar puncture mannikin. Faculty taught students in the traditional high-fidelity group in small groups before practicing on the high-fidelity manikin. Students in both groups then performed the procedure entirely with performance assessed via a standardized checklist. Number of attempts to achieve a predetermined minimum passing score were recorded, and students were also surveyed regarding their experience with both simulations.

RESULTS

Fifty-nine percent of students felt comfortable using mixed reality as a training modality with less than 6% having any prior extended reality experience. More than 55% felt the mixed-reality experience was as effective as the traditional high-fidelity simulation training. Many students (72%) desired more mixed-reality opportunities. In terms of performance, 58.8% of students in the mixed-reality group vs 65.5% in the traditional group completed the checklist in 1 attempt (P = .79).

CONCLUSIONS

We demonstrate that mixed reality is an acceptable and effective modality to train health care professionals in lumbar punctures with opportunities to further enhance the immersive learning experience.

Manual Uterine Aspiration Simulation for Emergency Medicine Learners

Introduction

Manual uterine aspiration is a potentially lifesaving procedure for treating patients with hemorrhagic complications of early pregnancy loss. While early pregnancy loss is a common diagnosis seen in the emergency department, manual uterine aspiration education is lacking for emergency medicine physicians.

Methods

We designed a 90-minute procedural skills training session for 30 emergency medicine learners. The session included a brief lecture and video demonstration, followed by two micro-skills stations before finally completing the simulated procedure in its entirety. At each station, learners were asked to verbalize the steps and landmarks for the procedure before performing them on models. Participants completed a combined pre-post survey evaluating their perceived knowledge of the procedure and self-efficacy in performing the procedure.

Results

Thirty learners who participated in the workshop were surveyed, with a 100% response rate. All participants reported increased comfort with the procedure and knowledge about the procedure. All participants completed a successful simulated procedure. Participants also indicated increased interest in learning more about manual uterine aspiration and its potential application within the emergency medicine physician's scope of practice.

Discussion

We developed a workshop to train emergency medicine learners in manual uterine aspiration to stop life-threatening hemorrhage in the setting of early pregnancy loss. The workshop was well received by learners and increased their self-efficacy and desire for additional training with this procedure. Similar curricula should be tried at other institutions.

Simulation for Systems Integration: A Win-Win to Achieve Your Education, Quality, and Safety Goals

Health care education is a cornerstone of clinical excellence, ensuring the highest level of readiness to achieve high-quality and safe care. Integrating simulation into health care systems can provide a modality to address educational, quality, and safety goals. Simulation is a methodology used to immerse individuals, teams, and medical systems into clinical scenarios or environments. Through facilitation and debriefing, the immersive experience can be used to provide vital education, develop optimal health care practices, and identify prospects for improvement. Simulation provides an opportunity to blend the needs of continuing multiprofessional education while addressing quality and safety goals, and ultimately, promoting a positive safety culture. Applied comprehensively, simulation for systems integration can both promote the transformation of the health care system into a learning organization, as well as translate to improved health care outcomes. [Pediatr Ann. 2024;53(11):e408-e413.].

The effectiveness of simulation-based learning (SBL) on students' knowledge and skills in nursing programs: a systematic review

BACKGROUND

Simulation-Based Learning (SBL) serves as a valuable pedagogical approach in nursing education, encompassing varying levels of fidelity. While previous reviews have highlighted the potential effectiveness of SBL in enhancing nursing students' competencies, a gap persists in the evidence-base addressing the long-term retention of these competencies. This systematic review aimed to evaluate the impact of SBL on nursing students' knowledge and skill acquisition and retention.

METHOD

A comprehensive search of electronic databases, including CINAHL, PubMed, Embase, Scopus, and Eric, was conducted from 2017 to 2023 to identify relevant studies. The Joanna Briggs critical appraisal tools were used to assess the methodological quality of the included studies. A total of 33 studies (15 RCTs and 18 quasi-experimental) met the inclusion criteria and were included in the review. A descriptive narrative synthesis method was used to extract relevant data.

RESULTS

The cumulative sample size of participants across the included studies was 3,670. Most of the studies focused on the impact of SBL on life-saving skills like cardiopulmonary resuscitation (CPR) or other life-support skills. The remaining studies examined the impact of SBL on critical care skills or clinical decision-making skills. The analysis highlighted consistent and significant improvements in knowledge and skills. However, the evidence base had several limitations, including the heterogeneity of study designs, risk of bias, and lack of long-term follow-up.

CONCLUSION

This systematic review supports the use of SBL as a potent teaching strategy within nursing education and highlights the importance of the ongoing evaluation and refinement of this approach. While current evidence indicates enhancing knowledge and skill acquisition, limited studies evaluated the retention beyond five months, constraining generalisable claims regarding durability. Further research is essential to build on the current evidence and address gaps in knowledge related to the retention, optimal design, implementation, and evaluation of SBL interventions in nursing education.

Optimizing peripheral regional anaesthesia: strategies for single shot and continuous blocks

PURPOSE OF REVIEW

Regional anaesthesia is increasingly prominent within anaesthesia, offering alternative analgesic options amidst concerns over opioid-based analgesia. Since Halsted's initial description, the field has burgeoned, with ultrasound visualization revolutionizing local anaesthetic spread assessment, leading to the development of numerous novel techniques. The benefits of regional anaesthesia have gained increasing evidence to support their application, leading to changes within training curricula. Consequently, regional anaesthesia is at a defining moment, embracing the development of core skills for the general anaesthesiologist, whilst also continuing the advancement of the specialty.

RECENT FINDINGS

Recent priority setting projects have focussed attention on key aspects of regional anaesthesia delivery, including pain management, conduct and efficacy, education, and technological innovation. Developments in our current understanding of anatomy and pharmacology, combined with strategies for optimizing the conduct and maximizing efficacy of techniques, minimizing complications, and enhancing outcomes are explored. In addition, advancements in education and training methodologies and the integration of progress in novel technologies will be reviewed.

SUMMARY

This review highlights recent scientific advances in optimizing both single-shot and continuous peripheral regional anaesthesia techniques. By synthesizing these developments, this review offers valuable insights into the evolving landscape of regional anaesthesia, aiming to improve clinical practice and patient care.

Assessing Chest Tube Insertion Skills Using a Porcine Rib Model-A Validity Study

INTRODUCTION

Assessments require sufficient validity evidence before their use. The Assessment for Competence in Chest Tube Insertion (ACTION) tool evaluates proficiency in chest tube insertion (CTI), combining a rating scale and an error checklist. The aim of this study was to collect validity evidence for the ACTION tool on a porcine rib model according to the Messick framework.

METHODS

A rib model, consisting of a porcine hemithorax that was placed in a wooden frame, was used as simulator. Participants were recruited from the departments of surgery, pulmonology, and emergency medicine. After familiarization with the rib model and the equipment, standardized instructions and clinical context were provided. They performed 2 CTIs while being scored with the ACTION tool. All performances were assessed live by 1 rater and by 3 blinded raters using video recordings. Generalizability-analysis was performed and mean scores and errors of both groups on the first performance were compared. A pass/fail score was established using the contrasting groups' method.

RESULTS

Nine novice and 8 experienced participants completed the study. Generalizability coefficients where high for the rating scale (0.92) and the error checklist (0.87). In the first CTI, novices scored lower than the experienced group (38.1/68 vs. 47.1/68, P = 0.042), but no difference was observed on the error checklist. A pass/fail score of 44/68 was established.

CONCLUSION

A solid validity argument for the ACTION tool's rating scale on a porcine rib model is presented, allowing formative and summative assessment of procedural skills during training before patient contact.

Shifting the Curve: Mastery Learning to Teach Nerve Conduction Studies in Physical Medicine and Rehabilitation Residency

ABSTRACT

Nerve conduction studies are a commonly performed procedure and a core competency for physical medicine and rehabilitation residents. Nerve conduction studies are complex to learn and no standardized training protocols exist across physical medicine and rehabilitation programs. The aims of this study are to standardize training and assessment of resident proficiency in nerve conduction studies skills and to evaluate the impact of an educational intervention on skills development. This was a repeated measures design wherein 36 physical medicine and rehabilitation residents underwent a pretest, followed by a mastery learning-based intervention, including deliberate practice. Residents were then expected to meet or exceed a minimum passing score at posttest. Performance improved from a median score of 4.5/66 on the pretest to a median score of 63/66 on the posttest. Thirty-three of 36 residents achieved the minimum passing score on the first attempt; three residents required additional deliberate practice and met the minimum passing score on the second attempt. A curriculum featuring deliberate practice dramatically increased checklist scores of residents performing nerve conduction studies. This mastery learning program shows a reliable and reproducible method to achieve procedural competency within a physical medicine and rehabilitation residency program and can shift the curve to allow residents to immediately start performing nerve conduction studies at the start of their clinical experience.

A Comparison of Telesimulation Using the Virtual Resus Room and In Situ Simulation in Pediatric Emergency Medicine

OBJECTIVES

During the COVID-19 pandemic, there was a marked shift toward telesimulation in medical education. Limited studies exist comparing the effectiveness of online and offline simulation education. The goals of this study are to evaluate active learners' perceived effectiveness of telesimulation versus in situ simulation and to identify potential shortcomings of existing online teaching platforms.

METHODS

Through participant evaluations after a simulation, we compared telesimulation using the Virtual Resus Room (VRR) to in situ simulation in the domains of (1) self-efficacy, (2) fidelity, (3) educational value, and (4) teaching quality. Study subjects included medical and pharmacy residents and medical students completing their pediatric emergency medicine rotation at two children's hospitals as well as nurses, nurse practitioners, and physician assistants who were recently hired and orienting to their new roles in the emergency department. Learners used a modified Michigan Standard Simulation Experience Scale to evaluate either a telesimulation or in situ simulation case. Survey responses were compared using Wilcoxon rank sum tests with Bonferroni correction for multiple comparisons.

RESULTS

In overall assessment, in situ simulation was rated higher than telesimulation. There were significant differences noted related to perceived realism, utility in training device-related skills, and utility in training team-building skills. All P values were less than 0.0036. There were no significant differences between simulation types in perception of physical examination fidelity, instructor adequacy, or self-efficacy.

CONCLUSIONS

Telesimulation using the VRR is comparable to in situ simulation in learners' perception of improvement in self-efficacy and of teaching quality for pediatric emergency medicine topics. However, participants felt less able to practice tactile and communication skills virtually. Further innovation is needed to improve learners' experience with fidelity and educational value.

Explor-A-Thora: A Novel Three-Dimensionally Printed Pleural Simulator

Background

For procedural education, the shift from the traditional apprenticeship model to simulation-based mastery has become increasingly accepted as the gold standard and has underscored the importance of high-fidelity, cost-effective training options. However, cost-effective pleural procedure simulators providing both realistic haptic feedback and ultrasound compatibility are lacking.

Objective

We aimed to create a pleural procedure simulator with characteristics of human tissue, at low cost and with ultrasound compatibility.

Methods

This work used design-based research principles and a collaborative rapid iteration approach in collaboration with the University of California, San Francisco, Makers Lab and design-based researchers at the University of California, Berkeley, which led to the creation of a three-dimensionally printed pleural procedure simulator.

Results

The needs assessment indicated significant discomfort with pleural procedures and a request for more accessible simulation opportunities. Iterative prototyping resulted in a three-dimensionally printed rib cage and a series of innovations in the fluid pocket and skin layers to provide realistic tactile feedback and ultrasound imaging compatibility. The final model costs significantly less than commercial simulators, with durable components and replaceable parts that can be reused multiple times.

Conclusion

The development of a low-cost, high-fidelity pleural procedure simulator addresses the current limitations of commercially available pleural simulators. By integrating three-dimensional printing technology and easily accessible materials, we were able to produce a simulator that closely replicates the feel of human tissue, allows ultrasound use, and is adaptable for different patient anatomies and clinical scenarios. This novel simulator is a scalable solution to elevate the standard of procedural education and ultimately positively affect patient care.

Survey of Current Simulation Based Training in the US Military Health System

INTRODUCTION

Simulation-based medical training has been shown to be effective and is widely used in civilian hospitals; however, it is unclear how widely and how effectively simulation is utilized in the U.S. Military Health System (MHS). The current operational state of medical simulation in the MHS is unknown, and there remains a need for a system-wide assessment of whether and how the advances in simulation-based medical training are employed to meet the evolving needs of the present-day warfighter. Understanding the types of skills and methods used within simulation programs across the enterprise is important data for leaders as they plan for the future in terms of curriculum development and the investment of resources. The aim of the present study is to survey MHS simulation programs in order to determine the prevalence of skills taught, the types of learners served, and the most common methodologies employed in this worldwide health care system.

MATERIALS AND METHODS

A cross-sectional survey of simulation activities was distributed to the medical directors of all 93 simulation programs in the MHS. The survey was developed by the authors based on lists of critical wartime skills published by the medical departments of the US Army, Navy, and Air Force. Respondents were asked to indicate the types of learners trained at their program, which of the 82 unique skills included in the survey are trained at their site, and for each skill the modalities of simulation used, i.e., mannequin, standardized patients, part task trainers, augmented/virtual reality tools, or cadaver/live tissue.

RESULTS

Complete survey responses were obtained from 75 of the 93 (80%) MHS medical simulation training programs. Across all skills included in the survey, those most commonly taught belonged predominantly to the categories of medic skills and nursing skills. Across all sites, the most common category of learner was the medic/corpsman (95% of sites), followed by nurses (87%), physicians (83%), non-medical combat lifesavers (59%), and others (28%) that included on-base first responders, law enforcement, fire fighters, and civilians. The skills training offered by programs included most commonly the tasks associated with medics/corpsmen (97%) followed by nursing (81%), advanced provider (77%), and General Medical Officer (GMO) skills (47%).

CONCLUSION

The survey demonstrated that the most common skills taught were all related to point of injury combat casualty care and addressed the most common causes of death on the battlefield. The availability of training in medic skills, nursing skills, and advanced provider skills were similar in small, medium, and large programs. However, medium and small programs were less likely to deliver training for advanced providers and GMOs compared to larger programs. Overall, this study found that simulation-based medical training in the MHS is focused on medic and nursing skills, and that large programs are more likely to offer training for advanced providers and GMOs. Potential gaps in the availability of existing training are identified as over 50% of skills included in the nursing, advanced provider, and GMO skill categories are not covered by at least 80% of sites serving those learners.

Using In Situ Simulation to Identify Latent Safety Threats in Emergency Medicine: A Systematic Review

SUMMARY STATEMENT

This review aimed to explore existing literature on the use of in situ simulation to identify latent safety threats in emergency medicine. Studies were required to take place in a clinically active emergency department and have either a primary or secondary aim of identifying latent safety threats. A total of 2921 articles were retrieved through database searches and a total of 15 were deemed to meet the inclusion criteria.Latent safety threats were detected by a variety of methods including documentation during debrief/discussion (66%), during the simulation itself (33%), participant surveys (20%), and video analysis (20%). Using a multimodality approach with input from observers and participants from different professional backgrounds yielded the highest number of threats per simulation case (43 per case). Equipment was the most commonly reported threat (83%), followed by teamwork/communication (67%). Some studies did not report on mitigation of identified risks; formal processes should be implemented for the management of latent safety threats identified by in situ simulation. Future research should focus on translational outcomes to further strengthen the position of in situ simulation in emergency medicine.

Automated performance metrics and surgical gestures: two methods for assessment of technical skills in robotic surgery

The objective of this study is to compare automated performance metrics (APM) and surgical gestures for technical skills assessment during simulated robot-assisted radical prostatectomy (RARP). Ten novices and six experienced RARP surgeons performed simulated RARPs on the RobotiX Mentor (Surgical Science, Sweden). Simulator APM were automatically recorded, and surgical videos were manually annotated with five types of surgical gestures. The consequences of the pass/fail levels, which were based on contrasting groups' methods, were compared for APM and surgical gestures. Intra-class correlation coefficient (ICC) analysis and a Bland-Altman plot were used to explore the correlation between APM and surgical gestures. Pass/fail levels for both APM and surgical gesture could fully distinguish between the skill levels of the surgeons with a specificity and sensitivity of 100%. The overall ICC (one-way, random) was 0.70 (95% CI: 0.34-0.88), showing moderate agreement between the methods. The Bland-Altman plot showed a high agreement between the two methods for assessing experienced surgeons but disagreed on the novice surgeons' skill level. APM and surgical gestures could both fully distinguish between novices and experienced surgeons in a simulated setting. Both methods of analyzing technical skills have their advantages and disadvantages and, as of now, those are only to a limited extent available in the clinical setting. The development of assessment methods in a simulated setting enables testing before implementing it in a clinical setting.

Adopting human factors in early phase and experimental medicine research: A nested pilot study observing controlled human infection with SARS-CoV-2

AIMS

The influence of human factors on safety in healthcare settings is well established, with targeted interventions reducing risk and enhancing team performance. In experimental and early phase clinical research participant safety is paramount and safeguarded by guidelines, protocolized care and staff training; however, the real-world interaction and implementation of these risk-mitigating measures has never been subjected to formal system-based assessment.

METHODS

Independent structured observations, systematic review of study documents, and interviews and focus groups were used to collate data on three key tasks undertaken in a clinical research facility (CRF) during a SARS CoV-2 controlled human infection model (CHIM) study. The Systems Engineering Initiative for Patient Safety (SEIPS) was employed to analyse and categorize findings, and develop recommendations for safety interventions.

RESULTS

High levels of team functioning and a clear focus on participant safety were evident throughout the study. Despite this, latent risks in both study-specific and CRF work systems were identified in all four SEIPS domains (people, environment, tasks and tools). Fourteen actionable recommendations were generated collaboratively. These included inter-organization and inter-study standardization, optimized checklists for safety critical tasks, and use of simulation for team training and exploration of work systems.

CONCLUSIONS

This pioneering application of human factors techniques to analyse work systems during the conduct of research in a CRF revealed risks unidentified by routine review and appraisal, and despite international guideline adherence. SEIPS may aid categorization of system problems and the formulation of recommendations that reduce risk and mitigate potential harm applicable across a trials portfolio.

Procedural simulation in venipuncture for medical undergraduates and its transfer to the bedside: a cluster randomized study

Simulation is accepted as an effective method of learning procedural skills. However, the translational outcomes of skills acquired through simulation still warrants investigation. We designed this study to assess if skills laboratory training in addition to bedside learning (intervention group [IG]) would provide better learning results than bedside learning alone (control group [CG]) in the context of venipuncture training. This prospective, cluster-randomized, single-blind study took place at the Faculty of Medicine, University of Kelaniya, Sri Lanka. Seventeen clusters of second-year medical students were randomly assigned to either IG or CG. The IG trained on venipuncture in the skills laboratory, receiving instruction after modified Payton's Four Step Method. Following the training, students of both IG and CG underwent bedside learning for one month. Afterward, students of both groups performed venipuncture on actual patients in a clinical setting. An independent, blinded assessor scored students' performance using the Integrated Procedural Protocol Instrument (IPPI) and a checklist. Patients assessed students' performance with the Communication Assessment Tool (CAT). Eight and nine clusters were randomized to the intervention and control groups, respectively. IG completed significantly more single steps of the procedure correctly (IG: 19.36 ± 3.87 for checklist items; CG: 15.57 ± 4.95; p < 0.001). IG also scored significantly better on IPPI ratings (median: IG: 27 (12) vs. CG: 21 (8); p < 0.001). Rated by patients, students' communication skills did not significantly differ between the two groups. Simulation-based venipuncture training enabled students to perform the procedure on actual patients with a higher technical accuracy than students who learned venipuncture at the bedside. Students were able to transfer the skills acquired through venipuncture training at the skill laboratory to the bedside.

Simulation as an educational tool to teach emergency medicine residents about unconscious bias

Medical training embraces simulation-based education. One important topic that has recently been added to the simulation curriculum at the University of Toronto is unconscious bias. This educational innovation project evaluates a simulation that could be used as a novel instructional design strategy to teach unconscious bias. The simulation involved two resuscitation scenarios with a similar clinical trajectory. Each resuscitation was led by standardized physicians of different binary genders, followed by a debrief with highly trained facilitators. Tangible educational takeaways on team dynamic variation between different gendered team leaders were discussed following the simulation, highlighting its benefit to resident education on bias awareness and response. Limitations include inevitable unconscious bias in facilitators and unequal gender representation in the learner participants, which may impact simulation effectiveness. The findings support translating this simulation to other forms of bias education in future simulation development.

Criteria to evaluate graduate nurse proficiencies in obtaining a health history and perform physical assessment in simulation-based education: A narrative review

BACKGROUND

Simulation is a technique being used increasingly in healthcare education which offers opportunities to evaluate nursing proficiencies. The use of valid and reliable instruments is recognised as the foundation for a robust assessment, however competency-based health assessment courses for graduate nurses can consequently become reductionist in measuring proficiencies.

OBJECTIVE

The specific review question was: In simulation-based education, what are the criteria that evaluate graduate nursing student's competence in obtaining a health history and performance of patient assessment?

METHODS

Eleven studies were included in the review. Papers were critically appraised with The Joanna Briggs Institute quasi-experimental studies checklist. Bloom's taxonomy was used to structure this narrative review.

RESULTS

Seven papers evaluated cognition through questionnaires and two papers used a Likert-scale to determine self-perceived knowledge. Six papers evaluated psychomotor skills with a behavioural checklist. Diversity of application was factored into the studies when testing affective skills. Three papers used Likert-scales to evaluate preparedness, six papers used Likert-scales to evaluate self-confidence and one used a Likert-scale to evaluate autonomy. Three papers used a checklist to evaluate professionalism. Four papers used faculty member/ standardised patient feedback.

CONCLUSION

Reductionist evaluation instruments create a barrier when evaluating competency. The limited validity and reliability of assessment instruments in simulation, as well as the lack of standardisation of affective skills assessment, presents a challenge in simulation research. Affective skills encompass attitudes, behaviours and communication abilities, which pose a significant challenge for standardised assessments due to their subjective nature. This review of the simulation literature highlights a lack of robustness in the evaluation of the affective domain. This paper proposes that simulation assessment instruments should include the standardisation of affective domain proficiencies such as: adaptation to patients' cognitive function, ability to interpret and synthesise relevant information, ability to demonstrate clinical judgement, readiness to act, recognition of professional limitations and faculty/standardised-simulated patient feedback. The incorporation of the affective domain in standardised assessment instruments is important to ensure comprehensive assessment of simulation particularly in the development of health history and physical assessment proficiencies. Attention to all of the domains in Blooms taxonomy during simulation assessment has the potential to better prepare professionals for the patient care setting.

The Rapid and International Expansion of Veterinary Clinical Skills Laboratories: A Survey to Establish Recent Developments

Veterinary clinical skills laboratories are used for teaching a wide range of practical, clinical, and surgical skills on models and simulators. A survey conducted in 2015 identified the role of such facilities in veterinary education in North America and Europe. The current study aimed to capture recent changes using a similar survey with three sections to collect data about the structure of the facility, its uses in teaching and assessment, and the staffing. The survey consisted of multiple choice and free text questions, was administered online using Qualtrics and was disseminated in 2021 via clinical skills networks and Associate Deans. Responses were received from 91 veterinary colleges in 34 countries; 68 had an existing clinical skills laboratory and 23 were planning to open one within 1-2 years. Collated information from the quantitative data described the facility, teaching, assessment, and staffing. Major themes emerged from the qualitative data relating to aspects of the layout, location, integration in the curriculum, contributions to student learning, and the team managing and supporting the facility. Challenges were associated with budgeting, the ongoing need for expansion and leadership of the program. In summary, veterinary clinical skills laboratories are increasingly common around the world and the contributions to student learning and animal welfare were well recognized. The information about existing and planned laboratories and the tips from those managing the facilities provides valuable guidance for anyone intending to open or expand an existing clinical skills laboratory.

A view from the top: A qualitative exploration of top-level health care leaders' perceptions on the implementation of simulation-based education in postgraduate medical education

INTRODUCTION

Although there is substantial evidence supporting the benefits of simulation-based education (SBE), its widespread and effective implementation remains challenging. The aim of this study was to explore the perceptions of top-level health care leaders regarding SBE and the barriers and facilitators that influence its wide implementation as part of the postgraduate surgical curricula in Denmark.

METHODS

We conducted semi-structured interviews with top-level health care leaders who were chosen based on their roles in ensuring high-quality patient care delivery and developing strategies to achieve the goals of the entire health care system. The interview transcripts were translated into English, and a thematic approach was used to code and inductively analyse the data. We used the Consolidated Framework for Implementation Research to identify and understand the determinants to a successful implementation of SBE.

RESULTS

We interviewed 13 participants from different political and administrative levels. We found that the participants had limited knowledge about SBE, which highlighted a disconnection between these leaders and the educational environment. This was further compounded by a lack of effective communication and inadequate information dissemination between simulation centres and higher-level organisations. While participants recognised the benefits of SBE for doctors in training, they expressed concerns about the implementation given the already strained health care system and limited resources. The need for evidence, particularly in the context of patient safety, was emphasised to facilitate SBE implementation. Although participants supported the implementation of SBE, it was unclear who should initiate action.

DISCUSSION

This study highlighted the perspectives of top-level health care leaders regarding SBE and identified the determinant factors for a successful implementation. Effective communication channels are crucial to enhance collaborations and reduce the disconnection between the different health care organisational levels. Strategic implementation processes, including the roles and responsibilities, should be defined and established. These will inform decisions regarding the implementation strategies to effectively integrate SBE into the residency training curricula.

Massive Hemoptysis Simulation Curriculum Improves Performance

BACKGROUND

Massive hemoptysis is a rare, high-acuity presentation, which requires the integration of both cognitive and procedural skills. Simulation has been recommended to improve preparation for high-acuity, low-occurrence procedures; however, the effect of a simulation curriculum for massive hemoptysis management has never been investigated.

RESEARCH QUESTION

Does simulation for hemoptysis management improve competence?

STUDY DESIGN AND METHODS

Kern's six steps for medical education curriculum design were used iteratively to develop a simulation curriculum for the management of massive hemoptysis. Pulmonary and critical care medicine fellows from the University of Colorado participated in a local needs assessment and a massive hemoptysis simulation curriculum. Using a manikin-based massive hemoptysis simulator developed for this curriculum, the simulation session used repetitive practice, clinical variation, a range of difficulties, and directed feedback in a group practice setting. Time to management and performance were assessed for each management attempt; competence was assessed using a combined metric of management-related priorities and global entrustment.

RESULTS

During the needs assessment, fellows viewed massive hemoptysis management skills as important, while expressing their current confidence as low. Nineteen fellows participated in a 90-min case-based hemoptysis simulation during which each was exposed to five different cases and acted as the primary manager for two cases. There was significant improvement in performance from the first to final simulation attempts measured by time to successful management (14.24 vs 10.26 min, P = .0067) and entrustment (Global Assessment Scale, 1 [should not perform] to 5 [independent]; 4.11 vs 4.61; P = .015). Fellow self-assessed knowledge and confidence in hemoptysis management and endobronchial blocker placement improved significantly after the simulation.

INTERPRETATION

Hemoptysis simulation experience improves fellow confidence and skill for management of this high-acuity, low-occurrence presentation.

Age and its impact on crisis management performance and learning after simulation-based education by acute care physicians: a multicentre prospective cohort study

BACKGROUND

Physiological changes associated with ageing could negatively impact the crisis resource management skills of acute care physicians. This study was designed to determine whether physician age impacts crisis resource management skills, and crisis resource management skills learning and retention using full-body manikin simulation training in acute care physicians.

METHODS

Acute care physicians at two Canadian universities participated in three 8-min simulated crisis (pulseless electrical activity) scenarios. An initial crisis scenario (pre-test) was followed by debriefing with a trained facilitator and a second crisis scenario (immediate post-test). Participants returned for a third crisis scenario 3-6 months later (retention post-test).

RESULTS

For the 48 participants included in the final analysis, age negatively correlated with baseline Global Rating Scale (GRS; r=-0.30, P<0.05) and technical checklist scores (r=-0.44, P<0.01). However, only years in practice and prior simulation experience, but not age, were significant in a subsequent stepwise regression analysis. Learning from simulation-based education was shown with a mean difference in scores from pre-test to immediate post-test of 2.28 for GRS score (P<0.001) and 1.69 for technical checklist correct score (P<0.001); learning was retained for 3-6 months. Only prior simulation experience was significantly correlated with a decreased change in learning (r=-0.30, P<0.05).

CONCLUSIONS

A reduced amount of prior simulation training and increased years in practice, but not age on its own, were significant predictors of low baseline crisis resource management performance. Simulation-based education leads to crisis resource management learning that is well retained for 3-6 months, regardless of age or years in practice.

Simulation-based assessment of upper abdominal ultrasound skills

BACKGROUND

Ultrasound is a safe and effective diagnostic tool used within several specialties. However, the quality of ultrasound scans relies on sufficiently skilled clinician operators. The aim of this study was to explore the validity of automated assessments of upper abdominal ultrasound skills using an ultrasound simulator.

METHODS

Twenty five novices and five experts were recruited, all of whom completed an assessment program for the evaluation of upper abdominal ultrasound skills on a virtual reality simulator. The program included five modules that assessed different organ systems using automated simulator metrics. We used Messick's framework to explore the validity evidence of these simulator metrics to determine the contents of a final simulator test. We used the contrasting groups method to establish a pass/fail level for the final simulator test.

RESULTS

Thirty seven out of 60 metrics were able to discriminate between novices and experts (p < 0.05). The median simulator score of the final simulator test including the metrics with validity evidence was 26.68% (range: 8.1-40.5%) for novices and 85.1% (range: 56.8-91.9%) for experts. The internal structure was assessed by Cronbach alpha (0.93) and intraclass correlation coefficient (0.89). The pass/fail level was determined to be 50.9%. This pass/fail criterion found no passing novices or failing experts.

CONCLUSIONS

This study collected validity evidence for simulation-based assessment of upper abdominal ultrasound examinations, which is the first step toward competency-based training. Future studies may examine how competency-based training in the simulated setting translates into improvements in clinical performances.

'Gear up and get ready': Collaborative curriculum for radiographers supporting percutaneous nephrolithotomy in the operating theatre

INTRODUCTION

Newly qualified radiographers often find working in the operating theatre (OT) challenging and intimidating. These perceptions, which inhibit confidence, may hinder their effectiveness in interprofessional teamwork, which may in turn adversely affect patient outcomes. A collaborative education programme was designed, building upon the foundations of competency-based education (CBE) and simulation-based mastery learning (SBML) to examine its potential in mitigating these perceptions. The objective of this research was to assess participants' experience and level of competency after attending the curated collaborative educational programme.

METHODS

The programme was developed based on the Analysis, Design, Development, Implementation, and Evaluation (ADDIE) model and comprises two teaching and learning phases: educational session and simulation. A collaborative approach was undertaken to develop an assessment checklist for the interprofessional simulation. Requirements for the simulation, such as scenario design, information and storyboard, task trainer, logistics, and learners' briefing, debrief, and feedback, were identified and assembled. The radiographers' performance was recorded using a practical skills assessment checklist and a theory assessment.

RESULTS

Twelve radiographers participated and showed improvement in their self-rating of learning objectives before and after the programme. The median (interquartile range) score achieved in the theory assessment, out of a possible of 11, was 9.00 (7.75-9.50). The median (interquartile range) score achieved in the simulation component, out of a possible of 16, was 15.00 (14.00-15.00). There was statistically significant difference in self-perceived performance in all learning objective domains.

CONCLUSION

The findings from the programme were promising. The use of simulation and an assessment checklist proved to be useful learning tools in preparing newly qualified radiographers for work in the OT.

IMPLICATIONS FOR PRACTICE

Assessment checklists are valuable tools that should be considered to facilitate teaching and learning. The use of interprofessional simulation activities can support radiographers in developing knowledge, professional skills, and clinical competency. It should be conducted in a timely manner to facilitate the introduction to role understanding and effective communication.

Powerful medical education improves health care quality and return on investment

INTRODUCTION

Powerful medical education (PME) involves the use of new technologies informed by the science of expertise that are embedded in laboratories and organizations that value evidence-based education and support innovation. This contrasts with traditional medical education that relies on a dated apprenticeship model that yields uneven results. PME involves an amalgam of features, conditions and assumptions, and contextual variables that comprise an approach to developing clinical competence grounded in education impact metrics including efficiency and cost-effectiveness.

METHODS

This article is a narrative review based on SANRA criteria and informed by realist review principles. The review addresses the PME model with an emphasis on mastery learning and deliberate practice principles drawn from the new science of expertise. Pub Med, Scopus, and Web of Science search terms include medical education, the science of expertise, mastery learning, translational outcomes, cost effectiveness, and return on investment. Literature coverage is comprehensive with selective citations.

RESULTS

PME is described as an integrated set of twelve features embedded in a group of seven conditions and assumptions and four context variables. PME is illustrated via case examples that demonstrate improved ventilator patient management learning outcomes compared to traditional clinical education and mastery learning of breaking bad news communication skills. Evidence also shows that PME of physicians and other health care providers can have translational, downstream effects on patient care practices, patient outcomes, and return on investment. Several translational health care quality improvements that derive from PME include reduced infections; better communication among physicians, patients, and families; exceptional birth outcomes; more effective patient education; and return on investment.

CONCLUSIONS

The article concludes with challenges to hospitals, health systems, and medical education organizations that are responsible for producing physicians who are expected to deliver safe, effective, and cost-conscious health care.

The Impact of Just-in-Time Simulation Training for Healthcare Professionals on Learning and Performance Outcomes: A Systematic Review

ABSTRACT

Although just-in-time training (JIT) is increasingly used in simulation-based health professions education, its impact on learning, performance, and patient outcomes remains uncertain. The aim of this study was to determine whether JIT simulation training leads to improved learning and performance outcomes. We included randomized or nonrandomized interventional studies assessing the impact of JIT simulation training (training conducted in temporal or spatial proximity to performance) on learning outcomes among health professionals (trainees or practitioners). Of 4077 citations screened, 28 studies were eligible for inclusion. Just-in-time training simulation training has been evaluated for a variety of medical, resuscitation, and surgical procedures. Most JIT simulation training occurred immediately before procedures and lasted between 5 and 30 minutes. Despite the very low certainty of evidence, this systematic review suggests JIT simulation training can improve learning and performance outcomes, in particular time to complete skills. There remains limited data on better patient outcomes and collateral educational effects.

Neurosurgical simulation models developed in Latin America and the Caribbean: a scoping review

Simulation training is an educational tool that provides technical and cognitive proficiency in a risk-free environment. Several models have recently been presented in Latin America and the Caribbean (LAC). However, many of them were presented in non-indexed literature and not included in international reviews. This scoping review aims to describe the simulation models developed in LAC for neurosurgery training. Specifically, it focuses on assessing the models developed in LAC, the simulated neurosurgical procedures, the model's manufacturing costs, and the translational outcomes. Simulation models developed in LAC were considered, with no language or time restriction. Cadaveric, ex vivo, animal, synthetic, and virtual/augmented reality models were included for cranial and spinal procedures. We conducted a review according to the PRISMA-ScR, including international and regional reports from indexed and non-indexed literature. Two independent reviewers screened articles. Conflicts were resolved by a third reviewer using Covidence software. We collected data regarding the country of origin, recreated procedure, type of model, model validity, and manufacturing costs. Upon screening 917 studies, 69 models were developed in LAC. Most of them were developed in Brazil (49.28%). The most common procedures were related to general neurosurgery (20.29%), spine (17.39%), and ventricular neuroendoscopy and cerebrovascular (15.94% both). Synthetic models were the most frequent ones (38.98%). The manufacturing cost ranged from 4.00 to 2005.00 US Dollars. To our knowledge, this is the first scoping review about simulation models in LAC, setting the basis for future research studies. It depicts an increasing number of simulation models in the region, allowing a wide range of neurosurgical training in a resource-limited setting.

Patient and Technical Factors Associated with Difficult Arterial Access and Ultrasound Use in the Operating Room

Arterial catheterization enables continuous hemodynamic monitoring but has been shown to cause severe complications, especially when multiple attempts are required. The aim of this study was to explore what factors were associated with multiple attempts and ultrasound use in the operating room. We performed a retrospective analysis of patients who had arterial catheters inserted at a tertiary care children's hospital from January 2018 to March 2022, identifying clinical factors that were associated with both outcomes. A total of 3946 successful arterial catheter insertions were included. Multivariable analysis showed multiple attempts were associated with noncardiac surgery: pediatric (OR: 1.79, 95% CI: 1.30-2.51), neurologic (OR: 2.63, 95% CI: 1.89-3.57), orthopedic (OR: 3.23, 95% CI: 2.27-4.55), and non-radial artery placement (OR: 5.00, 95% CI: 3.33-7.14) (all p < 0.001). Multivariable analysis showed ultrasound use was associated with neonates (OR: 9.6, 95% CI: 4.1-22.5), infants (OR: 6.98, 95% CI: 4.67-10.42), toddlers (OR: 6.10, 95% CI: 3.8-9.8), and children (OR: 2.0, 95% CI: 1.7-2.5) compared to teenagers, with cardiac surgery being relative to other specialties-pediatric (OR: 0.48, 95% CI: 0.3-0.7), neurologic (OR: 0.27, 95% CI: 0.18-0.40), and orthopedic (OR: 0.38, 95% CI: 0.25-0.58) (all p < 0.001). In our exploratory analysis, increased odds of first-attempt arterial catheter insertion success were associated with cardiac surgery, palpation technique, and radial artery placement. Younger patient age category, ASA III and IV status, cardiac surgery, and anesthesiologist placement were associated with increased odds of ultrasound use.

The "STARS" study: advanced preoperative rehearsal and intraoperative navigation in neurosurgical oncology

BACKGROUND

Neurosurgical 3D visualizers and simulators are innovative devices capable of defining a surgical strategy in advance and possibly making neurosurgery safer by rehearsing the phases of the operation beforehand. The aim of this study is to evaluate Surgical Theater™ (Surgical Theater LLC, Mayfield, OH, USA), a new 3D neurosurgical planning, simulation, and navigation system, and qualitatively assess its use in the operating room.

METHODS

Clinical data were collected from 30 patients harboring various types of brain tumors; Surgical Theater™ was used for the preoperative planning and intraoperative 3D navigation. Preoperative and postoperative questionnaires were completed by first and second operators to get qualitative feedback on the system's functionality. Furthermore, we measured and compared the impact of this technology on surgery duration.

RESULTS

Neurosurgeons were overall satisfied when using this rehearsal and navigation tool and found it efficient and easy to use; interestingly, residents considered this device more useful as compared to their more senior colleagues (with significantly higher scores, P<0.05), possibly because of their limited anatomical experience and spatial/surgical rehearsal ability. The length of the surgical procedure was not affected by this technology (P>0.05).

CONCLUSIONS

Surgical Theater™ system was found to be clinically useful in improving anatomical understanding, surgical planning, and intraoperative navigation, especially for younger and less experienced neurosurgeons.

Development of a Rubric to Evaluate Implementation Quality of Simulation-Based Courses: A Consensus Study

INTRODUCTION

Simulation-based education is a recognized way of developing medical competencies, and there is overwhelming scientific evidence to support its efficacy. However, it is still underused, which can often be related to poor implementation process. In addition, best practices for implementation of simulation-based courses based on implementation science are not widely known nor applied. The purpose of this study was to develop a rubric, the Implementation Quality Rubric for Simulation (IQR-SIM), to evaluate the implementation quality of simulation-based courses.

METHODS

A 3-round, modified Delphi process involving international simulation and implementation experts was initiated to gather and converge opinions regarding criteria for evaluating the implementation quality of simulation-based courses. Candidate items for Round 1 were developed based on the Adapted Implementation Model for Simulation. Items were revised and expanded to include descriptive anchors for evaluation in Round 2. Criterion for inclusion was 70% of respondents selecting an importance rating of 4 or 5/5. Round 3 provided refinement and final approval of items and anchors.

RESULTS

Thirty-three experts from 9 countries participated. The initial rubric of 32 items was reduced to 18 items after 3 Delphi rounds, resulting in the IQR-SIM: a 3-point rating scale, with nonscored options "Don't know/can't assess" and "Not applicable," and a comments section.

CONCLUSIONS

The IQR-SIM is an operational tool that can be used to evaluate the implementation quality of simulation-based courses and aid in the implementation process to identify gaps, monitor the process, and promote the achievement of desired implementation and learning outcomes.