Linking simulation-based educational assessments and patient-related outcomes: a systematic review and meta-analysis

Building Global Partnerships: A Qualitative Exploration of In-Person Training for HealthCare Simulation Educators Working in Low-Resource Settings

INTRODUCTION

Effective educational initiatives can elevate the quality of patient care globally. Simulation-based education is widely used in high-resource settings, but barriers exist to its widespread use in low-resource settings. Vital Anesthesia Simulation Training (VAST) overcomes these barriers by offering immersive, low-cost portable simulation along with simulation facilitator training. During the COVID-19 pandemic, in-person courses were stopped for more than 2 years. Postpandemic, a 3-day VAST SIMposium was hosted in Rwanda to unite 42 VAST facilitators from 12 widespread countries to introduce new and revised course materials and to rejuvenate dormant skills. The purpose of this study was to explore how the VAST SIMposium influenced perceived development of skills, confidence, and engagement in a community of practice for simulation educators working in low-resource settings.

METHODS

This qualitative study involved in-depth interviews with a purposive sample of 16 VAST SIMposium attendees. Transcripts of audio recordings were analyzed using inductive thematic analysis.

RESULTS

Six themes were identified during data analysis: 1) Diversity with shared passion for medical education; 2) Supportive in-person learning environment; 3) Simulation-based education relevant to low-resource settings; 4) Camaraderie; 5) Building knowledge, skills, and confidence; and 6) Being part of a community of practice.

CONCLUSION

The SIMposium rejuvenated passion, knowledge, skills, and, most importantly, strengthened global connections and partnerships. These collaborations will benefit areas that are underrepresented in simulation and will ultimately improve patient outcomes. A SIMposium of this format is an efficient and effective way to foster sustainable global dissemination of simulation-based global health education.

Effect of a competence based medical education program on training quality in Intensive Care Medicine. COBALIDATION TRIAL

OBJECTIVES

The main objective of this study was to evaluate whether the implementation of CoBaTrICE (Competency-Based Training in Intensive Care Medicine in Europe) provides higher levels of competency in comparison with the current official time-based program in Intensive Care Medicine in Spain. Secondary objectives were: 1) To determine the percentage of critical essential performance elements (CEPE) accomplished, 2) To determine compliance with workplace-based assessments (wba).

DESIGN

Multicenter cluster randomized trial.

SETTING

Thirteen Spanish ICU Departments.

PARTICIPANTS

Thirty-six residents INTERVENTION: The implementation of CoBaTrICE included: (1) Training the trainers; (2) Wba; (3) The use of an electronic portfolio. The level of competency achieved by each participant was determined by a simulation-based Objective Structured Clinical Exam (OSCE) performed at the end of the 5th year of training period.

MAIN VARIABLES OF INTEREST

Total scoring in the five scenarios, CEPE completed, level of competency (1-5) achieved.

RESULTS

A total of 119 performances from 26 residents (17 from CoBaTrICE group and 9 from control group) were analyzed in the OSCE. CoBaTrICE residents´ achieved higher levels of competency [2 (1-5) vs. 2 (1-3), p = 0.07) and higher percentages of CEPE´s accomplishment than the control group (78% vs. 71%, p = 0.09).

CONCLUSIONS

The CoBaTrICE group showed a better performance trend in comparison to the control group, but the differences were not statistically significant. Since the number of Wba performed was low, additional research is needed to determine the potential superiority of CoBaTrICE.

Are serious games seriously good at preparing students for clinical practice?: A randomized controlled trial

PURPOSE

Serious games (SGs) have great potential for pediatric medical education. This study evaluated the efficacy of a SG in improving learner satisfaction, knowledge, and behavior.

MATERIALS AND METHODS

This was an investigator-blinded randomized controlled trial (RCT) comparing a SG against two controls: (i) adaptive tutorial (AT), and (ii) low-stimulus control (LSC). SG is a highly immersive role-playing game in a virtual hospital. AT delivers interactive web-based lessons. LSC is paper-based clinical practice guidelines. Metropolitan senior medical students at UNSW were eligible. A total of 154 enrolled and were block randomized to one intervention. Participants had access to one intervention for 8 weeks which taught pediatric acute asthma and seizure assessment and management. Satisfaction was assessed with Likert-scale responses to 5 statements and 2 free-text comments. Knowledge was assessed with 10 multiple-choice questions (MCQs). Clinical behavior was assessed during a 30-point simulated clinical management scenario (CMS). Primary analysis was performed on a modified intention-to-treat basis and compared: (1) SG vs. AT; and (2) SG vs. LSC.

RESULTS

A total of 118 participants were included in the primary analysis (modified intention-to-treat model). No significant differences in MCQ results between the SG and control groups. SG group outperformed the LSC group in the CMS, with a moderate effect (score out of 30: 20.8 (3.2) vs. 18.7 (3.2), respectively, d = 0.65 (0.2-1.1), p = 0.005). No statistically significant difference between SG and AT groups in the CMS (score: 20.8 (3.2) vs. 19.8 (3.1), respectively, d = 0.31 (-0.1 to 0.8), p = 0.18). A sensitivity analysis (per-protocol model) was performed with similar outcomes.

CONCLUSIONS

This is the first investigator-blinded RCT assessing the efficacy of a highly immersive SG on learner attitudes, knowledge acquisition, and performance in simulated pediatric clinical scenarios. The SG demonstrated improved translation of knowledge to a simulated clinical environment, particularly compared to LSC. SGs show promise in pediatric medical education.

Video Feedback Improves Anesthesia Residents' Communication Skill and Performance on Showing Empathy During Preoperative Interviews

OBJECTIVES

To determine the impact of scenario-based lecture and personalized video feedback on anesthesia residents' communication skills during preoperative visits.

METHODS

A total of 24 anesthesia residents were randomly divided into a video group and a control group. Residents in both groups took part in a simulated interview and received a scenario-based lecture on how to communicate with patients during preoperative visits. Afterwards, residents in the video group received personalized video feedback recorded during the simulated interview. One week later all the residents undertook another simulated interview. The communication skills of all the residents were assessed using the Consultation and Relational Empathy measure (CARE) scale by two examiners and one standardized patient (SP), both of whom were blinded to the group allocation.

RESULTS

CARE scores were comparable between the two groups before training, and significantly improved after training in both groups (all P < 0.05). The video group showed significantly greater increase in CARE score after the training than the control group, especially assessed by the SP (t = 6.980, P <0.001). There were significant correlations between the examiner-assessed scores and SP-assessed scores (both P = 0.001).

CONCLUSIONS

Scenario-based lectures with simulated interviews provide a good method for training communication skills of anesthesia residents, and personalized video feedback can enhance their performance on showing empathy during preoperative interview.

PEARLS debriefing for social justice and equity: integrating health advocacy in simulation-based education

Addressing health inequities in health professions education is essential for preparing healthcare workers to meet the demands of diverse communities. While simulation has become a widely recognized and effective method for providing safe and authentic clinical learning experiences, there has been limited attention towards the power of simulation in preparing health practitioners to work with groups who experience health disparities due to systems of inequality. Balancing technical proficiency with educational approaches that foster critical reflection and inform action oriented towards social accountability is essential. Transformational learning promotes the development of critical consciousness through critical reflection. Debriefing plays a crucial role in fostering learning in this direction by providing a structured opportunity to critically reflect on taken for granted assumptions, examine power and privilege embedded within systems and structures, and empower learners to take action toward changing those conditions. Building on the Promoting Excellence and Reflective Learning in Simulation (PEARLS) Healthcare Debriefing Tool, we propose a PEARLS Debriefing for Social Justice and Equity (DSJE) Tool that specifically directs attention to systems of inequality that contribute to health disparities for vulnerable groups across a range of simulation scenarios. This approach has two aims: (a) to transform debriefings into a critically reflective space by engaging learners in dialogue about social and structural determinants of health that may create or perpetuate inequities and (b) to foster critical reflection on what actions can be taken to improve the health and well-being of identified at risk and vulnerable groups. From this perspective, we can use the adapted PEARLS Tool to incorporate conversations about systems of inequality, equity, diversity, and inclusion (EDI) into our existing educational practices, and make concentrated efforts towards community-driven and socially conscious simulation-based education (SBE).

Teaching Procedural Skills: Integrating Educational Theory Into Practice

Introduction

Teaching procedural skills is an essential part of health professions education, yet formal training is often lacking from traditional curricula.

Methods

A workshop on teaching procedural skills was developed as part of a clinician educator track at a large health professions university. Participants included medical residents and fellows (postgraduate years 2-6) from various training programs. The 90-minute, interactive training integrated Gagne's model of instructional design with evidence-based teaching practices. Workshop outcomes were evaluated with pre- and postworkshop surveys. Learner reactions (Kirkpatrick level 1) were assessed via course evaluation. Learning and behavior (Kirkpatrick level 2) were evaluated via attitudinal and knowledge-based questions. Data were collected from three cohorts of participants (2022-2024). Responses were analyzed by paired t test.

Results

Residents and fellows from 11 different disciplines participated in the workshops. The survey response rate was 30 out of 35 (86%). Course evaluations were positive for all six questions (mean scores: 4.8-5.0 on a 5-point Likert scale [1 = strongly disagree, 5 = strongly agree]). Postworkshop scores improved significantly for all five attitudinal questions, including "I can apply instructional design theory when teaching procedural skills" (pre: 2.2 vs. post: 4.3 on a 5-point scale, p < .01). The number of participants correctly answering the knowledge-based questions also increased following the workshop.

Discussion

A workshop focused on evidence-based teaching of procedural skills was well reviewed and improved participants' attitudes and knowledge. Strengths of the workshop include its appeal to a broad range of medical trainees, integration of educational theory, and interactive design.

Simulation in podiatry teaching and learning: A scoping review

BACKGROUND

In podiatry, there are a variety of clinical tasks that require precision and skill and it is expected that clinicians will obtain these skills during their training. Simulation is a dynamic teaching tool used in healthcare to enhance skill and knowledge acquisition. Currently, the extent and nature of the research on the use of simulation in podiatry teaching and learning are not clear.

AIM

A scoping review was conducted to identify the extent and nature of research activity on the use of simulation in podiatry teaching and learning and identify gaps in the existing literature.

METHODS

Any research relating to simulation use in podiatry teaching including various designs and focusing on simulations aimed at improving podiatry teaching or learning were eligible for inclusion. A systematic search was conducted on February 14, 2024 of the following databases: Embase (via Embase.com), MEDLINE (via PubMed), CINAHL, and the Web of Science. Additional papers were identified via bibliographies of included studies. Content analysis of content relating to podiatry teaching and learning was performed and grouped into broad themes, then further narrowing to six themes.

RESULTS

A total of 21 research studies were deemed eligible for inclusion focusing on diverse aspects of podiatry simulation utilized in high-income countries exclusively. Conducted between 1997 and 2023, these studies were categorized into six key themes: skill improvement, communication and professionalism, clinical competencies and patient safety, educational enhancement, and anatomy and histology education. The simulations, carried out by or assessed for podiatry professionals, staff, or students, ranged from high-fidelity medical mannequins to low-fidelity simulations such as a grapefruit model of a diabetes-related foot ulcer.

CONCLUSION

Overall, the findings suggest that simulation teaching in podiatry, whether through direct skill enhancement or through educational impact assessments, holds potential in improving competency, confidence, and educational outcomes in podiatry practice. This scoping review identified a small yet diverse evidence base for simulation modalities in podiatry education, demonstrating gaps in long-term effects and comparative effectiveness studies. It highlights the urgent need for research focused on longitudinal impacts, evaluating various simulation technologies and standardizing best practices to improve podiatry education and align with clinical and patient care needs.

Simulation-based medical education in Canadian anesthesiology academic institutions: a national survey

PURPOSE

Simulation-based medical education (SBME) is provided by all anesthesiology residency programs in Canada. The purpose of this study was to characterize SBME in Canadian anesthesiology residency training programs.

METHODS

We administered a 21-question survey to the simulation director/coordinator for all 17 Canadian academic departments of anesthesiology from October 2019 to January 2020. The survey consisted of questions pertaining to the characteristics of the simulation centres, their faculty, learners, curriculum, and assessment processes.

RESULTS

All 17 residency training programs participated in the survey and reported large variability in the number and formal training of simulation faculty and in content delivery. Five programs (29%) did not provide faculty recognition for curriculum design and running simulation sessions. Most programs offered one to four simulation sessions per academic year for each year of residency. All programs offered mannequin-based and part-task trainers for teaching technical and nontechnical skills. Fourteen programs (82%) offered interprofessional and interdisciplinary simulation sessions, and ten programs (59%) did not include in situ simulation training. Commonly reported barriers to faculty involvement were lack of protected time (12 programs, 71%), lack of financial compensation (ten programs, 59%), and lack of appreciation for SBME (seven programs, 41%).

CONCLUSION

Large variability exists in the delivery of SBME in Canadian anesthesiology residency simulation programs, in part because of differences in financial/human resources and educational content. Future studies should explore whether training and patient outcomes differ between SBME programs and, if so, whether additional standardization is warranted.

Assessing Healthcare Simulation Facilitation: A Scoping Review of Available Tools, Validity Evidence, and Context Suitability for Faculty Development in Low-Resource Settings

SUMMARY STATEMENT

Assessment tools support simulation facilitation skill development by guiding practice, structuring feedback, and promoting reflective learning among educators. This scoping review followed a systematic process to identify facilitation assessment tools used in postlicensure healthcare simulation. Secondary objectives included mapping of the validity evidence to support their use and a critical appraisal of their suitability for simulation faculty development in low-resource settings. Database searching, gray literature searching, and stakeholder engagement identified 11,568 sources for screening, of which 72 met criteria for full text review. Thirty sources met inclusion; 16 unique tools were identified. Tools exclusively originated from simulation practice in high-resource settings and predominantly focused on debriefing. Many tools have limited validity evidence supporting their use. In particular, the validity evidence supporting the extrapolation and implications of assessment is lacking. No current tool has high context suitability for use in low-resource settings.

The impact of surgical simulation and training technologies on general surgery education

The landscape of general surgery education has undergone a significant transformation over the past few years, driven in large part by the advent of surgical simulation and training technologies. These innovative tools have revolutionized the way surgeons are trained, allowing for a more immersive, interactive, and effective learning experience. In this review, we will explore the impact of surgical simulation and training technologies on general surgery education, highlighting their benefits, challenges, and future directions. Enhancing the technical proficiency of surgical residents is one of the main benefits of surgical simulation and training technologies. By providing a realistic and controlled environment, With the use of simulations, residents may hone their surgical skills without compromising patient safety. Research has consistently demonstrated that training with simulations enhances surgical skills., reduces errors, and enhances overall performance. Furthermore, simulators can be programmed to mimic a wide range of surgical scenarios, enabling residents to cultivate the essential critical thinking and decision-making abilities required to manage intricate surgical cases. Another area of development is incorporating simulation-based training into the wider surgical curriculum. As simulation technologies become more widespread, they will need to be incorporated into the fabric of surgical education, rather than simply serving as an adjunct to traditional training methods. This will require a fundamental shift in the way surgical education is delivered, with a greater emphasis on simulation-based training and assessment.

Extrapolative Validity Evidence of the Anastomosis Objective Structured Assessment of Technical Skill (A-OSATS) for Robotic Ileocolic Anastomosis

OBJECTIVE

To collect validity evidence for the use of the Anastomosis Objective Structured Assessment of Technical Skills (A-OSATS) instrument, which has been developed to evaluate performance of a minimally invasive side-to-side bowel anastomosis with hand-sewn common enterotomy.

DESIGN

Residents performed a robotic ileocolic anastomosis simulation on an ex vivo porcine model. Faculty scored each resident with the A-OSATS and performed a provocative leak test on the completed anastomoses. Residents were reassessed on the sewing sub-score 1 month later. Data were compared with parametric and nonparametric analysis.

SETTING

Single academic general surgery residency PARTICIPANTS: PGY-4 and -5 general surgery residents (n = 17) RESULTS: PGY-5s performed better than PGY-4s in repeat A-OSATS sewing sub-score (mean 55/55 ± 0 vs 43 ± 4.9, p < 0.001) and time to complete (minutes, mean 14.5 ± 4.9 vs 21.2 ± 3.9, p = 0.01). There was a strong correlation between A-OSATS score and time (r = -0.67, p = 0.005). For the initial assessment, there was no significant difference in mean A-OSATS score between anastomoses that leaked and those that did not leak (137.3 ± 14.5 vs 150.1 ± 11.2, p = 0.098), but on repeat assessment, intact anastomoses had a higher mean A-OSATS sewing sub-score than those that leaked (52.2 ± 4.7 vs 39 ± 3.5, p = 0.007). There was no significant difference between initial A-OSATS score and repeat score (p = 0.14).

CONCLUSIONS

We provide extrapolative validity evidence for the A-OSATS instrument by comparing A-OSATS score to time to sew, provocative leak test, and discrimination between PGY-4s and PGY-5s. Generalizability validity evidence is provided by test-retest reliability. Further refinement is needed for the A-OSATS tool to be used for high-stakes entrustment decisions in resident-performed robotic ileocolic anastomoses.

Disparities in Adherence to Pediatric Diabetic Ketoacidosis Management Guidelines Across a Spectrum of Emergency Departments in the State of Indiana: An Observational In Situ Simulation-Based Study

BACKGROUND

Diabetic ketoacidosis (DKA) is a common presentation to an emergency department (ED), with the majority presenting to community EDs. Adherence to clinical guidelines in these EDs can reduce morbidity and mortality. Few methods to describe practice gaps for DKA management have been reported.

OBJECTIVES

We hypothesized that high-fidelity in situ simulation can be used to measure and compare the quality of the care provided to pediatric patients with DKA presenting to community EDs in the state of Indiana.

METHODS

This observational study examined multiprofessional teams caring for a simulated pediatric patient who presented with DKA to community EDs. The primary outcome was overall adherence to pediatric DKA guidelines as measured by a validated performance checklist. A composite adherence score (CAS) was calculated using the sum of 9 checklist performance parameters. Multivariable logistic regression was used to examine the impact of ED volume and characteristics on the scores.

RESULTS

A 49 multiprofessional teams from 13 sites were enrolled. Of the 252 participants, 26 (10.3%) were physicians, 143 (56.7%) registered nurses, 25 (9.9%) respiratory therapists, and 58 (23.0%) were other. The overall CAS for all sites was 55.6% (25th, 75th interquartile range, 44.4%, 66.7%). Excessive intravenous fluid boluses were given by 53.1%, whereas 30.6% and 26.5% incorrectly administered insulin and sodium bicarbonate boluses, respectively. Only 10.2% used an appropriate intravenous fluid rate, and 57.1% performed an hourly glucose. No significant difference in the CAS was found due to pediatric ED volume or presence of an inpatient pediatric service.

CONCLUSIONS

Using validated in situ simulation; we revealed high variability in adherence to the pediatric DKA management guidelines at a wide range of community EDs. A statewide education initiative focused on decreasing variation and improving adherence to pediatric DKA guidelines is necessary for patient safety.

Executive Summary of the Spanish Society of Anesthesiology, Reanimation and Pain Therapy (SEDAR) Spanish Society of Emergency and Emergency Medicine (SEMES) and Spanish Society of Otolaryngology, Head and Neck Surgery (SEORL-CCC) Guideline for difficult airway management

The Airway section of the Spanish Society of Anesthesiology, Reanimation and Pain Therapy (SEDAR), Spanish Society of Emergency and Emergency Medicine (SEMES) and Spanish Society of Otolaryngology, Head and Neck Surgery (SEORL-CCC) present the Guidelines for the integral management of difficult airway in adult patients. This document provides recommendations based on current scientific evidence, theoretical-educational tools and implementation tools, mainly cognitive aids, applicable to the treatment of the airway in the field of anesthesiology, critical care, emergencies and prehospital medicine. Its principles are focused on the human factors, cognitive processes for decision-making in critical situations and optimization in the progression of the application of strategies to preserve adequate alveolar oxygenation in order to improve safety and quality of care.

Teaching medical students ultrasound-guided needle aspiration of synthetic cysts: effect of a formalin-embalmed cadaver simulation model

BACKGROUND

Standard training for ultrasound-guided cyst needle aspiration is currently performed on live patients during residency. This practice presents risk of iatrogenic injury to patients and provides a high-stress learning environment for medical trainees. Simulation training using synthetic cysts in a formalin-embalmed cadaver model may allow for realistic, practical, and effective training free from patient risk.

METHODS

Thirty first-year medical students viewed an orientation video, then attended a skills workshop to perform cyst needle aspiration in formalin-embalmed cadaver tissue under ultrasound guidance. Participants were randomly assigned to one of three ultrasound-trained instructor-types which included a medical student, clinical anatomist, or an ultrasound fellowship trained emergency medicine physician. After training, participants underwent a 5-min skills test to assess their ability to drain a synthetic cyst independently. Pre- and post-training self-confidence surveys were administered.

RESULTS

Ultrasound images of synthetic cysts in formalin-embalmed tissue were clear and realistic in appearance, and sonographic needle visualization was excellent. Participants took an average of 161.5 s and 1.9 attempts to complete the procedure. Two of the 30 participants could not complete the procedure within the time limit. Participants' self-reported confidence with respect to all aspects of the procedure significantly increased post-training. Mean confidence scores rose from 1.2 (95% CI 0.96 to 1.39) to 4.4 (95% CI 4.09 to 4.53) (P < 0.0001) Procedure time, number of attempts, performance scores, and self-confidence outcomes were not significantly affected by instructor type.

CONCLUSIONS

The use of synthetic cysts in formalin-embalmed cadaveric tissue is feasible, realistic, and efficacious for the teaching of ultrasound-guided needle aspiration to novice medical trainees. This simulation training method can be delivered effectively by multiple instructor types and may allow medical trainees to increase their tactical skill and self-confidence prior to performing ultrasound-guided cyst needle aspiration on live patients.

Procedural simulation in venipuncture for medical undergraduates and its transfer to the bedside: a cluster randomized study

Simulation is accepted as an effective method of learning procedural skills. However, the translational outcomes of skills acquired through simulation still warrants investigation. We designed this study to assess if skills laboratory training in addition to bedside learning (intervention group [IG]) would provide better learning results than bedside learning alone (control group [CG]) in the context of venipuncture training. This prospective, cluster-randomized, single-blind study took place at the Faculty of Medicine, University of Kelaniya, Sri Lanka. Seventeen clusters of second-year medical students were randomly assigned to either IG or CG. The IG trained on venipuncture in the skills laboratory, receiving instruction after modified Payton's Four Step Method. Following the training, students of both IG and CG underwent bedside learning for one month. Afterward, students of both groups performed venipuncture on actual patients in a clinical setting. An independent, blinded assessor scored students' performance using the Integrated Procedural Protocol Instrument (IPPI) and a checklist. Patients assessed students' performance with the Communication Assessment Tool (CAT). Eight and nine clusters were randomized to the intervention and control groups, respectively. IG completed significantly more single steps of the procedure correctly (IG: 19.36 ± 3.87 for checklist items; CG: 15.57 ± 4.95; p < 0.001). IG also scored significantly better on IPPI ratings (median: IG: 27 (12) vs. CG: 21 (8); p < 0.001). Rated by patients, students' communication skills did not significantly differ between the two groups. Simulation-based venipuncture training enabled students to perform the procedure on actual patients with a higher technical accuracy than students who learned venipuncture at the bedside. Students were able to transfer the skills acquired through venipuncture training at the skill laboratory to the bedside.

Linking assessment to real life practice - comparing work based assessments and objective structured clinical examinations using mystery shopping

Objective Structured Clinical Examinations (OSCEs) and Work Based Assessments (WBAs) are the mainstays of assessing clinical competency in health professions' education. Underpinned by the extrapolation inference in Kane's Validity Framework, the purpose of this study is to determine whether OSCEs translate to real life performance by comparing students' OSCE performance to their performance in real-life (as a WBA) using the same clinical scenario, and to understand factors that affect students' performance. A sequential explanatory mixed methods approach where a grade comparison between students' performance in their OSCE and WBA was performed. Students were third year pharmacy undergraduates on placement at a community pharmacy in 2022. The WBA was conducted by a simulated patient, unbeknownst to students and indistinguishable from a genuine patient, visiting the pharmacy asking for health advice. The simulated patient was referred to as a 'mystery shopper' and the process to 'mystery shopping' in this manuscript. Community pharmacy is an ideal setting for real-time observation and mystery shopping as staff can be accessed without appointment. The students' provision of care and clinical knowledge was assessed by the mystery shopper using the same clinical checklist the student was assessed from in the OSCE. Students who had the WBA conducted were then invited to participate in semi-structured interviews to discuss their experiences in both settings. Overall, 92 mystery shopper (WBA) visits with students were conducted and 36 follow-up interviews were completed. The median WBA score was 41.7% [IQR 28.3] and significantly lower compared to the OSCE score 80.9% [IQR 19.0] in all participants (p < 0.001). Interviews revealed students knew they did not perform as well in the WBA compared to their OSCE, but reflected that they still need OSCEs to prepare them to manage real-life patients. Many students related their performance to how they perceived their role in OSCEs versus WBAs, and that OSCEs allowed them more autonomy to manage the patient as opposed to an unfamiliar workplace. As suggested by the activity theory, the performance of the student can be driven by their motivation which differed in the two contexts.

Correlating Simulation Training and Assessment With Clinical Performance: A Feasibility Study

OBJECTIVE

Simulation education and assessment are increasingly used in prehospital curriculums. The objective of this study was to assess the challenges and feasibility of correlating evaluation data from an airway management simulation assessment with clinical performance.

METHODS

This study was undertaken in Ontario, the most populous province in Canada, where 13 bases are distributed in geographically diverse areas, from urban to rural and remote locations. This is a retrospective cohort study of paramedics who had completed simulation education and assessment in rapid sequence intubation. Logistic regression was used to assess for correlation between assessment scores (ie, the global score and the overall score and the definitive airway sans hypoxia/hypotension on the first attempt [DASH-1A] success in the field).

RESULTS

DASH-1A success when grouped by base varied from 25% to 100%. The odds of DASH-1A success increased for paramedics who had a higher overall score (odds ratio [OR]: 1.03; 95% confidence interval [CI], 0.96-1.11) and for paramedics who had a higher global rating (OR: 1.27; CI, 0.73-2.21) when accounting for base intubation frequency. The odds of DASH-1A success increased for paramedics who had a higher overall score (OR: 1.01; CI, 0.93-1.09) and decreased for paramedics who had a higher global rating (OR: 0.96; CI, 0.47-1.96) when accounting for base geography.

CONCLUSION

Although this study lacked a sample size large enough to draw conclusions, it provides a foundation and areas to improve in future work exploring the relationship between simulation assessments and clinical performance.

Innovative Research Methods: Using Simulation to Evaluate Health Care Policy

SUMMARY STATEMENT

Health care policies have the potential to improve patient outcomes, access to care, and reduce health disparities. However, new policy is often tested in the field, where unintended consequences are paid for by patients. In this perspective, we argue that health care simulations, which can elucidate the potential for policy to hinder clinicians' ability to provide high-quality care, are a complement to large-scale policy evaluations in the field.

Spanish Society of Anesthesiology, Reanimation and Pain Therapy (SEDAR), Spanish Society of Emergency and Emergency Medicine (SEMES) and Spanish Society of Otolaryngology, Head and Neck Surgery (SEORL-CCC) Guideline for difficult airway management. Part II

The Airway Management section of the Spanish Society of Anesthesiology, Resuscitation, and Pain Therapy (SEDAR), the Spanish Society of Emergency Medicine (SEMES), and the Spanish Society of Otorhinolaryngology and Head and Neck Surgery (SEORL-CCC) present the Guide for the comprehensive management of difficult airway in adult patients. Its principles are focused on the human factors, cognitive processes for decision-making in critical situations, and optimization in the progression of strategies application to preserve adequate alveolar oxygenation in order to enhance safety and the quality of care. The document provides evidence-based recommendations, theoretical-educational tools, and implementation tools, mainly cognitive aids, applicable to airway management in the fields of anesthesiology, critical care, emergencies, and prehospital medicine. For this purpose, an extensive literature search was conducted following PRISMA-R guidelines and was analyzed using the GRADE methodology. Recommendations were formulated according to the GRADE methodology. Recommendations for sections with low-quality evidence were based on expert opinion through consensus reached via a Delphi questionnaire.

Future directions for simulation in healthcare: A critical review

There was a widespread discontinuation of simulation programs during and after the COVID-19 pandemic. The objective is to explore how to facilitate greater integration of simulation in healthcare organizations. A literature review was conducted in PubMed, MEDES, IBECS and DOCUMED databases. Twenty-three articles published after the pandemic were selected, categorized in seven themes and critically reviewed. In order to consistently and fully integrate simulation into the organizational culture it is recommended to prioritize the development of new strategies that enhance the efficiency and safety of healthcare delivery. And also strategies that enhance the satisfaction and well-being of all stakeholders.

Artificial Intelligence Improves Novices' Bronchoscopy Performance: A Randomized Controlled Trial in a Simulated Setting

BACKGROUND

Navigating through the bronchial tree and visualizing all bronchial segments is the initial step toward learning flexible bronchoscopy. A novel bronchial segment identification system based on artificial intelligence (AI) has been developed to help guide trainees toward more effective training.

RESEARCH QUESTION

Does feedback from an AI-based automatic bronchial segment identification system improve novice bronchoscopists' end-of-training performance?

STUDY DESIGN AND METHODS

The study was conducted as a randomized controlled trial in a standardized simulated setting. Novices without former bronchoscopy experience practiced on a mannequin. The feedback group (n = 10) received feedback from the AI, and the control group (n = 10) trained according to written instructions. Each participant decided when to end training and proceed to performing a full bronchoscopy without any aids.

RESULTS

The feedback group performed significantly better on all three outcome measures (median difference, P value): diagnostic completeness (3.5 segments, P < .001), structured progress (13.5 correct progressions, P < .001), and procedure time (-214 seconds, P = .002).

INTERPRETATION

Training guided by this novel AI makes novices perform more complete, more systematic, and faster bronchoscopies. Future studies should examine its use in a clinical setting and its effects on more advanced learners.

Powerful medical education improves health care quality and return on investment

INTRODUCTION

Powerful medical education (PME) involves the use of new technologies informed by the science of expertise that are embedded in laboratories and organizations that value evidence-based education and support innovation. This contrasts with traditional medical education that relies on a dated apprenticeship model that yields uneven results. PME involves an amalgam of features, conditions and assumptions, and contextual variables that comprise an approach to developing clinical competence grounded in education impact metrics including efficiency and cost-effectiveness.

METHODS

This article is a narrative review based on SANRA criteria and informed by realist review principles. The review addresses the PME model with an emphasis on mastery learning and deliberate practice principles drawn from the new science of expertise. Pub Med, Scopus, and Web of Science search terms include medical education, the science of expertise, mastery learning, translational outcomes, cost effectiveness, and return on investment. Literature coverage is comprehensive with selective citations.

RESULTS

PME is described as an integrated set of twelve features embedded in a group of seven conditions and assumptions and four context variables. PME is illustrated via case examples that demonstrate improved ventilator patient management learning outcomes compared to traditional clinical education and mastery learning of breaking bad news communication skills. Evidence also shows that PME of physicians and other health care providers can have translational, downstream effects on patient care practices, patient outcomes, and return on investment. Several translational health care quality improvements that derive from PME include reduced infections; better communication among physicians, patients, and families; exceptional birth outcomes; more effective patient education; and return on investment.

CONCLUSIONS

The article concludes with challenges to hospitals, health systems, and medical education organizations that are responsible for producing physicians who are expected to deliver safe, effective, and cost-conscious health care.

Perceptions of paramedic educators on assessments used in the first year of a paramedic programme: a qualitative exploration

BACKGROUND

In Ireland, there are currently three educational institutions (recognised institutions- RIs) providing paramedic programmes, accredited by the regulator, the Pre-Hospital Emergency Care Council (PHECC). Each RI assesses their students in-house, and in order to acquire a licence to practice, students must also pass summative assessments provided by PHECC. These assessments comprise multiple choice questions, short answer questions and skills assessments. The objective of this study was to explore the perceptions and experiences of paramedic educators of assessments used within their institution and by the regulator to provide insights that could inform the future design of paramedic assessments.

METHODS

A qualitative study with an interpretivist approach and purposive sampling strategy was performed. Semi-structured interviews were conducted with educators from one RI, across their three sites. Data were analysed using an inductive approach to thematic analysis.

RESULTS

Four major themes were identified in the data: improving assessments by enhancing authenticity, modifying the current process of assessment, aligning the PHECC and RI examinations and opportunities to use assessment as learning.

CONCLUSIONS

This study identifies perceived deficits and opportunities in the assessments currently used for paramedic students and ways in which these assessments could be improved. While participants were relatively content with their own RI assessments, they identified ways to improve both the RI and PHECC assessments. Modifying some of the current methods could be a useful first step. In particular, assessments used by PHECC could be improved by reflecting 'real-world' practice. The inclusion of additional assessment methods by PHECC, a continuous assessment process or devolvement of the entire assessment suite, to the RI/University has the potential to enhance assessments, particularly summative assessments, for paramedic students.

Evaluating the Academic Performance of Mustansiriyah Medical College Teaching Staff vs. Final-Year Students Failure Rates

Background: Evaluating medical colleges' teaching staff performance is an essential topic for evaluating academic performance. Objective: To assess the academic performance of the teaching staff in the pediatric, surgical, and gynecology branches in comparison to the medicine branch performance from 2014 to 2018. Methods: The total number and the number of failed students being examined in final years were obtained officially from the examination committee in Mustansiriyah Medical College for the final 6th-year students for the four main branches (medicine, pediatrics, surgery, and obstetrics and gynecology). The students' number in the medicine branch was used as the control group against which the performance of other branches was compared. We utilized the odds ratio from meta-analysis statistics and compared student failure and success rates. Results: The odd ratio of pediatric branch performance versus medicine branch was 1.02 with a 95% confidence interval of 0.68–1.53, while for surgery, it was 0.67 with a 95% CI (0.46 to 0.98) and 3.13 with a 95% CI (1.79 to 5.47) for the obstetrics and the gynecology departments. Conclusion: The performance of the pediatric branch was the only one compatible with the performance of the medicine branch. In contrast, both the surgery and gynecology branches significantly deviated from the medicine branch's performance. Further research is needed to pinpoint the causes of these performance deviations.

Distance simulation in the health professions: a scoping review

BACKGROUND

Distance simulation is defined as simulation experiences in which participants and/or facilitators are separated from each other by geographic distance and/or time. The use of distance simulation as an education technique expanded rapidly with the recent COVID-19 pandemic, with a concomitant increase in scholarly work.

METHODS

A scoping review was performed to review and characterize the distance simulation literature. With the assistance of an informationist, the literature was systematically searched. Each abstract was reviewed by two researchers and disagreements were addressed by consensus. Risk of bias of the included studies was evaluated using the Risk of Bias 2 (RoB 2) and Risk of Bias in Non-randomized Studies of Interventions (ROBINS-I) tools.

RESULTS

Six thousand nine hundred sixty-nine abstracts were screened, ultimately leading to 124 papers in the final dataset for extraction. A variety of simulation modalities, contexts, and distance simulation technologies were identified, with activities covering a range of content areas. Only 72 papers presented outcomes and sufficient detail to be analyzed for risk of bias. Most studies had moderate to high risk of bias, most commonly related to confounding factors, intervention classification, or measurement of outcomes.

CONCLUSIONS

Most of the papers reviewed during the more than 20-year time period captured in this study presented early work or low-level outcomes. More standardization around reporting is needed to facilitate a clear and shared understanding of future distance simulation research. As the broader simulation community gains more experience with distance simulation, more studies are needed to inform when and how it should be used.

Comparison of practice vs theory model training for vacuum-assisted delivery: A randomized controlled trial

OBJECTIVE

To assess the effect of two modalities of simulation training on acquisition/retention of skills for performing operative vaginal delivery.

DESIGN

Randomized, controlled, single-centre study.

SETTING

A tertiary referral hospital in Italy.

PARTICIPANTS

Twenty residents from a single university programme and two young specialists.

INTERVENTION

Group 1 had an individual training session with a single senior specialist using a fixed simulator model. After the session, trainees watched a pre-recorded 1-h lecture on vacuum-assisted operative vaginal delivery. Group 1 repeated the simulator session using the same test after 8-12 weeks and 12 months. Group 2 watched the pre-recorded lecture then undertook the same sessions as Group 1. Video recordings of all test performances were evaluated by five specialists in a blinded manner.

MAIN OUTCOME MEASURE

Each procedure was evaluated using a Global Rating Scale (GRS), scored with 0-5 points for each item on an Objective Structured Assessment of Technical Skills (OSATS) dataset with seven items (total 35 points). OSATS were evaluated over time, compared for the whole population, and weighted for route and year of residency. The primary outcome was comparison of the effectiveness of training between the two groups based on year of residency by assessing videos of the baseline test and GRS for OSATS scores. The secondary outcome was overall retention of skills at 8-12 weeks and 12 months.

RESULTS

Twenty-two participants were recruited and randomized to either Group 1 (n = 11) or Group 2 (n = 11). Five participants did not complete follow-up. The primary outcome of GRS for OSATS scores at time 0 differed significantly between groups for total GRS score, and 'Call for help' and 'Explanation to woman and relatives' item scores (p = 0.002 and p = 0.007, respectively). In a multiple linear regression analysis, OSATS scores were not independently influenced by year of residency. At 8-12-week follow-up, the 'Call for help' item score showed a significant improvement in Group 1 (p = 0.018), although this was not confirmed when year of residency was included as an independent variable. At 12-month follow-up, none of the item scores demonstrated a significant change (p = 0.033). Year of residency did not influence the difference between groups. One-way analysis of variance found significant differences between the groups for 'Localization of the flexion point' (p = 0.005), 'Traction of vacuum cup' (p = 0.039) and 'Use of second hand of the operator' (p = 0.009) item scores and total GRS score (p = 0.007). The values weighted by year of residency did not demonstrate any significant difference. The secondary outcomes evaluated all the candidates of both groups for retention of technical skills over time. A significant effect of time was found for total GRS score (p < 0.001) and OSATS item scores.

CONCLUSIONS

Independent of the sequence of theoretical teaching and simulation training, trainees demonstrated high retention - and, actually, improvement - of technical skills for operative vaginal delivery at 12-month follow-up.

Effectiveness of Flexible Bronchoscopy Simulation-Based Training: A Systematic Review

BACKGROUND

The implementation of simulation-based training (SBT) to teach flexible bronchoscopy (FB) skills to novice trainees has increased during the last decade. However, it is unknown whether SBT is effective to teach FB to novices and which instructional features contribute to training effectiveness.

RESEARCH QUESTION

How effective is FB SBT and which instructional features contribute to training effectiveness?

STUDY DESIGN AND METHODS

We searched Embase, PubMed, Scopus, and Web of Science for articles on FB SBT for novice trainees, considering all available literature until November 10, 2022. We assessed methodological quality of included studies using a modified version of the Medical Education Research Study Quality Instrument, evaluated risk of bias with relevant tools depending on study design, assessed instructional features, and intended to correlate instructional features to outcome measures.

RESULTS

We identified 14 studies from an initial pool of 544 studies. Eleven studies reported positive effects of FB SBT on most of their outcome measures. However, risk of bias was moderate or high in eight studies, and only six studies were of high quality (modified Medical Education Research Study Quality Instrument score ≥ 12.5). Moreover, instructional features and outcome measures varied highly across studies, and only four studies evaluated intervention effects on behavioral outcome measures in the patient setting. All of the simulation training programs in studies with the highest methodological quality and most relevant outcome measures included curriculum integration and a range in task difficulty.

INTERPRETATION

Although most studies reported positive effects of simulation training programs on their outcome measures, definitive conclusions regarding training effectiveness on actual bronchoscopy performance in patients could not be made because of heterogeneity of training features and the sparse evidence of training effectiveness on validated behavioral outcome measures in a patient setting.

TRIAL REGISTRATION

PROSPERO; No.: CRD42021262853; URL: https://www.crd.york.ac.uk/prospero/.

Academic half days, noon conferences and classroom-based education in postgraduate medical education: a scoping review

BACKGROUND

Classroom-based education (CBE) is ubiquitous in postgraduate medical education (PGME), but to date no studies have synthesized the literature on the topic. We conducted a scoping review focusing on academic half days and noon conferences.

METHODS

We searched 4 databases (MEDLINE [OVID], Embase [OVID], ERIC [EBSCO] and Web of Science) from inception to December 2021, performed reference and citation harvesting, and applied predetermined inclusion and exclusion criteria to our screening. We used 2 frameworks for the analysis: "experiences, trajectories and reifications" and "description, justification and clarification."

RESULTS

We included 90 studies, of which 55 focused on resident experiences, 29 on trajectories and 6 on reification. We classified 44 studies as "description," 38 as "justification" and 8 as "clarification." In the description studies, 12 compared academic half days with noon conferences, 23 described specific teaching topics, and 9 focused on resources needed for CBE. Justification studies examined the effects of CBE on outcomes, such as examination scores (17) and use of teaching strategies in team-based learning, principles of adult learning and e-learning (15). Of the 8 clarification studies, topics included the role of CBE in PGME, stakeholder perspectives and transfer of knowledge between classroom and workplace.

INTERPRETATION

Much of the existing literature is either a description of various aspects of CBE or justification of particular teaching strategies. Few studies exist on how and why CBE works; future studies should aim to clarify how CBE facilitates resident learning within the sociocultural framework of PGME.

Adaptation of a Simulation Model and Checklist to Assess Pediatric Emergency Care Performance by Prehospital Teams

INTRODUCTION

Simulation tools to assess prehospital team performance and identify patient safety events are lacking. We adapted a simulation model and checklist tool of individual paramedic performance to assess prehospital team performance and tested interrater reliability.

METHODS

We used a modified Delphi process to adapt 3 simulation cases (cardiopulmonary arrest, seizure, asthma) and checklist to add remote physician direction, target infants, and evaluate teams of 2 paramedics and 1 physician. Team performance was assessed with a checklist of steps scored as complete/incomplete by raters using direct observation or video review. The composite performance score was the percentage of completed steps. Interrater percent agreement was compared with the original tool. The tool was modified, and raters trained in iterative rounds until composite performance scoring agreement was 0.80 or greater (scale <0.20 = poor; 0.21-0.39 = fair, 0.40-0.59 = moderate; 0.60-0.79 = good; 0.80-1.00 = very good).

RESULTS

We achieved very good interrater agreement for scoring composite performance in 2 rounds using 6 prehospital teams and 4 raters. The original 175 step tool was modified to 171 steps. Interrater percent agreement for the final modified tool approximated the original tool for the composite checklist (0.80 vs. 0.85), cardiopulmonary arrest (0.82 vs. 0.86), and asthma cases (0.80 vs. 0.77) but was lower for the seizure case (0.76 vs. 0.91). Most checklist items (137/171, 80%) had good-very good agreement. Among 34 items with fair-moderate agreement, 15 (44%) related to patient assessment, 9 (26%) equipment use, 6 (18%) medication delivery, and 4 (12%) cardiopulmonary resuscitation quality.

CONCLUSIONS

The modified checklist has very good agreement for assessing composite prehospital team performance and can be used to test effects of patient safety interventions.

Technology-enhanced simulation in emergency medicine: Updated systematic review and meta-analysis 1991-2021

Background

Over the past decade, the use of technology-enhanced simulation in emergency medicine (EM) education has grown, yet we still lack a clear understanding of its effectiveness. This systematic review aims to identify and synthesize studies evaluating the comparative effectiveness of technology-enhanced simulation in EM.

Methods

We searched MEDLINE, EMBASE, PsycINFO, CINAHL, ERIC, Web of Science, and Scopus to identify EM simulation research that compares technology-enhanced simulation with other instructional modalities. Two reviewers screened articles for inclusion and abstracted information on learners, clinical topics, instructional design features, outcomes, cost, and study quality. Standardized mean difference (SMD) effect sizes were pooled using random effects.

Results

We identified 60 studies, enrolling at least 5279 learners. Of these, 23 compared technology-enhanced simulation with another instructional modality (e.g., living humans, lecture, small group), and 37 compared two forms of technology-enhanced simulation. Compared to lecture or small groups, we found simulation to have nonsignificant differences for time skills (SMD 0.33, 95% confidence interval [CI] -0.23 to 0.89, n = 3), but a large, significant effect for non-time skills (SMD 0.82, 95% CI 0.18 to 1.46, n = 8). Comparison of alternative types of technology-enhanced simulation found favorable associations with skills acquisition, of moderate magnitude, for computer-assisted guidance (compared to no computer-assisted guidance), for time skills (SMD 0.50, 95% CI -1.66 to 2.65, n = 2) and non-time skills (SMD 0.57, 95% CI 0.33 to 0.80, n = 6), and for more task repetitions (time skills SMD 1.01, 95% CI 0.16 to 1.86, n = 2) and active participation (compared to observation) for time skills (SMD 0.85, 95% CI 0.25 to 1.45, n = 2) and non-time skills (SMD 0.33 95% CI 0.08 to 0.58, n = 3).

Conclusions

Technology-enhanced simulation is effective for EM learners for skills acquisition. Features such as computer-assisted guidance, repetition, and active learning are associated with greater effectiveness.

Introducing the "SIMline"-A Simulation Course in the Management of Severe Burns as a Tool in Undergraduate Medical Education

Background: Management of burn injuries presents a complex and challenging situation for medical staff, especially for inexperienced young doctors. However, training on how to treat burn victims in the clinical setting is rarely taught in undergraduate medical classes. We have created the "SIMline", a simulation training program explicitly designed for coaching medical students in burn management. Methods: A total of 43 students participated in the "SIMline" course, which took place at the training facility at the Medical University of Graz, between 2018 and 2019. The course provided theoretical classes, practical exercises, and a full-scale care process simulation training. The learning progress of the students was monitored via a formative integrated test. Results: Students showed great progress throughout the course of the "SIMline" program, as their test scores improved by an average of 88%. The passing rate was 0% at the first exam (prior to course) as compared to 87% at the final exam, taken after the training. Conclusions: Comprehensive practical training programs in burn care are underrepresented in medical education. The "SIMline" course presents a novel and effective approach in training medical students in burn management. However, follow-up evaluation is necessary to confirm long-term educational benefits.

Computer-assisted simulated workplace-based assessment in surgery: application of the universal framework of intraoperative performance within a mixed-reality simulation

Objectives

Workplace-based assessment (WBA) is a key requirement of competency-based medical education in postgraduate surgical education. Although simulated workplace-based assessment (SWBA) has been proposed to complement WBA, it is insufficiently adopted in surgical education. In particular, approaches to criterion-referenced and automated assessment of intraoperative surgical competency in contextualized SWBA settings are missing.Main objectives were (1) application of the universal framework of intraoperative performance and exemplary adaptation to spine surgery (vertebroplasty); (2) development of computer-assisted assessment based on criterion-referenced metrics; and (3) implementation in contextualized, team-based operating room (OR) simulation, and evaluation of validity.

Design

Multistage development and assessment study: (1) expert-based definition of performance indicators based on framework's performance domains; (2) development of respective assessment metrics based on preoperative planning and intraoperative performance data; (3) implementation in mixed-reality OR simulation and assessment of surgeons operating in a confederate team. Statistical analyses included internal consistency and interdomain associations, correlations with experience, and technical and non-technical performances.

Setting

Surgical simulation center. Full surgical team set-up within mixed-reality OR simulation.

Participants

Eleven surgeons were recruited from two teaching hospitals. Eligibility criteria included surgical specialists in orthopedic, trauma, or neurosurgery with prior VP or kyphoplasty experience.

Main outcome measures

Computer-assisted assessment of surgeons' intraoperative performance.

Results

Performance scores were associated with surgeons' experience, observational assessment (Objective Structured Assessment of Technical Skill) scores and overall pass/fail ratings. Results provide strong evidence for validity of our computer-assisted SWBA approach. Diverse indicators of surgeons' technical and non-technical performances could be quantified and captured.

Conclusions

This study is the first to investigate computer-assisted assessment based on a competency framework in authentic, contextualized team-based OR simulation. Our approach discriminates surgical competency across the domains of intraoperative performance. It advances previous automated assessment based on the use of current surgical simulators in decontextualized settings. Our findings inform future use of computer-assisted multidomain competency assessments of surgeons using SWBA approaches.

Simulation-based summative assessment in healthcare: an overview of key principles for practice

BACKGROUND

Healthcare curricula need summative assessments relevant to and representative of clinical situations to best select and train learners. Simulation provides multiple benefits with a growing literature base proving its utility for training in a formative context. Advancing to the next step, "the use of simulation for summative assessment" requires rigorous and evidence-based development because any summative assessment is high stakes for participants, trainers, and programs. The first step of this process is to identify the baseline from which we can start.

METHODS

First, using a modified nominal group technique, a task force of 34 panelists defined topics to clarify the why, how, what, when, and who for using simulation-based summative assessment (SBSA). Second, each topic was explored by a group of panelists based on state-of-the-art literature reviews technique with a snowball method to identify further references. Our goal was to identify current knowledge and potential recommendations for future directions. Results were cross-checked among groups and reviewed by an independent expert committee.

RESULTS

Seven topics were selected by the task force: "What can be assessed in simulation?", "Assessment tools for SBSA", "Consequences of undergoing the SBSA process", "Scenarios for SBSA", "Debriefing, video, and research for SBSA", "Trainers for SBSA", and "Implementation of SBSA in healthcare". Together, these seven explorations provide an overview of what is known and can be done with relative certainty, and what is unknown and probably needs further investigation. Based on this work, we highlighted the trustworthiness of different summative assessment-related conclusions, the remaining important problems and questions, and their consequences for participants and institutions of how SBSA is conducted.

CONCLUSION

Our results identified among the seven topics one area with robust evidence in the literature ("What can be assessed in simulation?"), three areas with evidence that require guidance by expert opinion ("Assessment tools for SBSA", "Scenarios for SBSA", "Implementation of SBSA in healthcare"), and three areas with weak or emerging evidence ("Consequences of undergoing the SBSA process", "Debriefing for SBSA", "Trainers for SBSA"). Using SBSA holds much promise, with increasing demand for this application. Due to the important stakes involved, it must be rigorously conducted and supervised. Guidelines for good practice should be formalized to help with conduct and implementation. We believe this baseline can direct future investigation and the development of guidelines.

A Decade Later-Progress and Next Steps for Pediatric Simulation Research

SUMMARY STATEMENT

A decade ago, at the time of formation of the International Network for Pediatric Simulation-based Innovation, Research, and Education, the group embarked on a consensus building exercise. The goal was to forecast the facilitators and barriers to growth and maturity of science in the field of pediatric simulation-based research. This exercise produced 6 domains critical to progress in the field: (1) prioritization, (2) research methodology and outcomes, (3) academic collaboration, (4) integration/implementation/sustainability, (5) technology, and (6) resources/support/advocacy. This article reflects on and summarizes a decade of progress in the field of pediatric simulation research and suggests next steps in each domain as we look forward, including lessons learned by our collaborative grass roots network that can be used to accelerate research efforts in other domains within healthcare simulation science.

Taking Aim at the Injury Prevention Curriculum: Educating Residents on Talking to Patients About Firearm Injury

The medical community recognizes that firearm injury is a public health problem. Yet we lack both the tools for and the implementation of evidence-based firearm injury screening and counseling techniques. One reason for these deficits is the lack of clinical training related to engaging patients in firearm injury risk reduction. In this issue, Rickert et al describe a pre-post evaluation of a 2-part firearm injury prevention training curriculum for first-year medical residents at a single academic medical center. Their manuscript serves an important, but still preliminary, step forward for the field of postgraduate medical education on firearm injury and its prevention. Important elements of this project and paper consist of the inclusion of multiple medical disciplines and the use of standardized patients to evaluate participants' learning. This project also points to the need for further growth. We must commit to consistently and conscientiously framing injuries from guns not as "gun violence" but rather as "firearm injuries" to ensure that they are considered squarely in both the public health space and the clinical space. An ideal curriculum would also highlight the importance of trauma-informed care, cultural competency, and antiracist medical practice while countering implicit biases (e.g., toward gun owners, victims of firearm violence, perpetrators of violence). It should address barriers, as well as facilitators, to change. And most importantly, future educational work must evaluate the effect of these trainings on actual clinical practice-and, even better, the efficacy of education in changing behavior and patient-level outcomes.

Can virtual-reality simulation ensure transthoracic echocardiography skills before trainees examine patients?

OBJECTIVES

This study aimed to develop and gather the validity evidence for a standardised simulation-based skills test in transthoracic echocardiography and to establish a credible pass/fail score.

METHODS

Experts developed a virtual-reality simulator test in cardiology, medical education and simulation-based education. Thirty-six physicians with different experiences in transthoracic echocardiography completed the test at Odense University Hospital, Denmark. The performances of novice, intermediate and experienced participants were compared using the Bonferroni post hoc test. Cronbach's alpha was used to determine the internal consistency reliability of the test. The consistency of performance was analysed using the intraclass correlation coefficient. A pass/fail score was established using the contrasting groups' standard-setting method.

RESULTS

We developed a test with high consistent reliability (Alpha = .81), 95% CI [.69, .89]. In both cases, the performers' level was consistent, fitting others at the same level of experience (intraclass correlation r₍₃₅₎=.81, p<.001). A pass/fail score of 48/50 points was established based on the mean test score of novice and experienced physicians.

CONCLUSIONS

We developed a standardised virtual-reality simulation-based test of echocardiography skills with the ability to distinguish between participants with different levels of transthoracic echocardiography experience. This test could direct a mastery learning training program where trainees practise until they reach the pre-defined level and secure a higher level of competency to ensure quality and safety for patients.

Developing an integrated national simulation-based educational programme for Scottish junior doctors through structured, multistep action research cycles

OBJECTIVES

Simulation is widely employed to teach a range of skills, across healthcare professions and is most effective when embedded within a standarised curriculum. Although recommended by many governing bodies, establishing a national programme of simulation presents many challenges. Successful implementation requires a clear understanding of the priorities and needs of those it seeks to serve yet there are limited examples of how best to do this. This study aimed to develop an integrated national simulation-based educational programme for junior doctors in Scotland through a structed, multistep prioritisation process.

DESIGN

A series of action research cycles were undertaken to develop and evaluate a national simulation programme. This paper describes cycle 1, which employed a six-step structured approach to understand and prioritise learner needs.

SETTING

The study considered the educational needs of Scottish junior doctors in the UK Foundation Programme (UKFP).

PARTICIPANTS

Multiple stakeholder groups were involved in each stage of the process including recent Scottish UKFP graduates, clinical educators, UKFP programme directors and postgraduate deans.

RESULTS

Key stakeholders reviewed the 370 competencies in the UKFP curriculum and identified 18 initial competency areas. These 18 areas were subsequently prioritised through the analytical hierarchy process, resulting in a carefully ordered list of 12 competencies from which a targeted simulation-based educational programme could be developed.

CONCLUSIONS

To our knowledge, this is the first study to outline the methods of competency prioritisation to create a simulation curriculum that is integrated within a national curriculum in the medical education context. As well as demonstrating the practical steps of such a process, key implications for practice are identified. This robust approach to educational design also resulted in unexpected benefits, including educator and clinician acceptance and programme funding sustainability.

Comparison of multimodal active learning and single-modality procedural simulation for central venous catheter insertion for incoming residents in anesthesiology: a prospective and randomized study

BACKGROUND

Active learning methods, including low-fidelity simulation, are useful but the incremental learning effect of each method is however limited. We designed this study to assess if combining flipped classroom and the modified Peyton's « 4-steps» method during procedural simulation (intervention group [IG]) would provide better learning results than simulation alone (control group [CG]) in the context of central venous catheter insertion training.

METHODS

This prospective, single-center, and randomized study took place in 2017 in a single simulation center. All first year Anesthesiology residents of Ile de France area at the start of their residency were randomly included either in the IG or CG during a seminar aimed at providing initial procedural skills with low-fidelity simulation. A composite learning score which included knowledge MCQ and a questionnaire assessing satisfaction and value of the training session was recorded after training (primary outcome, /100). A randomized sub-group of learners of each group were video-recorded during central venous catheter insertion at the end of training and their skills were evaluated with validated tools, including a specific checklist and a global rating scale (GRS).

RESULTS

Among 89 anesthesiology residents, 48 and 41 learners were randomized in the intervention and control groups respectively. Of the IG residents, 40/48 (83%) had read the learning material prior to the session. There was no significant difference regarding the composite outcome ([IG]= 81.1 vs [CG] = 80.5 /100 (p = 0.68)). Results of the post-session MCQ knowledge questionnaire were also non-significantly different. Residents were similarly satisfied and described a significant improvement of their knowledge and skills after training. Learners highly valued the training session as a mean to improve their future practice. No significant differences regarding checklist and GRS scores were observed.

CONCLUSIONS

A multimodal active learning strategy of procedural learning did not provide better learning outcomes when compared to a traditional simulation method. In both groups, satisfaction was high and perception of the acquired theoretical and practical knowledge was improved after training.

National Implementation of Simulator Training Improves Transurethral Resection of Bladder Tumours in Patients

Background

Objective

Design, setting, and participants

Outcome measurements and statistical analyses

Results and limitations

Conclusions

Patient summary

Expertise in colonoscopy intubation does not predict diagnostic accuracy: a simulation-based study

Background and study aims  Studies have linked cecal intubation rate with adenoma detection rate; however, the direct association between technical performance during colonoscopy intubation and withdrawal has never been explored. Thus, it remains unclear whether gentle and efficient intubation predicts superior mucosal inspection. The aim of this study was to investigate the correlation between performance during intubation and withdrawal in a simulation-based setup. Methods  Twenty-four physicians with various experience in colonoscopy performed twice on the Endoscopy Training System (ETS). Intubation skills were evaluated by assessing tasks on the ETS related to intubation (scope manipulation and loop management) and use of a computerized assessment tool called the 3D-Colonoscopy Progression Score (3D-CoPS). Diagnostic accuracy was defined by the number of polyps found during the ETS task of mucosal inspection. Pearson's correlations were calculated to explore associations between intubation skill and diagnostic accuracy. Results  The correlation analysis between 3D-CoPS and number of polyps found during mucosal inspection revealed a weak and insignificant correlation (0.157, P  = 0.3). Likewise, an insignificant correlation was seen between ETS intubation and number of polyps found (0.149, P  = 0.32). Conclusions  We found no evidence to support that technical performance during intubation is correlated with mucosal inspection performance in a simulation-based setting.

Improving Pediatric Readiness and Clinical Care in General Emergency Departments: A Multicenter Retrospective Cohort Study

OBJECTIVE

To evaluate the impact of a collaborative initiative between general emergency departments (EDs) and the pediatric academic medical center on the process of clinical care in a group of general EDs.

STUDY DESIGN

This retrospective cohort study assessed the process of clinical care delivered to critically ill children presenting to 3 general EDs. Our previous multifaceted intervention included the following components: postsimulation debriefing, designation of a pediatric champion, customized performance reports, pediatric resources toolkit, and ongoing interactions. Five pediatric emergency care physicians conducted chart reviews and scored encounters using the Pediatric Emergency Care Research Network's Quality of Care Implicit Review Instrument, which assigns scores between 5 and 35 across 5 domains. In addition, safety metrics were collected for medication, imaging, and laboratory orders.

RESULTS

A total of 179 ED encounters were reviewed, including 103 preintervention and 76 postintervention encounters, with an improvement in mean total quality score from 23.30 (SD 5.1) to 24.80 (4.0). In the domain of physician initial treatment plan and initial orders, scores increased from a mean of 4.18 (0.13) to 4.61 (0.15). In the category of safety, administration of wrong medications decreased from 28.2% to 11.8% after the intervention.

CONCLUSION

A multifaceted collaborative initiative involving simulation and enhanced pediatric readiness was associated with improvement in the processes of care in general EDs. This work provides evidence that innovative collaborations between academic medical centers and general EDs may serve as an effective strategy to improve pediatric care.

Quality of Resuscitative Care Provided to an Infant With Abusive Head Trauma in Community Emergency Departments: An In Situ, Prospective, Simulation-Based Study

OBJECTIVES

Abusive head trauma (AHT) is a very common and serious form of physical abuse, and a major cause of mortality and morbidity for young children. Early Recognition and supportive care of children with AHT is a common challenge in community emergency department (CEDs). We hypothesized that standardized, in situ simulation can be used to measure and compare the quality of resuscitative measures provided to children with AHT in a diverse set of CEDs.

METHODS

This prospective, simulation-based study measured teams' performance across CEDs. The primary outcome was overall adherence to AHT using a 15-item performance assessment checklist based on the number of tasks performed correctly on the checklist.

RESULTS

Fifty-three multiprofessional teams from 18 CEDs participated in the study. Of 270 participants, 20.7% were physicians, 65.2% registered nurses, and 14.1% were other providers. Out of all tasks, assessment of airway/breathing was the most successfully conducted task by 53/53 teams (100%). Although 43/53 teams (81%) verbalized the suspicion for AHT, only 21 (39.6%) of 53 teams used hyperosmolar agent, 4 (7.5%) of 53 teams applied cervical spine collar stabilization, and 6 (11.3%) of 53 teams raised the head of the bed. No significant difference in adherence to the checklist was found in the CEDs with an inpatient pediatric service or these with designated adult trauma centers compared with CEDs without. Community emergency departments closer to the main academic center outperformed CEDs these that are further away.

CONCLUSIONS

This study used in situ simulation to describe quality of resuscitative care provided to an infant presenting with AHT across a diverse set of CEDs, revealing variability in the initial recognition and stabilizing efforts and provided and targets for improvement. Future interventions focusing on reducing these gaps could improve the performance of CED providers and lead to improved patient outcomes.

Competency-based medical education in radiotherapy treatment planning

PURPOSE

To develop a technology-enhanced education methodology with competency-based evaluation for radiotherapy treatment planning. The education program is designed for integration in the existing framework of Commission on Accreditation of Medical Physics Education Programs (CAMPEP) accredited medical physics residency programs.

MATERIALS AND METHODS

This education program pairs an accessible, multi-institutional infrastructure with established medical education evaluation tools to modernize treatment planning education. This program includes three evaluation components: (i) competency-based evaluation, (ii) inter- and intra-modality comparison, and (iii) learner feedback. For this study, synchronous bilateral breast cancer was selected to demonstrate a complex treatment site and non-standardized technique. Additionally, an online study was made available to a public cohort of worldwide participants of certified Medical Dosimetrists and Medical Physicists to benchmark performance. Prior to evaluation, learners were given a disease site-specific education session on potential clinical treatment strategies. During the assessment, learners generated treatment plans in their institutional planning system under the direct observation of an expert evaluator. Qualitative proficiency was evaluated for all learners on a five-point scale of graduated task independence. Quantitative dosimetry was compared between the learner cohort and public cohort. A feedback session provided learners context of multi-institutional experience through multimodality and technique comparison. After study completion, learners were provided a survey that was used to gauge their perception of the education program.

RESULTS

In the public study, 34 participants submitted treatment plans. Across three CAMPEP-accredited residency programs, six learners participated in the education and evaluation program. All learners successfully completed treatment plans that met the dosimetric constraints described in the case study. All learners favourably reviewed the study either comprehensively or in specified domains.

CONCLUSION

The competency-based education and evaluation program developed in this work has been incorporated in CAMPEP-accredited residency programs and is adaptable to other residency programs with minimal resource commitment.

Simulation and Active Learning Decreases Training Time of an Emergency Triage Assessment and Treatment Course in Pilot Study in Malawi: Implications for Increasing Efficiency and Workforce Capacity in Low-Resource Settings

OBJECTIVES

The aim of the Emergency Triage Assessment and Treatment (ETAT) plus trauma course is to improve the quality of care provided to infants and children younger than 5 years. The curriculum was revised and shortened from 5 to 2.5 days by enhancing simulation and active learning opportunities. The aim of this study was to examine the feasibility and value of the new short-form ETAT course by assessing postcourse knowledge and satisfaction.

METHODS

We delivered the short-form ETAT course to a group of interdisciplinary health workers in Malawi. Precourse and postcourse knowledge was assessed using a standardized 20 questions short answer test used previously in the 5-day courses. A 13-statement survey with 2 open-ended questions was used to examine participant satisfaction.

RESULTS

Participants' postcourse knowledge improved significantly (P < 0.001) after the shorter ETAT course. Participants reported high levels of satisfaction with the short-form ETAT.

CONCLUSIONS

Simulation and other active learning strategies reduced training time by 50% in the short-form ETAT course. Participants with and without previous ETAT training improved their knowledge after participating in the short-form ETAT course. Reduced training time is beneficial in settings already burdened by scarce human resources, may facilitate better access to in-service training, and build capacity while conserving resources in low-resource settings.

Simulation-based assessment of care for infant cardiogenic shock in the emergency department

BACKGROUND

Congenital heart disease (CHD) is the leading cause of infant deaths associated with birth defects. Neonates with undiagnosed CHD often present to general emergency departments (GEDs) for initial resuscitation that are less prepared than paediatric centres, resulting in disparities in the quality of care. Neonates with undiagnosed CHD represent a challenge; thus, it is necessary for GEDs to be prepared for this population.

AIM

To evaluate the process of resuscitative care provided to a neonate in cardiogenic shock due to CHD in the GEDs in a simulated setting and to describe the impact of teams and GED variables on the process of care.

METHODS

This is a prospective simulation-based assessment of the process of care provided to a neonate with coarctation of the aorta in cardiogenic shock. Simulation sessions were conducted at participating GEDs utilizing each GED's interdisciplinary team and resources. The primary outcome was adherence to best practice, as measured by a 15-item overall composite adherence score (CAS). In addition, we stratified the overall CAS into CHD-critical items and the general resuscitation items CAS. The secondary outcome was the impact of the team's and GED's characteristics on the scores.

FINDINGS

This study enrolled 32 teams from 12 GEDs. Among 161 participants, 103 (63.97%) were registered nurses, 33 (20.50%) were physicians, 17 (10.56%) were respiratory therapists, and 8 (4.97%) were other medical professionals. The overall median CAS was 84, with the CHD-critical items having a median CAS of 34.5. The most underperformed tasks are checking pulses on the upper and lower extremities (44%), obtaining blood pressure in the upper and lower extremities (25%), and administering prostaglandin E1 (22%).

CONCLUSIONS

Using in situ simulation in a set of GEDs, we revealed gaps in the resuscitation care of neonates with CHD in cardiogenic shock.

RELEVANCE TO CLINICAL PRACTICE

These findings highlight the importance of targeted improvement programs for high-stakes illnesses in GED.

Harnessing the power of simulation for assessment: Consensus recommendations for the use of simulation-based assessment in emergency medicine

OBJECTIVES

To address the increasing demand for the use of simulation for assessment, our objective was to review the literature pertaining to simulation-based assessment and develop a set of consensus-based expert-informed recommendations on the use of simulation-based assessment as presented at the 2019 Canadian Association of Emergency Physicians (CAEP) Academic Symposium on Education.

METHODS

A panel of Emergency Medicine (EM) physicians from across Canada, with leadership roles in simulation and/or assessment, was formed to develop the recommendations. An initial scoping literature review was conducted to extract principles of simulation-based assessment. These principles were refined via thematic analysis, and then used to derive a set of recommendations for the use of simulation-based assessment, organized by the Consensus Framework for Good Assessment. This was reviewed and revised via a national stakeholder survey, and then the recommendations were presented and revised at the consensus conference to generate a final set of recommendations on the use of simulation-based assessment in EM.

CONCLUSION

We developed a set of recommendations for simulation-based assessment, using consensus-based expert-informed methods, across the domains of validity, reproducibility, feasibility, educational and catalytic effects, acceptability, and programmatic assessment. While the precise role of simulation-based assessment will be a subject of continued debate, we propose that these recommendations be used to assist educators and program leaders as they incorporate simulation-based assessment into their programs of assessment.

The role of weight bias and role-modeling in medical students' patient-centered communication with higher weight standardized patients

OBJECTIVE

Patients with obesity may experience less patient-centered care. We assessed whether medical students' implicit/explicit weight-related attitudes and perceptions of normative attitudes are associated with patient-centered care for patients with obesity.

METHODS

Third and fourth year medical students (N = 111) at one medical school completed a survey and participated in a patient care scenario with a standardized patient with obesity. Encounters were coded for patient-centered behavior. Predictors of patient-centered behaviors were assessed.

RESULTS

Student perceptions that negative attitudes about patients with obesity are normative in medical school were significantly associated with poorer patient-centered behaviors, including lower attentiveness (b=-0.19, p = 0.01), friendliness (b=-0.28, p < 0.001), responsiveness (b=-0.21, p = 0.002), respectfulness (b=-0.17, p = 0.003), interactivity (b=-0.22, p = 0.003), likelihood of being recommended by observers (b=-0.34, p < 0.001), and patient-centeredness index scores (b=-0.16, p = 0.002). Student reported faculty role-modeling of discrimination against patients with obesity predicted lower friendliness (b=-0.16, p = 0.03), recommendation likelihood (b=-0.22, p = 0.04), and patient-centeredness index score (b=-0.12, p = 0.03).

CONCLUSIONS

Negative normative attitudes and behaviors regarding obesity in the medical school environment may adversely influence the quality of patient-centered behaviors provided to patients with obesity.

PRACTICE IMPLICATIONS

Efforts to improve patient-centered communication quality among medical trainees may benefit from intervention to improve group normative attitudes about patients with obesity.

Qualities of Effective Vital Anaesthesia Simulation Training Facilitators Delivering Simulation-Based Education in Resource-Limited Settings

BACKGROUND

Lack of access to safe and affordable anesthesia and surgical care is a major contributor to avoidable death and disability across the globe. Effective education initiatives are a viable mechanism to address critical skill and process gaps in perioperative teams. Vital Anaesthesia Simulation Training (VAST) aims to overcome barriers limiting widespread application of simulation-based education (SBE) in resource-limited environments, providing immersive, low-cost, multidisciplinary SBE and simulation facilitator training. There is a dearth of knowledge regarding the factors supporting effective simulation facilitation in resource-limited environments. Frameworks evaluating simulation facilitation in high-income countries (HICs) are unlikely to fully assess the range of skills required by simulation facilitators working in resource-limited environments. This study explores the qualities of effective VAST facilitators; knowledge gained will inform the design of a framework for assessing simulation facilitators working in resource-limited contexts and promote more effective simulation faculty development.

METHODS

This qualitative study used in-depth interviews to explore VAST facilitators' perspectives on attributes and practices of effective simulation in resource-limited settings. Twenty VAST facilitators were purposively sampled and consented to be interviewed. They represented 6 low- and middle-income countries (LMICs) and 3 HICs. Interviews were conducted using a semistructured interview guide. Data analysis involved open coding to inductively identify themes using labels taken from the words of study participants and those from the relevant literature.

RESULTS

Emergent themes centered on 4 categories: Persona, Principles, Performance and Progression. Effective VAST facilitators embody a set of traits, style, and personal attributes (Persona) and adhere to certain Principles to optimize the simulation environment, maximize learning, and enable effective VAST Course delivery. Performance describes specific practices that well-trained facilitators demonstrate while delivering VAST courses. Finally, to advance toward competency, facilitators must seek opportunities for skill Progression.Interwoven across categories was the finding that effective VAST facilitators must be cognizant of how context, culture, and language may impact delivery of SBE. The complexity of VAST Course delivery requires that facilitators have a sensitive approach and be flexible, adaptable, and open-minded. To progress toward competency, facilitators must be open to self-reflection, be mentored, and have opportunities for practice.

CONCLUSIONS

The results from this study will help to develop a simulation facilitator evaluation tool that incorporates cultural sensitivity, flexibility, and a participant-focused educational model, with broad relevance across varied resource-limited environments.