A critical review of simulation-based mastery learning with translational outcomes

Validation of a Novel Simulation-Based Test in Robot-Assisted Radical Prostatectomy

Purpose: To investigate validity evidence for a simulator-based test in robot-assisted radical prostatectomy (RARP). Materials and Methods: The test consisted of three modules on the RobotiX Mentor VR-simulator: Bladder Neck Dissection, Neurovascular Bundle Dissection, and Ureterovesical Anastomosis. Validity evidence was investigated by using Messick's framework by including doctors with different RARP experience: novices (who had assisted for RARP), intermediates (robotic surgeons, but not RARP surgeons), or experienced (RARP surgeons). The simulator metrics were analyzed, and Cronbach's alpha and generalizability theory were used to explore reliability. Intergroup comparisons were done with mixed-model, repeated measurement analysis of variance and the correlation between the number of robotic procedures and the mean test score were examined. A pass/fail score was established by using the contrasting groups' method. Results: Ten novices, 11 intermediates, and 6 experienced RARP surgeons were included. Six metrics could discriminate between groups and showed acceptable internal consistency reliability, Cronbach's alpha = 0.49, p < 0.001. Test-retest reliability was 0.75, 0.85, and 0.90 for one, two, and three repetitions of tests, respectively. Six metrics were combined into a simulator score that could discriminate between all three groups, p = 0.002, p < 0.001, and p = 0.029 for novices vs intermediates, novices vs experienced, and intermediates vs experienced, respectively. Total number of robotic operations and the mean score of the three repetitions were significantly correlated, Pearson's r = 0.74, p < 0.001. Conclusion: This study provides validity evidence for a simulator-based test in RARP. We determined a pass/fail level that can be used to ensure competency before proceeding to supervised clinical training.

Educating Anesthesiologists During the Coronavirus Disease 2019 Pandemic and Beyond

The coronavirus disease 2019 (COVID-19) pandemic has altered approaches to anesthesiology education by shifting educational paradigms. This vision article discusses pre-COVID-19 educational methodologies and best evidence, adaptations required under COVID-19, and evidence for these modifications, and suggests future directions for anesthesiology education. Learning management systems provide structure to online learning. They have been increasingly utilized to improve access to didactic materials asynchronously. Despite some historic reservations, the pandemic has necessitated a rapid uptake across programs. Commercially available systems offer a wide range of peer-reviewed curricular options. The flipped classroom promotes learning foundational knowledge before teaching sessions with a focus on application during structured didactics. There is growing evidence that this approach is preferred by learners and may increase knowledge gain. The flipped classroom works well with learning management systems to disseminate focused preclass work. Care must be taken to keep virtual sessions interactive. Simulation, already used in anesthesiology, has been critical in preparation for the care of COVID-19 patients. Multidisciplinary, in situ simulations allow for rapid dissemination of new team workflows. Physical distancing and reduced availability of providers have required more sessions. Early pandemic decreases in operating volumes have allowed for this; future planning will have to incorporate smaller groups, sanitizing of equipment, and attention to use of personal protective equipment. Effective technical skills training requires instruction to mastery levels, use of deliberate practice, and high-quality feedback. Reduced sizes of skill-training workshops and approaches for feedback that are not in-person will be required. Mock oral and objective structured clinical examination (OSCE) allow for training and assessment of competencies often not addressed otherwise. They provide formative and summative data and objective measurements of Accreditation Council for Graduate Medical Education (ACGME) milestones. They also allow for preparation for the American Board of Anesthesiology (ABA) APPLIED examination. Adaptations to teleconferencing or videoconferencing can allow for continued use. Benefits of teaching in this new era include enhanced availability of asynchronous learning and opportunities to apply universal, expert-driven curricula. Burdens include decreased social interactions and potential need for an increased amount of smaller, live sessions. Acquiring learning management systems and holding more frequent simulation and skills sessions with fewer learners may increase cost. With the increasing dependency on multimedia and technology support for teaching and learning, one important focus of educational research is on the development and evaluation of strategies that reduce extraneous processing and manage essential and generative processing in virtual learning environments. Collaboration to identify and implement best practices has the potential to improve education for all learners.

American Society of Regional Anesthesia and Pain Medicine expert panel recommendations on point-of-care ultrasound education and training for regional anesthesiologists and pain physicians-part II: recommendations

Point-of-care ultrasound (POCUS) is a critical skill for all regional anesthesiologists and pain physicians to help diagnose relevant complications related to routine practice and guide perioperative management. In an effort to inform the regional anesthesia and pain community as well as address a need for structured education and training, the American Society of Regional Anesthesia and Pain Medicine Society (ASRA) commissioned this narrative review to provide recommendations for POCUS. The recommendations were written by content and educational experts and were approved by the guidelines committee and the Board of Directors of the ASRA. In part II of this two-part series, learning goals and objectives were identified and outlined for achieving competency in the use of POCUS, specifically, airway ultrasound, lung ultrasound, gastric ultrasound, the focus assessment with sonography for trauma exam, and focused cardiac ultrasound, in the perioperative and chronic pain setting. It also discusses barriers to POCUS education and training and proposes a list of educational resources. For each POCUS section, learning goals and specific skills were presented in the Indication, Acquisition, Interpretation, and Medical decision-making framework.

Short-term Retention of Patient and Caregiver Ventricular Assist Device Self-care Skills after Simulation-based Mastery Learning

Background

We developed a simulation-based mastery learning (SBML) curriculum that boosted self-care skills for patients with a ventricular assist device (VAD). In this study, we evaluated short-term skills retention.

Methods

We assessed skill retention among patients and caregivers who participated in VAD self-care SBML at a tertiary care center. We compared discharge skills tests (immediately after completing SBML) to 1- and 3-month follow-up tests to assess skill retention.

Results

Fifteen patients and 15 caregivers completed discharge and follow-up testing. Skills were largely retained at 1- and 3-months.

Conclusions

SBML results in short-term retention of VAD self-care skills.

Evaluating the Clinical Implications of an Innovative Anatomy Lab Suturing Curriculum for Physician Assistant Students

PURPOSE

The purpose of this study was to evaluate the outcomes of a new suturing curriculum incorporated throughout the anatomy lab for Yale Physician Associate (YPA) students.

METHODS

This controlled before-and-after study evaluated the class of 2017 (n = 39) as the control group and the class of 2018 (n = 37) as the intervention group. Suturing competency data were collected on all students from their clinical preceptor evaluations. Students completed surveys to measure perceived confidence in suturing skills.

RESULTS

Preceptor evaluations showed a 14.98% increase in suturing competence between the control and intervention groups (P < .05). Student surveys showed no significant difference in self-perceived confidence in suturing skills between the 2 cohorts.

CONCLUSIONS

After initiation of an innovative anatomy lab suturing curriculum, YPA students demonstrated improvement in preceptor-perceived suturing competency during clinical rotations.

Training Thoracic Ultrasound Skills: A Randomized Controlled Trial of Simulation-Based Training versus Training on Healthy Volunteers

INTRODUCTION

As ultrasound becomes more accessible, the use of point-of-care ultrasound examinations performed by clinicians has increased. Sufficient theoretical and practical skills are prerequisites to integrate thoracic ultrasound into a clinical setting and to use it as supplement in the clinical decision-making. Recommendations on how to educate and train clinicians for these ultrasound examinations are debated, and simulation-based training may improve clinical performance.

OBJECTIVES

The aim of this study was to explore the effect of simulation-based training in thoracic ultrasound compared to training on healthy volunteers.

METHOD

A total of 66 physicians with no previous experience in thoracic ultrasound completed a training program and assessment of competences from November 2018 to May 2019. After a theoretical session in ultrasound physics, sonoanatomy, and thoracic ultrasound, the physicians were randomized into one of three groups for practical training: (1) simulation-based training, (2) training on a healthy volunteer, or (3) no training (control group). Primary outcome was difference in the clinical performance score after the training period.

RESULTS

Using a multiple comparison, ANOVA with Bonferroni correction for multiplicity, there was no statistical significant difference between the two trained groups' performance score: 45.1 points versus 41.9 points (minimum 17 points, maximum 68 points; p = 0.38). The simulation-based training group scored significantly higher than the control group without hands-on training, 36.7 points (p = 0.009).

CONCLUSIONS

The use of simulation-based training in thoracic ultrasound does not improve the clinical performance score compared to conventional training on healthy volunteers. As focused, thoracic ultrasound is a relatively uncomplicated practical procedure when taught; focus should mainly be on the theoretical part and the supervised clinical training in a curriculum. However, simulation can be used instead or as an add-on to training on simulated patients.

Implementing the transvaginal ultrasound simulation training (TRUSST) programme for obstetric registrars

BACKGROUND

Transvaginal ultrasound (TVUS) training opportunities are limited due to its intimate nature; however, TVUS is an important component of early pregnancy assessment. Simulation can bridge this learning gap.

AIM

To describe and measure the effect of a transvaginal ultrasound simulation programme for obstetric registrars.

MATERIALS AND METHODS

The transvaginal ultrasound simulation training (TRUSST) curriculum consisted of supported practice using virtual reality transvaginal simulators (ScanTrainer, Medaphor) and communication skills training to assist obstetric registrars in obtaining required competencies to accurately and holistically care for women with early pregnancy complications. Trainee experience of live transvaginal scanning was evaluated with a questionnaire. Programme evaluation was by pre-post self-reported confidence level and objective pre-post training assessment using Objective Structured Assessment of Ultrasound Skills (OSAUS) and modified Royal Australian and New Zealand College of Obstetrics and Gynaecology assessment scores. Quantitative data was compared using paired t tests.

RESULTS

Fifteen obstetric registrars completed the programme. Numbers of performed live transvaginal ultrasound by trainees were low. Participants reported an increase in confidence level in performing a TVUS following training: mean pre score 1.6/5, mean post score 3/5. Objective assessments improved significantly across both OSAUS and RANZCOG scores following training; mean improvement scores 7.6 points (95% CI 6.2-8.9, p < 0.05) and 32.5 (95% CI 26.4-38.6, p < 0.05) respectively. It was noted that scores for a systematic approach and documentation were most improved: 1.9 (95% CI 1.4-2.5, p < 0.05) and 2.1 (95% CI 1.5-2.7, p < 0.05) respectively.

CONCLUSION

The implementation of a simulation-based training curriculum resulted in improved confidence and ability in TVUS scanning, especially with regard to a systematic approach and documentation.

Simulation-based mastery improves nursing skills in BSc nursing students: a quasi-experimental study

BACKGROUND

Clinical education is an essential part of nursing education. Selected clinical teaching methods influence the quality of education. Simulation-based mastery learning has been used to improve clinical skills among nursing students and may provide a novel way to enhance nursing skills. This study aimed to assess the effect of simulation-based mastery learning on the clinical skills of undergraduate nursing students from 2017 to 2019.

METHODS

This quasi-experimental study was conducted with two groups (the control and intervention). A hundred and five students were selected by random convenience sampling, and written consent was obtained. The intervention group participated in a simulation-based mastery learning intervention, and the control group received no intervention except for traditional training. The students of both groups completed the demographic information questionnaire and the checklist before and after the intervention. The results were analyzed by SPSS version 21 and descriptive and inferential statistics.

RESULTS

The results showed no significant differences between the two groups before the intervention (p> 0.05). In addition, students' performance in the intervention and control groups improved significantly at the post-test compared with the baseline (p< 0.05). Furthermore, the Cohen test implied that the simulation-based mastery model used by the intervention group was significantly more effective than the traditional training used by the control.

CONCLUSION

These findings showed that mastery learning was more effective in improving clinical skills in undergraduate nursing students. The results suggest that other nursing and health programs can be developed by implementing a mastery-based learning model.

Simulation in Pediatrics: A Learning Lab for Education, Quality Improvement, and Patient Safety

Simulation-based medical education is an experiential modality that has evolved over the last 60 years, amassing evidence as an efficacious tool for skill acquisition and care improvement. We review the underlying theory, core defining principles, and applications of medical simulation broadly and in pediatrics in hopes that it can be accessible to every pediatric clinician regardless of practice environment and resources. Any situation where there is risk of harm to a patient or clinician can be simulated for practice, reflection, and re-practice. Whether preparing for clinic-based emergencies, new hospital units, or new daily workflows, simulation is valuable to novice and master clinicians for individual and team care enhancement. [Pediatr Ann. 2021;50(1):e13-e18.].

Changing the view: impact of simulation-based mastery learning in pediatric tracheal intubation with videolaryngoscopy

OBJECTIVE

To determine the effect of a training program using simulation-based mastery learning on the performance of residents in pediatric intubations with videolaryngoscopy.

METHOD

Retrospective cohort study carried out in a tertiary pediatric hospital between July 2016 and June 2018 evaluating a database that included the performance of residents before and after training, as well as the outcome of tracheal intubations. A total of 59 pediatric residents were evaluated in the pre-training with a skills' checklist in the scenario with an intubation simulator; subsequently, they were trained individually using a simulator and deliberate practice in the department itself. After training, the residents were expected to have a minimum passing grade (90/100) in a simulated scenario. The success of the first attempted intubation, use of videolaryngoscopy, and complications in patients older than 1year of age during the study period were also recorded in clinical practice.

RESULTS

Before training, the mean grade was 77.5/100 (SD 15.2), with only 23.7% (14/59) of residents reaching the minimum passing grade of 90/100. After training, 100% of the residents reached the grade, with an average of 94.9/100 (SD 3.2), p<0.01, with only 5.1% (3/59) needing more practice time than that initially allocated. The success rate in the first attempt at intubation in the emergency department with videolaryngoscopy was 77.8% (21/27). The rate of adverse events associated with intubations was 26% (7/27), representing a serious event.

CONCLUSIONS

Simulation-based mastery learning increased residents' skills related to intubation and allowed safe tracheal intubations with video laryngoscopy.

Simulation in Clinical Nursing Education

Background:

Simulation constitutes a teaching method and a strategy for learning and understanding theoretical knowledge and skills in the nursing and medical field.

Objective:

To review and present modern data related to this issue.

Methods:

Literature review of data related to the issue derived from Medline, Cinhal, and Scopus databases, in English, using the following keywords: nursing, simulation, simulator, nursing laboratory.

Results:

The implementation of simulation enables students to practice their clinical and decision-making skills for some significant issues they may face in their daily work. The protected environment and the sense of security enhance students’ self-esteem and confidence, thus promoting learning. In this way, the gap between theory and practice is substantially reduced.

Conclusion:

The further development of simulation, along with other instructional techniques, can significantly help the efforts made by the students to become integrated and successful healthcare professionals.

Change is never easy: How management theories can help operationalise change in medical education

CONTEXT

Medical education is neither simple nor stable, and is highly contextualised. Hence, ways of perceiving multiple connections and complexity are fundamental when seeking to describe, understand and address concerns and questions related to change.

PROPOSAL

In response to calls in the literature, we introduce three examples of contemporary organisational theory which can be used to understand and operationalise change within medical education. These theories, institutional logics, paradox theory and complexity leadership theory, respectively, are relatively unknown in medical education. However, they provide a way of making sense of the complexity of change creatively. Specifically, they cross-cut different levels of analysis and allow us to 'zoom in' to micro levels, as well as to 'zoom out' and connect what is happening at the individual level (the micro level) to what happens at a wider institutional and even national or international level (the macro level), thereby providing a means of understanding the interactions among individuals, teams, organisations and systems. We highlight the potential value of these theories, provide a brief discussion of the few studies that have used them in medical education, and then briefly critique each theory.

CONCLUSIONS

We hope that by drawing the attention of readers to the potential of these management theories, we can unlock some of the complexity of change in medical education, support new ways of thinking and open new avenues for research.

Non-technical skills progression during anesthesiology residency in Portugal: the impact of a National Pedagogical Plan

Background Simulation is known as an important tool for the learning of technical and non-technical skills without endangering patient safety. In Portugal, a National Pedagogical Plan for Anesthesiology Residents was created based on simulation training. This plan was designed according to the objectives set forth by the Portuguese Board of Anesthesiology. This study aimed to evaluate the impact of simulation training courses on the non-technical skills of medical residents in Anesthesiology. Methods Confidential questionnaires, pre- and post-course, were answered by all the residents that attended the different modules of the simulation training program at Centro Hospitalar e Universitário de Coimbra Biomedical Simulation Centre, Portugal, from February 2011 to March 2018. Results A total of 344 questionnaires were answered. In the group of questions regarding the need for help, mistakes, and self-efficacy over time, students recognized an increase over time in the need for support and the self-assessment of the number of mistakes (p < 0.001). Regarding the self-evaluation of safety culture and communication skills, at the end of the residency, almost all the students recognized that they did not feel bad when asking for help or expressing their opinion, even when they disagreed with the consultant anesthesiologist. This was significantly different from the values of the self-assessment at the beginning of residency (p < 0.001). The evolution of preparation, knowledge, and training also showed a positive evolution over the simulation modules (p < 0.001). Finally, the evaluation of the behavioral component in the clinical setting showed a significant positive evolution over time (p < 0.001): in the end, all the students strongly agreed that behavioral competencies are crucial. Conclusions The impact of simulation on anesthesiology non-technical skills during residency is positive and recognized by the students. Moreover, simulation also helps in the recognition of error, enriching the value of self-confidence and the crucial role of behavioral skills. ABBREVIATIONS BSC-CHUC: Biomedical Simulation Centre from Centro Hospitalar e Universitário de Coimbra.

Physiotherapy students can be educated to portray realistic patient roles in simulation: a pragmatic observational study

BACKGROUND

Simulation-based education (SBE) has many benefits for learners, but costs can limit embedding SBE in health professional curricula. Peer simulation involves students portraying patient roles, and may reduce costs while still providing the benefits of other SBE experiences. However, the quality of the SBE may be impacted if students cannot portray authentic and realistic patient roles. The aim of this study was to investigate whether targeted education was associated with observable changes to physiotherapy students' abilities to portray patient roles in SBE.

METHODS

Second year pre-registration physiotherapy students (n = 40) participated. Students completed online and face-to-face education about SBE, patient portrayal skills, and how to portray a specific patient role. Students were video-recorded portraying patient roles in practical exams before and after the program. Three blinded independent assessors rated the overall quality of portrayals using a purpose-developed assessment instrument.

RESULTS

Twenty-three sets of pre- and post-program videos were analysed. Correlations between assessor scores spanned 0.62 to 0.82 for analyses of interest, which justified using average assessor ratings in analysis. Statistically significant higher scores were seen for post-program assessments for overall portrayal scores (mean difference 6.5, 95%CI [1.51-11.45], p = 0.013), accuracy (mean difference 3.4, 95%CI [0.69-6.13], p = 0.016) and quality (mean difference 3.1, 95%CI [0.64-5.49], p = 0.016).

CONCLUSIONS

Physiotherapy students appear capable of playing realistic patient roles. Peer simulation can be embedded into health professional programs, and education in patient role portrayal appears to be associated with improvements in portrayal quality and realism. Given these findings, further investigation, including testing program effects in a randomised study, is warranted.

Impact of repeated simulation on learning curve characteristics of residents exposed to rare life threatening situations

Background

Little is known about the learning curve characteristics of residents undertaking simulation-based education. It is important to understand the time for acquisition and decay of knowledge and skills needed to manage rare and difficult clinical situations.

Method

Ten anaesthesiology residents underwent simulation-based education to manage a cannot intubate cannot ventilate scenario during general anaesthesia for caesarean section. Their performance was measured using an assessment tool and debriefed by two experienced anaesthesiologists. The parameters against which the performance was judged were grouped into preoperative assessment, preoperative patient care, equipment availability, induction sequence, communication and adherence to airway algorithm protocol. The scenario was repeated at 6 and 12 months thereafter. The residents’ acquisition of knowledge, technical and non-technical skills were assessed and compared at baseline, 6 months and end of 12 months.

Result

The skills of preoperative assessment, preoperative care and communication quickly improved but the specific skill of managing a difficult airway as measured by adherence to an airway algorithm required more than 6 months (CI at 6 vs 12 months: −3.4 to –0.81, p=0.016). The skills of preoperative assessment and preoperative care improved to a higher level quickly and were retained at this improved level. Communication (CI at 0 vs 6 months: −3.78 to −0.22, p=0.045 and at 6 vs 12 months : −3.39 to −1.49, p=0.007) and difficult airway management skill were slower to improve but continued to do so over the 12 months. The compliance to machine check was more gradual and showed an improvement at 12 months.

Conclusion

Our study is unique in analysing the learning curve characteristics of different components of a failed obstetric airway management skill. Repeated simulations over a longer period of time help in better reinforcement, retention of knowledge, recapitulation and implementation of technical and non-technical skills.

1HR ON-CALL - Using Simulated ON-CALL to Underpin Experiential Learning in Final Year Medical Students

A survey of Blackpool Foundation Year One (FY1) doctors found limited training about being on-call. We could not find any direct mention in Tomorrow’s Doctors for preparing undergraduates for this.¹¹

Tomorrow’s Doctors: Outcomes and standards for undergraduate medical education [Internet]. 1st ed. 2009 [cited 30 February 2015]. Available from: http://www.gmc-uk.org/Tomorrow_s_Doctors_1214.pdf_48905759.pdf

Working out of hours, on-call and with a reduced workforce is a known area of anxiety among junior doctors. With few examples in literature,²²

Dickinson M, Pimblett M, Hanson J, Davis M. Reflecting reality: pager simulations in undergraduate education. The Clinical Teacher. 2014;11(6):421–424.

^,33

Fisher J, Martin R, Tate D. Hands on + hands free: simulated on-call interaction. The Clinical Teacher. 2014;11(6):425–428.

we developed a novel approach to aiding final-year medical students prepare for this. A simulated teaching programme allowed students to experience the pressures of working on-call. We hoped to imitate stressors within a safe environment. Students were each given a bleep for an hour. Supervisors role-playing a concerned nurse “bleeped” the students. Each task was held in a folder on different wards (no patient interaction or information was involved). They were relatively simple and designed to stimulate resourcefulness, communication and triage skills. Various resources were available including the number for the medical registrar, played by supervisors. The final station was always the unwell patient aimed at drawing the student immediately to this scenario. A facilitated feedback session explored students’ positive and negative experiences, concerns and coping mechanisms. Over the three years of this running, results were resoundingly positive with students taking great confidence from the programme. During the open feedback session, students valued using open wards and having to navigate in an unfamiliar hospital as a realistic preparation for next year. Being on-call is an inevitable part of a junior doctor’s work and we believe there is scope for better preparation within undergraduate training. We have developed an effective and sustainable simulation that has shown excellent results. Due to the positive reaction and low maintenance of the project, we aim to cement our teaching programme as a permanent feature for undergraduate students at Blackpool Victoria Hospital.

[Simulation curricular content in postgraduate emergency medicine: A multicentre Delphi study]

OBJECTIVES

There is increasing evidence to support integration of simulation into medical training; however, no national emergency medicine (EM) simulation curriculum exists. Using Delphi methodology, we aimed to identify and establish content validity for adult EM curricular content best suited for simulation-based training, to inform national postgraduate EM training.

METHODS

A national panel of experts in EM simulation iteratively rated potential curricular topics, on a 4-point scale, to determine those best suited for simulation-based training. After each round, responses were analyzed. Topics scoring <2/4 were removed and remaining topics were resent to the panel for further ratings until consensus was achieved, defined as Cronbach α ≥ 0.95. At conclusion of the Delphi process, topics rated ≥ 3.5/4 were considered "core" curricular topics, while those rated 3.0-3.5 were considered "extended" curricular topics.

RESULTS

Forty-five experts from 13 Canadian centres participated. Two hundred eighty potential curricular topics, in 29 domains, were generated from a systematic literature review, relevant educational documents and Delphi panellists. Three rounds of surveys were completed before consensus was achieved, with response rates ranging from 93-100%. Twenty-eight topics, in eight domains, reached consensus as "core" curricular topics. Thirty-five additional topics, in 14 domains, reached consensus as "extended" curricular topics.

CONCLUSIONS

Delphi methodology allowed for achievement of expert consensus and content validation of EM curricular content best suited for simulation-based training. These results provide a foundation for improved integration of simulation into postgraduate EM training and can be used to inform a national simulation curriculum to supplement clinical training and optimize learning.

Variation in Intensive Care Unit Intubation Practices in Pulmonary Critical Care Medicine Fellowship

Background: Endotracheal intubation in the intensive care unit (ICU) is a high-risk procedure. Competence in endotracheal intubation is a requirement for Pulmonary and Critical Care Medicine (PCCM) training programs, but fellow experience as the primary operator in intubating ICU patients has not been described on a large scale.

Objective: We hypothesized that significant variation surrounding endotracheal intubation practices in medical ICUs exists in U.S. PCCM training programs.

Methods: We administered a survey to a convenience sample of U.S. PCCM fellows to elicit typical intubation practices in the medical ICU.

Results: Eighty-nine discrete U.S. PCCM and Internal Medicine Critical Care Medicine training programs (77% response rate) were represented. At 43% of programs, the PCCM fellow was “always or almost always” designated the primary operator for intubation of a medical ICU patient, whereas at 21% of programs, the PCCM fellow was “rarely or never” the primary operator responsible for intubating in the ICU. Factors influencing this variation included time of day, hospital policies, attending skill or preference, ICU census and acuity, and patient factors. There was an association between location of the training program, but not program size, and whether the PCCM fellow was the primary operator.

Conclusion: There is significant variation in whether PCCM fellows are the primary operators to intubate medical ICU patients during training. Further work should explore how this variation affects fellow career development and competence in intubation.

A systematic review of simulation-based training tools for technical and non-technical skills in ophthalmology

To evaluate all simulation models for ophthalmology technical and non-technical skills training and the strength of evidence to support their validity and effectiveness. A systematic search was performed using PubMed and Embase for studies published from inception to 01/07/2019. Studies were analysed according to the training modality: virtual reality; wet-lab; dry-lab models; e-learning. The educational impact of studies was evaluated using Messick's validity framework and McGaghie's model of translational outcomes for evaluating effectiveness. One hundred and thirty-one studies were included in this review, with 93 different simulators described. Fifty-three studies were based on virtual reality tools; 47 on wet-lab models; 26 on dry-lab models; 5 on e-learning. Only two studies provided evidence for all five sources of validity assessment. Models with the strongest validity evidence were the Eyesi Surgical, Eyesi Direct Ophthalmoscope and Eye Surgical Skills Assessment Test. Effectiveness ratings for simulator models were mostly limited to level 2 (contained effects) with the exception of the Sophocle vitreoretinal surgery simulator, which was shown at level 3 (downstream effects), and the Eyesi at level 5 (target effects) for cataract surgery. A wide range of models have been described but only the Eyesi has undergone comprehensive investigation. The main weakness is in the poor quality of study design, with a predominance of descriptive reports showing limited validity evidence and few studies investigating the effects of simulation training on patient outcomes. More robust research is needed to enable effective implementation of simulation tools into current training curriculums.

Changing the view: Video versus direct laryngoscopy for intubation in the pediatric emergency department

The aim of this study was to compare the success of first-attempt tracheal intubation in pediatric patients >1-year old performed using video versus direct laryngoscopy and compare the frequency of tracheal intubation-associated events and desaturation among these patients.Prospective observational cohort study conducted in an Academic pediatric tertiary emergency department. We compared 50 children intubated with Mcgrath Mac video laryngoscope (VL group) and an historical series of 141 children intubated with direct laryngoscopy (DL group). All patients were aged 1 to 18 years.The first attempt success rates were 68% (34/50) and 37.6% (53/141) in the VL and DL groups (P < .01), respectively. There was a lower proportion of tracheal intubation-associated events in the VL group (VL, 31.3% [15/50] vs DL, 67.8% [97/141]; P < .01) and no significant differences in desaturation (VL, 35% [14/50] vs DL 51.8% [72/141]; P = .06). The median number of attempts was 1 (range, 1-5) for the VL group and 2 (range, 1-8) for the DL group (P < .01). Multivariate logistic regression showed that video laryngoscope use was associated with higher chances of first-attempt intubation with an odds ratio of 4.5 (95% confidence interval, 1.9-10.4, P < 0.01).Compared with direct laryngoscopy, VL was associated with higher success rates of first-attempt tracheal intubations and lower rates of tracheal intubation-associated events.

Identifying optimal program structure, motivations for and barriers to peer coaching participation for surgeons in practice: a qualitative synthesis

BACKGROUND

Continuous advancement of surgical skills is of utmost importance to surgeons in practice, but traditional learning activities without personalized feedback often do not translate into practice changes in the operating room. Peer coaching has been shown to lead to very high rates of practice changes and utilization of new skills. The purpose of this study was to explore the opinions of practicing surgeons regarding the characteristics of peer coaching programs, in order to better inform future peer coaching program design.

METHODS

Using a convenience sample, practicing general surgeons were invited to participate in focus group interviews. Allocation into groups was according to years in practice. The interviews were conducted using open-ended questions by trained facilitators. Audio recordings were transcribed and coded into themes by two independent reviewers using a grounded theory approach.

RESULTS

Of 52 invitations, 27 surgeons participated: 74% male; years in practice: < 5 years: 33%; 5-15 years: 26%; > 15 years: 41%. Three main themes emerged during coding: ideal program structure, motivations for participation, and barriers to implementation. For the ideal structure of a peer coaching program all groups agreed coaching programs should be voluntary, involve bidirectional learning, and provide CME credits. Live, in situ coaching was preferred. Motivations for coaching participation included: desire to learn new techniques (48%), remaining up to date with the evolution of surgical practice (30%) and improvement of patient outcomes (18%). Barriers to program implementation were categorized as: surgical culture (42%), perceived lack of need (26%), logistical constraints (23%) and issues of coach-coachee dynamics (9%).

CONCLUSION

Peer coaching to refine or acquire new skills addresses many shortcomings of traditional, didactic learning modalities. This study revealed key aspects of optimal program structure, motivations and barriers to coaching which can be used to inform the design of successful peer coaching programs in the future.

Reaching for Peak Performance During Surgical Training: The Value in Assessment Tools and Critical Performance Measures

Peak performance is the ability to achieve optimal outcomes or performance of a given task in a consistent manner. Commonly studied in athletes and musicians, this topic is less often studied in medicine and rarely broached for trainees. However, residency is a crucial period of training where foundations for future success are sewn into habit and daily practice. As educators, we should aim to equip trainees with the tools necessary to achieve peak performance. Studies in athletes and musicians focus on achieving peak performance by setting goals, mastering skills through deliberate practice, perseverance, grit, and discipline. Mentorship is also an important element to streamline efforts for the trainee and help them to focus. In the following article, we review the current literature on applying these principles to surgical trainees to achieve peak performance.

Increasing the Effectiveness of "Stop the Bleed" Training Through Stepwise Mastery Learning with Deliberate Practice

OBJECTIVE

The Stop the Bleed (STB) Campaign supported by the American College of Surgeons Committee on Trauma (ACSCT) and numerous other national organizations aspires to translate lifesaving military successes into reductions in civilian hemorrhagic deaths. While a curricular framework has been described, precise approaches to hands-on training are not specified and training success rates are not yet optimized. Our aim was to test the feasibility and effectiveness of an STB program enhanced by stepwise mastery learning with deliberate practice.

STUDY DESIGN

Learners participated in an STB program combining evidence-based training models: the Peyton 4-stage model and simulation-based mastery learning with deliberate practice. ASCTC-certified STB coaches used a 3-point, behaviorally explicit checklist to test 4 skills: apply direct pressure; apply standard and improvised tourniquets; pack a wound. An anonymous questionnaire was administered.

SETTING

Simulation Center, Emory University School of Medicine.

PARTICIPANTS

College students (N = 30) with no previous trauma training.

RESULTS

100% of participants reached mastery level for all 4 hemorrhage control skills within 4 tries. Additionally, 87% could state a definitive sign of life-threatening bleeding. 76% predicted comfort using a tourniquet in a real-life emergency; among 6 who would be very uncomfortable, 5 nonetheless would definitely recommend the course.

CONCLUSIONS

We demonstrate feasibility and increased effectiveness of an STB course using evidence-based procedural training techniques. Adopting these techniques in current STB programs could close the current trainee performance gap and substantially increase the annual number of successfully trained laypersons over current reported levels with no increase required in enrollees, programs, or resources. Future studies should address the challenges of knowledge retention and skill decay, just-in-time innovations, implementation science methods to broaden access, and barriers to responding to real-life crisis events. Surgery education leaders can close performance gaps and make a unique contribution to the Hartford Consensus principle: No one should die from uncontrolled bleeding.

Rapid cycle system improvement for COVID-19 readiness: integrating deliberate practice, psychological safety and vicarious learning

Introduction

In the face of a rapidly advancing pandemic with uncertain pathophysiology, pop-up healthcare units, ad hoc teams and unpredictable personal protective equipment supply, it is difficult for healthcare institutions and front-line teams to invent and test robust and safe clinical care pathways for patients and clinicians. Conventional simulation-based education was not designed for the time-pressured and emergent needs of readiness in a pandemic. We used ‘rapid cycle system improvement’ to create a psychologically safe learning oasis in the midst of a pandemic. This oasis provided a context to build staff technical and teamwork capacity and improve clinical workflows simultaneously.

Methods

At the Department of Anaesthesia and Intensive Care in Prince of Wales Hospital, a tertiary institution, in situ simulations were carried out in the operating theatres and intensive care unit (ICU). The translational simulation design leveraged principles of psychological safety, rapid cycle deliberate practice, direct and vicarious learning to ready over 200 staff with 51 sessions and achieve iterative system improvement all within 7 days. Staff evaluations and system improvements were documented postsimulation.

Results/Findings

Staff in both operating theatres and ICU were significantly more comfortable and confident in managing patients with COVID-19 postsimulation. Teamwork, communication and collective ability to manage infectious cases were enhanced. Key system issues were also identified and improved.

Discussion

To develop readiness in the rapidly progressing COVID-19 pandemic, we demonstrated that ‘rapid cycle system improvement’ can efficiently help achieve three intertwined goals: (1) ready staff for new clinical processes, (2) build team competence and confidence and (3) improve workflows and procedures.

Real-time light-guided vocal fold injection as a simulation-based training tool

OBJECTIVE

Cricothyroid (CT) approach for vocal fold injection (VFI) has advantages of a low complication rate, suitability for in-office practice, and good patient compliance. However, it requires a high level of experience and a steep learning curve due to invisibility of needle. Recently, real-time light-guided VFI (RL-VFI) was developed for safe and precise injection into laryngeal structures under light guidance. Herein, we describe the development of a simulation-based training (SBT) program using RL-VFI for CT approach and report its preliminary application with in-training otolaryngologists.

METHODS

The workshop comprised 3 sessions: mini-lectures, and two hands-on training courses of conventional VFI and RL-VFI. Excised canine larynges and the device for RL-VFI were prepared for hands-on courses. Comfort levels for VFI was evaluated using visual analogue scale after each session. Trainees were requested to identify the needle tip on the target point lateral to vocal process. The time (s) to reach the target point was measured in all procedures. After workshop, all participants filled out questionnaires regarding their future preference for conventional VFI and RL-VFI.

RESULTS

Eleven otolaryngology residents participated in the study. The mean comfort levels were 1.7 ± 1.6, 5.5 ± 2.6, 4.8 ± 1.7, and 7.5 ± 1.6 for pre-workshop, post-lecture, post-conventional VFI, and post-RL-VFI (P < .001). The mean time (s) to reach the target point were 146.4 ± 90.1 and 42.7 ± 40.5 for conventional VFI and RL-VFI (P = .004). The mean preference scores were 4.2 ± 1.3 and 8.7 ± 1.3 for conventional VFI and RL-VFI (P = .004).

CONCLUSION

SBT program using RL-VFI might improve the comfort levels of trainees for VFI with CT approach. It would be helpful for trainees to practice VFI before trying it on actual patients.

LEVEL OF EVIDENCE

N/A.

Development and initial evaluation of a novel simulation model for comprehensive brain tumor surgery training

Background

Increasing technico-manual complexity of procedures and time constraints necessitates effective neurosurgical training. For this purpose, both screen- and model-based simulations are under investigation. Approaches including 3D printed brains, gelatin composite models, and virtual environments have already been published. However, quality of brain surgery simulation is limited due to discrepancies in visual and haptic experience. Similarly, virtual training scenarios are still lacking sufficient real-world resemblance. In this study, we introduce a novel simulator for realistic neurosurgical training that combines real brain tissue with 3D printing and augmented reality.

Methods

Based on a human CT scan, a skull base and skullcap were 3D printed and equipped with an artificial dura mater. The cerebral hemispheres of a calf’s brain were placed in the convexity of the skullcap and tumor masses composed of aspic, water, and fluorescein were injected in the brain. The skullcap and skull base were placed on each other, glued together, and filled up with an aspic water solution for brain fixation. Then, four surgical scenarios were performed in the operating room as follows: (1) simple tumor resection, (2) complex tumor resection, (3) navigated biopsy via burr hole trepanation, and (4) retrosigmoidal craniotomy. Neuronavigation, augmented reality, fluorescence, and ocular—as well as screen-based (exoscopic)—surgery were available for the simulator training. A total of 29 participants performed at least one training scenario of the simulator and completed a 5-item Likert-like questionnaire as well as qualitative interviews. The questionnaire assessed the realism of the tumor model, skull, and brain tissue as well as the capability for training purposes.

Results

Visual and sensory realism of the skull and brain tissue were rated,”very good,” while the sensory and visual realism of the tumor model were rated “good.” Both overall satisfaction with the model and eligibility of the microscope and neurosurgical instruments for training purposes were rated with “very good.” However, small size of the calf’s brain, its limited shelf life, and the inability to simulate bleedings due to the lack of perfusion were significant drawbacks.

Conclusion

The combination of 3D printing and real brain tissue provided surgical scenarios with very good real-life resemblance. This novel neurosurgical model features a versatile setup for surgical skill training and allows for efficient training of technological support like image and fluorescence guidance, exoscopic surgery, and robotic technology.

Electronic supplementary material

The online version of this article (10.1007/s00701-020-04359-w) contains supplementary material, which is available to authorized users.

Mastery Learning Ensures Correct Personal Protective Equipment Use in Simulated Clinical Encounters of COVID-19

INTRODUCTION

The correct use of personal protective equipment (PPE) limits transmission of serious communicable diseases to healthcare workers, which is critically important in the era of coronavirus disease 2019 (COVID-19). However, prior studies illustrated that healthcare workers frequently err during application and removal of PPE. The goal of this study was to determine whether a simulation-based, mastery learning intervention with deliberate practice improves correct use of PPE by physicians during a simulated clinical encounter with a COVID-19 patient.

METHODS

This was a pretest-posttest study performed in the emergency department at a large, academic tertiary care hospital between March 31-April 8, 2020. A total of 117 subjects participated, including 56 faculty members and 61 resident physicians. Prior to the intervention, all participants received institution-mandated education on PPE use via an online video and supplemental materials. Participants completed a pretest skills assessment using a 21-item checklist of steps to correctly don and doff PPE. Participants were expected to meet a minimum passing score (MPS) of 100%, determined by an expert panel using the Mastery Angoff and Patient Safety standard-setting techniques. Participants that met the MPS on pretest were exempt from the educational intervention. Testing occurred before and after an in-person demonstration of proper donning and doffing techniques and 20 minutes of deliberate practice. The primary outcome was a change in assessment scores of correct PPE use following our educational intervention. Secondary outcomes included differences in performance scores between faculty members and resident physicians, and differences in performance during donning vs doffing sequences.

RESULTS

All participants had a mean pretest score of 73.1% (95% confidence interval [CI], 70.9-75.3%). Faculty member and resident pretest scores were similar (75.1% vs 71.3%, p = 0.082). Mean pretest doffing scores were lower than donning scores across all participants (65.8% vs 82.8%, p<0.001). Participant scores increased 26.9% (95% CI of the difference 24.7-29.1%, p<0.001) following our educational intervention resulting in all participants meeting the MPS of 100%.

CONCLUSION

A mastery learning intervention with deliberate practice ensured the correct use of PPE by physician subjects in a simulated clinical encounter of a COVID-19 patient. Further study of translational outcomes is needed.

Promoting Readiness for Residency: Embedding Simulation-Based Mastery Learning for Breaking Bad News Into the Medicine Subinternship

PURPOSE

It is challenging to add rigorous, competency-based communication skills training to existing clerkship structures. The authors embedded a simulation-based mastery learning (SBML) curriculum into a medicine subinternship to demonstrate feasibility and determine the impact on the foundational skill of breaking bad news (BBN).

METHOD

All fourth-year students enrolled in a medicine subinternship at Northwestern University Feinberg School of Medicine from September 2017 through August 2018 were expected to complete a BBN SBML curriculum. First, students completed a pretest with a standardized patient using a previously developed BBN assessment tool. Learners then participated in a 4-hour BBN skills workshop with didactic instruction, focused feedback, and deliberate practice with simulated patients. Students were required to meet or exceed a predetermined minimum passing standard (MPS) at posttest. The authors compared pretest and posttest scores to evaluate the effect of the intervention. Participant demographic characteristics and course evaluations were also collected.

RESULTS

Eighty-five students were eligible for the study, and 79 (93%) completed all components. Although 55/79 (70%) reported having personally delivered serious news to actual patients, baseline performance was poor. Students' overall checklist performance significantly improved from a mean of 65.0% (SD = 16.2%) items correct to 94.2% (SD = 5.9%; P < .001) correct. There was also statistically significant improvement in scaled items assessing quality of communication, and all students achieved the MPS at mastery posttest. All students stated they would recommend the workshop to colleagues.

CONCLUSIONS

It is feasible to embed SBML into a required clerkship. In the context of this study, rigorous SBML resulted in uniformly high levels of skill acquisition, documented competency, and was positively received by learners.

Relationship between ratings of performance in the simulated and workplace environments among emergency medicine residents

Objectives

The Emergency Medicine (EM) Specialty Committee of the Royal College of Physicians and Surgeons of Canada (RCPSC) specifies that resuscitation entrustable professional activities (EPAs) can be assessed in the workplace and simulated environments. However, limited validity evidence for these assessments in either setting exists. We sought to determine if EPA ratings improve over time and whether an association exists between ratings in the workplace v. simulation environment.

Methods

All Foundations EPA1 (F1) assessments were collected for first-year residents (n = 9) in our program during the 2018–2019 academic year. This EPA focuses on initiating and assisting in the resuscitation of critically ill patients. EPA ratings obtained in the workplace and simulation environments were compared using Lin's concordance correlation coefficient (CCC). To determine whether ratings in the two environments differed as residents progressed through training, a within-subjects analysis of variance was conducted with training environment and month as independent variables.

Results

We collected 104 workplace and 36 simulation assessments. No correlation was observed between mean EPA ratings in the two environments (CCC(8) = -0.01; p = 0.93). Ratings in both settings improved significantly over time (F(2,16) = 18.8; p < 0.001; η2= 0.70), from 2.9 ± 1.2 in months 1–4 to 3.5 ± 0.2 in months 9–12. Workplace ratings (3.4 ± 0.1) were consistently higher than simulation ratings (2.9 ± 0.2) (F(2,16) = 7.2; p = 0.028; η2= 0.47).

Conclusions

No correlation was observed between EPA F1 ratings in the workplace v. simulation environments. Further studies are needed to clarify the conflicting results of our study with others and build an evidence base for the validity of EPA assessments in simulated and workplace environments.

Simulation past, present and future—a decade of progress in simulation-based education in the UK

This review will present developments in simulation-based education (SBE) over the past decade with a focus on activity in the UK’s National Health Service and the role of the national society (the Association for Simulation Practice in Healthcare). The article covers the evolution of strategic changes for the use of SBE in the UK and the operational challenges faced by clinicians and other faculty in SBE. The expansion of the evidence base to support SBE in healthcare both in technical skills and, more broadly, in interprofessional team training is explored. Finally, the wider role of simulation in patient safety and healthcare systems, including testing pathways and the development of cognitive aids and involvement of patients in SBE is considered.

Characteristics of Effective Teachers of Invasive Bedside Procedures: A Multi-institutional Qualitative Study

BACKGROUND

Faculty supervision of invasive bedside procedures (IBPs) in the ICU may enhance procedural education and ensure patient safety. However, there is limited research on teaching effectiveness in the ICU, and there are no best teaching practices regarding the supervision of IBPs.

RESEARCH QUESTION

We conducted a multi-institutional qualitative study of pulmonary and critical care medicine faculty and fellows to better understand characteristics of effective IBP teachers.

STUDY DESIGN AND METHODS

Separate focus groups (FGs) were conducted with fellows and faculty at four large academic institutions that were geographically distributed across the United States. FGs were facilitated by a trained investigator, audio-recorded, and transcribed verbatim for analysis. Themes were identified inductively and compared with constructs from social and situated learning theories. Data were analyzed between and across professional groups. Qualitative research software (NVivo; QSR International) was used to facilitate data organization and create an audit trail of the analysis. A multidisciplinary research team was engaged to minimize interpretive bias.

RESULTS

Thirty-three faculty and 30 fellows participated. Inductive analysis revealed three categories of themes among successful IBP teachers: traits, behaviors, and context. Traits included calm demeanor, trust, procedural competence, and effective communication. Behaviors included leading preprocedure huddles to assess learners' experiences and define expectations; debriefing to provide feedback; and allowing appropriate autonomy. Context included learning climate, levels of distraction, patient acuity, and institutional culture.

INTERPRETATION

We identified specific traits and behaviors of effective IBP teachers that intersect with the practice environment, which highlights the challenge of teaching IBPs. Notably, FG participants emphasized interpersonal, more than technical, aspects of successful IBP teachers. These findings should inform future curricula on teaching IBPs in the ICU, standardize IBP teaching for pulmonary and critical care medicine fellows, and reduce patient injury from procedural complications.

Educational Interventions to Enhance Situation Awareness: A Systematic Review and Meta-Analysis

STATEMENT

We conducted a systematic review to evaluate the comparative effectiveness of educational interventions on health care professionals' situation awareness (SA). We searched MEDLINE, CINAHL, HW Wilson, ERIC, Scopus, EMBASE, PsycINFO, psycARTICLES, Psychology and Behavioural Science Collection and the Cochrane library. Articles that reported a targeted SA intervention or a broader intervention incorporating SA, and an objective outcome measure of SA were included. Thirty-nine articles were eligible for inclusion, of these 4 reported targeted SA interventions. Simulation-based education (SBE) was the most prevalent educational modality (31 articles). Meta-analysis of trial designs (19 articles) yielded a pooled moderate effect size of 0.61 (95% confidence interval = 0.17 to 1.06, P = 0.007, I = 42%) in favor of SBE as compared with other modalities and a nonsignificant moderate effect in favor of additional nontechnical skills training (effect size = 0.54, 95% confidence interval = 0.18 to 1.26, P = 0.14, I = 63%). Though constrained by the number of articles eligible for inclusion, our results suggest that in comparison with other modalities, SBE yields better SA outcomes.

In-situ Simulation-Based Module to Train Interns in Resuscitation Skills During Cardiac Arrest

PURPOSE

Competency to perform cardiopulmonary resuscitation is essential to improve the outcome during an event of cardiac arrest. Medical internship is a "transition phase" beyond which physicians are exposed to the unpreparedness and anxiety while dealing with life-threatening conditions which need rapid actions. Experiential learning, deliberate practice and immediate feedback are the primary advantages of simulation-based medical education. In-situ simulation occurs in a real patient care environment utilizing the available resources. Our aim was to introduce in-situ simulation-based resuscitation skills training for medical interns.

METHODOLOGY

This was a prospective quasi-experimental study with mixed research design conducted in the emergency department. The knowledge score questionnaire, skill-score sheet, post-simulation feedback from the interns and staff were designed, validated through experts, peer-reviewed and pilot tested. All the interns posted in the emergency department participated in the training. Appropriate statistical analyses were applied for quantitative data. Thematic analysis of the data from the open-ended questions was used to identify the themes.

RESULTS

Six groups of interns (n=35) participated in at least one of the 40 in-situ simulation sessions. The means of the pre- and post-simulation knowledge scores were 5.89± 2.8 SD and 11.74± 1.8 SD, respectively, and the difference was statistically significant [t (35) = 16.705, p<0.001]. The skill score had a positive correlation (Pearson correlation coefficient r=0.985, p=0.01) with the number of simulations performed in each group. The thematic analysis uncovered the themes related to the simulation methodology, reflective learning, enhancement of the non-technical skills and effective feedback. The self-reported enhancement in teamwork skills and the confidence level was significant. The feedback of the staff in the Emergency Department showed a positive attitude towards in-situ simulation.

CONCLUSION

This study demonstrated an increase in the knowledge and skills of the interns after participation in the in-situ simulation sessions. Based on the results, the study recommends that in-situ simulation-based teaching should be incorporated in the existing medical education system.

Simulation-based education for paediatric surgeons: Does it really improve technical skills?

In recent years, there has been an increasing interest in employing simulation methodology for teaching surgical skills. Supposedly, skills achieved in the simulation arena, should lead to an improvement of surgical performance in the operating room. In this article we aim to summarise those areas of simulation-based research that relate to paediatric surgery; we present the current evidence behind low-fidelity simulation and computer-based simulation in improving the acquisition of surgical skills.

Consensus-Based Expert Development of Critical Items for Direct Observation of Point-of-Care Ultrasound Skills

BACKGROUND

Point-of-care ultrasound (POCUS) is increasingly used in a number of medical specialties. To support competency-based POCUS education, workplace-based assessments are essential.

OBJECTIVE

We developed a consensus-based assessment tool for POCUS skills and determined which items are critical for competence. We then performed standards setting to set cut scores for the tool.

METHODS

Using a modified Delphi technique, 25 experts voted on 32 items over 3 rounds between August and December 2016. Consensus was defined as agreement by at least 80% of the experts. Twelve experts then performed 3 rounds of a standards setting procedure in March 2017 to establish cut scores.

RESULTS

Experts reached consensus for 31 items to include in the tool. Experts reached consensus that 16 of those items were critically important. A final cut score for the tool was established at 65.2% (SD 17.0%). Cut scores for critical items are significantly higher than those for noncritical items (76.5% ± SD 12.4% versus 53.1% ± SD 12.2%, P < .0001).

CONCLUSIONS

We reached consensus on a 31-item workplace-based assessment tool for identifying competence in POCUS. Of those items, 16 were considered critically important. Their importance is further supported by higher cut scores compared with noncritical items.

An Event-based Approach to Measurement: Facilitating Observational Measurement in Highly Variable Clinical Settings

BACKGROUND

Translational research in medical education requires the ability to rigorously measure learner performance in actual clinical settings; however, current measurement systems cannot accommodate the variability inherent in many patient care  environments. This is especially problematic in emergency medicine, where patients represent a wide spectrum of severity for a single clinical presentation. Our objective is to describe and implement EBAM, an event-based approach to measurement that can be applied to actual emergency medicine clinical events.

METHODS

We used a four-step event-based approach to create an emergency department trauma resuscitation patient care measure. We applied the measure to a database of 360 actual trauma resuscitations recorded in a Level I trauma center using trained raters. A subset (n = 50) of videos was independently rated in duplicate to determine inter-rater reliability. Descriptive analyses were performed to describe characteristics of resuscitation events and Cohen's kappa was used to calculate reliability.

RESULTS

The methodology created a metric containing both universal items that are applied to all trauma resuscitation events and conditional items that only apply in certain situations. For clinical trauma events, injury severity scores ranged from 1 to 75 with a mean (±SD) of 21 (±15) and included both blunt (254/360; 74%) and penetrating (86/360; 25%) traumatic injuries, demonstrating the diverse nature of the clinical encounters. The mean (±SD) Cohen's kappa for patient care items was 0.7 (±0.3).

CONCLUSION

We present an event-based approach to performance assessment that may address a major gap in translational education research. Our work centered on assessment of patient care behaviors during trauma resuscitation. More work is needed to evaluate this approach across a diverse array of clinical events.

Achieving expert performance through simulation-based education and application of mastery learning principles

Medical education has undergone significant change as we have moved from a purely apprenticeship model to one of competence. Simulation-based education can and does play a significant role in the development of an expert surgeon. Proven concepts of simulation-based education include distributed deliberate practice, content that challenges the learner, formative and summative feedback. The application of mastery learning principles is critical to achieve the best retention and outcomes related to an educational program. In this manuscript, we explore the methodology and justification for Mastery Learning as the gold standard of surgical education.

Formal Training Efforts to Develop Simulation Educators: An Integrative Review

STATEMENT

Formal training for educators who use simulation-based education (SBE) is required by standards of best practice, simulation guidelines, regulatory, and accrediting bodies. Training efforts to establish educator competency for SBE are being offered. However, a systematic review of this body of literature has yet to be conducted. The purpose of this integrative review was to appraise formal training efforts of educators who use SBE. The aims were to summarize the training topics, describe the structure of training programs, and explore evaluation methods of educators. The New World Kirkpatrick Model guided the review. A PRISMA search approach yielded 2007 citations of which 38 met inclusion criteria. Analysis supports a formalized training process that uses a combination of didactic material, time for repetitive practice, and ongoing feedback with longitudinal and scaffolded delivery approaches. An identified gap in the literature is threshold levels for determining competency of educators. Recommendations for planning simulation training programs are provided.

Exploring transformative learning when developing medical students' non-technical skills

CONTEXT

Non-technical skills (NTS) training should be incorporated into medical students' education and simulation-based approaches are often utilised to facilitate this. Such experiences have the potential to foster transformative learning by facilitating a reassessment of one's prior assumptions and a significant shift in one's outlook, referred to as the process of perspective transformation. The aim of this research was to explore how NTS training might facilitate transformative learning in final-year medical students.

METHODS

Following ethical approval, medical student volunteers from four medical schools (Aberdeen, Dundee, Edinburgh and Glasgow) participated in simulation sessions, were debriefed with an emphasis on NTS using a behavioural marker system and then took part in focus groups. Focus group discussions were semi-structured and questions were based on the phases of perspective transformation identified by Jack Mezirow. Focus group discussions were audiorecorded, transcribed verbatim, anonymised and analysed using template analysis.

RESULTS

A total of 33 medical students took part in five focus groups. There was evidence of the following stages of perspective transformation: Phase 2 (self-examination with emotional disturbance, including fear, anxiety, guilt, shame and frustration); Phase 3 (critical assessment of assumptions, including the undervaluing of NTS, recognising that technical skills alone are insufficient, and recognising that it is possible to improve one's NTS); Phase 5 (exploring options for new roles, relationships and actions), and Phase 6 (planning a course of action for future simulations, as a medical student and as a doctor).

CONCLUSIONS

This study deepens our understanding of how exposure to NTS training in simulation-based education influences the learning of medical students and shows that such exposure can result in the cognitive phases of transformative learning. It provides us with valuable insights into medical students' perspectives on their learning of NTS at a pivotal stage in training and represents an interesting way of assessing the educational impact of such sessions.