Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains

Video-assisted reflection: improving OSCE feedback

BACKGROUND

Objective Structured Clinical Examinations (OSCEs) are commonly used to provide feedback to students on their performance in formative examinations. However, students are often unable to act independently on the feedback they receive. This study explored how the use of video-assisted reflection in OSCEs can enhance students' ability to reflect and engage in sustainable feedback.

METHODS

Twenty-one students undertaking a mock-final OSCE consented to have one of their examination stations filmed. Participants completed a series of reflective forms immediately after the OSCE, after verbal feedback from an examiner and finally, after watching the video of their own performance. Students were asked to predict their overall grade as well as list areas for improvement. Pearson r correlations examined the relationship between the examiners' grades and the candidates' self-predicted grades. Wilcoxon signed-rank tests were used to compare the length of reflections at each stage. Semi-structured interviews were conducted to explore students' beliefs on self-efficacy and how the video-assisted reflection altered their ability to act on feedback.

RESULTS

The students' ability to self-assess and gauge their own performance improved significantly after undertaking the video-assisted reflection (p < 0.01). Furthermore, video-assisted reflection significantly increased the length of the student's reflections. In interviews, participants described multiple ways in which the video-assisted reflection improved their confidence and ability to act on feedback, highlighting a clear enhancement in self-efficacy.

DISCUSSION

Video-assisted reflection of recorded OSCE stations represents an effective approach to increase student self-efficacy and subsequently improve engagement in sustainable feedback practice.

Twelve tips on guiding preparation for both high-stakes exams and long-term learning

High-stakes exams including admissions, licensing, and maintenance of certification examinations are commonplace in health professions education. Although exam scores and performance can often serve gate-keeping purposes, the broader goal of health professions education is to foster deep, self-directed, meaningful, motivated learning. Establishing strong support structures that emphasize deep learning and understanding rather than exam scores can be beneficial to preparing learners who have the knowledge base to be excellent practitioners. This article offers guidance that can be used by academic support centres, medical educators, learning specialists, and faculty advisors, or even test-takers, to help learners to balance score achievement and knowledge development, while simultaneously cultivating more efficient and motivated studying and increasingly self-regulated learning. This series of tips details considerations for building academic success supports, fostering a growth mindset, planning efficient and effective studying efforts, utilizing test-enhanced learning strategies, exam-taking skills practice, and other support structures that can help strengthen learning experiences overall.

Identification and Management of Pediatric Sepsis: A Medical Student Curricular Supplement for PICU and NICU Rotations

INTRODUCTION

Medical students frequently report lack of confidence and skill in managing ill pediatric patients. We aimed to implement targeted learning interventions to address these knowledge gaps, specifically focusing on pediatric sepsis. Our objective was to create a curriculum to advance knowledge and confidence in identifying and managing pediatric sepsis.

METHODS

We designed this curriculum to augment medical student pediatric ICU (PICU) and neonatal ICU (NICU) rotations. We first emailed students a pretest and upon completion, we emailed students a series of brief educational videos. Students then participated in a simulation experience designed to assess their ability to diagnose and manage severe sepsis. We provided immediate debriefing after each session. Upon completion of the simulation experience, we emailed students a posttest (identical to the pretest). The pre-/posttest included multiple-choice questions to assess the students' ability to recognize and manage pediatric sepsis and septic shock, as well as Likert-scale questions assessing confidence levels in diagnosis and management of pediatric sepsis. We performed paired Student t tests comparing knowledge-based question scores and Likert-scale results.

RESULTS

Of students, 40 enrolled in and 30 completed the curriculum between 2015 and 2020. When comparing pre- and posttest results, we found a significant improvement in knowledge scores (33% mean increase, 95% CI [22%-45%], p < .001) and confidence levels (mean increase in Likert scale score of 1.5, 95% CI [1.2-1.7], p < .001).

DISCUSSION

Results suggested that the curriculum advanced students' knowledge scores and improved self-reported confidence in managing theoretical pediatric patients with sepsis.

Skull Base Training and Mentorship

OBJECTIVE

To review teaching and mentoring techniques of experienced skull base surgeons and educators STUDY DESIGN:: Expert commentary.

SETTING

8th Quadrennial International Conference on vestibular schwannoma and other CPA tumors, panel on teaching, and mentoring.

MAIN OUTCOME MEASURES

Experiences and opinions of experienced skull base surgeons, both neurosurgeons and neurotologists, presented and discussed at the conference.

CONCLUSIONS

Obtaining surgical mastery is essential for the teachers of skull base surgery. Hard work and practice with immediate and constant feedback on performance is an essential component to success. Creating a patient-centered culture that encourages academic achievement is an accelerator for success of a training program. Both the mentor and the mentee must play an intentional and active role to maximize learning.

Gamifying health literacy: how can digital technology optimize patient outcomes in surgery?

The digital age is entrenched in our society, with constant innovation driving change in the way clinicians and patients manage their health concerns. Health literacy is emerging as an important modifiable factor that can affect clinical and patient outcomes, yet traditional forms of patient education have shown mixed results. Digital media and technologies, the concept of gamification as a means to improve patient health literacy, and its potential for misuse will be explored in this review, in the context of a digital, gamified tool that could support patients along their surgical journey.

Does high-variation training facilitate transfer of training in paediatric transthoracic echocardiography?

BACKGROUND

Factors that facilitate transfer of training in paediatric echocardiography remain poorly understood. This study assessed whether high-variation training facilitated successful transfer in paediatric echocardiography.

METHODS

A mixed-methods study of transfer of technical and interpretive skill application amongst postgraduate trainees. Trainees were randomised to a low or high-variation training group. After a period of 8 weeks intensive echocardiography training, we video-recorded how trainees completed an echocardiogram in a complex cardiac lesion not previously encountered. Blinded quantitative analysis and scoring of trainee performance (echocardiogram performance, report, and technical proficiency) were performed using a validated assessment tool by a blinded cardiologist and senior cardiac physiologist. Qualitative interviews of the trainees were recorded to ascertain trainee experiences during the training and transfer process.

RESULTS

Sixteen trainees were enrolled in the study. For the cumulative score for all three components tested (echocardiogram performance, report, and technical proficiency), χ2 = 8.223, p = .016, which showed the high-variation group outperformed the low-variation group. Two common themes which assisted in the transfer emerged from interviews are as follows: (1) use of strategies described in variation theory to describe abnormal hearts, (2) the use of formative live feedback from trainers during hands-on training.

CONCLUSION

Training strategies exposing trainees to high-variation training may aid transfer of paediatric echocardiography skills.

Unravelling the polyphony in clinical reasoning research in medical education

RATIONALE

Clinical reasoning lies at the heart of medical practice and has a long research tradition. Nevertheless, research is scattered across diverse academic disciplines with different research traditions in a wide range of scientific journals. This polyphony is a source of conceptual confusion.

AIMS AND OBJECTIVES

We sought to explore the underlying theoretical assumptions of clinical reasoning aiming to promote a comprehensive conceptual and theoretical understanding of the subject area. In particular, we asked how clinical reasoning is defined and researched and what conceptualizations are relevant to such uses.

METHODS

A scoping review of the clinical reasoning literature was undertaken. Using a "snowball" search strategy, the wider scientific literature on clinical reasoning was reviewed in order to clarify the different underlying conceptual assumptions underlying research in clinical reasoning, particularly to the field of medical education. This literature included both medical education, as well as reasoning research in other academic disciplines outside medical education, that is relevant to clinical reasoning. A total of 124 publications were included in the review.

RESULTS

A detailed account of the research traditions in clinical reasoning research is presented. In reviewing this research, we identified three main conceptualisations of clinical reasoning: "reasoning as cognitive activity," "reasoning as contextually situated activity," and "reasoning as socially mediated activity." These conceptualisations reflected different theoretical understandings of clinical reasoning. Each conceptualisation was defined by its own set of epistemological assumptions, which we have identified and described.

CONCLUSIONS

Our work seeks to bring into awareness implicit assumptions of the ongoing clinical reasoning research and to hopefully open much needed channels of communication between the different research communities involved in clinical reasoning research in the field.

Conjunctive standards in OSCEs: The why and the how of number of stations passed criteria

INTRODUCTION

Many institutions require candidates to achieve a minimum number of OSCE stations passed (MNSP) in addition to the aggregate pass mark. The stated rationale is usually that this conjunctive standard prevents excessive degrees of compensation across an assessment. However, there is a lack of consideration and discussion of this practice in the medical education literature.

METHODS

We consider the motivations for the adoption of the MNSP from the assessment designer perspective, outlining potential concerns about the complexity of what the OSCE is trying to achieve, particularly around the blueprinting process and the limitations of scoring instruments. We also introduce four potential methods for setting an examinee-centred MNSP standard, and highlight briefly the theoretical advantages and disadvantages of these approaches.

DISCUSSION AND CONCLUSION

There are psychometric arguments for and against the limiting of compensation in OSCEs, but it is clear that many stakeholders value the application of an MNSP standard. This paper adds to the limited literature on this important topic and notes that current MNSP practices are often problematic in high stakes settings. More empirical work is needed to develop understanding of the impact on pass/fail decision-making of the proposed standard setting methods developed in this paper.

Effectiveness of a serious game addressing guideline adherence: cohort study with 1.5-year follow-up

BACKGROUND

Patients presenting with acute shortness of breath and chest pain should be managed according to guideline recommendations. Serious games can be used to train clinical reasoning. However, only few studies have used outcomes beyond student satisfaction, and most of the published evidence is based on short-term follow-up. This study investigated the effectiveness of a digital simulation of an emergency ward regarding appropriate clinical decision-making.

METHODS

In this prospective trial that ran from summer 2017 to winter 2018/19 at Göttingen Medical University Centre, a total of 178 students enrolled in either the fourth or the fifth year of undergraduate medical education took six 90-min sessions of playing a serious game ('training phase') in which they managed virtual patients presenting with various conditions. Learning outcome was assessed by analysing log-files of in-game activity (including choice of diagnostic methods, differential diagnosis and treatment initiation) with regard to history taking and patient management in three virtual patient cases: Non-ST segment elevation myocardial infarction (NSTEMI), pulmonary embolism (PE) and hypertensive crisis. Fourth-year students were followed up for 1.5 years, and their final performance was compared to the performance of students who had never been exposed to the game but had otherwise taken the same five-year undergraduate course.

RESULTS

During the training phase, overall performance scores increased from 57.6 ± 1.1% to 65.5 ± 1.2% (p < 0.001; effect size 0.656). Performance remained stable over 1.5 years, and the final assessment revealed a strong impact of ever-exposure to the game on management scores (72.6 ± 1.2% vs. 63.5 ± 2.1%, p < 0.001; effect size 0.811). Pre-exposed students were more than twice as likely to correctly diagnose NSTEMI and PE and showed significantly greater adherence to guideline recommendations (e.g., troponin measurement and D-dimer testing in suspected PE).

CONCLUSIONS

The considerable difference observed between previously exposed and unexposed students suggests a long-term effect of using the game although retention of specific virtual patient cases rather than general principles might partially account for this effect. Thus, the game may foster the implementation of guideline recommendations.

Automated assessment of cortical mastoidectomy performance in virtual reality

INTRODUCTION

Cortical mastoidectomy is a core skill that Otolaryngology trainees must gain competency in. Automated competency assessments have the potential to reduce assessment subjectivity and bias, as well as reducing the workload for surgical trainers.

OBJECTIVES

This study aimed to develop and validate an automated competency assessment system for cortical mastoidectomy.

PARTICIPANTS

Data from 60 participants (Group 1) were used to develop and validate an automated competency assessment system for cortical mastoidectomy. Data from 14 other participants (Group 2) were used to test the generalisability of the automated assessment.

DESIGN

Participants drilled cortical mastoidectomies on a virtual reality temporal bone simulator. Procedures were graded by a blinded expert using the previously validated Melbourne Mastoidectomy Scale: a different expert assessed procedures by Groups 1 and 2. Using data from Group 1, simulator metrics were developed to map directly to the individual items of this scale. Metric value thresholds were calculated by comparing automated simulator metric values to expert scores. Binary scores per item were allocated using these thresholds. Validation was performed using random sub-sampling. The generalisability of the method was investigated by performing the automated assessment on mastoidectomies performed by Group 2, and correlating these with scores of a second blinded expert.

RESULTS

The automated binary score compared with the expert score per item had an accuracy, sensitivity and specificity of 0.9450, 0.9547 and 0.9343, respectively, for Group 1; and 0.8614, 0.8579 and 0.8654, respectively, for Group 2. There was a strong correlation between the total scores per participant assigned by the expert and calculated by the automatic assessment method for both Group 1 (r = .9144, P < .0001) and Group 2 (r = .7224, P < .0001).

CONCLUSION

This study outlines a virtual reality-based method of automated assessment of competency in cortical mastoidectomy, which proved comparable to the assessment provided by human experts.

European Resuscitation Council Guidelines 2021: Education for resuscitation

These European Resuscitation Council education guidelines, are based on the 2020 International Consensus on Cardiopulmonary Resuscitation Science with Treatment Recommendations. This section provides guidance to citizens and healthcare professionals with regard to teaching and learning the knowledge, skills and attitudes of resuscitation with the ultimate aim of improving patient survival after cardiac arrest.

A Model for an Online Learning Management System for Simulation-Based Acquisition of Psychomotor Skills in Health Professions Education

The current coronavirus disease (COVID-19) pandemic has shifted traditional educational approaches in health professions education (HPE) from in-person to remote learning. Although pedagogical strategies have been developed and implemented rapidly to support cognitive and affective domains of learning in HPE, less progress has occurred in psychomotor skills acquisition. Psychomotor skills, referred to as technical skills training, are underpinned by educational theories and conceptual frameworks. Considering the widening gap in learning domains, this editorial provides an overview and recommendations for developing and implementing remote training supported by educational theories, such as deliberate practice, and conceptual frameworks in technical skills acquisition in HPE. We begin by discussing the unique curricular needs for remote psychomotor skills in medical teaching-learning contexts and subsequently present a theory-driven and evidence-based model for remote psychomotor skills acquisition.

Can ultrasound novices develop image acquisition skills after reviewing online ultrasound modules?

BACKGROUND

Point-of-care ultrasound is becoming a ubiquitous diagnostic tool, and there has been increasing interest to teach novice practitioners. One of the challenges is the scarcity of qualified instructors, and with COVID-19, another challenge is the difficulty with social distancing between learners and educators. The purpose of our study was to determine if ultrasound-naïve operators can learn ultrasound techniques and develop the psychomotor skills to acquire ultrasound images after reviewing SonoSim® online modules.

METHODS

This was a prospective study evaluating first-year medical students. Medical students were asked to complete four SonoSim® online modules (aorta/IVC, cardiac, renal, and superficial). They were subsequently asked to perform ultrasound examinations on standardized patients utilizing the learned techniques/skills in the online modules. Emergency Ultrasound-trained physicians evaluated medical students' sonographic skills in image acquisition quality, image acquisition difficulty, and overall performance. Data are presented as means and percentages with standard deviation. All P values are based on 2-tailed tests of significance.

RESULTS

Total of 44 medical students participated in the study. All (100%) students completed the hands-on skills evaluation with a median score of 83.7% (IQR 76.7-88.4%). Thirty-three medical students completed all the online modules and quizzes with median score of 87.5% (IQR 83.8-91.3%). There was a positive association between module quiz performance and the hands-on skills performance (R-squared = 0.45; p < 0.001). There was no statistically significant association between module performance and hands-on performance for any of the four categories individually. In all four categories, the evaluators' observation of the medical students' difficulty obtaining views correlated with hands-on performance scores.

CONCLUSIONS

Our study findings suggest that ultrasound-naïve medical students can develop basic hands-on skills in image acquisition after reviewing online modules.

Validated specialty-specific models for multi-disciplinary microsurgery training laboratories: a systematic review

BACKGROUND

Laboratory simulation is increasingly important for teaching microsurgical skills. Training microsurgeons of different specialties within the same simulation laboratory increases efficiency of resource use. For maximal benefit, simulations should be available for trainees to practice specialty-specific, higher-order skills. Selection of appropriate simulations requires knowledge of the efficacy and validity of the numerous described laboratory models. Here we present a systematic review of validated training models that may serve as useful adjuncts to achieving competency in specialty elements of microsurgery, and appraise the evidence behind them.

METHODS

In setting up a multi-disciplinary microsurgery training course, we performed a systematic review according to preferred reporting items for systematic reviews and meta-analyses guidelines. EMBASE, MEDLINE, Cochrane and PubMed databases were searched for studies describing validated, microscope-based, specialty-specific simulations, and awarded a level of evidence and level of recommendation based on a modified Oxford Centre for Evidence-Based Medicine classification.

RESULTS

A total of 141 papers describing specialty-specific microsimulation models were identified, 49 of which included evidence of validation. Eleven were in the field of neurosurgery, 21 in otolaryngology/head and neck surgery, two in urology/gynaecology and 15 plastic and reconstructive surgery. These papers described synthetic models in 19 cases, cadaveric animals in 10 cases, live animals in 12 cases and human cadaveric material in 10 cases.

CONCLUSION

Numerous specialty-specific models for use in the microscope laboratory are available, but the quality of evidence for them is poor. Provision of models that span numerous specialties may encourage use of a microscope lab whilst still enabling more specific skills training over a 'one-size-fits-all' approach.

The Impact of Subspecialty Fellows on Orthopaedic Resident Surgical Experience: A Multicenter Study of 51,111 Cases

INTRODUCTION

Meaningful participation in surgery is important for orthopaedic resident education. This study aimed to quantify the effect of fellows on resident surgical experience. We hypothesized that as fellowship programs expanded, resident caseload would decrease, whereas "double-scrubbed" cases would increase.

METHODS

This multicenter retrospective study included 9 years of surgical caselog data from two orthopaedic residency programs. Six subspecialty services on which fellow number varied over time were included (trauma, spine, foot and ankle, adult reconstruction, and hand). Case volume and personnel composition per case were extracted. Statistical analysis was performed with two-sample equal variance Student t-tests.

RESULTS

A total of 51,111 cases were assessed. Surgical volume increased across all sites/services over time. Fellow numbers did not affect average resident caseload. However, in years with more fellows, an 11% decrease in one-on-one resident-attending cases (P = 0.002) and a 17% increase in resident-fellow-attending "double-scrubbed" cases was observed (P < 0.001).

DISCUSSION

Increasing orthopaedic fellows did not affect resident case volume but resulted in fewer one-on-one cases with the attending and more "double-scrubbed" cases with a fellow. The implications of these findings to resident education require further study, but orthopaedic educators should be aware of these findings to try to maximize educational opportunities.

LEVEL OF EVIDENCE

Level III.

The effect of expertise, target usefulness and image structure on visual search

Experts outperform novices on many cognitive and perceptual tasks. Extensive training has tuned experts to the most relevant information in their specific domain, allowing them to make decisions quickly and accurately. We compared a group of fingerprint examiners to a group of novices on their ability to search for information in fingerprints across two experiments-one where participants searched for target features within a single fingerprint and another where they searched for points of difference between two fingerprints. In both experiments, we also varied how useful the target feature was and whether participants searched for these targets in a typical fingerprint or one that had been scrambled. Experts more efficiently located targets when searching for them in intact but not scrambled fingerprints. In Experiment 1, we also found that experts more efficiently located target features classified as more useful compared to novices, but this expert-novice difference was not present when the target feature was classified as less useful. The usefulness of the target may therefore have influenced the search strategies that participants used, and the visual search advantages that experts display appear to depend on their vast experience with visual regularity in fingerprints. These results align with a domain-specific account of expertise and suggest that perceptual training ought to involve learning to attend to task-critical features.

Introduction of an ENT emergency-safe boot camp into postgraduate surgical training in the Republic of Ireland

Background

Otolaryngology (ENT) is a craft surgical specialty, which presents significant challenges for the novice surgical trainee (NST), when managing patients with potentially life-threatening conditions. We assess current evidence behind intensive, simulation-rich training programmes or “boot camps”. We outline an ENT emergency-safe boot camp for NSTs and the impact this can have on trainees’ competence when dealing with ENT emergencies.

Methods

We designed a standardized “ENT emergency-safe boot camp”, aimed at NSTs with a view to up-skilling them in the management of 3 critical clinical scenarios namely post-thyroidectomy haematoma, post-tonsillectomy haemorrhage and epistaxis. All core surgical trainees (CSTs) (n = 54) on the National Surgical Training programme at the Royal College of Surgeons (RCSI) were enrolled on the boot camp. A pre- and post-boot camp anonymised self-assessment was completed by all participants.

Results

Trainees’ self-assessment demonstrated an increase in median competence scores from 3/5 to 4/5 for management of post-thyroidectomy haematoma and epistaxis. There was a greater increase in self-assessed competence scores from 2/5 to 4/5 for the management of post-tonsillectomy haemorrhage. Ninety-two percent of respondents said the course added to their previous knowledge, 70% of respondents said it improved their technical skills and 85% of participants said the course gave them more confidence with handling ENT related on-call problems.

Conclusion

Study participants expressed an increase in confidence when dealing with specific ENT emergencies, following boot camp training. This correlates with published data. Patient safety should be enhanced, as a result of participation in this dedicated ENT emergency-safe boot camp.

Performance Improvement With Implementation of a Surgical Skills Curriculum

OBJECTIVE

To assess the efficacy of an intern surgical skills curriculum involving a boot camp for core open and laparoscopic skills, self-guided practice with positive and negative incentives, and semiannual performance evaluations.

DESIGN

Longitudinal cohort study.

SETTING

Academic tertiary care center.

PARTICIPANTS

Intervention group (n = 15): residents who completed the intern surgical skills curriculum and had performance evaluations in fall of intern year, spring of intern year, and fall of second year. Control group (n = 8): second-year residents who were 1 year ahead of the intervention group in the same residency program, did not participate in the curriculum, and had performance evaluations in fall of second year.

RESULTS

In fall of second year of residency, the intervention group had better performance (presented as median values with interquartile ranges) than the control group on one-hand ties (left hand: 9.1 [6.3-10.1] vs 14.6 [13.5-15.4] seconds, p = 0.007; right hand: 8.7 [8.5-9.6] vs 11.5 [9.9-16.8] seconds, p = 0.039). The intervention group also had better performance on all open suturing skills, including mattress suturing (vertical: 33.4 [30.0-40.0] vs 55.8 [50.0-67.6] seconds, p = 0.001; horizontal: 28.7 [27.3-39.9] vs 52.7 [40.7-57.8] seconds, p = 0.003), and a water-filled glove clamp, divide, and ligate task (28.0 [25.0-31.0] vs 59.1 [53.0-93.0] seconds, p < 0.001). Finally, the intervention group had better performance on all laparoscopic skills, including peg transfer (66.0 [59.0-82.0] vs 95.2 [87.5-101.5] seconds, p = 0.018), circle cut (82.0 [69.0-124.0] seconds vs 191.8 [155.5-231.5] seconds, p = 0.002), and intracorporeal suturing (195.0 [117.0-200.0] seconds vs 359.5 [269.0-450.0] seconds, p = 0.002).

CONCLUSIONS

Implementation of a comprehensive surgical skills curriculum was associated with improved performance on core open and laparoscopic skills. Further research is needed to understand and optimize motivational factors for deliberate practice and surgical skill acquisition.

Preparation for Challenging Cases: What Differentiates Expert From Novice Surgeons?

OBJECTIVE

The study of expert performance provides a rich field for exploration in the surgical literature. This study aimed to examine the difference between expert and novice surgeons in their preparation for challenging cases.

DESIGN

Expert (attending) and novice (postgraduate-year 2) surgeons were presented two cases of complicated cholecystitis and were asked how they would prepare, what they would expect to encounter intraoperatively, and how they would deal with these challenges. Their responses were recorded, transcribed verbatim and analyzed using thematic analysis.

SETTING

Academic teaching hospitals.

PARTICIPANTS

Two group of expert and novice surgeons.

RESULTS

Nine experts and eleven novices from two academic centers participated. The majority of novices focused on patient history, work-up, preoperative optimization, anatomy, and anticipation of intraoperative challenges. In addition to the patient's presentation and preoperative optimization, most experts' thoughts were directed toward preparation for surgery (level of urgency, required skills in surgical team, case difficulty, and risk of conversion to open). Experts would involve the patient in the decision-making and were more likely to communicate with the operating room team. While novices attempted to predict challenges depending on gallbladder condition and intra-abdominal adhesions, the experts highlighted the importance of various elements of the operative field, the detail of the technique and possible challenges, and their troubleshooting plans. Regarding operative planning to address anticipated challenges, novices would tailor their plan to patient characteristics and verbalized an analytical "if-then" approach for all possibilities they might encounter. Experts would start with their standard technique regardless of case complexity and would deal with contingencies as they arise. Safety was a critical part of expert surgeons' plans.

CONCLUSIONS

Novices mostly conveyed descriptive knowledge based on presented facts while experts demonstrated an ability to paint a richer mental image of possible future events by creating comprehensive anticipation of the operative field. Further studies are needed to validate the results of this study.

Twelve tips for embedding assessment for and as learning practices in a programmatic assessment system

Programmatic assessment supports the evolution from assessment of learning to fostering assessment for learning and as learning practices. A well-designed programmatic assessment system aligns educational objectives, learning opportunities, and assessments with the goals of supporting student learning, making decisions about student competence and promotion decisions, and supporting curriculum evaluation. We present evidence-based guidance for implementing assessment for and as learning practices in the pre-clinical knowledge assessment system to help students learn, synthesize, master and retain content for the long-term so that they can apply knowledge to patient care. Practical tips are in the domains of culture and motivation of assessment, including how an honour code and competency-based grading system can support an assessment system to develop student self-regulated learning and professional identity, curricular assessment structure, such as how and when to utilize low-stakes and cumulative assessment to drive learning, exam and question structure, including what authentic question and exam types can best facilitate learning, and assessment follow-up and review considerations, such exam retake processes to support learning, and academic success structures. A culture change is likely necessary for administrators, faculty members, and students to embrace assessment as most importantly a learning tool for students and programs.

Piloting the FIRE: A Novel Error Management Training Simulation Curriculum for Fasciotomy Instruction

BACKGROUND

Multiple studies have demonstrated poor performance of lower extremity fasciotomy (LEF), highlighted by missed and/or inadequately released compartments. Incorporating error management training (EMT) into surgical simulation has been promoted as a way to gain deeper understanding of procedural errors and overall performance. The purpose of this study was to evaluate LEF performance using a Fasciotomy Improvement through Recognition of Errors (FIRE) simulation training curriculum to train novice surgical trainees.

METHODS

A mastery learning-based EMT curriculum was developed, and surgical residents were enrolled and pretested with a multiple-choice question (MCQ) written test, and a simulated fasciotomy using a lower leg model. Each trainee then watched a 15-minute narrated presentation followed by 2 rounds of fasciotomy error recognition and management training exercises to a mastery standard. During each round, trainees performed hands-on assessment of unique premade fasciotomy leg models containing a variable number of procedural errors. They were required to identify and propose corrective action for all errors. Serial rounds of remediation were implemented until the mastery standard was attained on both error identification rounds. All trainees were post-tested with the same MCQ and another simulated fasciotomy.

RESULTS

All 14 residents had minimal experience with only 0.3 ± 0.6 fasciotomies performed prior to instruction. There were 3 ± 1.6 missed or inadequately released compartments on the pretest. Residents examined 14 ± 2.5 legs, including 2 ± 2.5 legs during remediation to attain mastery. All residents demonstrated significant improvement following the FIRE of Error curriculum for the MCQ (57% ± 16% vs 78% ± 13%; p = 0.01; Cohen's d = 1.4), fasciotomy score (10 ± 7.1 vs 28 ± 1.9; p < 0.001; Cohen's d = 3.6), and achieving a complete fasciotomy (14% ± 36% vs 93% ± 27%; p < 0.001; Cohen's d = 2.5). Only a single cumulative compartment was missed on post-testing.

CONCLUSIONS

Implementation of a mastery learning-based EMT curriculum for fasciotomy simulation training results in significant improvement in fasciotomy technique without reliance on repeated procedure performance nor clinical fasciotomy exposure. This curriculum is a highly effective option for surgical trainees lacking fasciotomy training during residency.

The embodiment of practice thresholds: from standardization to stabilization in surgical education

Surgeons practice their own variations on a procedure. Residents experience shifting thresholds between variations that one surgeon holds firmly as principle and another takes more lightly as preference. Such variability has implications for surgical education, but the impact is not well understood. This is a critical problem to investigate as programs seek to define procedures for competency-based medical education (CBME) and improve learning through deliberate practice. Our study analyzes the emergence of procedural variation in an early-adopter CBME program through a situational analysis of tonsillectomy, a foundation level procedure in this otolaryngology, head and neck surgical program. An earlier phase of the study identified frequent variations (n = 12) on tonsillectomy among co-located surgeons who routinely perform this procedure (n = 6). In the phase reported here we interviewed these surgeons (n = 4) and residents at different stages of training (n = 3) about their experiences of these variations to map the relations of contributing social and material actors. Our results show that even a basic procedure resists standardization. This study contributes a sociomaterial grounded theory of surgical practice as an embodied response to conditions materialized by intra-relations of human and more-than-human actors. Shifting root metaphors about practice in surgical education from standardization to stabilization can help residents achieve stable-for-now embodiments of performance as their practice thresholds continue to emerge.

Development and Assessment of the Wisconsin Surgical Coaching Rubric

Importance

Surgical coaching continues to gain momentum as an innovative method for continuous professional development. A tool to measure the performance of a surgical coach is needed to provide formative feedback to coaches for continued skill development and to assess the fidelity of a coaching intervention for future research and dissemination.

Objective

To evaluate the validity of the Wisconsin Surgical Coaching Rubric (WiSCoR), a novel tool to assess the performance of a peer surgical coach.

Design, Setting, and Participants

Surgical coaching sessions from November 2014 through February 2018 conducted by 2 statewide peer surgical coaching programs were audio recorded and transcribed. Twelve raters used WiSCoR to rate the performance of the surgical coach for each session. The study included peer surgical coaches in the Wisconsin Surgical Coaching Program (n = 8) and the Michigan Bariatric Surgery Collaborative coaching program (n = 15). The data were analyzed in 2019.

Interventions or Exposures

Use of WiSCoR to rate peer surgical coaching sessions.

Main Outcomes and Measures

There were 282 WiSCoR ratings from the 106 coaching sessions included in the study. WiSCoR was evaluated using a framework, including inter-rater reliability assessed with Gwet weighted agreement coefficent. Descriptive statistics of WiSCoR were calculated.

Results

Eight coaches (35%) and 11 coachees (29%) were from the Wisconsin Surgical Program and 15 coaches (65%) and 27 coachees (71%) were from the Michigan Bariatric Surgery Collaborative. The validity of WiSCoR is supported by high interrater reliability (Gwet weighted agreement coefficient, 0.87) as well as a weakly positive correlation of WiSCoR to coachee ratings of coaches (r = 0.22; P = .04), rigorous content development, consistent rater training, and the association of WiSCoR with coach and coaching program development. The mean (SD) overall coach performance rating using WiSCoR was 3.23 (0.82; range, 1-5).

Conclusions and Relevance

WiSCoR is a reliable measure that can assess the performance of a surgical coach, inform fidelity to coaching principles, and provide formative feedback to surgical coaches. While coachee ratings may reflect coachee satisfaction, they are not able to determine the quality of a coach.

An analysis of core EPAs reveals a gap between curricular expectations and medical school graduates' self-perceived level of competence

BACKGROUND

Entrustable Professional Activities (EPAs) are being implemented worldwide as a means to promote competency-based medical education. In Switzerland, the new EPA-based curriculum for undergraduate medical education will be implemented in 2021. The aim of our study was to analyze the perceived, self-reported competence of graduates in 2019. The data represent a pre-implementation baseline and will provide guidance for curriculum developers.

METHODS

Two hundred eighty-one graduates of the Master of Human Medicine program of the University of Zurich who had passed the Federal Licensing Exam in September 2019 were invited to complete an online survey. They were asked to rate their needed level of supervision ("observe only", "direct, proactive supervision", "indirect, reactive supervision") for 46 selected EPAs. We compared the perceived competence with the expected competence of the new curriculum.

RESULTS

The response rate was 54%. The need for supervision expressed by graduates varied considerably by EPA. The proportion of graduates rating themselves at expected level was high for "history taking", "physical examination" "and documentation"; medium for "prioritizing differential diagnoses", "interpreting results" and "developing and communicating a management plan"; low for "practical skills"; and very low for EPAs related to "urgent and emergency care".

CONCLUSIONS

Currently, there are significant gaps between the expectations of curriculum developers and the perceived competences of students. This is most obvious for practical skills and emergency situations. The new curriculum will either need to fill this gap or expectations might need to be revised.

Effect of a positive pressure ventilation-refresher program on ventilation skill performance during simulated newborn resuscitation

Aim

Clinical staff highly proficient in neonatal resuscitation are essential to ensure prompt, effective positive pressure ventilation (PPV) for infants that do not breathe spontaneously after birth. However, it is well-documented that resuscitation competency is transient after standard training. We hypothesized that brief, repeated PPV psychomotor skill refresher training would improve PPV performance for newborn care nurses.

Methods

Subjects completed a blinded baseline and post PPV-skills assessment. Data on volume and rate for each ventilation was recorded. After baseline assessment, subjects completed PPV-Refreshers over 3 months consisting of psychomotor skill training using a newborn manikin with visual feedback. Subjects provided PPV until they could deliver ≥30 s of PPV meeting targets for volume (10-21 mL) and rate (40-60 ventilations per minute [vpm]). Baseline and post assessments were compared for total number PPV delivered, number target PPV delivered (volume 10-21 mL), mean volume and mean rate (Wilcoxon signed-rank test, median[IQR]).

Results

Twenty-six subjects were enrolled and completed a baseline assessment; 24 (92%) completed a post-assessment; 2 (8%) were lost to follow-up. Over 3 months, a mean 3.2 (range 1-6) PPV-Refreshers/subject were completed. Compared to baseline, subjects demonstrated significant improvement for total (57 [36-74] vs. 33 [26-46]; p = 0.0007) and target PPV (23 [13-23] vs. 11 [5-21]; p = 0.024), and a significant change in mean volume (mL) (11.5 [10.2-13] vs. 13.4 [11-16]; p = 0.02) and mean rate (vpm) (54 [45-61] vs. 40 [28-49]; p = 0.019).

Conclusions

A PPV-Refresher program with brief, repeated psychomotor skill practice significantly improved PPV performance with the greatest improvement in total PPV and target PPV. Additional investigation is warranted to determine optimal PPV-Refresher frequency.Registered at ClinicalTrials.gov #NCT02347241.

What does it take to become a neurosurgeon? A mixed methods cross-sectional analysis of professional identity formation amongst neurosurgical trainees

INTRODUCTION

The concept of professional identity (as distinct from technical skills development), and how it can be cultivated by trainees in neurosurgical training has not been explored before.

AIMS

This study aimed to assess neurosurgical trainees' perceptions of professional identity, how it develops and how it might be enhanced during the course of training.

METHODS

A mixed methods cross-sectional analysis was conducted consisting of semi-structured, in-depth interviews with a small group of neurosurgical trainees to qualitatively identify common themes around professional identity. These themes were then quantitatively analysed via a national on-line questionnaire survey amongst the wider United Kingdom neurosurgical trainee cohort.

RESULTS

Interviewed participants were fiveteen British neurosurgical trainees spanning across junior (n = 5), intermediate (n = 5) and senior (n = 5) levels of training at two University teaching hospitals representing high-volume tertiary centres for Neurosurgery in the United Kingdom. The on-line questionnaire survey returned complete responses by 80 trainees (30% response rate). These data demonstrated that at different stages of training, neurosurgical trainees self-identified differently, and engaged in different practices to develop their identity. However, all trainees irrespective of level appeared to perceive a common set of qualities that define the identity of a fully-fledged neurosurgeon.

CONCLUSION

A model has been constructed that describes professional identity formation amongst neurosurgical trainees at different stages of training, and how these feed into an aspired core identity profile of a Neurosurgeon. Based on this model, suggestions have been made to potentially improve professional identity formation amongst neurosurgical trainees.

Development and Validation of a Bovine Castration Model and Rubric

Veterinary students require deliberate practice to reach competence in surgical bovine castration, but animal availability limits opportunities for practice. We sought to create and validate a surgical bovine castration model consisting of a molded silicone scrotum and testicles to allow students to practice this skill without the use of live animals. We sought to validate the model and associated scoring rubric for use in a veterinary clinical skills course. A convenience sample of third-year veterinary students (n = 19) who had never castrated a bovine were randomized into two groups. The traditionally trained (T) group performed castration on a live bull calf after a 50-minute instructional lecture. The model-trained (M) group received the same lecture and a 2-hour clinical skills session practicing bovine castration using the model. All students were subsequently digitally recorded while castrating a live bull calf. Performance recordings were scored by an investigator blinded to group. Survey data were collected from the students and from expert veterinarians testing the model (n = 8). Feedback from both groups was positive. The M group had higher performance scores than the T group (M group, M = 80.6; T group, M = 68.2; p = .005). Reliability of rubric scores was adequate at .74. No difference was found in surgical time (M group, M = 4.5 min; T group, M = 5.5 min; p = .12). Survey feedback indicated that experts and students considered the model useful. Model training improved students' performance scores and provided evidence for validation of the model and rubric.

Team-based Learning to Promote the Development of Metacognitive Awareness and Monitoring in Pharmacy Students

Objective. To evaluate the metacognitive abilities of pharmacy students and determine whether introducing the concept along with team-based learning (TBL) enhances metacognition.Methods. Pharmacy students completed a Metacognitive Awareness Inventory (MAI) and a low-stakes pretest during the first class that evaluated students' knowledge about the therapeutic concepts that would be taught through TBL. The same questions were administered on the comprehensive final examination for the course. For each of the course assessments, students were asked to indicate their understanding of the topic and predict their performance. Actual performance was measured as a result of each assessment.Results. The pre-MAI composite score was 77.3%. Scores significantly improved by the end of the course to 84.6%. There were significant differences in both declarative knowledge and conditional knowledge when evaluating performance groups. Students in the middle performance group demonstrated the greatest ability to predict their performance on the final examination. Though these were not significant, students in the low group overestimated their performance, while students in the high group underestimated their performance. Baseline grade point average was the only factor predictive of the final examination score and the final course grade.Conclusion. Pedagogies such as TBL may support development of metacognitive skills in pharmacy students. However, intentional guidance provided by an instructor is required to improve pharmacy students' regulation of cognition skills.

Exploring the use of virtues to facilitate identity construction among management students

This paper explores the challenges and opportunities facing educators who wish to facilitate management students' identity construction as a means to foster their students' emergent professional identities and post-graduate career attainments. We look to medical and law schools' recent advancements, alongside their traditional dissemination of knowledge and technical skills, in better aligning students' developing identities and related behaviors with respective professional ideals. Drawing on insights from these professional schools, we examine whether the concept of "professional virtues" that's proven valuable in these contexts might also be used to facilitate students' professional identity construction within management education.

Integrating Concept Maps into a Medical Student Oncology Curriculum

Expanding frontiers of knowledge have prompted medical schools to reconsider how best to promote learning in the face of information overload. Concept mapping (CM) promotes knowledge retention and integration. Students have perceived CM positively in prior studies, but the feasibility and utility of integrating CM into a medical student oncology curriculum as a learning and assessment tool have not been described. At the University of California, San Francisco, 152 medical students in a second-year hematology/oncology course produced concept maps about a single cancer type over 4 weeks. We collected student evaluations about CM. Two of three graders independently scored each map using a standard rubric. We compared CM scores with USMLE Step 1 scores and clerkship grades using regression. All students produced a concept map. Student perception was mixed, and students provided feedback to improve CM utility as a learning tool. Grading was feasible, and inter-rater reliability was high. CM scores did not predict USMLE Step 1 scores or clerkship grades. CM was feasible as a learning tool, and we present strategies based on student feedback and literature review to improve utility. CM was feasible and reliable as an assessment tool; additional validity evidence may improve utility. Future studies should explore whether CM integrated into medical student oncology curricula early, serially, and collaboratively, with iterative practice and feedback, may predict meaningful learning and performance outcomes.

Consensus statement on the content of clinical reasoning curricula in undergraduate medical education

INTRODUCTION

Effective clinical reasoning is required for safe patient care. Students and postgraduate trainees largely learn the knowledge, skills and behaviours required for effective clinical reasoning implicitly, through experience and apprenticeship. There is a growing consensus that medical schools should teach clinical reasoning in a way that is explicitly integrated into courses throughout each year, adopting a systematic approach consistent with current evidence. However, the clinical reasoning literature is 'fragmented' and can be difficult for medical educators to access. The purpose of this paper is to provide practical recommendations that will be of use to all medical schools.

METHODS

Members of the UK Clinical Reasoning in Medical Education group (CReME) met to discuss what clinical reasoning-specific teaching should be delivered by medical schools (what to teach). A literature review was conducted to identify what teaching strategies are successful in improving clinical reasoning ability among medical students (how to teach). A consensus statement was then produced based on the agreed ideas and the literature review, discussed by members of the consensus statement group, then edited and agreed by the authors.

RESULTS

The group identified 30 consensus ideas that were grouped into five domains: (1) clinical reasoning concepts, (2) history and physical examination, (3) choosing and interpreting diagnostic tests, (4) problem identification and management, and (5) shared decision making. The literature review demonstrated a lack of effectiveness for teaching the general thinking processes involved in clinical reasoning, whereas specific teaching strategies aimed at building knowledge and understanding led to improvements. These strategies are synthesised and described.

CONCLUSION

What is taught, how it is taught, and when it is taught can facilitate clinical reasoning development more effectively through purposeful curriculum design and medical schools should consider implementing a formal clinical reasoning curriculum that is horizontally and vertically integrated throughout the programme.

Using psychometric ability to improve education in ultrasound-guided regional anaesthesia: a multicentre randomised controlled trial

The learning curve for novices developing regional anaesthesia skills, such as real-time ultrasound-guided needle manipulation, may be affected by innate visuospatial ability, as this influences spatial cognition and motor co-ordination. We conducted a multinational randomised controlled trial to test if novices with low visuospatial ability would perform better at an ultrasound-guided needling task with deliberate practice training than with discovery learning. Visuospatial ability was evaluated using the mental rotations test-A. We recruited 140 medical students and randomly allocated them into low-ability control (discovery learning), low-ability intervention (received deliberate practice), high-ability control, and high-ability intervention groups. Primary outcome was the time taken to complete the needling task, and there was no significant difference between groups: median (IQR [range]) low-ability control 125 s (69-237 [43-600 s]); low-ability intervention 163 s (116-276 [44-600 s]); high-ability control 130 s (80-210 [41-384 s]); and high-ability intervention 177 s (113-285 [43-547 s]), p = 0.06. No difference was found using the global rating scale: mean (95%CI) low-ability control 53% (95%CI 46-60%); low-ability intervention 61% (95%CI 53-68%); high-ability control 63% (95%CI 56-70%); and high-ability intervention 66% (95%CI 60-72%), p = 0.05. For overall procedure pass/fail, the low-ability control group pass rate of 42% (14/33) was significantly less than the other three groups: low-ability intervention 69% (25/36); high-ability control 68% (25/37); and high-ability intervention 85% (29/34) p = 0.003. Further research is required to determine the role of visuospatial ability screening in training for ultrasound-guided needle skills.

Contemporary training methods in regional anaesthesia: fundamentals and innovations

Over the past two decades, regional anaesthesia and medical education as a whole have undergone a renaissance. Significant changes in our teaching methods and clinical practice have been influenced by improvements in our theoretical understanding as well as by technological innovations. More recently, there has been a focus on using foundational education principles to teach regional anaesthesia, and the evidence on how to best teach and assess trainees is growing. This narrative review will discuss fundamentals and innovations in regional anaesthesia training. We present the fundamentals in regional anaesthesia training, specifically the current state of simulation-based education, deliberate practice and curriculum design based on competency-based progression. Moving into the future, we present the latest innovations in web-based learning, emerging technologies for teaching and assessment and new developments in alternate reality learning systems.

From Theory to Practice: The Application of Cognitive Load Theory to the Practice of Medicine

Cognitive load theory has become a leading model in educational psychology and has started to gain traction in the medical education community over the last decade. The theory is rooted in our current understanding of human cognitive architecture in which an individual's limited working memory and unlimited long-term memory interact during the process of learning. Though initially described as primarily a theory of learning, parallels between cognitive load theory and broader aspects of medical education as well as clinical practice are now becoming clear. These parallels are particularly relevant and evident in complex clinical environments, like resuscitation medicine. The authors have built on these connections to develop a recontextualized version of cognitive load theory that applies to complex professional domains and in which the connections between the theory and clinical practice are made explicit, with resuscitation medicine as a case study. Implications of the new model for medical education are also presented along with suggested applications.

Comparison of Simulation-Based versus Cadaveric-Tissue-Based Ocular Trauma Training on Novice Ophthalmologists: Repair of Corneal Laceration Model

Purpose  The aim of this study was to evaluate whether the simulated tissue models may be used in place of animal-based model for corneal laceration repair for surgical skills acquisition. Design  Prospective randomized controlled trial. Participants  Seventy-nine military and civilian 2nd- and 3rd-year ophthalmology residents and 16 staff ophthalmologists participating in the Tri-Service Ocular Trauma Skills Laboratory at the Uniformed Services University (Bethesda, MD). Methods  Resident ophthalmologists underwent preliminary evaluation of their ability to close a 5-mm linear, full-thickness corneal laceration involving the visual axis. They then were randomized to undergo 90 to 120 minutes of either simulator-based (SIM) or swine cadaveric-tissue-based (CADAVER) corneal laceration repair. The same evaluation was performed post training. On a more limited basis, the study was repeated for attending ophthalmologists to act as a pilot for future analysis and test efficacy for "refresher" training. Main Outcome Measures  Successful wound closure with secondary outcomes of suture length, tension, depth, and orientation, as graded by attending ophthalmologists. Results  No significant difference in CADAVER versus SIM groups in the primary outcome of watertight wound closure of the corneal laceration. CADAVER group performed better than SIM group for certain metrics (suture depth, p  = 0.009; length, p  = 0.003; and tension, p  = 0.043) that are associated with poor wound closure and increased amount of induced corneal astigmatism. For attending ophthalmologists, six of the eight in each group (SIM and CADAVER) retained or improved their skills. Conclusions  For resident ophthalmologists, SIM training is sufficient for achieving the primary outcome of watertight wound closure. However, CADAVER training is superior for wound metrics for the ideal closure. For attending ophthalmologists, SIM training may be useful for retention of skills.

Assessing Performance in Simulated Cleft Palate Repair Using a Novel Video Recording Setup

OBJECTIVE

To test the feasibility of implementing a high-fidelity cleft palate simulator during a workshop in Santiago, Chile, using a novel video endoscope to assess technical performance.

DESIGN

Sixteen cleft surgeons from South America participated in a 2-day cleft training workshop. All 16 participants performed a simulated repair, and 13 of them performed a second simulated repair. The repairs were recorded using a low-cost video camera and a newly designed camera mouth retractor attachment. Twenty-nine videos were assessed by 3 cleft surgeons using a previously developed cleft palate objective structured assessment of technical skill (CLOSATS with embedded overall score assessment) and global rating scale. The reliability of the ratings and technical performance in relation to minimum acceptable scores and previous experience was assessed.

RESULTS

The video setup provided acceptable recording quality for the purpose of assessment. Average intraclass correlation coefficient for the CLOSATS, global, and overall performance score was 0.69, 0.75, and 0.82, respectively. None of the novice surgeons passed the CLOSATS and global score for both sessions. One participant in the intermediate group, and 2 participants in the advanced group passed the CLOSATS and global score for both sessions. There were highly experienced participants who failed to pass the CLOSATS and global score for both sessions.

CONCLUSIONS

The cleft palate simulator can be practically implemented with video-recording capability to assess performance in cleft palate repair. This technology may be of assistance in assessing surgical competence in cleft palate repair.

Take-Home Laparoscopy Simulators in Pediatric Surgery: Is More Expensive Better?

Background: To increase complex minimally invasive skills (MIS), frequent training outside the clinical setting is of uttermost importance. This study compares two low-cost pediatric MIS simulators, which can easily be used preclinically. Materials and Methods: The LaparoscopyBoxx is a portable simulator without a tracking system, with costs ranging from €90 to €315. The EoSim simulator has a built-in camera and tracking system and costs range from €780 to €1800. During several pediatric surgical conferences and workshops (January 2017-December 2018), participants were asked to use both simulators. Afterward, they completed a questionnaire regarding their opinion on realism and didactic value, scored on a five-point Likert scale. Results: A total of 50 participants (24 experts and 25 target group, one unknown) evaluated one or both simulators. Both simulators scored well on the questionnaire. The LaparoscopyBoxx scored significantly better regarding the "on screen representation of the instrument actions" (mean 4.2 versus 3.5, P = .001), "training tool for pediatric surgery" (mean 4.4 versus 3.9, P = .005), and "appealing take-home simulator" (mean 4.6 versus 4.0, P = .002). Conclusion: The simulators tested in this study were both regarded an appealing take-home simulator. The LaparoscopyBoxx scored significantly better than the EoSim, even though this is a low budget simulator without tracking capabilities.

Tips for teaching procedural skills

The teaching of procedural skills required for clinical practice remains an ongoing challenge in healthcare education. Health professionals must be competent to perform a wide range of clinical skills, and are also regularly required to teach these clinical skills to their peers, junior staff, and students. Teaching of procedural skills through the use of frameworks, observation and provision of feedback, with opportunities for repeated practice assists in the learners' acquisition and retention of skills. With a focus on the teaching of non-complex skills, this paper explores how skills are learned; ways to improve skill performance; determining competency; and the provision of effective feedback.

Effectiveness of blended learning versus lectures alone on ECG analysis and interpretation by medical students

BACKGROUND

Most medical students lack confidence and are unable to accurately interpret ECGs. Thus, better methods of ECG instruction are being sought. Current literature indicates that the use of e-learning for ECG analysis and interpretation skills (ECG competence) is not superior to lecture-based teaching. We aimed to assess whether blended learning (lectures supplemented with the use of a web application) resulted in better acquisition and retention of ECG competence in medical students, compared to conventional teaching (lectures alone).

METHODS

Two cohorts of fourth-year medical students were studied prospectively. The conventional teaching cohort (n = 67) attended 4 hours of interactive lectures, covering the basic principles of Electrocardiography, waveform abnormalities and arrhythmias. In addition to attending the same lectures, the blended learning cohort (n = 64) used a web application that facilitated deliberate practice of systematic ECG analysis and interpretation, with immediate feedback. All participants completed three tests: pre-intervention (assessing baseline ECG competence at start of clinical clerkship), immediate post-intervention (assessing acquisition of ECG competence at end of six-week clinical clerkship) and delayed post-intervention (assessing retention of ECG competence 6 months after clinical clerkship, without any further ECG training). Diagnostic accuracy and uncertainty were assessed in each test.

RESULTS

The pre-intervention test scores were similar for blended learning and conventional teaching cohorts (mean 31.02 ± 13.19% versus 31.23 ± 11.52% respectively, p = 0.917). While all students demonstrated meaningful improvement in ECG competence after teaching, blended learning was associated with significantly better scores, compared to conventional teaching, in immediate (75.27 ± 16.22% vs 50.27 ± 17.10%, p <  0.001; Cohen's d = 1.58), and delayed post-intervention tests (57.70 ± 18.54% vs 37.63 ± 16.35%, p <  0.001; Cohen's d = 1.25). Although diagnostic uncertainty decreased after ECG training in both cohorts, blended learning was associated with better confidence in ECG analysis and interpretation.

CONCLUSION

Blended learning achieved significantly better levels of ECG competence and confidence amongst medical students than conventional ECG teaching did. Although medical students underwent significant attrition of ECG competence without ongoing training, blended learning also resulted in better retention of ECG competence than conventional teaching. Web applications encouraging a stepwise approach to ECG analysis and enabling deliberate practice with feedback may, therefore, be a useful adjunct to lectures for teaching Electrocardiography.

Impact of a web-based module on trainees' ability to interpret neonatal cranial ultrasound

BACKGROUND

Accurate interpretations of neonatal cranial ultrasound (CUS) studies are essential skills for physicians in neonatal intensive care units (NICUs) in order to properly diagnose and manage brain injury. However, these skills are not formally taught to pediatric and neonatal-perinatal medicine (NPM) trainees in Canada. Therefore, our study describes the design, implementation, and evaluation of a new web-based learning (WBL) module that focuses on teaching these skills.

METHODS

Trainees' needs assessment survey, sent to all NPM and pediatrics trainees (n = 62), concluded that most of them feel uncomfortable with their ability to interpret CUS, highlighting the need for a new educational intervention. The needs assessment informed the development of the WBL module, which we evaluated using questionnaires and pre-and post-testing methods to measure participants' satisfaction, knowledge gain, skills development, and behaviour changes. Only trainees rotating through the NICU over 6 months (n = 23) were invited to participate in all the evaluation steps. We used the ADDIE instructional design model as a framework for this project.

RESULTS

Respondents were very satisfied with the module, and their baseline knowledge increased significantly after studying and engaging with the module. The post-test score was 76% (p < 0.001) compared to the pre-test mean score of 42%. Tests for CUS interpretation skills assessment showed that 49% of pre-test answers were incorrect compared to 8% in the post-test (p < 0.001). Seventy-eight percent of trainees (n = 18) responded to a survey conducted a year after implementation, and 78% of the respondents (n = 14) reported that they still used these skills and shared this knowledge with junior trainees.

CONCLUSION

A WBL module for teaching neonatal CUS interpretation considerably improved trainees' knowledge and enhanced their skills in interpreting neonatal CUS.

Procedural frequency: Results from 18 academic, community and freestanding emergency departments

BACKGROUND

Emergency physicians must maintain procedural skills, but clinical opportunities may be insufficient. We sought to determine how often practicing emergency physicians in academic, community and freestanding emergency departments (EDs) perform 4 procedures: central venous catheterization (CVC), tube thoracostomy, tracheal intubation, and lumbar puncture (LP).

METHODS

This was a retrospective study evaluating emergency physician procedural performance over a 12-month period. We collected data from the electronic records of 18 EDs in one healthcare system. The study EDs included higher and lower volume, academic, community and freestanding, and trauma and non-trauma centers. The main outcome measures were median number of procedures performed. We examined differences in procedural performance by physician years in practice, facility type, and trauma status.

RESULTS

Over 12 months, 182 emergency physicians performed 1582 of 2805 procedures (56%) and supervised (resident, nurse practitioner or physician assistant) an additional 1223 of the procedures they did not perform (43%). Median (interquartile range) physician performance for each procedure was CVC 0 [0, 2], tube thoracostomy 0 [0, 0], tracheal intubation 3 [0.25, 8], and LP 0 [0, 2]. The percentage of emergency physicians who did not perform at least one of each procedure during the 1-year time frame ranged from 25.3% (tracheal intubation) to 76.4% (tube thoracostomy). Physicians who work at high-volume EDs (>50,000 visits per year) performed nearly twice as many tracheal intubations, CVCs, and LPs than those at low-volume EDs or freestanding EDs when normalized per 1000 visits. Years out of training were inversely related to total number of procedures performed. Emergency physicians at trauma centers performed almost 3 times as many tracheal intubations and almost 4 times as many CVCs compared to non-trauma centers.

CONCLUSION

In a large healthcare system, regardless of ED type, emergency physicians infrequently performed the 4 procedures studied. Physicians in high-volume EDs, trauma centers, and recent graduates performed more procedures. Our study adds to a growing body of research that suggests clinical frequency alone may be insufficient for all emergency physicians to maintain competency.

Development of a pilot procedural skills training course for preclerkship medical students

OBJECTIVE

Despite procedural skills being recognized as an important component of medical school education, medical students are not confident in their ability to carry out a range of medical procedures. We conducted an institutional needs assessment and used the results to inform the creation of a procedure-based preclinical elective for first- and second-year students.

METHODS

We surveyed second-, third-, and fourth-year medical students at Alpert Medical School as well as select program directors to guide selection of a list of procedures to be taught in the elective. We then created an extracurricular 10-week procedural skills course for preclerkship medical students utilizing a hands-on, flipped classroom practice model.  Volunteer preceptors were recruited from the Department of Emergency Medicine to participate with a student-to-faculty ratio not exceeding 5:1. Knowledge and skill acquisition were assessed using a multiple-choice knowledge exam and 4-station practical exam, respectively. Pre- and post-course online surveys were used to assess self-perceived confidence for all procedures.

RESULTS

We implemented our procedural skills training course for first- and second-year medical students in the fall of 2015. Forty-four students applied for the first iteration of the course and 15 students were selected to participate. Fourteen students ultimately completed the elective as well as the subsequent course surveys, multiple-choice exam, and practical exam. Students who participated in the elective had increased levels of self-reported confidence at the conclusion of the elective and performed better on a practical exam and multiple-choice exam compared to students who participated in only the standard curriculum.

CONCLUSION

A longitudinal preclerkship procedural course early during medical school is a feasible method of teaching procedural skills to a cohort of learners. A number of adjustments could be made to the course in order to scale up and include a larger cohort of students at our own or another institution.