The testing effect on skills learning might last 6 months

Test-Only Learning via Virtual Patients to Improve Surgical Illness Scripts of Preclinical Medical Students as a Preparation for Clinical Clerkships: An Experimental Study

OBJECTIVE

The aim was to determine whether preclinical medical students can develop their illness scripts to a level comparable to that of clerkship students through test-only learning using repeated formative online testlets.

METHODS

In this experimental study, participants were 52 preclinical and 53 clinical medical students. The intervention group consisted of preclinical medical students, and the control group consisted of clinical medical students. The intervention group responded to online testlets containing feedback, an innovative formative assessment method called ContExtended Questions, on general surgery for 8 days by spending no more than 30 minutes each day. The control group completed the general surgery clerkship. The performances were assessed using 20 Key-Feature Question items. The intervention group was assessed twice: immediately after the intervention (the immediate test), and again 1 month later (the delayed test). The control group was assessed once, immediately after the clerkship. All performance tests were identical.

RESULTS

The preclinical students had a significantly higher mean score on the immediate test (83.1 ± 9.6) compared to the clinical students (75.4 ± 8.9), P < .001. The effect size (Cohen's d) was .83. However, the mean score in the delayed test (76.9 ± 13.6) was not significantly different from clinical students' mean score (75.4 ± 8.9), P > .05.

CONCLUSIONS

Test-only learning as a spaced repetition of online formative testlets is effective in preparing preclinical medical students to the clinical clerkship. Through using this approach in preclinical period, they can prepare themselves for the clinical environment to optimize the benefits derived from clerkships.

Using Evidence-Based Learning Strategies to Improve Medical Education

Medical education research has been adopting principles from psychology to improve student learning. Here is an overview and illustrative examples of six evidence-based learning strategies that have been thoroughly researched and validated in the psychology literature: spacing, interleaving, retrieval practice, elaboration, dual coding, and concrete examples. For each of these, their use within medical education and considerations that may influence efficacy are discussed. Medical education researchers should collaborate more with psychology researchers in transdisciplinary teams to better implement these strategies and more directly benefit from advances made in the psychology literature.

Association of Training in Basic Life Support with the Evolution of Cardiopulmonary Resuscitation Performed by Firefighters

Introduction

This study aimed to compare the results of professional technical and anthropometric anamnesis data with the transmission of external chest compressions performed by military firefighters at different execution times.

Objective

The objective was to evaluate the performance and perceived effort of the sequence of external chest compressions performed in two minutes, as well as the evolution of the technique over time.

Materials and Methods

This was a descriptive, correlational study involving adult firefighters who were members of a specific firefighter group, comprising a population of 105 individuals with a voluntary sample of 44 participants. The study used a Bayesian statistical approach to provide probabilistic expressions.

Results

The participants had an average work experience of 17 years, an average age of 38.6 years, an average weight of 81.48 kilograms, an average height of 176 centimeters, and an average of 2.5 qualifications. The results indicated that the firefighters performed external chest compressions with excellent technique and a moderate level of perceived effort in a two-minute evaluation. The evaluation of the evolution of the technique over time showed that the participants were able to maintain high-quality compressions for an average of 6 minutes, with a maximum of 20 uninterrupted minutes.

Conclusion

The study underscores the critical role of professional firefighters in performing and maintaining high-quality external chest compressions, which has the potential to reduce morbidity and mortality in cases of cardiorespiratory arrest.

Active Retrieval Improves Procedural Learning in Orthopedic Surgery

OBJECTIVE

Tests are shown to enhance learning: this is known as the "testing effect". The benefit of testing is theorized to be through "active retrieval", which is the effortful process of recalling stored knowledge. This differs from "passive studying", such as reading, which is a low effort process relying on recognition. The testing effect is commonly studied in random word list scenarios and is thought to disappear as complexity of material increases. Little is known about the testing effect in complex situations such as procedural learning. Therefore, we investigated if testing improves procedural learning of fracture fixation as compared to "passive studying".

DESIGN, SETTING, AND PARTICIPANTS

Fifty participants watched an instructional video of an open reduction internal fixation of a Sawbones™ femur. Participants then performed the procedure under guided supervision (pretest). After randomization, they either read the steps (passive studying group), or wrote down the steps from memory (active retrieval group) for a period of 15 minutes. After a washout period, all participants performed the procedure without guidance (posttest) and then once more, 1 week after the initial testing (retention test). The participants were assessed using the Objective Structured Assessment of Technical Skill. Each performance was video recorded for data analysis purposes.

RESULTS

Participants in the passive studying group had significantly higher Objective Structured Assessment of Technical Skill scores during immediate assessment compared to the active retrieval group (p = 0.001), especially with respect to remembering the correct order of the steps (p = 0.002). The percentage of information forgotten was significantly less in the active retrieval group (p = 0.02) at the retention test.

CONCLUSION

We demonstrated that, compared to passive studying, testing with active retrieval through writing resulted in better retention of fracture fixation knowledge (i.e., less forgetting). These findings can easily be applied and incorporated in existing curricula. Future studies are needed to determine the effects of different kinds of active retrieval methods such as verbal retrieval (e.g., dictating) in surgical practice.

Written-Based Progress Testing: A Scoping Review

PURPOSE

Progress testing is an increasingly popular form of assessment in which a comprehensive test is administered to learners repeatedly over time. To inform potential users, this scoping review aimed to document barriers, facilitators, and potential outcomes of the use of written progress tests in higher education.

METHOD

The authors followed Arksey and O'Malley's scoping review methodology to identify and summarize the literature on progress testing. They searched 6 databases (Academic Search Complete, CINAHL, ERIC, Education Source, MEDLINE, and PsycINFO) on 2 occasions (May 22, 2018, and April 21, 2020) and included articles written in English or French and pertaining to written progress tests in higher education. Two authors screened articles for the inclusion criteria (90% agreement), then data extraction was performed by pairs of authors. Using a snowball approach, the authors also screened additional articles identified from the included reference lists. They completed a thematic analysis through an iterative process.

RESULTS

A total of 104 articles were included. The majority of progress tests used a multiple-choice and/or true-or-false question format (95, 91.3%) and were administered 4 times a year (38, 36.5%). The most documented source of validity evidence was internal consistency (38, 36.5%). Four major themes were identified: (1) barriers and challenges to the implementation of progress testing (e.g., need for additional resources); (2) established collaboration as a facilitator of progress testing implementation; (3) factors that increase the acceptance of progress testing (e.g., formative use); and (4) outcomes and consequences of progress test use (e.g., progress testing contributes to an increase in knowledge).

CONCLUSIONS

Progress testing appears to have a positive impact on learning, and there is significant validity evidence to support its use. Although progress testing is resource- and time-intensive, strategies such as collaboration with other institutions may facilitate its use.

Training non-intensivist doctors to work with COVID-19 patients in intensive care units

Background

Due to an expected surge of COVID‐19 patients in need of mechanical ventilation, the intensive care capacity was doubled at Rigshospitalet, Copenhagen, in March 2020. This resulted in an urgent need for doctors with competence in working with critically ill COVID‐19 patients. A training course and a theoretical test for non‐intensivist doctors were developed. The aims of this study were to gather validity evidence for the theoretical test and explore the effects of the course.

Methods

The 1‐day course was comprised of theoretical sessions and hands‐on training in ventilator use, hemodynamic monitoring, vascular access, and use of personal protective equipment. Validity evidence was gathered for the test by comparing answers from novices and experts in intensive care. Doctors who participated in the course completed the test before (pretest), after (posttest), and again within 8 weeks following the course (retention test).

Results

Fifty‐four non‐intensivist doctors from 15 different specialties with a wide range in clinical experience level completed the course. The test consisted of 23 questions and demonstrated a credible pass–fail standard at 16 points. Mean pretest score was 11.9 (SD 3.0), mean posttest score 20.6 (1.8), and mean retention test score 17.4 (2.2). All doctors passed the posttest.

Conclusion

Non‐intensivist doctors, irrespective of experience level, can acquire relevant knowledge for working in the ICU through a focused 1‐day evidence‐based course. This knowledge was largely retained as shown by a multiple‐choice test supported by validity evidence. The test is available in appendix and online.

Rethinking Assessment Design: Evidence-Informed Strategies to Boost Educational Impact in the Anatomical Sciences

University assessment is in the midst of transformation. Assessments are no longer designed solely to determine that students can remember and regurgitate lecture content, nor in order to rank students to aid with some future selection process. Instead, assessments are expected to drive, support, and enhance learning and to contribute to student self-assessment and development of skills and attributes for a lifetime of learning. While traditional purposes of certifying achievement and determining readiness to progress remain important, these new expectations for assessment can create tensions in assessment design, selection, and deployment. With the recognition of these tensions, three contemporary approaches to assessment in medical education are described. These approaches include careful consideration of the educational impact of assessment-before, during (test or recall enhanced learning) and after assessments; development of student (and staff) assessment literacy; and planning of cohesive systems of assessment (with a range of assessment tools) designed to assess the various competencies demanded of future graduates. These approaches purposefully straddle the cross purposes of assessment in modern health professions education. The implications of these models are explored within the context of medical education and then linked with contemporary work in the anatomical sciences in order to highlight current synergies and potential future innovations when using evidence-informed strategies to boost the educational impact of assessments.

Learning Outcome After Different Combinations of Seven Learning Activities in Basic Life Support on Laypersons in Workplaces: a Cluster Randomised, Controlled Trial

BACKGROUND

The goal for laypersons after training in basic life support (BLS) is to act effectively in an out-of-hospital cardiac arrest situation. However, it is still unclear whether BLS training targeting laypersons at workplaces is optimal or whether other effective learning activities are possible.

AIM

The primary aim was to evaluate whether there were other modes of BLS training that improved learning outcome as compared with a control group, i.e. standard BLS training, six months after training, and secondarily directly after training.

METHODS

In this multi-arm trial, lay participants (n = 2623) from workplaces were cluster randomised into 16 different BLS interventions, of which one, instructor-led and film-based BLS training, was classified as control and standard, with which the other 15 were compared. The learning outcome was the total score for practical skills in BLS calculated using the modified Cardiff Test.

RESULTS

Four different training modes showed a significantly higher total score compared with standard (mean difference 2.3-2.9). The highest score was for the BLS intervention including a preparatory web-based education, instructor-led training, film-based instructions, reflective questions and a chest compression feedback device (95% CI for difference 0.9-5.0), 6 months after training.

CONCLUSION

BLS training adding several different combinations of a preparatory web-based education, reflective questions and chest compression feedback to instructor-led training and film-based instructions obtained higher modified Cardiff Test total scores 6 months after training compared with standard BLS training alone. The differences were small in magnitude and the clinical relevance of our findings needs to be further explored.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT03618888. Registered August 07, 2018-Retrospectively registered, https://clinicaltrials.gov/ct2/show/NCT03618888.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-020-01160-3.

Education, Implementation, and Teams: 2020 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations

For this 2020 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations, the Education, Implementation, and Teams Task Force applied the population, intervention, comparator, outcome, study design, time frame format and performed 15 systematic reviews, applying the Grading of Recommendations, Assessment, Development, and Evaluation guidance. Furthermore, 4 scoping reviews and 7 evidence updates assessed any new evidence to determine if a change in any existing treatment recommendation was required. The topics covered included training for the treatment of opioid overdose; basic life support, including automated external defibrillator training; measuring implementation and performance in communities, and cardiac arrest centers; advanced life support training, including team and leadership training and rapid response teams; measuring cardiopulmonary resuscitation performance, feedback devices, and debriefing; and the use of social media to improve cardiopulmonary resuscitation application.

Education, Implementation, and Teams: 2020 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations

For this 2020 International Consensus on Cardiopulmonary Resuscitation and Emergency Cardiovascular Care Science With Treatment Recommendations, the Education, Implementation, and Teams Task Force applied the population, intervention, comparator, outcome, study design, time frame format and performed 15 systematic reviews, applying the Grading of Recommendations, Assessment, Development, and Evaluation guidance. Furthermore, 4 scoping reviews and 7 evidence updates assessed any new evidence to determine if a change in any existing treatment recommendation was required. The topics covered included training for the treatment of opioid overdose; basic life support, including automated external defibrillator training; measuring implementation and performance in communities, and cardiac arrest centers; advanced life support training, including team and leadership training and rapid response teams; measuring cardiopulmonary resuscitation performance, feedback devices, and debriefing; and the use of social media to improve cardiopulmonary resuscitation application.

Spaced learning versus massed learning in resuscitation - A systematic review

AIM

Skill decay is a recognised problem in resuscitation training. Spaced learning has been proposed as an intervention to optimise resuscitation skill performance compared to traditional massed learning. A systematic review was performed to answer 'In learners taking resuscitation courses, does spaced learning compared to massed learning improve educational outcomes and clinical outcomes?'

METHODS

This systematic review followed the PRISMA guidelines. We searched bibliographic databases (Embase, MEDLINE and the Cochrane Library (CENTRAL)) from inception to 2 December 2019. Randomised controlled trials and non-randomised studies were eligible for inclusion. Two reviewers independently scrutinized studies for relevance, extracted data and assessed quality of studies. Risk of bias of studies and quality of evidence were assessed using RoB, ROBINS-I tool and GRADEpro respectively. Educational outcomes studied were skill retention and performance 1 year after completion of training; skill performance between completion of training and 1 year; and knowledge at course conclusion. Clinical outcomes were skill performance at actual resuscitation, patient survival to discharge with favourable neurological outcome. This systematic review was registered in PROSPERO (CRD42019150358).

RESULTS

From 2,042 references, we included data from 17 studies (13 randomised studies, 4 cohort studies) in courses with manikins and simulation in the narrative synthesis. Eight studies reported results from basic life support training (with or without automatic external defibrillator); three studies reported from paediatric life support training; five were in neonatal resuscitation and one study reported results from a bespoke emergency medicine course which included resuscitation teaching. Fifteen out of seventeen studies reported improved performance with the use of spaced learning. The overall certainty of evidence was rated as very low for all outcomes primarily due to a very serious risk of bias. Heterogeneity across studies precluded any meta-analyses. There was a lack of data on the effectiveness of spaced learning on skill acquisition compared to maintaining skill performance and/or preventing skill decay. There was also insufficient data to examine the effectiveness of spaced learning on laypeople compared to healthcare providers.

CONCLUSIONS

Despite the very low certainty of evidence this systematic review suggests that spaced learning can improve skill performance at 1 year post course conclusion and skill performance between course conclusion and 1 year. There is a lack of data from this educational intervention on skill performance in clinical resuscitation and patient survival at discharge with favourable neurological outcomes.

Experience and Challenges of Objective Structured Clinical Examination (OSCE): Perspective of Students and Examiners in a Clinical Department of Ethiopian University

Background

Invented nearly half a century ago, Objective Structured Clinical Examination (OSCE) is overwhelmingly accepted clinical skills assessment tool and has been used worldwide for evaluating and teaching learners' competences in health care disciplines. Regardless of factors affecting the attributes, OSCE is considered as reliable and powerful tool with certain validity evidences. In spite of its advantages and various promotion efforts, the progress of OSCE implementation in Ethiopian public universities has not been satisfactory. Therefore, the objective of this study was to explore the experience and challenges of OSCE implementation from the perspective of clinical year-II medical students and their examiners in Ob-Gyn Department of Jimma University.

Methods

Forty-nine students and seven examiners voluntarily participated in Ob-Gyn Department where OSCE has been used as one of summative assessment methods. Qualitative study design using structured open-ended questionnaire as a tool and descriptive phenomenology as underpinning method were employed. Collaizzi's descriptive analysis was used as phenomenological analysis approach.

Result

Poor organization, inadequate student preparation time, and inadequate number and duration of stations were thematically emerged as umbrellas of factors negatively affecting OSCE implementation. Satisfaction with OSCE was the only theme with findings that encourage OSCE implementation.

Conclusion

There should be team approach, shared responsibility and proper planning among faculty to minimize hindering factors of OSCE implementation. Besides faculty development on OSCE, the department should improve skill lab utilization arranging schedule for both students and faculty members to increase guided students' exposure to simulation-based learning and ultimately enhance OSCE implementation.

Influence of the Instructional Approach "Mastery Learning" versus "See One, Do One" on Acquiring Competencies in Abdomen Sonography: A Comparative Effectiveness Analysis

Ultrasound is an important diagnostic tool in patients with abdominal pain and after injury. However, it is highly dependent on the skills and training of the examiner. Thus, ultrasound competencies should be acquired early during medical education. The instructional approach affects the retention and performance of skills. A promising approach is "mastery learning." The aim of the study was to evaluate the effectiveness of "mastery learning" compared with the "see one, do one" approach by performing a focused assessment of sonography for trauma (FAST) in undergraduate medical students based using an academic assessment tool (Objective Structured Clinical Examination [OSCE]). In a prospective controlled trial, 146 participants were randomly allocated to two groups (see one, do one and mastery learning) and trained in a 90-min module. In the see one, do one group, the trainer demonstrated the complete FAST routine, and then the students trained each other on it under supervision and received direct oral feedback from the tutors. In the mastery learning group, each student received a routing slip. The routing slip contained five levels of competence for the FAST routine, each of which had to be achieved (e.g., choosing the correct probe) and verified by the trainer before working toward the next competency level. The acquired competencies were assessed after training using the OSCE, which is a standardized practical exam using checklists. The mastery learning group attained 40.69 ± 5.6 points on average (of a maximum of 46 points), and the see one, do one group, 33.85 ± 7.7 points (p < 0.001). Mastery learning is an effective teaching method for undergraduate medical students performing FAST and is superior to the see one, do one approach, as assessed with the OSCE.

Training in obstetric and neonatal emergencies in Mexico: effect on knowledge and self-efficacy by gender, age, shift, and profession

BACKGROUND

Continuing education is essential for healthcare workers. Education interventions can help to maintain and improve competency and confidence in the technical skills necessary to address adverse events. However, characteristics of the health provider such as age (related to more critical and reflexive attitude); sex (relationship with gender socialization), profession and work conditions might have an influence on the effect of continuing education efforts.

METHODS

A training in the management of obstetric and neonatal emergencies (PRONTO, Spanish acronym for Neonatal and Obstetric Rescue Program: Optimal and Timely treatment) was implemented in 14 hospitals in six Mexican states between 2013 and 2014, with a before-after evaluation design. A total of 351 health providers including physicians, interns, nurses and midwives completed the training and were included in the analytic sample. Mixed-effects regression models were fitted to model changes in knowledge and self-efficacy scores after the training for each training topic. Interaction terms of training with age, gender, profession, and shift were included to evaluate possible heterogeneities of effect. All models considered the within-hospital clustering of participants.

RESULTS

After training, all participants showed a significant knowledge gain by an average of 19 percentage points for hemorrhage, 23 for neonatal resuscitation, 19 for shoulder dystocia, and 15 for preeclampsia/eclampsia (p < 0.001). Participants who worked night shifts showed lower scores for overall knowledge, compared with morning shift workers. Interns perceived the lowest self-efficacy while they scored very high in knowledge. Self-efficacy in managing obstetric and neonatal emergencies increased significantly by 16 percentage points in average.

CONCLUSIONS

Our results show that PRONTO is generally successful in increasing knowledge and self-efficacy on all topics but knowledge and self efficacy levels vary greatly by factors such as work shift. Training should be particularly aimed at personnel working during weekends and night shifts, as well as interns and nurses.

Baseline Cardiopulmonary Resuscitation Skill Performance of Nursing Students Is Improved After One Resuscitation Quality Improvement Skill Refresher

This article reports the results of baseline cardiopulmonary resuscitation (CPR) skills performance measurements from 467 nursing students. All participants had completed a CPR course. Baseline measurements were compared to performance after one 10-minute refresher training session on the Resuscitation Quality Improvement system. Significant improvements were made after the computer- and practice-based refresher. Findings suggest that staff developers should evaluate the use of audio and visual feedback devices to improve the quality of CPR provided by clinical staff.

Learners and Luddites in the Twenty-first Century: Bringing Evidence-based Education to Anesthesiology

Anesthesiologists are both teachers and learners and alternate between these roles throughout their careers. However, few anesthesiologists have formal training in the methodologies and theories of education. Many anesthesiology educators often teach as they were taught and may not be taking advantage of current evidence in education to guide and optimize the way they teach and learn. This review describes the most up-to-date evidence in education for teaching knowledge, procedural skills, and professionalism. Methods such as active learning, spaced learning, interleaving, retrieval practice, e-learning, experiential learning, and the use of cognitive aids will be described. We made an effort to illustrate the best available evidence supporting educational practices while recognizing the inherent challenges in medical education research. Similar to implementing evidence in clinical practice in an attempt to improve patient outcomes, implementing an evidence-based approach to anesthesiology education may improve learning outcomes.

Toward Practice-Based Continuing Education Protocols: Using Testing to Help Physicians Update Their Knowledge

INTRODUCTION

Using assessment to facilitate learning is a well-established priority in education but has been associated with variable effectiveness for continuing professional development. What factors modulate the impact of testing in practitioners are unclear. We aimed to improve capacity to support maintenance of competence by exploring variables that influence the value of web-based pretesting.

METHODS

Family physicians belonging to a practice-based learning program studied two educational modules independently or in small groups. Before learning sessions they completed a needs assessment and were assigned to either sit a pretest intervention or read a relevant review article. After the learning session, they completed an outcome test, indicated plans to change practice, and subsequently documented changes made.

RESULTS

One hundred twelve physicians completed the study, 92 in small groups. The average lag between tests was 6.3 weeks. Relative to those given a review article, physicians given a pretest intervention: (1) reported spending less time completing the assigned task (16.7 versus 25.7 minutes); (2) performed better on outcome test questions that were repeated from the pretest (65.9% versus 58.7%); and (3) when the learning module was completed independently, reported making a greater proportion of practice changes to which they committed (80.0% versus 45.0%). Knowledge gain was unrelated to physicians' stated needs.

DISCUSSION

Low-stakes formative quizzes, delivered with feedback, can influence the amount of material practicing physicians remember from an educational intervention independent of perceptions regarding the need to engage in continuing professional development on the particular topic.

The timing of testing influences skill retention after basic life support training: a prospective quasi-experimental study

BACKGROUND

Proper basic life support (BLS) is key in improving the survival of out-of-hospital cardiac arrest. BLS skills deteriorate in three to 6 months after training. One method to improve skill retention may be using the "testing effect" to test skills at the end of a BLS course. The aim of our study was to investigate whether either testing or the timing of such testing after BLS training have any influence on skill retention.

METHODS

This was a post-test only, partial coverage, prospective quasi-experimental study designed to evaluate a BLS training course among 464 fifth year medical students at Semmelweis University in the first semester of 2013/2014. Groups were systematically but non-randomly assigned to either a control group that took no exam or one of two experimental groups that took an exam (N = 179, NoExam group; N = 165, EndExam group - exam at the end of the BLS training; N = 120, 3mExam group - exam 3 months after the BLS training). The ability to perform ten prescribed essential BLS steps was evaluated during a skill retention assessment 2 months after the course in the NoExam, 2 months after the course (and the exam) in the EndExam and 5 months after the course (2 months after the exam) in the 3mExam group to measure skill retention and the effect of our intervention. Scores were calculated for each BLS step, and also summed up as a total score. We used Kruskal-Wallis test to assess differences in skill retention.

RESULTS

Overall, NoExam and EndExam groups showed similar skill retention. The mean total score (and many of the sub-scores) of students was significantly higher in the 3mExam group compared to both the NoExam and the EndExam groups, and there was no difference in the total score (and many of the sub-scores) of the latter two groups. The 3mExam group had less variability in total scores (and many of the sub-scores) than the other two groups.

CONCLUSION

Our study provides evidence that testing these skills 3 months after BLS training may be more effective than either testing immediately at the end of the course or no testing at all.

Using Testing as a Learning Tool

Objective. The purpose of this review is to discuss some principles from cognitive psychology regarding the benefits of testing and translate those findings into practical applications for instruction and studying. Findings. Testing or retrieval practice is superior to re-study for promoting long-term retention. The benefits of testing can be see with open-ended responses (eg, cued or free recall) and multiple choice questions. The use of multiple-choice questions during testing may have an additional benefit as it may stabilize information that is stored in memory but temporarily inaccessible due to disuse (eg, marginal knowledge). Summary. Testing can have multiple learning benefits. We emphasize that incorporating opportunities for retrieval after teaching is an essential component of lasting learning. In addition, retrieval practice can be incorporated in all aspects of instruction.

Resuscitation Education Science: Educational Strategies to Improve Outcomes From Cardiac Arrest: A Scientific Statement From the American Heart Association

The formula for survival in resuscitation describes educational efficiency and local implementation as key determinants in survival after cardiac arrest. Current educational offerings in the form of standardized online and face-to-face courses are falling short, with providers demonstrating a decay of skills over time. This translates to suboptimal clinical care and poor survival outcomes from cardiac arrest. In many institutions, guidelines taught in courses are not thoughtfully implemented in the clinical environment. A current synthesis of the evidence supporting best educational and knowledge translation strategies in resuscitation is lacking. In this American Heart Association scientific statement, we provide a review of the literature describing key elements of educational efficiency and local implementation, including mastery learning and deliberate practice, spaced practice, contextual learning, feedback and debriefing, assessment, innovative educational strategies, faculty development, and knowledge translation and implementation. For each topic, we provide suggestions for improving provider performance that may ultimately optimize patient outcomes from cardiac arrest.

The Impact of Medical Students' Individual Teaching Format Choice on the Learning Outcome Related to Clinical Reasoning

BACKGROUND

Repeated formative assessments using key feature questions have been shown to enhance clinical reasoning. Key feature questions augmented by videos presenting clinical vignettes may be more effective than text-based questions, especially in a setting where medical students are free to choose the format they would like to work with. This study investigated learning outcomes related to clinical reasoning in students using video- or text-based key feature questions according to their individual preferences.

OBJECTIVE

The aim of this study was to test the hypothesis that repeated exposure to video-based key feature questions enhances clinical reasoning to a greater extent than repeated exposure to text-based key feature questions if students are allowed to choose between those different formats on their own.

METHODS

In this monocentric, prospective, nonrandomized trial, fourth-year medical students attended 12 computer-based case seminars during which they worked on case histories containing key feature questions. Cases were available in a text- and a video-based format. Students chose their preferred presentation format at the beginning of each case seminar. Student performance in key feature questions was assessed in formative entry, exit, and retention exams and was analyzed with regard to preceding exposure to video- or text-based case histories.

RESULTS

Of 102 eligible students, 75 provided written consent and complete data at all study exams (response rate=73.5%). A majority of students (n=52) predominantly chose the text-based format. Compared with these, students preferring the video-based format achieved a nonsignificantly higher score in the exit exam (mean 76.2% [SD 12.6] vs 70.0% [SD 19.0]; P=.15) and a significantly higher score in the retention exam (mean 75.3% [SD 16.6] vs 63.4% [SD 20.3]; P=.02). The effect was independent of the video- or text-based presentation format, which was set as default in the respective exams.

CONCLUSIONS

Despite students' overall preference for text-based case histories, the learning outcome with regard to clinical reasoning was higher in students with higher exposure to video-based items. Time-on-task is one conceivable explanation for these effects as working with video-based items was more time-consuming. The baseline performance levels of students do not account for the results as the preceding summative exam results were comparable across the 2 groups. Given that a substantial number of students chose a presentation format that was less effective, students might need to be briefed about the beneficial effects of using video-based case histories to be able to make informed choices about their study methods.

Challenges & Issues: Evidence-Based Clinical Skills Teaching and Learning: What Do We Really Know?

The recent programmatic focus on skills development in veterinary medicine means that many programs are devoting increased time to formal clinical skills teaching. This expansion makes it essential that we use the time as effectively as possible. This review examines current practices and veterinary training principles using the broader field of evidence-based motor skills learning as a lens. In many areas, current practices may be hindering learning. Proposed practices include using videos and discussions for pre-laboratory training, focusing on a single complex skill at a time, using more near-peer instructors rather than faculty, including assessments in each teaching or practice session, and encouraging supervised distributed practice by incorporating practice sessions into the formal curriculum. Ensuring mastery of a few core skills rather than exposure to many may be the new goal. Further research is urgently needed on block versus spiral curricula, optimum instructor-to-student ratios, learning and practice schedules, hours required for proficiency, and the benefits of exercise on motor skills learning.

Utility of intermittent online quizzes as an early warning for residents at risk of failing the pediatric board certification examination

BACKGROUND

Traditionally, quizzes have been applied as a tool for summative assessment, though literature suggests their use as a formative assessment can improve motivation and content retention. With this premise, we implemented a series of intermittent, online quizzes known as the Board Examination Simulation Exercise (BESE). We sought to demonstrate an association between BESE participation and scores and performance on the American Board of Pediatrics (ABP) Certifying Examination (CE).

METHODS

Residents were assigned online quizzes on a single topic at 2 week intervals that consisted of 20 multiple choice questions written by the study authors. This analysis includes graduates of 3 Pediatric and Internal Medicine-Pediatrics residency programs.

RESULTS

Data were available for 329 residents. The overall BESE score weakly correlated with ABP CE score (n = 287; r = 0.39, p < 0.0001). ABP CE pass rates increased from 2009 to 2016 at all programs combined (p = 0.0001). A composite BESE score ≤ 11 had sensitivity of 54% and specificity of 80% for predicting ABP CE failure on the first attempt. There was no difference in ABP CE failure rates or scores by number of completed quizzes.

CONCLUSION

Intermittent online quizzes implemented at three pediatric residency programs were associated with overall increasing ABP CE pass rates. BESE increased program emphasis on board preparation. Residents with lower BESE scores more often failed ABP CE. Though additional data are needed, BESE is a promising tool for pediatric resident learning and board preparation. It may also aid in earlier identification of residents at higher risk of failing the ABP CE and facilitate targeted interventions.

Assessment and recency drive skill acquisition

BACKGROUND

The assessment of medical students' clinical skill acquisition is variable and is driven by differences in curricula and health services requirements. This project aimed to ascertain whether different assessment models impact on students' performance of as intravenous (i.v.) cannulation. Recency of practice was also explored as a factor influencing performance.

METHODS

A total of 137 students in the first clinical year (years 2 or 3 of a 4-year course) of their medical degree, from four regional clinical schools, participated in a multiple-station mock objective structured clinical examination (MOSCE). Intravenous cannulation was one of the stations examined. Fifty-nine students came from a model that required the assessment of i.v. proficiency during their clinical year (model 1). Seventy-eight students came from a model that required no assessment of i.v. proficiency (model 2). Students reported their most recent clinical i.v. cannulation experience relative to the MOSCE.

RESULTS

The MOSCE pass rate of 73% for students in model 1 was significantly higher than the corresponding MOSCE pass rate of 45% for students in model 2. There was a highly significant association between assessment model and MOSCE pass/fail rate. The assessment model was also highly associated with recency of practice. The assessment of medical students' clinical skill acquisition is variable CONCLUSIONS: The results support an assessment model that promotes the continuing clinical practice of i.v. cannulation. Integration of this model will require innovative approaches by staff and collaboration with affiliated organisations.

Test-enhanced learning in health professions education: A systematic review: BEME Guide No. 48

BACKGROUND

Cognitive psychology studies demonstrate that subjects who attempt to recall information show better learning, retention, and transfer than subjects who spend the same time studying the same material (test-enhanced learning, TEL). We systematically reviewed TEL interventions in health professions education.

METHODS

We searched 13 databases, 14 medical education journals, and reference lists. Inclusion criteria included controlled studies of TEL that compared TEL to studying the same material or to a different TEL strategy. Two raters screened articles for inclusion, abstracted information, determined quality scores, and calculated the standardized mean difference (SMD) for the learning outcomes.

RESULTS

Inter-rater agreement was excellent for all comparisons. The 19 included studies reported 41 outcomes with data sufficient to determine a SMD. TEL interventions included short answer questions, multiple choice questions, simulation, and standardized patients. Five of six immediate learning outcomes (SMD 0.09-0.44), 21 of 23 retention outcomes (SMD 0.12-2.5), and all seven transfer outcomes (SMD 0.33-1.1) favored TEL over studying.

CONCLUSIONS

TEL demonstrates robust effects across health professions, learners, TEL formats, and learning outcomes. The effectiveness of TEL extends beyond knowledge assessed by examinations to clinical applications. Educators should include TEL in health professions curricula to enhance recall, retention, and transfer.

Comparison of a newly established emotional stimulus approach to a classical assessment-driven approach in BLS training: a randomised controlled trial

OBJECTIVE

The study objective was to implement two strategies (short emotional stimulus vs announced practical assessment) in the teaching of resuscitation skills in order to evaluate whether one led to superior outcomes.

SETTING

This study is an educational intervention provided in one German academic university hospital.

PARTICIPANTS

First-yearmedical students (n=271) during the first3 weeks of their studies.

INTERVENTIONS

Participants were randomly assigned to one of two groups following a sequence of random numbers: the emotional stimulus group (EG) and the assessment group (AG). In the EG, the intervention included watching an emotionally stimulating video prior to the Basic Life Support (BLS) course. In the AG, a practical assessment of the BLS algorithm was announced and tested within a 2 min simulated cardiac arrest scenario. After the baseline testing, a standardised BLS course was provided. Evaluation points were defined 1 week and 6 months after.

PRIMARY OUTCOME MEASURES

Compression depth (CD) and compression rate (CR) were recorded as the primary endpoints for BLS quality.

RESULTS

Within the study, 137 participants were allocated to the EG and 134 to the AG. 104 participants from EG and 120 from AG were analysed1 week after the intervention, where they reached comparable chest-compression performance without significant differences (CR P=0.49; CD P=0.28). The chest-compression performance improved significantly for the EG (P<0.01) and the AG (P<0.01) while adhering to the current resuscitation guidelines criteria for CD and CR.

CONCLUSIONS

There was no statistical difference between both groups' practical chest-compression-performance. Nevertheless, the 2 min video sequence used in the EG with its low production effort and costs, compared with the expensive assessment approach, provides broad opportunities for applicability in BLS training.

The effect of repeated testing vs repeated practice on skills learning in undergraduate dental education

OBJECTIVES

Recent studies in undergraduate medical education have demonstrated the advantage of repeated testing over repeated practice with regard to knowledge and skills retention. The aim of this study was to investigate whether this "testing effect" also applies to skills retention in undergraduate dental education.

METHODS

In this prospective, randomised controlled trial, fourth-year dental students at Göttingen University Medical Centre participated in a training session on surgical suturing in winter term 2014/2015. Following this, they were either assigned to two sessions of additional skills training (group A) or two sessions of skills assessment with feedback (group B). These sessions were spaced over a period of 4 weeks. Skills retention was assessed in a summative objective structured clinical examination (OSCE) at the end of term, that is 6 months after the initial teaching session.

RESULTS

A total of 32 students completed the study. With regard to suturing, OSCE performance was significantly better in group B than group A (81.9±13.1% vs 63.0±15.4%; P=0.001; Cohen's d=1.33). There was no significant OSCE performance difference in the two groups with regard to other learning objectives that were addressed in the end-of-term examination. Thus, the group difference was specific to suturing skills.

CONCLUSIONS

This is the first study to demonstrate that in dental education, repeated testing produces more favourable skills retention than repeated practice. Test-enhanced learning might be a viable concept for skills retention in undergraduate dentistry education.

Can a simulation-based training program impact the use of evidence based routine practices at birth? Results of a hospital-based cluster randomized trial in Mexico

Background

In Mexico, although the majority of births are attended in hospitals, reports have emerged of obstetric violence, use of unsafe practices, and failure to employ evidence-based practices (EBP). Recent attention has refocused global efforts towards provision of quality care that is both patient-centered and evidence-based. Scaling up of local interventions should rely on strong evidence of effectiveness.

Objective

To perform a secondary analysis to evaluate the impact of a simulation and team-training program (PRONTO) on the performance of EBP in normal births.

Methods

A pair-matched cluster randomized controlled trial of the intervention was designed to measure the impact of the program (PRONTO intervention) on a sample of 24 hospitals (12 hospitals received the PRONTO training and 12 served as controls) in the states of Chiapas, Guerrero, and Mexico. We estimated the impact of receiving the intervention on the probability of birth practices performance in a sample of 641 observed births of which 318 occurred in the treated hospitals and 323 occurred in control hospitals. Data was collected at 4 time points (baseline, 4^th, 8^th and 12^th months after the training). Women were blinded to treatment allocation but observers and providers were not. Estimates were obtained by fitting difference-in-differences logistic regression models considering confounding variables. The trial is registered at clinicaltrials.gov: # NCT01477554.

Results

Significant changes were found following the intervention. At 4 months post-intervention an increase of 20 percentage points (p.p.) for complete Active Management of Third Stage of Labor (AMTSL) (p = 0.044), and 16 p.p. increase for Skin-to-Skin Contact (p = 0.067); at 12 months a 25 p.p. increase of the 1st step of AMTSL (p = 0.026) and a 42 p.p. increase of Delayed Cord Clamping (p = 0.004); at 4 months a 30 (p = 0.001) and at 8 months a 22 (p = 0.010) p.p. decrease for Uterine Sweeping.

Conclusions

The intervention has an impact on adopting EBP at birth, contributing to an increased quality of care. Long lasting impacts on these practices are possible if there were to be a widespread adoption of the training techniques including simulation, team-training and facilitated discussions regarding routine care.

Intraosseous access can be taught to medical students using the four-step approach

BACKGROUND

The intraosseous (IO) access is an alternative route for vascular access when peripheral intravascular catheterization cannot be obtained. In Denmark the IO access is reported as infrequently trained and used. The aim of this pilot study was to investigate if medical students can obtain competencies in IO access when taught by a modified Walker and Peyton's four-step approach.

METHODS

Nineteen students attended a human cadaver course in emergency procedures. A lecture was followed by a workshop. Fifteen students were presented with a case where IO access was indicated and their performance was evaluated by an objective structured clinical examination (OSCE) and rated using a weighted checklist. To evaluate the validity of the checklist, three raters rated performance and Cohen's kappa was performed to assess inter-rater reliability (IRR). To examine the strength of the overall IRR, Randolph's free-marginal multi rater kappa was used.

RESULTS

A maximum score of 15 points was obtained by nine (60%) of the participants and two participants (13%) scored 13 points with all three raters. Only one participant failed more than one item on the checklist. The expert rater rated lower with a mean score of 14.2 versus the non-expert raters with mean 14.6 and 14.3. The overall IRR calculated with Randolph's free-marginal multi rater kappa was 0.71.

CONCLUSION

The essentials of the IO access procedure can be taught to medical students using a modified version of the Walker and Peyton's four-step approach and the checklist used was found reliable.

A study on the knowledge and attitudes on advanced life support among medical students and medical officers in a tertiary care hospital in Sri Lanka

Background

Advanced life support (ALS) and cardio pulmonary resuscitation, provided at the right time is essential for improving mortality in medical emergencies. Accurate knowledge and skills on this regard, in all medical personals is an essential part of medical education and it should be up to date with varying protocols. The aim of this study is to assess the knowledge and attitudes among the undergraduate medical students and medical officers in the Teaching Hospital Peradeniya and provide suggestions to improve the training programme on ALS.

Methods

A standardized self-administered questionnaire regarding knowledge and attitudes on ALS was filled by 4th and final year medical students, and medical officers, and the data was analyzed.

Results

There were 411 eligible candidates and of them 130 (31.6 %) were 4th year medical students, 221 (53.8 %) were final year medical students and 60 (14.6 %) were medical officers. Of the medical officers, only 15.8 % indicated that the internship training was adequate to handle an emergency confidently. Approximately 45 % of the medical officers and 34.6 % of the final year medical students were confident of saving lives with their current ALS knowledge. However, only 22 % of 4th year medical students were confident in saving the life of a patient.

Conclusions

Overall, just over 10 % of participants demonstrated inadequate ALS knowledge scores. A significantly higher proportion of final year medical students had good knowledge, compared to medical officers and 4th year students. Only one-third of participants were confident in saving a life with their current ALS knowledge. Nearly all participants thought that the ALS course should be reevaluated frequently.

Towards a program of assessment for health professionals: from training into practice

Despite multifaceted attempts to "protect the public," including the implementation of various assessment practices designed to identify individuals at all stages of training and practice who underperform, profound deficiencies in quality and safety continue to plague the healthcare system. The purpose of this reflections paper is to cast a critical lens on current assessment practices and to offer insights into ways in which they might be adapted to ensure alignment with modern conceptions of health professional education for the ultimate goal of improved healthcare. Three dominant themes will be addressed: (1) The need to redress unintended consequences of competency-based assessment; (2) The potential to design assessment systems that facilitate performance improvement; and (3) The importance of ensuring authentic linkage between assessment and practice. Several principles cut across each of these themes and represent the foundational goals we would put forward as signposts for decision making about the continued evolution of assessment practices in the health professions: (1) Increasing opportunities to promote learning rather than simply measuring performance; (2) Enabling integration across stages of training and practice; and (3) Reinforcing point-in-time assessments with continuous professional development in a way that enhances shared responsibility and accountability between practitioners, educational programs, and testing organizations. Many of the ideas generated represent suggestions for strategies to pilot test, for infrastructure to build, and for harmonization across groups to be enabled. These include novel strategies for OSCE station development, formative (diagnostic) assessment protocols tailored to shed light on the practices of individual clinicians, the use of continuous workplace-based assessment, and broadening the focus of high-stakes decision making beyond determining who passes and who fails. We conclude with reflections on systemic (i.e., cultural) barriers that may need to be overcome to move towards a more integrated, efficient, and effective system of assessment.

On Shaky Ground: Learner Response and Confidence After Tabletop Earthquake Simulation

OBJECTIVE

The aim of this study was to evaluate residents' confidence and attitudes related to management of earthquake victims during a tabletop simulation and 6 months after the intervention.

METHODS

Pediatric residents from 4 training programs were recruited via e-mail. The tabletop simulation involved 3 pediatric patients (crush injury, head injury, and a nonverbal patient with minor injuries). A facilitated debriefing took place after the simulation. The same simulation was repeated 6 months later. A survey was administered before the simulation, immediately after, and after the 6-month repeat simulation to determine participants' self-rated confidence and willingness to respond in the event of a disaster. A 5-point Likert scale that ranged through novice, advanced beginner, competent, proficient, and expert was used.

RESULTS

Ninety-nine participants completed the survey before the initial simulation session. Fifty-one residents completed the immediate postsurvey, and 75 completed the 6-month postsurvey. There was a statistically significant improvement in self-rated confidence identifying and managing victims of earthquake disasters after participating in the simulation, with 3% rating themselves as competent on the presurvey and 33% rating themselves as competent on the postsurvey (P < 0.05). There was a nonstatistically significant improvement in confidence treating suspected traumatic head injury as well as willingness to deploy to both domestic and international disasters.

CONCLUSIONS

Tabletop simulation can improve resident comfort level with rare events, such as caring for children in the aftermath of an earthquake. Tabletop can also be easily integrated into resident curriculum and may be an effective way to provide disaster medical response training for trainees.

Taking the sting out of assessment: is there a role for progress testing?

CONTEXT

It has long been understood that assessment is an important driver for learning. However, recently, there has been growing recognition that this powerful driving force of assessment has the potential to undermine curricular efforts. When the focus of assessment is to categorise learners into competent or not (i.e. assessment of learning), rather than being a tool to promote continuous learning (i.e. assessment for learning), there may be unintended consequences that ultimately hinder learning. In response, there has been a movement toward constructing assessment not only as a measurement problem, but also as an instructional design problem, and exploring more programmatic models of assessment across the curriculum. Progress testing is one form of assessment that has been introduced, in part, to attempt to address these concerns. However, in order for any assessment tool to be successful in promoting learning, careful consideration must be given to its implementation.

OBJECTIVE

The purpose of this paper is to consider the implications of implementing progress testing within practice, and how this might promote or impede learning in the three phases of assessment (pre-test, pure-test and post-test).

METHODS

We will examine the literature on how assessment drives learning and how this might apply to progress testing. We will also explore the distinction between assessment of learning and assessment for learning, including ways in which they overlap and differ. We end by discussing how the properties of an assessment tool can be harnessed to optimise learning.

CONCLUSIONS

Progress tests are one potential solution to the problem of removing (or at least lessening) the sting associated with assessment. If implemented with careful thought and consideration, progress tests can be used to support the type of deep, meaningful and continuous learning that we are trying to instill in our learners.

Development and validation of a theoretical test in non-anaesthesiologist-administered propofol sedation for gastrointestinal endoscopy

OBJECTIVE

Safety with non-anaesthesiologist-administered propofol sedation (NAAP) during gastrointestinal (GI) endoscopy is related to theoretical knowledge. A summative testing of knowledge before attempting supervised nurse-administered propofol sedation (NAPS) in the clinic is advised. The aims of this study were to develop a theoretical test about propofol sedation, to gather validity evidence for the test and to measure the effect of a NAPS-specific training course.

MATERIAL AND METHODS

A three-phased psychometric study on multiple choice questionnaire (MCQ) test development, gathering of validity evidence and evaluation of the effect of a specific NAAP course on the test result. A MCQ containing 86 questions was developed and administered 113 times to 91 participants representing novices, intermediates and experienced.

RESULTS

Question difficulty analyses revealed 50 level I and II questions. The 50 MCQs showed mean (SD) intergroup differences (p = 0.001) between novices = 28.6 (4.82), intermediates = 36.8 (5.43) and experienced = 41.8 (4.65) and provided a pass score of 35.2. The course with pre-course test had significant effect on the knowledge of nurses (18% increase) and physicians (19% increase; p = 0.001 and 0.001, respectively).

CONCLUSIONS

Data supported the validity of the developed MCQ test. The NAPS-specific course with pre-course testing adds theoretical knowledge to already well-prepared participants.

The benefits of testing for learning on later performance

Testing has been shown to enhance retention of learned information beyond simple studying, a phenomena known as test-enhanced learning (TEL). Research has shown that TEL effects are greater for tests that require the production of responses [e.g., short-answer questions (SAQs)] relative to tests that require the recognition of correct answers [e.g., multiple-choice questions (MCQs)]. High stakes licensure examinations have recently differentiated MCQs that require the application of clinical knowledge (context-rich MCQs) from MCQs that rely on the recognition of "facts" (context-free MCQs). The present study investigated the influence of different types of educational activities (including studying, SAQs, context-rich MCQs and context-free MCQs) on later performance on a mock licensure examination. Fourth-year medical students (n = 224) from four Quebec universities completed four educational activities: one reading-based activity and three quiz-based activities (SAQs, context-rich MCQs, and context-free MCQs). We assessed the influence of the type of educational activity on students' subsequent performance in a mock licensure examination, which consisted of two types of context-rich MCQs: (1) verbatim replications of previous items and (2) items that tested the same learning objective but were new. Mean accuracy scores on the mock licensure exam were higher when intervening educational activities contained either context-rich MCQs (Mean z-score = 0.40) or SAQs (M = 0.39) compared to context-free MCQs (M = -0.38) or study only items (M = -0.42; all p < 0.001). Higher mean scores were only present for verbatim items (p < 0.001). The benefit of testing was observed when intervening educational activities required either the generation of a response (SAQs) or the application of knowledge (context-rich MCQs); however, this effect was only observed for verbatim test items. These data provide evidence that context-rich MCQs and SAQs enhance learning through testing compared to context-free MCQs or studying alone. The extent to which these findings generalize beyond verbatim questions remains to be seen.

Can formative quizzes predict or improve summative exam performance?

OBJECTIVE

Despite wide use, the value of formative exams remains unclear. We evaluated the possible benefits of formative assessments in a physical examination course at our chiropractic college.

METHODS

Three hypotheses were examined: (1) Receiving formative quizzes (FQs) will increase summative exam (SX) scores, (2) writing FQ questions will further increase SE scores, and (3) FQs can predict SX scores. Hypotheses were tested across three separate iterations of the class.

RESULTS

The SX scores for the control group (Class 3) were significantly less than those of Classes 1 and 2, but writing quiz questions and taking FQs (Class 1) did not produce significantly higher SX scores than only taking FQs (Class 2). The FQ scores were significant predictors of SX scores, accounting for 52% of the SX score. Sex, age, academic degrees, and ethnicity were not significant copredictors.

CONCLUSION

Our results support the assertion that FQs can improve written SX performance, but students producing quiz questions didn't further increase SX scores. We concluded that nonthreatening FQs may be used to enhance student learning and suggest that they also may serve to identify students who, without additional remediation, will perform poorly on subsequent summative written exams.

The effects of test-enhanced learning on long-term retention in AAN annual meeting courses

OBJECTIVE

We measured the long-term retention of knowledge gained through selected American Academy of Neurology annual meeting courses and compared the effects of repeated quizzing (known as test-enhanced learning) and repeated studying on that retention.

METHODS

Participants were recruited from 4 annual meeting courses. All participants took a pretest. This randomized, controlled trial utilized a within-subjects design in which each participant experienced 3 different postcourse activities with each activity performed on different material. Each key information point from the course was randomized in a counterbalanced fashion among participants to one of the 3 activities: repeated short-answer quizzing, repeated studying, and no further exposure to the materials. A final test covering all information points from the course was taken 5.5 months after the course.

RESULTS

Thirty-five participants across the 4 courses completed the study. Average score on the pretest was 36%. Performance on the final test showed that repeated quizzing led to significantly greater long-term retention relative to both repeated studying (55% vs 46%; t[34] = 3.28, SEM = 0.03, p = 0.01, d = 0.49) and no further exposure (55% vs 44%; t[34] = 3.16, SEM = 0.03, p = 0.01, d = 0.58). Relative to the pretest baseline, repeated quizzing helped participants to retain almost twice as much of the knowledge acquired from the course compared to repeated studying or no further exposure.

CONCLUSIONS

Whereas annual meeting continuing medical education (CME) courses lead to long-term gains in knowledge, when repeated quizzing is added, retention is significantly increased. CME planners may consider adding repeated quizzing to increase the impact of their courses.

Test-enhanced learning and its effect on comprehension and diagnostic accuracy

CONTEXT

In health professions education, tests have traditionally been used to assess the skills and knowledge of learners. More recently, research in psychology and education has shown that tests can also be used to enhance student memory; a phenomenon called the 'testing effect'. Much of the research in this domain has focused on enhancing rote memory of simple facts, and not on the deeper comprehension and application of complex theoretical knowledge necessary to diagnose and manage patients. The purpose of this study was to examine the effects of testing on students' comprehension of the basic science mechanisms and diagnostic accuracy.

METHODS

Undergraduate dental and dental hygiene students (n = 112) were taught the radiographic features and pathophysiology underlying four intrabony abnormalities. Participants were divided into two groups: the test-enhanced (TE) condition and the study (ST) condition. Following the learning phase, the TE condition group completed an interventional test that tested the basic science mechanisms and the ST condition group was given additional study passages without being tested. Participants in both groups then completed a diagnostic test and a memory test immediately after the learning phase and 1 week later.

RESULTS

A main effect of testing condition was found. The participants in the TE condition group outperformed those in the ST condition group on immediate and delayed testing. Unlike the diagnostic test, the memory test showed no difference between the groups.

CONCLUSION

The inclusion of the basic science test appears to have improved the students' understanding of the underlying disease mechanisms learned and also improved their performance on a test of diagnostic accuracy.

The effectiveness of a 'train the trainer' model of resuscitation education for rural peripheral hospital doctors in Sri Lanka

BACKGROUND

Sri Lankan rural doctors based in isolated peripheral hospitals routinely resuscitate critically ill patients but have difficulty accessing training. We tested a train-the-trainer model that could be utilised in isolated rural hospitals.

METHODS

Eight selected rural hospital non-specialist doctors attended a 2-day instructor course. These "trained trainers" educated their colleagues in advanced cardiac life support at peripheral hospital workshops and we tested their students in resuscitation knowledge and skills pre and post training, and at 6- and 12-weeks. Knowledge was assessed through 30 multiple choice questions (MCQ), and resuscitation skills were assessed by performance in a video recorded simulated scenario of a cardiac arrest using a Resuci Anne Skill Trainer mannequin.

RESULTS/DISCUSSION/CONCLUSION

Fifty seven doctors were trained. Pre and post training assessment was possible in 51 participants, and 6-week and 12-week follow up was possible for 43, and 38 participants respectively. Mean MCQ scores significantly improved over time (p<0.001), and a significant improvement was noted in "average ventilation volume", "compression count", and "compressions with no error", "adequate depth", "average depth", and "compression rate" (p<0.01). The proportion of participants with compression depth ≥40mm increased post intervention (p<0.05) and at 12-week follow up (p<0.05), and proportion of ventilation volumes between 400-1000mls increased post intervention (p<0.001). A significant increase in the proportion of participants who "checked for responsiveness", "opened the airway", "performed a breathing check", who used the "correct compression ratio", and who used an "appropriate facemask technique" was also noted (p<0.001). A train-the-trainer model of resuscitation education was effective in improving resuscitation knowledge and skills in Sri Lankan rural peripheral hospital doctors. Improvement was sustained to 12 weeks for most components of resuscitation knowledge and skills. Further research is needed to identify which components of training are most effective in leading to sustained improvement in resuscitation.

The overall impact of testing on medical student learning: quantitative estimation of consequential validity

Given medical education's longstanding emphasis on assessment, it seems prudent to evaluate whether our current research and development focus on testing makes sense. Since any intervention within medical education must ultimately be evaluated based upon its impact on student learning, this report seeks to provide a quantitative accounting of the learning gains attained through educational assessments. To approach this question, we estimate achieved learning within a medical school environment that optimally utilizes educational assessments. We compare this estimate to learning that might be expected in a medical school that employs no educational assessments. Effect sizes are used to estimate testing's total impact on learning by summarizing three effects; the direct effect, the indirect effect, and the selection effect. The literature is far from complete, but the available evidence strongly suggests that each of these effects is large and the net cumulative impact on learning in medical education is over two standard deviations. While additional evidence is required, the current literature shows that testing within medical education makes a strong positive contribution to learning.

The importance of seeing the patient: test-enhanced learning with standardized patients and written tests improves clinical application of knowledge

Previous research has shown that repeated retrieval with written tests produces superior long-term retention compared to repeated study. However, the degree to which this increased retention transfers to clinical application has not been investigated. In addition, increased retention obtained through written testing has not been compared to other forms of testing, such as simulation testing with a standardized patient (SP). In our study, 41 medical students learned three clinical topics through three different learning activities: testing with SPs, testing using written tests, and studying a review sheet. Students were randomized in a counter-balanced fashion to engage in one learning activity per topic. They participated in four weekly testing/studying sessions to learn the material, engaging in the same activity for a given topic in each session. Six months after initial learning, they returned to take an SP test on each topic, followed by a written test on each topic 1 week later. On both forms of final testing, we found that learning through SP testing and written testing generally produced superior long-term retention compared to studying a review sheet. SP testing led to significantly better performance on the final SP test relative to written testing, but there was no significant difference between the two testing conditions on the final written test. Overall, our study shows that repeated retrieval practice with both SPs and written testing enhances long-term retention and transfer of knowledge to a simulated clinical application.