Testing Test-Enhanced Continuing Medical Education: A Randomized Controlled Trial

Educational Method to Promote Recognition, Diagnosis, and Treatment of Eating Disorders: A Cluster-Randomized Study of the Effects of Retrieval Practice Amongst Medical Trainees

Background Eating disorder (ED) education during medical training is lacking. Few medical trainees feel comfortable managing EDs, and an alarming 78% of healthcare providers report feeling insecure in treating EDs. Recognizing EDs early is crucial as the standardized mortality ratio for patients with anorexia nervosa is more than five times higher than that of the general population. Retrieval practice is a powerful tool in producing meaningful learning of complex concepts in education. We investigated the effectiveness of retrieval practice in ED education among medical trainees to improve recognition, diagnosis, and treatment of EDs among adolescent patients. Methods This exploratory, prospective, cluster-randomized trial enrolled residents and medical students over 14, four-week blocks. Participants were randomized by block to either the conventional lecture-based format (control group) or the retrieval-based educational format (intervention group). The control group received case-based lectures. The intervention group received education via 21 case-based quiz questions over the block with immediate feedback. Groups completed nine-item, multiple choice pre (T1)- and post (T2)-rotation knowledge tests, covering recognizing, diagnosing, and treating EDs. All participants also completed pre- and post-rotation surveys designed to measure self-perceived comfort, confidence in training, knowledge, and skills on the topic of EDs. Results The study's primary outcome was the difference between T1 and T2 scores between study groups. The intervention group showed greater improvement from T1 to T2 (5.8 to 7.4, respectively) than the control group (5.1 to 6.0, respectively). The difference between mean T1 and T2 scores in the control group versus the intervention group was significant (p=0.020). Despite the control group reporting improvements in confidence regarding training on EDs, this increased confidence was inversely correlated with scores on T2 (r=-0.502, p=0.011). Conclusions Trainees benefit from retrieval practice to improve knowledge acquisition regarding EDs. Standard lectures may confer false confidence to learners, which may not accurately align with actual knowledge acquisition.

Enhancing Knowledge and Attitudes Regarding Opioid Use Disorder Among Private Primary Care Clinics: A Quality Improvement Project

ABSTRACT

Opioid use disorder (OUD) continues to impact communities worldwide. British Columbia specifically declared a public health emergency in April 2016. It is known that patients with OUD often experience barriers in access to care, including limited knowledge and training among providers, as well as persisting stigma in the medical community. The Doctor of Nursing Practice quality improvement project sought to provide barrier-targeted OUD education while using multiple effective teaching methods, such as test-enhanced learning, to family nurse practitioners (FNPs) working among private primary care clinics to assess the impact on knowledge and attitudes. In review of an experience survey, zero participants had received prior education on OUD (N = 7). The Drug and Drug Problems Perceptions Questionnaire was used to assess attitudes. In review of the data, attitudes before receiving education (Mdn = 74) improved after receiving barrier-targeted education (Mdn = 66), W = 0, p < .05. Knowledge was tested at three time points. After a review of unique identifiers, four participant tests were successfully linked. It was found that knowledge after receiving education (M = 7.75, Mdn = 7.5) improved in comparison with baseline knowledge (M = 6, Mdn = 6) and further improved after a 1-month time frame (M = 8.5, Mdn = 8.5). Although the project was limited by sample size, providing education to FNPs who have not received prior education on OUD, and using modalities such as test-enhanced learning, showed a favorable impact on knowledge and attitudes. In light of the opioid epidemic, nursing leaders must continue to actively engage practicing FNPs and students with OUD education. FNPs are well positioned to be champions in this area and may mobilize teams to overcome barriers among private primary care clinics and increase access to care.

Improving mastery and retention of knowledge and complex skills among sterile processing professionals: A pilot study on borescope training and competency testing

BACKGROUND

Healthcare is shifting toward minimally invasive procedures requiring increasingly complex instruments and sophisticated processing technologies. Effective training methods are needed to ensure sterile processing professionals acquire and retain essential skills.

OBJECTIVE

This study aimed to develop and evaluate a new training model that supports mastery and retention of complex key skills.

METHODS

The model was pilot-tested with training focused on visual inspection of endoscopes. Pre- and post-training tests were administered to enhance learning during a face-to-face workshop that interspersed lectures and hands-on practice, followed by structured homework, and an online booster session. Surveys assessed satisfaction and confidence levels.

RESULTS

Mean test scores for nine certified sterile processing employees increased significantly following the workshop (41% vs. 84%, p<0.001). After the workshop, all trainees identified actionable visible defects on patient-ready endoscopes in their facilities. Test scores remained high after two months (90%), and trainees reported higher technical confidence and satisfaction levels after training.

CONCLUSIONS

This study demonstrated effectiveness and clinical relevance of a new, evidence-based model for training sterile processing professionals that incorporated pre-testing, lectures, hands-on practice, a training booster, and post-testing to enhance learning. This model may be applicable to other complex skills necessary for infection prevention and patient safety.

The Importance of Sociocultural Factors in Moderating the Applicability of Test-Enhanced Learning to Continuing Professional Development

Many studies have shown that repeated testing promotes learning, an effect known as test-enhanced learning. Although test-enhanced learning has been well-established within health care trainees, there are mixed findings of whether testing facilitates learning within continuing professional development (CPD) contexts. This piece argues that for testing to support learning in CPD, there is need to understand the various social, cultural, and structural factors that influence clinicians' willingness to participate in regular testing activities. The solution to this problem is conceptual in nature, whereby CPD programs must consider how learning cultures shape practicing clinicians' perspectives toward testing.

Knowledge Retention Using "Relay": A Novel Active-Learning Technique

INTRODUCTION

Many residency programs utilize passive didactic lectures despite mixed evidence for knowledge retention. This prospective study investigated the efficacy of "relay," an active-learning technique, as measured by residents' knowledge retention and attitudes compared to traditional format.

METHODS

Faculty presented lectures to four family medicine residency programs on a randomized schedule. Lectures were followed by a 15-minute question-and-answer (Q&A) session or relay session. A relay is a collaborative, question-based game. The primary outcome was knowledge retention at 3 months, comparing Q&A to relay sessions as measured by a multiple-choice assessment. Responses were only included if a given resident completed knowledge assessments for both Q&A and relay sessions, to allow for intraresident adjustments, in addition to program, training year, and lecturer/topic. Secondary outcomes included residents' self-perceived knowledge and engagement as surveyed by an ordinal scale immediately following the learning session.

RESULTS

The primary analysis included 51 responses from 18 unique residents. The adjusted mean knowledge assessment score at 3 months was not statistically different after the relay sessions compared to Q&A (67% vs 60%, respectively; 7% difference, 95% CI: -4 to 18%, P=.20). For the secondary outcomes of learner attitudes (n=143 responses), learners reported greater engagement after the relay sessions compared to Q&A (51% vs 28% "very engaged"; overall P=.003), but self-perceived knowledge was not significantly different (overall P=.05, rounded down).

CONCLUSIONS

The relay technique did not show significant difference in 3-month knowledge retention, nor immediate self-perceived knowledge, despite greater learner self-perceived engagement.

Experiences from Two Ways of Integrating Pre- and Post-course Multiple-choice Assessment Questions in Educational Events for Surgeons

To examine how to optimise the integration of multiple-choice questions (MCQs) for learning in continuing professional development (CPD) events in surgery, we implemented and evaluated two methods in two subspecialities over multiple years. The same 12 MCQs were administered pre- and post-event in 66 facial trauma courses. Two different sets of 10 MCQs were administered pre- and post-event in 21 small animal fracture courses. We performed standard psychometric tests on responses from participants who completed both the pre- and post-event assessment. The average difficulty index pre-course was 57% with a discrimination index of 0.20 for small animal fractures and 53% with a discrimination index of 0.15 for facial trauma. For the majority of the individual MCQs, the scores were between 30%-70% and the discrimination index was >0.10. The difficulty index post-course increased in both groups (to 75% and 62%). The pre-course MCQs resulted in an average score in the expected range for both formats suggesting they were appropriate for the intended level of difficulty and an appropriate pre-course learning activity. Post-course completion resulted in increased scores with both formats. Both delivery methods worked well in all regions and overall quality depends on applying a solid item development and validation process.

Science of Learning Strategy Series: Article 2, Retrieval Practice

Retrieval practice is an evidence-based, science of learning strategy that is relevant to the planning and implementation of continuing professional development (CPD). Retrieval practice requires one to examine long-term memory to work with priority information again in working memory. Retrieval practice improves learning in two ways. It improves memory for the information itself (direct benefit), and retrieval practice provides feedback about what needs additional effort (indirect). Both benefits contribute significantly to durable learning. Research from cognitive psychology and neuroscience provides the rationale for retrieval practice, and examples of its implementation in health professions education are increasingly available in the literature. Through appropriate utilization, CPD participants can benefit from retrieval practice by making more-informed educational choices, and CPD planners can benefit in efforts to improve educational activities.

Training in colorectal surgery in Europe and 20 years of the European Board of Surgical Qualification coloproctology examination

AIM

Training in colorectal surgery across Europe is not yet standardized. The European Board of Surgical Qualification (EBSQ) coloproctology examination has been held annually since 1998. The aims of this study were to illustrate the current situation of coloproctology specialization in Europe and to analyse the EBSQ examinations held over the last 20 years.

METHOD

A survey, focused on current training and education in colorectal surgery in Europe, was conducted among all national representatives of the European Society of Coloproctology (ESCP) in 2018. Candidate demographics (1998-2018) and the results of the EBSQ examination (2007-2018) were analysed.

RESULTS

In Europe, there are currently 26 national colorectal societies, 27 national annual colorectal meetings, 16 national specialized training programmes and 13 national colorectal fellowships. Six countries have board certification in colorectal surgery and five a dedicated examination. During the last 20 years, 475 candidates from 29 countries, of whom 88 (19%) were women, passed the EBSQ examination. The pass rate was higher in younger applicants (< 42 years, P = 0.01). The success rate was higher for candidates with academic experience (more than five publications or presentations) and with an academic title (thesis) (P = 0.01).

CONCLUSION

Colorectal surgical training is still not standardized in Europe, although efforts have been made to recognize colorectal surgery as an independent speciality. The number of holders of the EBSQ Diploma has increased over the years, demonstrating the acceptance of the examination among European surgeons. Young candidates with an academic profile are the most successful.

Toward Practice-Based Continuing Education Protocols: Using Testing to Help Physicians Update Their Knowledge

INTRODUCTION

Using assessment to facilitate learning is a well-established priority in education but has been associated with variable effectiveness for continuing professional development. What factors modulate the impact of testing in practitioners are unclear. We aimed to improve capacity to support maintenance of competence by exploring variables that influence the value of web-based pretesting.

METHODS

Family physicians belonging to a practice-based learning program studied two educational modules independently or in small groups. Before learning sessions they completed a needs assessment and were assigned to either sit a pretest intervention or read a relevant review article. After the learning session, they completed an outcome test, indicated plans to change practice, and subsequently documented changes made.

RESULTS

One hundred twelve physicians completed the study, 92 in small groups. The average lag between tests was 6.3 weeks. Relative to those given a review article, physicians given a pretest intervention: (1) reported spending less time completing the assigned task (16.7 versus 25.7 minutes); (2) performed better on outcome test questions that were repeated from the pretest (65.9% versus 58.7%); and (3) when the learning module was completed independently, reported making a greater proportion of practice changes to which they committed (80.0% versus 45.0%). Knowledge gain was unrelated to physicians' stated needs.

DISCUSSION

Low-stakes formative quizzes, delivered with feedback, can influence the amount of material practicing physicians remember from an educational intervention independent of perceptions regarding the need to engage in continuing professional development on the particular topic.

Commentary: Paradigms, Axiology, and Praxeology in Medical Education Research

This supplement includes 11 research papers accepted by the 2018 Research in Medical Education Committee. In this Commentary, the authors draw the attention of medical educators and researchers to questions about the researcher's philosophical paradigm. Such questions fundamentally guide research-the choice of conceptual frameworks, methods, and methodology. The authors provide a brief overview of research paradigms and the related concept of axiology and praxeology in medical education research. The authors map the 11 accepted research articles, describing the researchers' stated or implied worldviews and the impact on chosen methods. The authors close by encouraging researchers to state the research paradigm behind their research, comment on how they ensured that the methods used displayed consistency with that paradigm, and highlight the value the research adds to everyday education.