Application of Frequent, Spaced Multiple-choice Questions as an Educational Tool in the Pediatric Emergency Department

Engagement and learning in an electronic spaced repetition curriculum companion for a paediatrics academic half-day curriculum

Postgraduate residencies utilize academic half-days to supplement clinical learning. Spaced repetition reinforces taught content to improve retention. We leveraged spaced repetition in a curriculum companion for a paediatric residency program's half-day. One half-day lecture was chosen weekly for reinforcement (day 0). Participants received 3 key points on day 1 and a multiple-choice question (MCQ) on day 8. On day 29, they received two MCQs to test reinforced and unreinforced content from the same day 0. Thirty-one (79%) residents participated over 17 weeks, but only 14 (36%) completed more than half of the weekly quizzes. Of all quizzes, 37.4% were completed, with an average weekly engagement of 5.5 minutes. Helpfulness to learning was rated as 7.89/10 on a Likert-like scale. Reported barriers were missing related half-days and emails, or limited time. There was no significant difference in performance between reinforced (63.4%, [53.6-73.3]) and unreinforced (65.6%, [53.7-73.2]) questions. Spaced repetition is a proven strategy in learning science, but was not shown to improve performance. Operational barriers likely limited participation and underpowered our analysis, therefore future implementation must consider practical and individual barriers to facilitate success. Our results also illustrate that satisfaction alone is an inadequate marker of success.

Experiences from Two Ways of Integrating Pre- and Post-course Multiple-choice Assessment Questions in Educational Events for Surgeons

To examine how to optimise the integration of multiple-choice questions (MCQs) for learning in continuing professional development (CPD) events in surgery, we implemented and evaluated two methods in two subspecialities over multiple years. The same 12 MCQs were administered pre- and post-event in 66 facial trauma courses. Two different sets of 10 MCQs were administered pre- and post-event in 21 small animal fracture courses. We performed standard psychometric tests on responses from participants who completed both the pre- and post-event assessment. The average difficulty index pre-course was 57% with a discrimination index of 0.20 for small animal fractures and 53% with a discrimination index of 0.15 for facial trauma. For the majority of the individual MCQs, the scores were between 30%-70% and the discrimination index was >0.10. The difficulty index post-course increased in both groups (to 75% and 62%). The pre-course MCQs resulted in an average score in the expected range for both formats suggesting they were appropriate for the intended level of difficulty and an appropriate pre-course learning activity. Post-course completion resulted in increased scores with both formats. Both delivery methods worked well in all regions and overall quality depends on applying a solid item development and validation process.