Spaced radiology: encouraging durable memory using spaced testing in pediatric radiology

Integration of artificial intelligence in radiology education: a requirements survey and recommendations from faculty radiologists, residents, and medical students

BACKGROUND

To investigate the perspectives and expectations of faculty radiologists, residents, and medical students regarding the integration of artificial intelligence (AI) in radiology education, a survey was conducted to collect their opinions and attitudes on implementing AI in radiology education.

METHODS

An online questionnaire was used for this survey, and the participant anonymity was ensured. In total, 41 faculty radiologists, 38 residents, and 120 medical students from the authors' institution completed the questionnaire.

RESULTS

Most residents and students experience different levels of psychological stress during the initial stage of clinical practice, and this stress mainly stems from tight schedules, heavy workloads, apprehensions about making mistakes in diagnostic report writing, as well as academic or employment pressures. Although most of the respondents were not familiar with how AI is applied in radiology education, a substantial proportion of them expressed eagerness and enthusiasm for the integration of AI into radiology education. Especially among radiologists and residents, they showed a desire to utilize an AI-driven online platform for practicing radiology skills, including reading medical images and writing diagnostic reports, before engaging in clinical practice. Furthermore, faculty radiologists demonstrated strong enthusiasm for the notion that AI training platforms can enhance training efficiency and boost learners' confidence. Notably, only approximately half of the residents and medical students shared the instructors' optimism, with the remainder expressing neutrality or concern, emphasizing the need for robust AI feedback systems and user-centered designs. Moreover, the authors' team has developed a preliminary framework for an AI-driven radiology education training platform, consisting of four key components: imaging case sets, intelligent interactive learning, self-quiz, and online exam.

CONCLUSIONS

The integration of AI technology in radiology education has the potential to revolutionize the field by providing innovative solutions for enhancing competency levels and optimizing learning outcomes.

Spaced Repetition Learning in Radiology Education: Exploring Its Potential and Practical Application

In today's medical landscape, rapidly learning vast amounts of information requires innovative learning methods. Spaced repetition tools (like Anki) aid efficient knowledge absorption and retention among medical trainees. Yet, adoption of these tools in radiology medical student education lags despite proven effectiveness. This article highlights spaced repetition as a learning tool alongside other evidence-based educational practices, aiming to revolutionize radiology education among medical students. We (1) describe the educational theory and current application of spaced repetition in the setting of other learning techniques often found in undergraduate medical education; (2) underscore the underutilization of tools such as Anki in radiology education; and (3) offer practical guidance for educators interested in integrating spaced repetition into their teaching methodologies.

Optimizing Radiological Education: The Role of Learning Spacing via Test sets in Enhancing Diagnostic Proficiency in Breast Screening Readers

RATIONALE AND OBJECTIVES

Integrating learning spacing in medicine has shown promise in enhancing knowledge retention and diagnostic proficiency. While studies demonstrate the effectiveness of spaced learning in various fields, limited research exists on its application in radiological training. This study aims to investigate the impact of intervals in spaced training on radiologists' and trainees' diagnostic performance via mammogram test sets.

METHODS

54 radiologists and 101 radiology trainees completed 207 and 458 first-time readings of 9 mammogram test sets between 2019 and 2023. Each test set comprised of 60 mammograms (20 cancer and 40 normal), sourced retrospectively from BreastScreen Australia. Each radiologist evaluated mammograms using the BIRADS lexicon. Readers' performance was compared with truth data and evaluated in terms of specificity, case sensitivity, lesion sensitivity, ROC AUC and JAFROC FOM. The progress of readers' performances in following test sets after the first one was analyzed using Wilcoxon Signed Rank test. The association of participants' performances and the intervals among test sets' completions was investigated using Pearson's test.

RESULTS

Significant positive correlations were found between intervals and radiologists' improvement in specificity and JAFROC FOM (P < 0.05). The separation of 4 to 10 days showed the most improvement among radiologists across all metrics, while intervals exceeding 90 days related to highest increase in case sensitivity (5.15%), lesion sensitivity (6.55%), ROC AUC (3.05%) and JAFROC FOM (6.3%). Trainees completing test sets in one day showed positive correlations with their ROC AUC (R=0.45; P = 0.008) and JAFROC FOM (R=0.43; P = 0.02), while those taking a longer time to complete showed negative impacts on case sensitivity (P = 0.009) and ROC AUC (P = 0.02). Remarkable progress in trainees was found in case sensitivity (6.15%), lesion sensitivity (11.6%), ROC AUC (3.5%) and JAFROC FOM (4.35%) with test set intervals of 31-90 days.

CONCLUSIONS

Radiologists demonstrated superior performance when the training test sets were spaced over longer intervals, whereas trainees exhibited proficiency with shorter time separations. By optimizing the spacing of reviewing and practicing radiological concepts, mammogram readers can bolster memory retention and diagnostic decision-making skills.

Enhancing students' agency in learning anatomy vocabulary with a formative intervention design

One of the major challenges for health science students is the rapid acquisition of a new vocabulary in anatomy comprising several hundred new words. Research has shown that vocabulary learning can be improved when students are directed to vocabulary strategies. This paper reported a study with a formative intervention design inspired by Vygotsky's method of double stimulation. In this design, the students were put in a structured situation that invited them to identify the challenges in learning anatomy and then provided them with active guidance and a range of anatomy vocabulary learning strategies that scaffolded them to work out a solution to the challenge and develop their individualized anatomy learning resources. The data were collected from surveys, pre and postquiz results, and group discussion transcripts. The results revealed students perceived one of the main challenges in learning anatomy was learning, memorizing, and remembering many new words. A key finding in our study was that the formative intervention enhanced students' agency in creating resources for learning anatomy vocabulary. In addition, the development of their understanding showed a recursive form: from concrete experiences to abstract concepts and then to concrete new practices.

Digital flashcards and medical physiology performance: a dose-dependent effect

Use of digital flashcards promotes active recall, spaced repetition, and self-assessment academic principles. This work explores the association and dose-dependent effect of this study method and locomotor (LP) and cardiovascular physiology (CP) grades. A single-faculty cohort study of medical LP and CP students was conducted, and 155 and 676 flashcards, respectively, were created through Moodle. An exploratory analysis examined three exam results (2019), and a confirmatory study used a fourth exam (2021) in another CP cohort. Of 685 students enrolled, 558 participated in the exploratory analysis: 319 (69%) for LP and 311 (84%) for CP, of which 203 LP and 267 CP students were flashcard users. Median grades were higher among flashcard users, and the number of cards reviewed was positively correlated with grades (r = 0.275 to 0.388 for LP and r = 0.239 to 0.432 for CP, P < 0.001). Multiple linear regression models confirmed a positive dose-dependent association between results and the number of flashcards studied: for every 100 LP cards reviewed, exam grades increased 0.44-0.75 on a 0-20 scale range (P < 0.001), and for every 1,000 CP flashcards, results raised 0.81-1.08 values (P < 0.05). These findings were confirmed in the 2021 CP cohort of 269 participants, of whom 67% were flashcard users. Digital flashcard revision has a consistent positive dose-dependent association on LP and CP grades.NEW & NOTEWORTHY Implementing flashcard-based strategies is a feasible way to promote active recall, spaced repetition, and self-assessment, and students are highly adherent to these initiatives. There is a positive dose-dependent association between the number of flashcards reviewed and physiology grades. These results are consistent across different physiology subjects, under different cohorts, over short and medium terms.

Anki Flashcards for Radiology Education

From online textbooks to educational social media posts, digital learning tools are transforming medical education. Spaced repetition programs are a popular tool, with 70% of first-year medical students reporting using the free, open-source flashcard application Anki.1 The Anki program pairs active recall and spaced repetition learning principles with a simple but versatile user interface.2 Spaced repetition flashcard programs such as Anki are uniquely suited to radiology education because they strengthen image-based recall, a necessary skill for radiology trainees to develop. In this review, we will describe the science behind active recall and spaced repetition programs, introduce the Anki platform, and describe future directions for its potential as an educational resource for radiology.

The Effectiveness of Spaced Learning, Interleaving, and Retrieval Practice in Radiology Education: A Systematic Review

PURPOSE

Radiology is a highly complex field that requires mastery over an ever-expanding body of knowledge. Spaced learning, interleaving, and retrieval practice are evidence-based learning strategies that enhance long-term retention of information. The aim of this systematic review is to assess the effectiveness of these interventions in the setting of radiology education.

METHODS

The authors searched MEDLINE, Embase, PsycInfo, ERIC, and forward and backward citations for studies published between database inception and February 19, 2023. Eligibility criteria for included studies were randomized and quasi-randomized controlled trials that investigated the impact of spaced, interleaved, or retrieval practice on knowledge retention of medical trainees after education related to medical imaging as assessed by postinterventional examination scores.

RESULTS

Of 1,316 records reviewed, 8 studies met eligibility criteria. Two studies investigated spaced learning, two studies interleaving, and six studies retrieval practice, including two trials that evaluated interventions incorporating both spaced learning and retrieval practice. Five of eight studies reported statistically significant differences between interventional and control groups on either immediate or delayed postinterventional examinations.

CONCLUSIONS

Despite extensive evidence in support of spaced, interleaved, and retrieval practice within the broader literature, few studies have examined the effectiveness of these strategies in radiology education. Additional trials are required to evaluate the usefulness of incorporating these techniques into educational programs related to medical imaging.

Study Smarter: Applying the Science of Learning to Radiology

Lifelong learning is critical to a successful career in radiology, but many learners use inefficient or ineffective studying strategies developed from personal experience. Decades of cognitive psychology research have identified several techniques that consistently improve knowledge consolidation and retrieval. To improve their knowledge and ultimately patient care, radiologists should strive to understand and adopt these learning techniques. The first part of this article reviews several evidence-based learning principles, including active retrieval and the testing effect, spaced repetition, interleaving, deliberate practice, and growth mind-set. The second part provides practical suggestions on how to incorporate these principles into radiology learning, both during training and beyond.

Artificial Intelligence for Personalised Ophthalmology Residency Training

Residency training in medicine lays the foundation for future medical doctors. In real-world settings, training centers face challenges in trying to create balanced residency programs, with cases encountered by residents not always being fairly distributed among them. In recent years, there has been a tremendous advancement in developing artificial intelligence (AI)-based algorithms with human expert guidance for medical imaging segmentation, classification, and prediction. In this paper, we turned our attention from training machines to letting them train us and developed an AI framework for personalised case-based ophthalmology residency training. The framework is built on two components: (1) a deep learning (DL) model and (2) an expert-system-powered case allocation algorithm. The DL model is trained on publicly available datasets by means of contrastive learning and can classify retinal diseases from color fundus photographs (CFPs). Patients visiting the retina clinic will have a CFP performed and afterward, the image will be interpreted by the DL model, which will give a presumptive diagnosis. This diagnosis is then passed to a case allocation algorithm which selects the resident who would most benefit from the specific case, based on their case history and performance. At the end of each case, the attending expert physician assesses the resident's performance based on standardised examination files, and the results are immediately updated in their portfolio. Our approach provides a structure for future precision medical education in ophthalmology.

Attitudes of an international student cohort to the Quizlet study system employed in an advanced clinical health care review course

Computer-based learning applications and mobile technology have transformed many aspects of the educational experience over the last decade, producing software aimed at improving learning efficiency and streamlining the presentation of course materials. One such class of software, purpose-created to take advantage of spaced learning and spaced testing principles, are electronic flashcard applications. We provide a perspective on the novel use of the Quizlet flashcard application in a tertiary educational setting. To reduce cognitive load for international graduate dental students taking a pharmacology review course, we implemented Quizlet, which integrates both spaced learning and self-testing, to improve the student learning experience. This study assessed students' perceptions of the Quizlet flashcard system in a student cohort comprised of two consecutive years' classes (n = 51 students in total). Results indicated broad acceptance of Quizlet based on ease of use of the software and ease of study of the material. Our data provide insight into the use of this common software in a professional healthcare tertiary education setting and further demonstrate the successful application of electronic flashcards for a mixed international student cohort. Further research should include an assessment of the impact of flashcard on long-term knowledge retention in this setting.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s10639-022-11371-3.

American Board of Physical Medicine and Rehabilitation Diplomate Customization Choices on the Longitudinal Assessment for Physical Medicine and Rehabilitation: A First-Year Experience

ABSTRACT

The American Board of Physical Medicine and Rehabilitation began administering the longitudinal assessment for physical medicine and rehabilitation for continuing certification in 2020. The longitudinal assessment for physical medicine and rehabilitation digitally delivers quarterly short assessments of content and repeats missed items to facilitate learning as well as serve as a summative certification assessment. With a goal of offering content relevant to an individual's practice and learning needs, diplomates choose how to customize the domains or topic areas of their question content on an annual basis. This report describes the first year of experience with customization of longitudinal assessment for physical medicine and rehabilitation. The American Board of Physical Medicine and Rehabilitation diplomate customization data are grouped and compared in a variety of ways to ascertain whether there are differences in customization choices. While customization choices were similar across several domains, significant differences were seen when comparing groups with specific areas of practice or subspecialty certification. Smaller differences were also seen when comparing question domain allocation choice between sexes, age groups, and practice setting. The results from this first full year of experience confirm an alignment of this innovative assessment approach to individual physician practice, a significant step in improving the relevance of continuing certification overall for participating physicians.

A spaced-repetition approach to enhance medical student learning and engagement in medical pharmacology

BACKGROUND

Pharmacology is a cornerstone of medical education as it underlies safe prescribing practices. However, medical students have reported unease regarding their perceived proficiency in medical pharmacology. Despite the significant impetus to improve student outcomes, there is little analysis available of the techniques used by medical students to learn, retain and apply medical pharmacology knowledge.

METHODS

A mixed-methods, student-focused approach was conducted to design and evaluate specific resources developed to address gaps in pharmacology education. This methodology comprised an anonymised scoping survey, followed by semi-structured focus group interviews. We then developed a flashcard resource as an intervention to support long-term revision for academic and clinical success. This resource was released to a cohort of 100 graduate entry preclinical medical students who were invited at the end of year to evaluate the intervention via a subsequent anonymous survey.

RESULTS

The scoping survey received 103 complete responses. Surveys and focus group interviews revealed that only 50% of students engage in ongoing revision. Amongst our cohort, we identified that the evidence-based technique of spaced-repetition was particularly well regarded. Hence, we developed and evaluated a bespoke resource utilising Anki™, an open-source, spaced-repetition flashcard program. A total of 1208 flashcards spanning 156 distinct classes of drugs with supplementary summary tables, diagrams and explanatory video and summary guides were created. Designed as a strategic revision tool to reinforce learning, evaluation showed students greatly appreciated the "comprehensive" and "well formatted" Anki™ resource that supported existing teaching modalities, with a global rating of 3.8 out of 5.

CONCLUSIONS

Strategic and personalised resources for medical pharmacology education that assist with in-semester revision and long-term retention are highly valued amongst students for examination preparation and preparedness for practice. Collectively, these results reflect a novel approach to identifying and addressing weaknesses in existing learning resources in a manner that is inclusive of, and acceptable to, medical students.

Rethinking Radiology: An Active Learning Curriculum for Head Computed Tomography Interpretation

Introduction: Head computed tomography (CT) interpretation is a vital skill for emergency physicians. Existing literature shows poor concordance between emergency physicians and radiologists in head CT interpretation. Prior studies have used passive learning methods to address this knowledge gap. We created an active learning curriculum for teaching head CT interpretation to emergency medicine (EM) residents and compared its effectiveness to a passive learning strategy. Methods: We conducted a prospective, randomized controlled study of EM residents at a single institution. Three educational sessions were delivered over a three-month period via video conference. The active learning cohort (ALC) scrolled through head CT teaching cases we designed on Pascbin, a web-based radiology picture archiving and communication system. The passive learning cohort (PLC) watched instructional videos that scrolled through the same cases. Both cohorts were given equal time to review the cases and ask an instructor questions. Residents took pre-intervention and post-intervention tests on head CT interpretation. We analyzed scores using paired and unpaired t-tests. Results: Forty-two residents took the pre-intervention test. Mean pre- and post-test scores for the ALC were 43.8% and 59.0% (P <0.001), and for the PLC were 41.7% and 45.3% (P = 0.29). The difference in ALC and PLC post-test scores was statistically significant (P = 0.009) with a large effect size (Cohen’s d = 1.34). Conclusion: Our active learning head CT curriculum using Pacsbin showed superior learning outcomes when compared to a passive learning strategy and required no additional time or resources. This intervention offers a more effective and learner-centric method for implementing radiology curricula in EM residency programs.

Training Radiology Residents, Bloom Style

Bloom's Taxonomy, an integral component of learning theory since its inception, describes cognitive skill levels in increasing complexity (Remember, Understand, Apply, Analyze, Evaluate, and Create). Considering Bloom's Taxonomy when writing learning objectives and lecture material, teaching residents at the workstation and creating multiple choice questions can increase an educator's effectiveness. The incorporation of higher Bloom levels aids in cultivating critical thinking skills vital to image interpretation and patient care, and becomes increasingly important as the radiologist's role evolves with the continued development of artificial intelligence. Following established tenets of multiple choice question writing, involving trainees in the question writing process, and incorporating audience response systems into lectures are all strategies in which higher Bloom level skills can be accomplished.

Novel Spaced Repetition Flashcard System for the In-training Examination for Obstetrics and Gynecology

INTRODUCTION

Electronic flashcards allow repeated information exposure over time along with active recall. It is increasingly used for self-study by medical students but remains poorly implemented for graduate medical education. The primary goal of this study was to determine whether a flashcard system enhances preparation for the in-training examination in obstetrics and gynecology (ob-gyn) conducted by the Council on Resident Education in Obstetrics and Gynecology (CREOG).

METHODS

Ob-gyn residents at Duke University were included in this study. A total of 883 electronic flashcards were created and distributed. CREOG scores and flashcard usage statistics, generated internally by interacting with the electronic flashcard system, were collected after the 2019 exam. The primary outcome was study aid usage and satisfaction. The secondary outcome was the impact of flashcard usage on CREOG exam scores.

RESULTS

Of the 32 residents, 31 (97%) participated in this study. Eighteen (58%) residents used the study's flashcards with a median of 276 flashcards studied over a median of 3.7 h. All of the flashcard users found the study aid helpful, and all would recommend them to another ob-gyn resident. Using the flashcards to study for the 2019 CREOG exam appeared to correlate with improvement in scores from 2018 to 2019, but did not achieve statistical significance after adjusting for post-graduate year (beta coefficient = 10.5; 95% confidence interval =  - 0.60,21.7; p = 0.06).

DISCUSSION

This flashcard resource was well received by ob-gyn residents for in-training examination preparation, though it was not significantly correlated with improvement in CREOG scores after adjusting for post-graduate year.

Rethinking gross anatomy in a compressed time frame: Clinical symptoms, not case studies, as the basis for introductory instruction

The goal of this observational study was to develop effective approaches to introduce first year medical students to gross anatomy/embryology in a compressed time frame. Pedagogical reorganization of anatomy instruction in the regions of Lower Extremity and Head and Neck was based upon core clinical conditions taught in second-year and USMLE Step 1 board review courses. These conditions were not presented as clinical problems, as many students had limited prior training in medical terminology, but focused upon clinical symptoms, allowing for direct correlation of structure and function. Instruction stressed vocabulary acquisition and was extended to prepare for laboratory dissections. Overall methodology was multimodal, including "flipped" and traditional lectures, study of prosections/radiographs and small group laboratory review sessions. Content was significantly reduced: knowledge of muscle actions and innervations was required, not muscle origins and insertions. Performance was evaluated by criterion-based written examinations that included a set of questions (34) asked repetitively over an 8 year period (n = 606 students) and by regional practical exams. Mean scores in all areas were sustained or numerically improved, despite the compression of instruction duration. Analysis showed no significant differences based upon question format or instructional modality. Despite the high performance levels, students needing assistance in learning could be identified by score distributions. A survey of students indicated that these changes effectively decreased stress and facilitated review for the USMLE Step 1 Board examination. These results suggest that training in gross anatomy can be modified to a compressed duration by instruction in the context of clinical symptomatology.

How massed practice improves visual expertise in reading panoramic radiographs in dental students: An eye tracking study

The interpretation of medical images is an error-prone process that may yield severe consequences for patients. In dental medicine panoramic radiography (OPT) is a frequently used diagnostic procedure. OPTs typically contain multiple, diverse anomalies within one image making the diagnostic process very demanding, rendering students’ development of visual expertise a complex task. Radiograph interpretation is typically taught through massed practice; however, it is not known how effective this approach is nor how it changes students’ visual inspection of radiographs. Therefore, this study investigated how massed practice–an instructional method that entails massed learning of one type of material–affects processing of OPTs and the development of diagnostic performance. From 2017 to 2018, 47 dental students in their first clinical semester diagnosed 10 OPTs before and after their regular massed practice training, which is embedded in their curriculum. The OPTs contained between 3 to 26 to-be-identified anomalies. During massed practice they diagnosed 100 dental radiographs without receiving corrective feedback. The authors recorded students’ eye movements and assessed the number of correctly identified and falsely marked low- and high prevalence anomalies before and after massed practice. Massed practice had a positive effect on detecting anomalies especially with low prevalence (p < .001). After massed practice students covered a larger proportion of the OPTs (p < .001), which was positively related to the detection of low-prevalence anomalies (p = .04). Students also focused longer, more frequently, and earlier on low-prevalence anomalies after massed practice (p_s < .001). While massed practice improved visual expertise in dental students with limited prior knowledge, there is still substantial room for improvement. The results suggest integrating massed practice with more deliberate practice, where, for example, corrective feedback is provided, and support is adapted to students’ needs.