Learner feedback and educational outcomes with an internet-based ambulatory curriculum: a qualitative and quantitative analysis

Mental health training in family medicine residencies: International curriculum overview

Integration of mental health into primary care has become a global trend, and many countries have developed mental health training in primary care. However, systematic mental health training for family physicians is insufficient in Japan. The newly established Japan Primary Care Association Mental Health Committee surveyed the current status of mental health training curricula in family medicine residency internationally. Participants were individuals involved in family medicine residency programs who were from Australia, Brazil, Hong Kong, the Philippines, Taiwan, the United Kingdom, and the United States. The results revealed that many academic societies have created competency lists and curriculum guidelines for mental health training; however, the implementation varied. This study is novel as it examined and compared different countries' curriculum; the findings of which can be used as a reference to develop future mental health training curriculum in Japan.

The development of a restorative Managed Clinical Network within the defence primary healthcare organisation

The United Kingdom Armed Forces introduced a Managed Clinical Network to transform care provision for military patients referred with complex restorative treatment needs. This article discusses the processes that underpinned this transformation of service, from assessment of populations needs to implementation of clinical delivery.

3D Printed Models for Teaching Orbital Anatomy, Anomalies and Fractures

PURPOSE

The aim of this study was to determine the efficacy of using 3D printing models in the learning process of orbital anatomy and pathology by ophthalmology residents.

METHODS

A quasi-experimental study was performed with 24 residents of ophthalmology at Mashhad University of Medical Sciences. Each stratum was randomized into two groups. The educational booklets were distributed, and various forms of orbital 3D models were printed from orbital computed tomography (CT) scans. Knowledge enhancement on the topics was measured by comparing pretest and posttest scores.

RESULTS

Thirteen residents who were trained using traditional methods were deemed the control group; while 11 residents who were trained using the 3D printed models were classed as the intervention group. The control group was younger than the intervention group (P = 0.047). The results showed that there was a statistically significant difference in the total posttest scores between the two groups. Based on the repeated measures of the analysis of variance (ANOVA), score variables were significant between the two groups (P = 0.008). Interestingly, the use of the 3D educational model was more effective and statistically significant with the year one residents as compared to the year two residents (P = 0.002).

CONCLUSION

This study is the first one in Iran quantifying the effects of learning using 3D printed models in medical education. In fact, 3D modeling training is seemingly effective in teaching ophthalmic residents. As residents have never encountered such technology before, their experience using 3D models proved to be satisfactory and had a surprising positive effect on the learning process through visual training.

Impact of online toxicology training on health professionals: the Global Educational Toxicology Uniting Project (GETUP)

OBJECTIVE

The Global Educational Toxicology Uniting Project (GETUP), supported by the American College of Medical Toxicology, links countries with and without toxicology services via distance education with the aim to improve education. Due to the lack of toxicology services in some countries there is a knowledge gap in the management of poisonings. We describe our experience with the worldwide delivery of an online introductory toxicology curriculum to emergency doctors and other health professionals treating poisoned patients.

METHODS

We delivered a 15-module introductory Internet-based toxicology curriculum to emergency doctors and health professionals, conducted from August to December 2016. This Internet-based curriculum was adapted from one used to teach emergency residents toxicology in the United States. Modules covered themes such as pharmaceutical (n = 8), toxidromes (n = 2) and agrochemicals (n = 5) poisoning. Participants completed pre-test and post-test multiple choice questions (MCQs) before and after completing the online module, respectively, throughout the course. We collected information on participant demographics, education and training, and perception of relevance of the curriculum. Participants gave feedback on the course and how it affected their practice.

RESULTS

One hundred and thirty-six health professionals from 33 countries participated in the course: 98 emergency doctors/medical officers, 25 physicians, eight pharmacists/poisons information specialists, two toxicologists, two medical students and one nurse. Median age of participants was 34 years. Median number of years postgraduate was seven. Ninety (65%) had access to either a poisons information centre over the phone or toxicologist and 48 (35%) did not. All participants expected the course to help improve their knowledge. Overall median pre-module MCQ scores were 56% (95%CI: 38, 75%) compared to post-module MCQ scores median 89% (95% CI: 67, 100%) (p < .0001).

CONCLUSIONS

Our participants demonstrated an increase in medical knowledge based on performance on MCQs. An online toxicology curriculum is an effective way to deliver education to health professionals treating poisoned patients and can help to bridge the knowledge gap and change practice in developed and developing countries.

Usage of 3D models of tetralogy of Fallot for medical education: impact on learning congenital heart disease

BACKGROUND

Congenital heart disease (CHD) is the most common human birth defect, and clinicians need to understand the anatomy to effectively care for patients with CHD. However, standard two-dimensional (2D) display methods do not adequately carry the critical spatial information to reflect CHD anatomy. Three-dimensional (3D) models may be useful in improving the understanding of CHD, without requiring a mastery of cardiac imaging. The study aimed to evaluate the impact of 3D models on how pediatric residents understand and learn about tetralogy of Fallot following a teaching session.

METHODS

Pediatric residents rotating through an inpatient Cardiology rotation were recruited. The sessions were randomized into using either conventional 2D drawings of tetralogy of Fallot or physical 3D models printed from 3D cardiac imaging data sets (cardiac MR, CT, and 3D echocardiogram). Knowledge acquisition was measured by comparing pre-session and post-session knowledge test scores. Learner satisfaction and self-efficacy ratings were measured with questionnaires filled out by the residents after the teaching sessions. Comparisons between the test scores, learner satisfaction and self-efficacy questionnaires for the two groups were assessed with paired t-test.

RESULTS

Thirty-five pediatric residents enrolled into the study, with no significant differences in background characteristics, including previous clinical exposure to tetralogy of Fallot. The 2D image group (n = 17) and 3D model group (n = 18) demonstrated similar knowledge acquisition in post-test scores. Residents who were taught with 3D models gave a higher composite learner satisfaction scores (P = 0.03). The 3D model group also had higher self-efficacy aggregate scores, but the difference was not statistically significant (P = 0.39).

CONCLUSION

Physical 3D models enhance resident education around the topic of tetralogy of Fallot by improving learner satisfaction. Future studies should examine the impact of models on teaching CHD that are more complex and elaborate.

A Qualitative Evaluation of an Online Expert-Facilitated Course on Tobacco Dependence Treatment

Qualitative evaluations of courses prove difficult due to low response rates. Online courses may permit the analysis of qualitative feedback provided by health care providers (HCPs) during and after the course is completed. This study describes the use of qualitative methods for an online continuing medical education (CME) course through the analysis of HCP feedback for the purpose of quality improvement. We used formative and summative feedback from HCPs about their self-reported experiences of completing an online expert-facilitated course on tobacco dependence treatment (the Training Enhancement in Applied Cessation Counselling and Health [TEACH] Project). Phenomenological, inductive, and deductive approaches were applied to develop themes. QSR NVivo 11 was used to analyze the themes derived from free-text comments and responses to open-ended questions. A total of 277 out of 287 participants (96.5%) completed the course evaluations and provided 690 comments focused on how to improve the program. Five themes emerged from the formative evaluations: overall quality, content, delivery method, support, and time. The majority of comments (22.6%) in the formative evaluation expressed satisfaction with overall course quality. Suggestions for improvement were mostly for course content and delivery method (20.4% and 17.8%, respectively). Five themes emerged from the summative evaluation: feedback related to learning objectives, interprofessional collaboration, future topics of relevance, overall modifications, and overall satisfaction. Comments on course content, website function, timing, and support were the identified areas for improvement. This study provides a model to evaluate the effectiveness of online educational interventions. Significantly, this constructive approach to evaluation allows CME providers to take rapid corrective action.

A Computer-Assisted, Real-Time Feedback System for Medical Students as a Tool for Web-Based Learning

Objectives Medical students sometimes do not receive proper feedback from their instructors. This study evaluated a newly developed automated and personalized real-time feedback system intended to address this issue. Methods Third- and fourth-year medical students participated in quizzes focusing on 17 learning objectives and a five-scale survey that queried their prior knowledge related to blood transfusions. Immediately after completing the quizzes, the students received automated and personalized, real-time feedback and were instructed to take part in self-directed learning. This activity was followed by a final quiz. After completion of the final quiz, the students responded to the five-scale survey that probed the usefulness of and satisfaction with the automated, personalized, real-time feedback system. Results Eighty students took part in this study. The third-year group had a higher score for prior knowledge and also on the first quiz (P= 0.008, P= 0.046, respectively). There was no significant difference in final quiz scores between the third- and fourth-year groups (P= 0.633). The scores for usefulness of and satisfaction with the automated, real-time feedback system were 4.45 and 4.34, and 4.55 and 4.40 in the third- and fourth-year students, respectively. Conclusions The automated, personalized, real-time feedback system provided timely and effective feedback for medical students and was helpful for their self-directed learning.

Internet Training Resulted in Improved Trainee Performance in a Simulated Opioid-Poisoned Patient as Measured by Checklist

INTRODUCTION

Opioid overdose is a leading cause of death in the USA. Internet-based teaching can improve medical knowledge among trainees, but there are limited data to show the effect of Internet-based teaching on clinical competence in medical training, including management of opioid poisoning.

METHODS

We used an ecological design to assess the effect of an Internet-based teaching module on the management of a simulated opioid-poisoned patient. We enrolled two consecutive classes of post-graduate year-1 residents from a single emergency medicine program. The first group (RA) was instructed to read a toxicology textbook chapter and the second group (IT) took a brief Internet training module. All participants subsequently managed a simulated opioid-poisoned patient. The participants' performance was evaluated with two types of checklist (simple and time-weighted), along with global assessment scores.

RESULTS

Internet-trained participants performed better on both checklist scales. The difference between mean simple checklist scores by the IT and RA groups was 0.23 (95 % CI, 0.016-0.44). The difference between mean time-weighted checklist scores was 0.27 (95 % CI, 0.048-0.49). When measured by global assessment, there was no statistically significant difference between RA and IT participants.

CONCLUSION

These data suggest that the Internet module taught basic principles of management of the opioid-poisoned patient. In this scenario, global assessment and checklist assessment may not measure the same proficiencies. These encouraging results are not sufficient to show that this Internet tool improves clinical performance. We should assess the impact of the Internet module on performance in a true clinical environment.

Engaging Physician Learners Through a Web-Based Platform: Individualized End-of-Life Education

BACKGROUND

Web-based modules provide a convenient and low-cost education platform, yet should be carefully designed to ensure that learners are actively engaged. In order to improve attitudes and knowledge in end-of-life (EOL) care, we developed a web-based educational module that employed hyperlinks to allow users access to auxiliary resources: clinical guidelines and seminal research papers.

METHODS

Participants took pre-test evaluations of attitudes and knowledge regarding EOL care prior to accessing the educational module, and a post-test evaluation following the module intervention. We recorded the type of hyperlinks (guideline or paper) accessed by learners, and stratified participants into groups based on link type accessed (none, either, or both). We used demographic and educational data to develop a multivariate mixed-effects regression analysis to develop adjusted predictions of attitudes and knowledge.

RESULTS

114 individuals participated. The majority had some professional exposure to EOL care (prior instruction 62%; EOL referral 53%; EOL discussion 56%), though most had no family (68%) or personal experience (51%). On bivariate analysis, non-partnered (p = .04), medical student training level (p = .03), prior palliative care referral (p = .02), having a family member (p = .02) and personal experience of EOL care (p < .01) were all associated with linking to auxiliary resources via hyperlinks. When adjusting for confounders, β coefficient estimates and least squares estimation demonstrated that participants clicking on both hyperlink types were more likely to score higher on all knowledge and attitude items, and demonstrate increased score improvements.

CONCLUSION

Auxiliary resources accessible by hyperlink are an effective adjunct to web-based learning in end-of-life care.

Concurrent Validity Between a Shared Curriculum, the Internal Medicine In-Training Examination, and the American Board of Internal Medicine Certifying Examination

BACKGROUND

A core objective of residency education is to facilitate learning, and programs need more curricula and assessment tools with demonstrated validity evidence.

OBJECTIVE

We sought to demonstrate concurrent validity between performance on a widely shared, ambulatory curriculum (the Johns Hopkins Internal Medicine Curriculum), the Internal Medicine In-Training Examination (IM-ITE), and the American Board of Internal Medicine Certifying Examination (ABIM-CE).

METHODS

A cohort study of 443 postgraduate year (PGY)-3 residents at 22 academic and community hospital internal medicine residency programs using the curriculum through the Johns Hopkins Internet Learning Center (ILC). Total and percentile rank scores on ILC didactic modules were compared with total and percentile rank scores on the IM-ITE and total scores on the ABIM-CE.

RESULTS

The average score on didactic modules was 80.1%; the percentile rank was 53.8. The average IM-ITE score was 64.1% with a percentile rank of 54.8. The average score on the ABIM-CE was 464. Scores on the didactic modules, IM-ITE, and ABIM-CE correlated with each other (P < .05). Residents completing greater numbers of didactic modules, regardless of scores, had higher IM-ITE total and percentile rank scores (P < .05). Resident performance on modules covering back pain, hypertension, preoperative evaluation, and upper respiratory tract infection was associated with IM-ITE percentile rank.

CONCLUSIONS

Performance on a widely shared ambulatory curriculum is associated with performance on the IM-ITE and the ABIM-CE.