Effect of an Internet-based curriculum on postgraduate education. A multicenter intervention

Knowledge of Pre- and Postexposure Prophylaxis for HIV Prevention Among Internal Medicine Residents in the United States

Prescription rates of pre-exposure prophylaxis (PrEP) have remained low among noninfectious disease providers in the United States despite almost a decade since their introduction. For future primary care doctors, residency is the optimal time to build practice patterns around HIV prevention. We assessed baseline knowledge of PrEP in specific pre- and post-exposure prophylaxis content areas among internal medicine trainees who completed the Physician Education and Assessment Center HIV learning module between 2013 to 2020 (N = 12,060). Resident baseline PrEP knowledge was universally low; despite rising awareness of antiretroviral therapy for PrEP in successive years following the nadir of 41% in 2014, still only 56% of residents affirmed this means of HIV prevention by 2020. Knowledge remained limited regardless of academic year, local HIV prevalence, or training program type. Online module completion increased competence across all content areas. There is still a deficit in HIV prevention knowledge across U.S. internal medicine residents, suggesting insufficient education and exposure to HIV-related care.

Assessment of Internal Medicine Resident Preparedness to Care for Lesbian, Gay, Bisexual, Transgender, and Queer/Questioning Patients

BACKGROUND

Recognizing the unique health needs of sexual and gender minorities (i.e., lesbian, gay, bisexual, transgender, queer/questioning individuals) is critical to providing competent and comprehensive healthcare.

OBJECTIVE

To assess resident knowledge of healthcare issues uniquely affecting sexual and gender minorities as well as the role of online case-based didactics to measure and improve knowledge in the diagnosis and treatment of these patients.

DESIGN

A multicenter online education intervention from December 2016 to April 2018.

PARTICIPANTS

The study population consisted of 833 PGY1-3 residents at 120 internal medicine residency programs in the USA who completed 1018 tests.

INTERVENTIONS

A 1-h online module addressing sexual and gender minority (SGM) health. The test evaluated each resident in four categories: (1) terminology relevant to SGM patients; (2) health disparities and preventive care issues affecting SGM patients; (3) substance use and mental health issues unique to SGM patients; and (4) common sexually transmitted illnesses affecting SGM populations.

MAIN MEASURES

Participants completed a pre-test assessing SGM health knowledge. A didactic module reviewing diagnosis and management of these diseases was then completed, followed by a post-test.

KEY RESULTS

Among 1018 resident respondents, there was no difference between post-graduate year pre-test performance (PGY-1 52%, PGY-2 50%, PGY-3 51%; p = 0.532) or post-test performance (PGY-1 80%, PGY-2 82%, PGY-3 82%; p = 0.285). Pre-test and post-test performance of an online didactic module was the same across test categories and patient populations for PGY-1 vs. PGY-2 vs. PGY-3. Residents demonstrated an improvement between pre- and post-test knowledge.

CONCLUSIONS

Baseline knowledge of health issues of sexual and gender minorities, as assessed by pre-test performance, did not change during residency training. An online didactic module introduced trainees to critical issues regarding the care of these vulnerable populations until such curricula are required in training. Health disparities in LGBTQ communities may improve with improved physician training on clinical care of LGBTQ patients and families.

From Modules to MOOCs: Application of the Six-Step Approach to Online Curriculum Development for Medical Education

Online curricula can make high-quality health professions education accessible in virtually any setting. They can enhance teaching and learning by both standardizing curricular resources and individualizing curricular experiences. Despite growing demand for and institutional interest in online curricula for medical education, many medical educators lack a framework for online curriculum development. Without rigorous and thoughtful development, online curricula can waste opportunity and resources by leading to education that is inferior to traditional methods. In this article, the authors describe a systematic approach to online curriculum development based on the Six-Step Approach for Curriculum Development for Medical Education, a widely used method that has led to successful implementation of a variety of traditional and online curricula. In each step, special considerations for curricula with larger and more diverse learner audiences-characteristic of many online curricula-are highlighted. Four common online curricular formats are also discussed: blended curricula, instructor-led fully online curricula, self-paced modules, and massive open online courses (MOOCs). The authors emphasize factors that differentiate one online format from another, including the budgetary, technical, and human resource requirements for each. The article concludes by urging medical educators to pursue opportunities to study and disseminate online curricular work.

Residents' Attitude, Knowledge, and Perceived Preparedness Toward Caring for Patients from Diverse Sociocultural Backgrounds

Purpose: Training residents to deliver care to increasingly diverse patients in the United States is an important strategy to help alleviate racial and ethnic disparities in health outcomes. Cross-cultural care training of residents continues to present challenges. This study sought to explore the associations among residents' cross-cultural attitudes, preparedness, and knowledge about disparities to better elucidate possible training needs. Methods: This cross-sectional study used web-based questionnaires from 2013 to 2014. Eighty-four internal medicine residency programs with 954 residents across the United States participated. The main outcome was perceived preparedness to care for sociocultural diverse patients. Key Results: Regression analysis showed attitude toward cross-cultural care (beta coefficient [β]=0.57, 95% confidence interval [CI]: 0.49-0.64, p<0.001) and report of serving a large number of racial/ethnic minorities (β=0.90, 95% CI: 0.56-1.24, p<0.001), and low-socioeconomic status patients (β=0.74, 95% CI: 0.37-1.10, p<0.001) were positively associated with preparedness. Knowledge of disparities was poor and did not differ significantly across postgraduate year (PGY)-1, PGY-2, and PGY-3 residents (mean scores: 56%, 58%, and 55%, respectively; p=0.08). Conclusion: Residents' knowledge of health and healthcare disparities is poor and does not improve during training. Residents' preparedness to provide cross-cultural care is directly associated with their attitude toward cross-cultural care and their level of exposure to patients from diverse sociocultural backgrounds. Future studies should examine the role of residents' cross-cultural care-related attitudes on their ability to care for diverse patients.

A Primer on Diabetes Mellitus: Foundations for the Incoming First-Year Resident

INTRODUCTION

Given the increasing prevalence of diabetes mellitus, trainees should have a strong foundation in the management of diabetes. Published literature on the knowledge base and comfort level of medical trainees in diabetes care describes varying levels of exposure to diabetes management in both inpatient and outpatient settings.

METHODS

This eight-module curriculum provides a foundation in the diagnosis, evaluation, and management of diabetes mellitus in the adult patient, as well as pharmacological treatment, patient education, and complications. Specifically, the modules consist of an introduction to diabetes, diagnosis and glycemic goals, patient education, basic nutrition, noninsulin therapies, insulin therapies, complications of diabetes, and financial considerations and cost. Each is a stand-alone presentation that may be viewed nonsequentially. We estimate each module taking 15 to 30 minutes to read. Students received a postsurvey.

RESULTS

We received responses from 23 (18%) of the total eligible residents over the course of 3 years. Approximately 50% of respondents completed an endocrinology elective as either a medical student or first-year resident. Overall, the majority of respondents felt that the modules had the correct amount of content, the online format was adequate, their understanding of diabetes was enhanced, and the curriculum led to altering their care.

DISCUSSION

This resource is unique to MedEdPORTAL as it includes basic information on diabetes education and medical-nutritional therapy. We have required completion of these modules by our internal medicine residents since the class that enrolled in 2013. The curriculum is directed towards incoming first-year internal medicine residents but may also be used by trainees in other primary care fields.

Use of extramural ambulatory care curricula in postgraduate medical training

INTRODUCTION

Extramural curricula developed for the purpose of sharing with other institutions have been designed to improve education on important topics in ambulatory care. We sought to assess the usage rates of these curricula among paediatric, internal medicine, and combined medicine-paediatrics residency programmes in the United States.

METHODS

Surveys on aspects of trainee continuity clinic were sent to paediatric and medicine-paediatrics programme directors in 2012. Surveys contained an item asking respondents about their use of extramural ambulatory care curricula. Since no similar recent data were available for internal medicine, and to verify the accuracy of the paediatric survey data, we queried the editors of four widely used curricula for subscription information. Descriptive and inferential statistics were calculated.

RESULTS

Responses from paediatric programmes indicated that 48 of 111 (43 %) were using an extramural curriculum, compared with 39 of 60 (65 %) medicine-paediatrics programmes (p = 0.007). Editor query revealed a collective subscription rate of internal medicine programmes (300 of 402, 75 %), which was greater than the subscription rate of paediatric programmes (90 of 201, 45 %) (p < 0.001).

DISCUSSION

Training programmes in paediatrics, internal medicine, and combined medicine-paediatrics utilize extramural curricula to guide education in ambulatory care, but internal medicine and medicine-paediatrics programmes employ these curricula at greater rates than paediatric programmes.

Concurrent Validity Between a Shared Curriculum, the Internal Medicine In-Training Examination, and the American Board of Internal Medicine Certifying Examination

BACKGROUND

A core objective of residency education is to facilitate learning, and programs need more curricula and assessment tools with demonstrated validity evidence.

OBJECTIVE

We sought to demonstrate concurrent validity between performance on a widely shared, ambulatory curriculum (the Johns Hopkins Internal Medicine Curriculum), the Internal Medicine In-Training Examination (IM-ITE), and the American Board of Internal Medicine Certifying Examination (ABIM-CE).

METHODS

A cohort study of 443 postgraduate year (PGY)-3 residents at 22 academic and community hospital internal medicine residency programs using the curriculum through the Johns Hopkins Internet Learning Center (ILC). Total and percentile rank scores on ILC didactic modules were compared with total and percentile rank scores on the IM-ITE and total scores on the ABIM-CE.

RESULTS

The average score on didactic modules was 80.1%; the percentile rank was 53.8. The average IM-ITE score was 64.1% with a percentile rank of 54.8. The average score on the ABIM-CE was 464. Scores on the didactic modules, IM-ITE, and ABIM-CE correlated with each other (P < .05). Residents completing greater numbers of didactic modules, regardless of scores, had higher IM-ITE total and percentile rank scores (P < .05). Resident performance on modules covering back pain, hypertension, preoperative evaluation, and upper respiratory tract infection was associated with IM-ITE percentile rank.

CONCLUSIONS

Performance on a widely shared ambulatory curriculum is associated with performance on the IM-ITE and the ABIM-CE.

Factors affecting learner satisfaction with an internet-based curriculum

OBJECTIVE

Online curricula are used increasingly for educating physicians, and evaluating educational outcomes can help improve their effectiveness. It is unknown how specific educational outcomes associate with each other among learners using online curricula. We set out to study how two educational outcomes, learner satisfaction and knowledge, and the learner's year of training and training hospital, were associated with one another among learners accessing a widely used online curriculum.

METHODS

Using data from the 2006-2007 academic year, learner satisfaction was compared with pretest knowledge, posttest knowledge, changes in knowledge, module topic, year of training, and training hospital among 3229 residents at 73 internal medicine residency training programs. A multivariable model was used to calculate the odds ratio of learner satisfaction relative to changes in knowledge.

RESULTS

Module topic, year of training, and hospital type were associated with learner satisfaction. Second-year residents were more satisfied with training modules (mean rating 4.01) than first- and third-year residents (mean ratings 3.97 and 3.95, respectively; P < 0.05). Learner satisfaction was greater among community hospital residents than university hospital residents (mean rating 4.0 vs 3.92; P < 0.05). Learner satisfaction was greater in residents with high pretest and high posttest knowledge (P < 0.05). In multivariate analyses, greater gains in knowledge were associated with greater learner satisfaction (P < 0.05).

CONCLUSIONS

Greater learner satisfaction is associated with greater baseline knowledge, greater knowledge after completing a curriculum, and greater improvement in knowledge while enrolled in a curriculum.

Efficacy of an internet-based tool for improving physician knowledge of chronic kidney disease: an observational study

Background

Early recognition and management of chronic kidney disease (CKD) are associated with better outcomes. Internal medicine residency should prepare physicians to diagnose and manage CKD.

Methods

To examine whether residency training and program characteristics were associated with CKD knowledge and investigate the effectiveness of an internet-based training module in improving CKD knowledge, we analyzed data from CKD training modules administered annually to U.S. internal medicine residents from July 1, 2005 to June 30, 2009. Baseline CKD knowledge was assessed using pre-tests. The modules’ effectiveness was evaluated by post-tests. Comparisons were performed using X² tests and paired t-tests.

Results

Of 4,702 residents, 38%, 33%, and 29% were program year (PGY)-1, PGY-2, and PGY-3, respectively. Baseline CKD knowledge was poor, with mean pre-test scores of 45.1-57.0% across the four years. The lowest pre-test performance was on CKD recognition. Pre-test scores were better with higher training levels (P-trend < 0.001 except 2005–2006 [P-trend = 0.35]). Affiliation with a renal fellowship program or program location within a region of high end-stage kidney disease prevalence was not associated with better baseline CKD knowledge. Completion of the CKD module led to significant improvements from pre- to post-test scores (mean improvement 27.8% [SD: 21.3%] which were consistent from 2005 to 2009.

Conclusions

Knowledge of diagnosis and management of CKD improves during residency training but remains poor among graduating residents. Web-based training can be effective in educating physicians on CKD-related issues. Studies are needed to determine whether knowledge gained from such an intervention translates to improved care of CKD patients.

Learner feedback and educational outcomes with an internet-based ambulatory curriculum: a qualitative and quantitative analysis

BACKGROUND

Online medical education curricula offer new tools to teach and evaluate learners. The effect on educational outcomes of using learner feedback to guide curricular revision for online learning is unknown.

METHODS

In this study, qualitative analysis of learner feedback gathered from an online curriculum was used to identify themes of learner feedback, and changes to the online curriculum in response to this feedback were tracked. Learner satisfaction and knowledge gains were then compared from before and after implementation of learner feedback.

RESULTS

37,755 learners from 122 internal medicine residency training programs were studied, including 9437 postgraduate year (PGY)1 residents (24.4 % of learners), 9864 PGY2 residents (25.5 %), 9653 PGY3 residents (25.0 %), and 6605 attending physicians (17.0 %). Qualitative analysis of learner feedback on how to improve the curriculum showed that learners commented most on the overall quality of the educational content, followed by specific comments on the content. When learner feedback was incorporated into curricular revision, learner satisfaction with the instructive value of the curriculum (1 = not instructive; 5 = highly instructive) increased from 3.8 to 4.1 (p < 0.001), and knowledge gains (i.e., post test scores minus pretest scores) increased from 17.0 % to 20.2 % (p < 0.001).

CONCLUSIONS

Learners give more feedback on the factual content of a curriculum than on other areas such as interactivity or website design. Incorporating learner feedback into curricular revision was associated with improved educational outcomes. Online curricula should be designed to include a mechanism for learner feedback and that feedback should be used for future curricular revision.

Use of an internet-based curriculum to teach internal medicine residents about addiction

OBJECTIVES

: Addiction is an important and common health problem. Many internal medicine training programs do not offer structured training in addiction; as a result, residents often report feeling unprepared in caring for patients with this problem. We developed an Internet-based curriculum to teach internal medicine residents about evaluating and treating patients with substance use disorders.

METHODS

: Three educational modules on addiction were developed and posted on an established Web site that provides an internal medicine curriculum for training programs throughout the United States. Baseline and posttest questions were tested and validated by having house officers and addiction medicine faculty members complete the tests. We compared baseline pretest scores between first (PGY-1) and third year (PGY-3) residents to assess baseline knowledge and pretest and posttest scores for the entire cohort to assess the impact of the modules.

RESULTS

: Each module was completed by over 1200 residents at 86 different training programs. Although overall baseline pretest scores were better among PGY-3 than PGY-1 residents (mean 58% vs 55%; P < 0.05), the difference between the 2 groups for individual modules was not significant. The mean baseline pretest score was 56.4% and posttest score was 74.8%, a difference that was statistically significant (P < 0.001). When asked to rate the educational value of the program, the residents gave it a mean score of 4.2 on a 5-point Likert scale (1 = not instructive; 5 = highly instructive).

CONCLUSIONS

: Internet-based curricula can be an effective tool to disseminate knowledge on addiction to trainees. Learners show an improvement in testing scores and rate these programs highly.

Evaluation of a case-based primary care pediatric conference curriculum

BACKGROUND

Our goal was to assess the impact of a standardized residency curriculum in ambulatory pediatrics on residents' participation, satisfaction, and confidence.

METHODS

A case-based curriculum for weekly primary care conference was developed to replace the existing free-form review of topics at the Yale Pediatrics Residency Program. Before the curricular switch, faculty preceptors and members of the academic year 2005-2006 intern class completed surveys designed to measure conference occurrence and resident attendance, participation, satisfaction, and confidence in clinical skills. One year after the curricular switch, identical surveys were completed by faculty preceptors and members of the academic year 2006-2007 intern class.

RESULTS

Faculty surveys indicated that conferences took place significantly more often after the curricular switch. The number of residents at conference each day (3.18 vs 4.50; P < .01) and the percentage who actually spoke during conference (45% vs 82%, P < .01) significantly increased. There were 18 demographically similar interns in each of the 2 classes. Members of the academic year 2006-2007 intern class, having trained exclusively with the standardized curriculum, were significantly more likely to respond favorably to survey items about participation, satisfaction, and confidence. In addition, they were more likely to endorse survey items that reflected explicit goals of the standardized curriculum and the Accreditation Council for Graduate Medical Education core competencies.

CONCLUSION

Implementation of a structured curriculum for ambulatory care improved interns' self-reported participation, satisfaction, and confidence. The primary care conference occurred more dependably after the curricular change, and improvements in attendance and participation were documented. Pediatric residency programs may make better use of conference time in the ambulatory setting through the use of structured, case-based educational material.

Changes in knowledge of diabetes guidelines during internal medicine residency training

We assessed the impact of residency training on knowledge of diabetes guidelines and found that knowledge of diagnosis and management of diabetes changes modestly during internal medicine residency training. Knowledge improved only between the first and second year of training, and significant gaps in knowledge remained among senior residents.

How to improve medical education website design

BACKGROUND

The Internet provides a means of disseminating medical education curricula, allowing institutions to share educational resources. Much of what is published online is poorly planned, does not meet learners' needs, or is out of date.

DISCUSSION

Applying principles of curriculum development, adult learning theory and educational website design may result in improved online educational resources. Key steps in developing and implementing an education website include: 1) Follow established principles of curriculum development; 2) Perform a needs assessment and repeat the needs assessment regularly after curriculum implementation; 3) Include in the needs assessment targeted learners, educators, institutions, and society; 4) Use principles of adult learning and behavioral theory when developing content and website function; 5) Design the website and curriculum to demonstrate educational effectiveness at an individual and programmatic level; 6) Include a mechanism for sustaining website operations and updating content over a long period of time.

SUMMARY

Interactive, online education programs are effective for medical training, but require planning, implementation, and maintenance that follow established principles of curriculum development, adult learning, and behavioral theory.

A multicenter study of internal medicine residents' perceptions of training, competence, and performance in outpatient HIV care

Routine HIV screening is recommended by the Centers for Disease Control and Prevention (CDC), but it is unknown how well internal medicine residents are trained in HIV risk assessment, testing, counseling, and initial management of HIV patients. We sought to determine internal medicine residents' attitudes about HIV training and the factors that influence their HIV care performance utilizing a cross-sectional survey of 321 second- and third-year internal medicine residents from four programs in Baltimore, Boston, Detroit, and New York City between March and June 2006. Measurements included HIV care experience; attitudes, competency, and adequacy of HIV training; and basic HIV care performance and factors impacting performance. Two hundred twenty-three residents (69%) completed the survey. While 50% of residents reported over 30 HIV inpatient encounters in the past year, the majority of residents had limited outpatient exposure providing care for only 1-5 HIV outpatients. Managing HIV patients was rated an excellent educational opportunity by 89% of residents and 77% planned to care for HIV patients in the future. However, 39% stated that they did not feel competent to provide HIV outpatient care. Higher rates of residents reported deficiency in outpatient HIV training compared to outpatient non-HIV training (p < 0.05) or inpatient HIV training (p < 0.05). Residents reported substandard HIV risk assessment, testing, counseling, and initial management performance. Self-reported proficiency correlated with the number of HIV outpatients cared for and perceived training adequacy. Current residency training in HIV care remains largely inpatient-based and residents frequently rate HIV outpatient training as inadequate.

Curriculum resource use and relationships with educational outcomes in an online curriculum

PURPOSE

Web-based continuing education (CE) offerings have increased dramatically, yet educators know little about factors influencing resource use within online curricula or relationships between resource use and educational outcomes.

METHOD

The authors conducted a study of online curriculum delivery to health care professionals in 2004 and 2005. The authors assessed knowledge and confidence regarding content (herbs and dietary supplements) at baseline and completion. They assessed hours spent and use of three resources (modules read, links accessed, and listserv participation) and how these effected change of knowledge and confidence.

RESULTS

Median time spent on the curriculum was 7 to 10 hours. Three quarters of participants read 36 to 40 modules; half accessed <30 of 335 Internet links. Listserv participation varied; 149 participants (19%) read <or=5 postings, and 168 (22%) read >or=41 postings. Those receiving modules incrementally across several weeks reported more hours spent, more modules read, and more links accessed, but less listserv participation than those receiving all modules at once (all P <or= .008). Those paying for CE credit invested more by all four measures (all P < .001). In multivariable analysis, modules read and hours spent had modest impacts on changes in knowledge and confidence, respectively, but less than paying for CE credit.

CONCLUSIONS

Greater resource use (i.e., time spent, modules read) modestly improved knowledge and confidence outcomes in this online curriculum. Paying for CE credit was associated with improved outcomes that were not mediated by spending more time on the curriculum. Incremental curriculum delivery increased resource use and merits further study.

Impact of medical residency on knowledge of asthma

OBJECTIVES

To assess resident physician knowledge of pathophysiology, diagnosis, and management of asthma and to assess the impact of an interactive curriculum on that knowledge.

PARTICIPANTS

A total of 720 resident and attending physicians at 15 internal medicine residency programs.

METHODS

An educational module and two multiple choice tests were developed using established methods of curriculum development and knowledge assessment, then disseminated online to 15 internal medicine residency programs. Baseline and post-intervention knowledge was analyzed according to year of training using Chi square to detect differences in group performance.

RESULTS

Baseline knowledge on asthma was poor. The average baseline score on all questions was 54.2%, and was worst on questions on diagnosis of asthma (47.5% correct) and questions on management of asthma (54.8% correct). Baseline knowledge was best on questions on the pathophysiology of asthma (71.5% correct). On specific concepts, only 41.9% correctly knew which pharmacotherapeutic agents were used as controller agents, and only 43.5% were able to correctly diagnose asthma severity. Knowledge on questions on diagnosis of asthma was no better in post-graduate year (PGY) 3 residents than in PGY1 residents (p = 0.054), but PGY3 residents performed better on questions about management of asthma than did PGY1 residents (p < 0.001). Knowledge improved for all concepts and at all levels of training after completion of an interactive module on asthma guidelines (p < 0.001).

CONCLUSION

Resident physician knowledge of asthma guidelines is poor and can be improved by an interactive curriculum.

A web-based simulation of a longitudinal clinic used in a 4-week ambulatory rotation: a cohort study

BACKGROUND

Residency training takes place primarily on inpatient wards. In the absence of a resident continuity clinic, internal medicine residents rely on block rotations to learn about continuity of care. Alternate methods to introduce continuity of care are needed.

METHODS

A web-based tool, Continuity of Care Online Simulations (COCOS), was designed for use in a one-month, postgraduate clinical rotation in endocrinology. It is an interactive tool that simulates the continuing care of any patient with a chronic endocrine disease. Twenty-three residents in internal medicine participated in a study to investigate the effects of using COCOS during a clinical rotation in endocrinology on pre-post knowledge test scores and self-assessment of confidence.

RESULTS

Compared to residents who did the rotation alone, residents who used COCOS during the rotation had significantly higher improvements in test scores (% increase in pre-post test scores +21.6 [standard deviation, SD, 8.0] vs. +5.9 [SD 6.8]; p < .001). Test score improvements were most pronounced for less commonly seen conditions. There were no significant differences in changes in confidence. Residents rated COCOS very highly, recommending its use as a standard part of the rotation and throughout residency.

CONCLUSION

A stand-alone web-based tool can be incorporated into an existing clinical rotation to help residents learn about continuity of care. It has the most potential to teach residents about topics that are less commonly seen during a clinical rotation. The adaptable, web-based format allows the creation of cases for most chronic medical conditions.

Evaluation of a web-based palliative care pain management module for housestaff

The objectives of this study were to determine internal medicine residents' knowledge of outpatient palliative care pain management, describe the association of level of training with knowledge, and evaluate the impact on knowledge of a web-based, interactive, evidence-based educational module. We developed the module using established educational principles, based on review of other educational materials, guidelines, and the medical literature. The module included pretest and post-test questions, case studies, didactic sections, and web links. Six hundred twelve housestaff in 35 training programs in 19 states completed the module during the 2005-2006 academic year (196 [32.0%] postgraduate year [PGY]-1, 200 [32.7%] PGY-2, and 216 [35.3%] PGY-3). The mean pretest score was 54.4% (range 31.1%-84.6%); scores were lowest for specific pain management knowledge questions, including appropriate titration of breakthrough opioid doses (mean 31.1% correct) and appropriate initial use of opioids (40.7% correct). Pretest scores were not significantly different by level of training (52.2% for PGY-1 and 56.7% for PGY-3). The mean post-test score was 72.8%, a statistically significant increase from the pretest overall (P<0.001) and for seven of the 10 learning objectives (P<0.001). These findings indicate that housestaff lacked knowledge in many areas of palliative care pain management, and knowledge did not increase with time spent in residency. The large increase in test scores after the module suggests that this may be an effective component of a comprehensive palliative care curriculum.

Medical residents' knowledge of dietary supplements

INTRODUCTION

The widespread use of complementary and alternative medicine modalities such as dietary supplements has prompted many medical schools to offer courses covering such topics. To date, little is known about the impact of these courses on medical graduates' knowledge. This study was designed to evaluate resident physicians' level of understanding of popular dietary supplement regulation and to determine whether an interactive online curriculum could aid in improving such knowledge.

METHODS

A multicenter online educational intervention was developed and administered to physicians at 15 internal medicine residency programs throughout the United States, between March 1, 2006 and June 30, 2006. Pretest performance was used to measure baseline knowledge of commonly used dietary supplements. Posttest performance compared with pretest performance measured the effectiveness of the educational intervention.

RESULTS

A total of 335 physicians completed the module. Baseline knowledge of dietary supplements was low (average pretest score 59.7%). More than one-third of respondents were unaware of the reasons for use of saw palmetto and black cohosh. Results for questions on safety and drug-supplement interactions were similarly low. Only 57% of physicians knew that kava kava has been associated with hepatitis. Only 15% were aware that St. John's Wort can lower cyclosporine levels. With regards to knowledge of efficacy, only 36% were aware that fish oil has been shown to lower triglyceride levels. After completion of the curriculum, scores improved significantly (P < 0.001) in all question/content areas.

CONCLUSIONS

Residents' knowledge of dietary supplements is poor. An online didactic module may improve knowledge and potentially enhance patient-physician communication regarding the use of such products.

Knowledge retention after an online tutorial: a randomized educational experiment among resident physicians

BACKGROUND

The time course of physicians' knowledge retention after learning activities has not been well characterized. Understanding the time course of retention is critical to optimizing the reinforcement of knowledge.

DESIGN

Educational follow-up experiment with knowledge retention measured at 1 of 6 randomly assigned time intervals (0-55 days) after an online tutorial covering 2 American Diabetes Association guidelines.

PARTICIPANTS

Internal and family medicine residents.

MEASUREMENTS

Multiple-choice knowledge tests, subject characteristics including critical appraisal skills, and learner satisfaction.

RESULTS

Of 197 residents invited, 91 (46%) completed the tutorial and were randomized; of these, 87 (96%) provided complete follow-up data. Ninety-two percent of the subjects rated the tutorial as "very good" or "excellent." Mean knowledge scores increased from 50% before the tutorial to 76% among those tested immediately afterward. Score gains were only half as great at 3-8 days and no significant retention was measurable at 55 days. The shape of the retention curve corresponded with a 1/4-power transformation of the delay interval. In multivariate analyses, critical appraisal skills and participant age were associated with greater initial learning, but no participant characteristic significantly modified the rate of decline in retention.

CONCLUSIONS

Education that appears successful from immediate posttests and learner evaluations can result in knowledge that is mostly lost to recall over the ensuing days and weeks. To achieve longer-term retention, physicians should review or otherwise reinforce new learning after as little as 1 week.

Continuity clinic satisfaction and valuation in residency training

BACKGROUND

Internal Medicine residency training in ambulatory care has been judged inadequate, yet how trainees value continuity clinic and which aspects of clinic affect attitudes are unknown.

OBJECTIVES

To determine the value that Internal Medicine residents place on continuity clinic and how clinic precepting, operations, and patient panels affect its valuation.

DESIGN AND MEASUREMENTS

A survey on ambulatory care was developed, including questions on career choice and the value of clinical training experiences. Independent variables were Likert-scale ratings (1 = disagree strongly/no value; 3 = neutral; 5 = agree strongly/high value) on preceptors, patients, operations, and resident characteristics. Odds ratios and stepwise multivariate logistic regression with clustering were used to evaluate associations between clinic valuation and independent variables.

SUBJECTS

Internal medicine residents at 3 residency programs.

RESULTS

218 of 260 residents (83.8%) completed the survey. Resident ratings were highest on diversity of illness seen (4.1), medical record systems used (4.1), and contact with preceptors who were receptive to questions (4.8). Resident ratings were lowest on economic diversity of patients (2.7), interruptions from inpatient wards (3.1), and contact with preceptors who taught history and physical exam skills (3.5). High ratings on all precepting issues and nearly all operational issues were associated with valuing clinic. With multivariate analysis, high ratings of preceptors as role models were most strongly associated with valuing clinic (corrected relative risk 3.44). A planned career in general Internal Medicine was not associated with valuing clinic.

CONCLUSIONS

Satisfaction with preceptors, particularly as role models, and clinic operations correlate with the value residents place on continuity clinic.

Video-based training increases sterile-technique compliance during central venous catheter insertion

OBJECTIVE

To evaluate the effect of an online training course containing video clips of central venous catheter insertions on compliance with sterile practice.

DESIGN

Prospective randomized controlled study.

SETTING

Admitting area of a university-based high-volume trauma center.

SUBJECTS

Surgical and emergency medicine residents rotating through the trauma services.

INTERVENTIONS

An online training course on recommended sterile practices during central venous catheter insertion was developed. The course contained short video clips from actual patient care demonstrating common noncompliant behaviors and breaks regarding recommended sterile practices. A 4-month study with a counterbalanced design compared residents trained by the video-based online training course (video group) with those trained with a paper version of the course (paper group). Residents who inserted central venous catheters but received neither the paper nor video training were used as a control group. Consecutive central venous catheter insertions from 12 noon to 12 midnight except Sundays were video recorded.

MEASUREMENTS AND MAIN RESULTS

Sterile-practice compliance was judged through video review by two surgeons blinded to the training status of the residents. Fifty residents inserted 73 elective central venous catheters (19, 31, and 23 by the video, paper, and control group operators, respectively) into 68 patients. Overall compliance with proper operator preparation, skin preparation, and draping was 49% (36 of 73 procedures). The training had no effect on selection of site and skin preparation agent. The video group was significantly more likely than the other two groups to fully comply with sterile practices (74% vs. 33%; odds ratio, 6.1; 95% confidence interval, 2.0-22.0). Even after we controlled for the number of years in residency training, specialty, number of central venous catheters inserted, and central venous catheter site chosen, the video group was more likely to comply with recommended sterile practices (p = .003).

CONCLUSIONS

An online training course, with short video clips of actual patient care demonstrating noncompliant behaviors, improved sterile-practice compliance for central venous catheter insertion. Paper handouts with equivalent content did not improve compliance.

Physician knowledge of national cholesterol guidelines before and after an interactive curriculum

An interactive curriculum on National Cholesterol Education Program (NCEP) III guidelines was developed and distributed online to 877 physicians at 37 internal medicine residency training programs. Baseline knowledge of NCEP guidelines was tested before the curriculum. A test after the curriculum was compared with baseline performance to determine the impact of the curriculum. Performance was compared based on year of training or attending status. Scores for baseline knowledge of NCEP III concepts were 21.8% for residents and 23.2% for attending physicians, a difference that was not statistically significant. Postgraduate-year (PGY)-3 residents performed better than PGY-1 residents (26.5% vs 17.8%, p <0.05). Scores for specific concepts were 29.2% for pharmacology, 14.2% for major cardiovascular risk factors, 32.9% for coronary heart disease risk-equivalent conditions, 15.2% for diagnosis of metabolic syndrome, and 17.3% for treatment of patients with metabolic syndrome. For hypothetical clinical cases, attending physicians performed better than residents (63.2% vs 42.5%, p <0.05) and PGY-3 residents performed better than PGY-1 residents (47.7% vs 36.5%, p <0.05). After the curriculum, knowledge increased for all concepts (22.5% before vs 61.3% after the test, p <0.05). In conclusion, resident and attending physician knowledge of NCEP III guidelines is poor and can be improved by an interactive curriculum delivered online.

Physician familiarity with diagnosis and management of hypertension according to JNC 7 guidelines

Physician knowledge of the Seventh Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure (JNC 7) guidelines is unknown and may contribute to the prevalence of uncontrolled hypertension. Our objective was to determine physician knowledge of JNC 7 guidelines and whether online instruction could improve knowledge. A pretest served as baseline knowledge, and comparison with a post-test after completing an online didactic demonstrated improvement in knowledge. Participants included 1280 physicians at 45 internal medicine residency training programs. Average baseline knowledge of six concepts of hypertension was 51.2%. Attending physicians performed better than trainees on some but not all concepts (p<0.05). Third-year post-graduate trainees performed better than first-year trainees on some but not all concepts (p<0.05). Knowledge increased significantly on all concepts after completing the curriculum (p<0.05). The authors demonstrated that physician knowledge of JNC 7 guidelines is poor but can be improved by an online curriculum. Further study is needed to determine the impact of physician education on clinical outcomes in individuals with hypertension.

Student perceptions of a virtual learning environment for a problem-based learning undergraduate medical curriculum

OBJECTIVE

To investigate the effects of a virtual learning environment (VLE) on group interaction and consultation of information resources during the preliminary phase, self-study phase and reporting phase of the problem-based learning process in an undergraduate medical curriculum.

METHODS

A questionnaire was administered to 355 medical students in Years 1 and 2 to ask them about the perceived usefulness of a virtual learning environment that was created with Blackboard for group interaction and the use of learning resources.

RESULTS

The students indicated that the VLE supported face-to-face interaction in the preliminary discussion and in the reporting phase but did not stimulate computer-mediated distance interaction during the self-study phase. They perceived that the use of multimedia in case presentations led to a better quality of group discussion than if case presentations were exclusively text-based. They also indicated that the information resources that were hyperlinked in the VLE stimulated the consultation of these resources during self-study, but not during the reporting phase.

CONCLUSIONS

Students indicated that the use of a VLE in the tutorial room and the inclusion of multimedia in case presentations supported processes of active learning in the tutorial groups. However, if we want to exploit the full potential of asynchronous computer-mediated communication to initiate in-depth discussion during the self-study phase, its application will have to be selective and deliberate. Students indicated that the links in the VLE to selected information in library repositories supported their learning.

Impact of self-assessment questions and learning styles in Web-based learning: a randomized, controlled, crossover trial

PURPOSE

To determine the effect of self-assessment questions on learners' knowledge and format preference in a Web-based course, and investigate associations between learning styles and outcomes.

METHOD

The authors conducted a randomized, controlled, crossover trial in the continuity clinics of the Mayo-Rochester internal medicine residency program during the 2003-04 academic year. Case-based self-assessment questions were added to Web-based modules covering topics in ambulatory internal medicine. Participants completed two modules with questions and two modules without questions, with sequence randomly assigned. Outcomes included knowledge assessed after each module, format preference, and learning style assessed using the Index of Learning Styles.

RESULTS

A total of 121 of 146 residents (83%) consented. Residents had higher test scores when using the question format (mean +/- standard error, 78.9% +/- 1.0) than when using the standard format (76.2% +/- 1.0, p = .006). Residents preferring the question format scored higher (79.7% +/- 1.1) than those preferring standard (69.5% +/- 2.3, p < .001). Learning styles did not affect scores except that visual-verbal "intermediate" learners (80.6% +/- 1.4) and visual learners (77.5% +/- 1.3) did better than verbal learners (70.9% +/- 3.0, p = .003 and p = .033, respectively). Sixty-five of 78 residents (83.3%, 95% CI 73.2-90.8%) preferred the question format. Learning styles were not associated with preference (p > .384). Although the question format took longer than the standard format (60.4 +/- 3.6 versus 44.3 +/- 3.3 minutes, p < .001), 55 of 77 residents (71.4%, 60.0-81.2%) reported that it was more efficient.

CONCLUSIONS

Instructional methods that actively engage learners improve learning outcomes. These findings hold implications for both Web-based learning and "traditional" educational activities. Future research, in both Web-based learning and other teaching modalities, should focus on further defining the effectiveness of selected instructional methods in specific learning contexts.

A web-based smoking cessation and prevention curriculum for medical students: why, how, what, and what next

This paper summarises some major developments in medical education relating to the health risks of tobacco and to training in tobacco cessation and prevention strategies, and discusses some of the barriers to training. We also describe a project whose purpose was to design, implement and evaluate a web-based self-study tobacco curriculum for medical students to teach medical students to assist smokers to quit and to counsel non-smoking adolescents not to start smoking. This curriculum addresses some of the barriers, namely lack of curriculum time, lack of access to materials and experts, and relevance of the materials. The project was designed and evaluated at two medical schools in Georgia: Morehouse School of Medicine and Mercer University School of Medicine. A curriculum on tobacco control strategies and techniques designed for use in clinical settings was made available to first-year medical students as interactive computer-based tutorials. The curriculum, based on the US Public Health Service Clinical Guideline for Treating Tobacco Use and Dependence, was divided into two parts: the tutorial and practicum sections. Pre- and post-exposure measures were collected for the evaluation. The mean differences for correctly answered knowledge items in the two schools were similar: mean for Morehouse was +2.07 and for Mercer +1.67, indicating improvements in knowledge for both schools. There were statistically significant improvements in all categories of self-rated ability to perform six counselling skills, except for Mercer students for the 'Ask' skill category (p=0.069). The amount of exposure (measured only at Morehouse) was not related to overall change in scores but was associated with self-reported improvement in skill in assisting patients to quit smoking, confidence in counselling patients not interested in quitting, and confidence in counselling teens. The web-based curriculum successfully improved the students' self-rated counselling skills. Given the need and desire on the part of practitioners for training, the curriculum may be found useful by practising physicians and other health-care professionals who wish to improve their skills in smoking prevention and cessation.

The research we still are not doing: an agenda for the study of computer-based learning

Media-comparative research-that is, the comparison of computer-based learning (CBL) to noncomputer instruction-is logically impossible because there are no valid comparison groups. Results from media-comparative studies are thus confounded and difficult to meaningfully interpret. In 1994, Friedman proposed that such research be supplanted by investigations into CBL designs, usage patterns, assessment methods, and integration. His proposal appears to have largely been ignored. In this article, the author updates the agenda for research in CBL (including Web-based learning). While media-comparative studies are confounded, CBL-CBL comparisons are often not. CBL instructional designs vary in configuration (e.g., discussion board or tutorial), instructional method (e.g., case-based learning, personalized feedback, or simulation), and presentation (e.g., screen layout, hyperlinks, or multimedia). Comparisons within one level (for example, comparing two instructional methods) facilitate evidence-based improvements, but comparisons between levels are confounded. Additional research questions within the CBL-CBL framework might include: Does adaptation of CBL in response to individual differences such as prior knowledge, computer experience, or learning style improve learning outcomes? Will integrating CBL with everyday clinical practice facilitate learning? How can simulations augment clinical training? And, how can CBL be integrated within and between institutions? In addressing these questions it is important to remember the most important outcome-effect on patients and practice-and outcomes specific to CBL including costs, cognitive structuring, and learning unique to the computer-based environment. CBL is not a panacea, but holds great promise. Realization of this potential requires that media-comparative studies be replaced by rigorous, theory-guided comparisons of CBL interventions.

Educational research at Johns Hopkins University School of Medicine: a grassroots development

The Divisions of General Internal Medicine at Johns Hopkins University School of Medicine have been a rich source of educational research. To better understand the facilitators and barriers to educational research within the divisions, the authors reviewed published educational research from the divisions published between 1995-2004 and examined the history, leadership decisions, and sentinel events that have allowed educational scholarship within the divisions to grow. The authors' analysis suggests a grassroots model of programmatic growth that includes a growing cadre of clinician-educator scholars, effective mentorship, a faculty development program, access to learners, access to research expertise, protected time for scholarship, some funding, and an institutional culture that stimulates scholarship. A medical education fellowship was integral to the model; fellows were first authors for 47% of reviewed manuscripts. Extramural funding has helped build an infrastructure that supports educational scholarship; however, only 12% of the publications have had extramural funding. Protected time for faculty is the characteristic of this model most at risk. While there has been a move toward more institutional support of educational research, it is clear that further growth in the educational research program will require noninstitutional resources.