The effect of a clinical teaching retreat on residents' teaching skills

Residents as teachers: Near peer learning in clinical work settings: AMEE Guide No. 106

This AMEE Guide provides a framework to guide medical educators engaged in the design and implementation of "Resident as Teacher" programs. The suggested approaches are based on established models of program development: the Program Logic model to guide program design, the Dundee three-circle model to inform a systematic approach to planning educational content and the Kirkpatrick pyramid, which forms the backbone of program evaluation. The Guide provides an overview of Resident as Teacher curricula, their benefits and impact, from existing literature supplemented by insights from the authors' own experiences, all of whom are engaged in teaching initiatives at their own institutions. A conceptual description of the Program Logic model is provided, a model that highlights an outcomes-based curricular design. Examples of activities under each step of this model are described, which would allow educational leaders to structure their own program based on the scope, context, institutional needs and resources available. Emphasis is placed on a modular curricular format to not only enhance the teaching skills of residents, but also enable development of future career educators, scholars and leaders. Application of the Dundee three-circle model is illustrated to allow for a flexible curricular design that can cater to varying levels of educational needs and interests. In addition, practical advice is provided on robust assessment of outcomes, both assessment of participants and program evaluation. Finally, the authors highlight the need for congruence between the formal and hidden curriculum through explicit recognition of the value of teaching by institutions, support for development of teaching programs, encouragement of evidence-based approach to education and rewards for all levels of teachers.

Residents-as-Teachers Publications: What Can Programs Learn From the Literature When Starting a New or Refining an Established Curriculum?

BACKGROUND

Teaching residents how to teach is a critical part of resident education because residents are often the major teachers of medical students. The importance of formal residents-as-teachers (RAT) curricula has been emphasized throughout the literature, yet not all residency programs have such a curriculum in place.

OBJECTIVE

The purpose of our study was to (1) review the medical education literature for established RAT curricula, (2) assess published curricula's reproducibility, (3) evaluate the type of outcomes achieved using the Kirkpatrick model of evaluation, and (4) identify curricula that training programs could feasibly adopt.

METHODS

We performed a literature review using PubMed, Medline, Scopus, PsycINFO, ERIC, and Embase. Key search words included residents, residents as teachers, teaching, internship and residency, and curriculum. In addition, a search of MedEdPORTAL was performed using the same key terms. Articles were evaluated based on the reproducibility of curricula and the assessment tools. Evaluation of educational outcomes was performed using the Kirkpatrick model.

RESULTS

Thirty-nine articles were deemed appropriate for review. Interventions and evaluation techniques varied greatly. Only 1 article from the literature was deemed to have both curricula and assessments that would be fully reproducible by other programs.

CONCLUSIONS

A literature review on RAT curricula found few articles that would be easily reproduced for residency programs that want to start or improve their own RAT curricula. It also demonstrated the difficulty and lack of rigorous outcome measurements for most curricula.

Evidence for the Validity of Grouped Self-Assessments in Measuring the Outcomes of Educational Programs

There is compelling empirical evidence in support of the use of grouped self-assessment data to measure program outcomes. However, other credible research has clearly shown that self-assessments are poor predictors of individual achievement such that the validity of self-assessments has been called into question. Based on the reanalysis of two previously published studies and an analysis of two original studies, we show that grouped self-assessments may be good predictors of and hence valid measures of performance at the group level, an outcome commonly used in program evaluation studies. We found statistically significant correlation coefficients (between 0.56 and 0.87), when comparing across performance items using the group means of self-assessments with the group means of individual achievement on criterion tests. We call for further research into the conditions and circumstances in which grouped self-assessments are used, so that they can be employed more effectively and confidently by program evaluators, decision makers, and researchers.

Impact of a Resident-as-Teacher Workshop on Teaching Behavior of Interns and Learning Outcomes of Medical Students

BACKGROUND

Residents and interns are recognized as important clinical teachers and mentors. Resident-as-teacher training programs are known to improve resident attitudes and perceptions toward teaching, as well as their theoretical knowledge, skills, and teaching behavior. The effect of resident-as-teacher programs on learning outcomes of medical students, however, remains unknown. An intervention cohort study was conducted to prospectively investigate the effects of a teacher-training workshop on teaching behavior of participating interns and on the clerkship learning outcomes of instructed fourth-year medical students.

METHODS

The House Officer-as-Teacher Training Workshop was implemented in November 2009 over 1.5 days and attended by all 34 interns from one teaching hospital. Subsequently, between February and August 2010, 124 fourth-year medical students rated the observable teaching behavior of interns during 6-week general surgery clerkships at this intervention hospital as well as at 2 comparable hospitals serving as control sites. Ratings were collected using an anonymous 15-item Intern Clinical Teaching Effectiveness Instrument. Student achievement of clerkship learning outcomes during this period was evaluated using a validated and centralized objective structured clinical examination.

RESULTS

Medical students completed 101 intern clinical teaching effectiveness instruments. Intern teaching behavior at the intervention hospital was found to be significantly more positive, compared with observed behavior at the control hospitals. Objective structured clinical examination results, however, did not demonstrate any significant intersite differences in student achievement of general surgery clerkship learning outcomes.

CONCLUSIONS

The House Officer-as-Teacher Training Workshop noticeably improved teaching behavior of surgical interns during general surgery clerkships. This improvement did not, however, translate into improved achievement of clerkship learning outcomes by medical students during the study period.

A novel resident-as-teacher training program to improve and evaluate obstetrics and gynecology resident teaching skills

BACKGROUND

Residents play a significant role in teaching, but formal training, feedback, and evaluation are needed.

AIMS

Our aims were to assess resident teaching skills in the resident-as-teacher program, quantify correlations of faculty evaluations with resident self-evaluations, compare resident-as-teacher evaluations with clinical evaluations, and evaluate the resident-as-teacher program.

METHOD

The resident-as-teacher training program is a simulated, videotaped teaching encounter with a trained medical student and standardized teaching evaluation tool. Evaluations from the resident-as-teacher training program were compared to evaluations of resident teaching done by faculty, residents, and medical students from the clinical setting.

RESULTS

Faculty evaluation of resident teaching skills in the resident-as-teacher program showed a mean total score of 4.5 ± 0.5 with statistically significant correlations between faculty assessment and resident self-evaluations (r = 0.47; p < 0.001). However, resident self-evaluation of teaching skill was lower than faculty evaluation (mean difference: 0.4; 95% CI 0.3-0.6). When compared to the clinical setting, resident-as-teacher evaluations were significantly correlated with faculty and resident evaluations, but not medical student evaluations. Evaluations from both the resident-as-teacher program and the clinical setting improved with duration of residency.

CONCLUSIONS

The resident-as-teacher program provides a method to train, give feedback, and evaluate resident teaching.

A Pilot Study of a "Resident Educator And Life-long Learner" Program: Using a Faculty Train-the-Trainer Program

PURPOSE

We sought to create a resident educator program using a Train-the-Trainer (TTT) approach with adaptable curricula at a large tertiary health care center with a medical school and 60 accredited residency programs.

METHODS

The Resident Educator And Life-long Learner (REALL) Program was designed as a 3-phase model. Phase 1 included centralized planning and development that led to the design of 7 teaching modules and evaluation tools for TTT and resident sessions. Phase 2 entailed the dissemination of the TTT modules (Learning Styles, Observational Skills, Giving Feedback, Communication Skills: The Angry Patient, Case-Based Teaching, Clinical Reasoning, Effective Presentations) to faculty trainers. In phase 3, specific modules were chosen and customized by the faculty trainers, and implemented for their residents. Evaluations from residents and faculty were collected throughout this process.

RESULTS

A total of 45 faculty trainers representing 27 residency programs participated in the TTT program, and 97% of trainers were confident in their ability to implement sessions for their residents. A total of 20 trainers from 11 residency programs implemented 33 modules to train 479 residents, and 97% of residents believed they would be able to apply the skills learned. Residents' comments revealed appreciation of discussion of their roles as teachers.

CONCLUSION

Use of an internal TTT program can be a strategy for dissemination of resident educator and life-long learner curricula in a large academic tertiary care center. The TTT model may be useful to other large academic centers.

A systematic review of resident-as-teacher programmes

CONTEXT

Residents in all disciplines serve as clinical teachers for medical students. Since the 1970s, there has been increasing evidence to demonstrate that residents wish to teach and that they respond positively to formal teacher training. Effective resident-as-teacher (RaT) programmes have resulted in improved resident teaching skills. Current evidence, however, is not clear about the specific features of an effective RaT programme.

OBJECTIVES

This study was performed in order to investigate the effectiveness of RaT programmes on resident teaching abilities and to identify the features that ensure success. Methods of assessment used to ascertain the effectiveness of RaT programmes are also explored.

METHODS

The literature search covered the period between 1971 and 2008. Articles focusing on improving resident teaching skills were included. Each study was reviewed by two reviewers and data were collected using a standard abstraction summary sheet. Study outcomes were graded according to a modified Kirkpatrick's model of educational outcomes.

RESULTS

Twenty-nine studies met review inclusion criteria. Interventions included workshops, seminars, lectures and teaching retreats. Twenty-six studies used a pre- and post-intervention outcome comparison method. Subjective outcome measures included resident self-evaluation of teaching skills or evaluation by medical students, peers and faculty members. Objective outcome measures included written tests, evaluation of teaching performance by independent raters and utilisation of objective structured teaching examinations. One study objectively measured learning outcomes at the level of medical students, utilising the results of an objective structured clinical examination. Overall resident satisfaction with RaT programmes was high. Participants reported positive changes in attitudes towards teaching. Participant knowledge of educational principles improved. Study methodologies allowed for significant risks of bias.

CONCLUSIONS

More rigorous study designs and the use of objective outcome measures are needed to ascertain the true effectiveness of RaT programmes. Future research should focus on determining the impact of RaT programmes on learning achievement at the level of medical students.

Residents-as-teachers curricula: a critical review

PURPOSE

Residents serve as medical students' primary teachers for practical clinical skills. The purpose of this study is to provide an updated systematic review of the literature on residents-as-teachers curricula to determine the most evidence-based curricula and evaluation strategy.

METHOD

In 2008, the authors performed a systematic review of the literature with PubMed using the MESH terms "internship and residency" and "teaching," as well as a key word search of the term "residents as teachers." The search was limited to publications in English from 1975 to 2008.

RESULTS

A total of 24 studies met inclusion criteria. Eleven (45.8%) were uncontrolled studies, seven (29.2%) were randomized control trials, and six (25%) were nonrandomized controlled trials. The mean sample size of all studies was 39.6. Evaluation was performed by a variety of means including objective structured teaching exams (5; 20.8%), videotape evaluations (6; 25.0%), learner evaluations (11; 45.8%), and self-questionnaires (7; 29.2%). The mean intervention length was 7.6 hours, and the most common intervention was based on the One-Minute Preceptor.

CONCLUSIONS

Research on residents-as-teachers curricula is limited by both the number of studies and their methodology. Despite this, the results demonstrated that residents-as-teachers curricula can significantly improve residents' teaching skills. In addition, the studies' methodologies have improved over time. Using these data, the authors recommend an evidence-based intervention and evaluation, which would include a three-hours-or-longer intervention (and, if possible, periodic reinforcement) based on the One-Minute Preceptor. The evaluation should be a randomized controlled trial using an objective structured teaching examination.

Training Tomorrow's Teachers Today: a national medical student teaching and leadership retreat

INTRODUCTION

Challenges to American academic medical centers have diverted attention and resources away from the core missions of teacher and leader development. To train the next generation of medical education leaders, substantial and institutionalized new programs are necessary.

METHODS

The American Medical Student Association, with the Mount Sinai School of Medicine, designed the Training Tomorrow's Teachers Today national medical student retreat. The week-long course endeavors to enhance participants' clinical teaching and academic leadership skills. The program also trains students to execute a required medical education project. At the end of the course, attendees evaluated their baseline and post-course teaching and leadership effectiveness. Long-term follow-up was also completed.

RESULTS

Attendees (n = 23) self-reported statistically significant (p < 0.05) improvement in teaching and leadership. The greatest improvements in teaching scores were in the area of communicating goals. The largest leadership score improvements were: defining goals; defining constituents, allies, and opponents; and delineating organization strengths/weaknesses. 13 participants engaged in medical education projects.

CONCLUSION

This program helps fill a gap in medical education training. Program data suggest enhancement of students' teaching and leadership skills. A competitive application process demonstrates interest in such training. These results suggest that medical schools should offer and/or fund undergraduate teaching and leadership training.

A Description of a Validated Effective Teacher-Training Workshop for Medical Residents

A teacher-training program for residents was designed and piloted in the St. Elisabeth Hospital in Curaçao, Netherlands Antilles. The program comprised of six modules namely: effective teaching, self-knowledge and teaching ability, feedback, assessing prior knowledge, trouble shooting and time management.

METHOD

Instruction was provided during a two-day workshop with eight hours instruction time per day. Residents in the first three years of training participated, and the instructors were experienced clinicians. Lectures, group discussions, case simulations, video presentations and role-plays were the forms of instruction.

RESULTS

Using standardized questionnaires, the participants rated the quality of the workshop highly. They considered it to be a feasible and appropriate educational intervention and that it had a positive impact on their teaching skills.

CONCLUSION

This workshop was developed based on careful analysis of medical residents' perceived educational needs and systematically implemented and evaluated. The results show that it is a suitable and effective educational intervention.

Quality of clinical supervision as perceived by specialist registrars in a university and district teaching hospital

BACKGROUND

Clinical supervision promotes the professional development of specialist registrars (SpRs). Our objective was to investigate and compare the perceived quality of supervision (PQS) in a university teaching hospital (UTH) and a district teaching hospital (DTH) and to identify aspects of supervision that could be improved.

METHODS

The Cleveland Clinic's teaching effectiveness instrument (CCTEI) was used to measure the quality of supervision of attending doctors. Fifteen items reflecting good teaching ability were rated on a 5-point Likert-scale (1-5 = poor-excellent).

RESULTS

SpRs rated 47 attending doctors using the CCTEI. A total of 416 ratings were obtained. Overall, the mean PQS was 3.85 (SD = 0.29) in the DTH and 3.56 (SD = 0.44) in the UTH (P = 0.02). A significant difference in PQS was found in 6 items. The supervisors in the DTH scored better on all these items. The best predictor of PQS was the item 'Organises time to allow for both teaching and care giving'.

CONCLUSIONS

Overall, PQS was better in the DTH compared to the UTH. In both settings, generating a good learning environment and respecting the autonomy of the SpRs scored favourably. Supervisory roles focusing on improving cost-effective practice and communicative skills need more emphasis.

Preparing fourth-year medical students to teach during internship

Interns are expected to teach medical students, yet there is little formal training in medical school to prepare them for this role. To enhance the teaching skills of our graduating students we initiated a 4-hour "teaching to teach" course as part of the end of the fourth-year curriculum. Course evaluations demonstrate that students strongly support this program (overall ratings 2000 to 2005: mean=4.4 [scale 1 to 5], n=224). When 2004 course participants were surveyed during the last month of their internship, 84%"agree" or "strongly agree" with the statement: "The teaching to teach course helped prepare me for my role as a teacher during internship" (2005: mean 4.2 [scale 1 to 5], n=45, response rate 60%). A course preparing fourth-year students to teach during internship is both feasible and reproducible, with a minimal commitment of faculty and resident time. Participants identify it as an important addition to their education and as useful during internship.

Fellows as Teachers

STUDY OBJECTIVES

Physicians undergoing medical subspecialty training, referred to as "fellows" in the United States, often serve as small group tutors. There are few reports, however, on whether fellows benefit from their experience teaching in the classroom. The purpose of this article is to describe the teacher-assistant requirement for fellows in a pulmonary and critical care training program and gain the perspectives of program graduates on their experiences as teacher assistants.

DESIGN, SETTING, AND PARTICIPANTS

The structure of the teacher-assistant requirement is described in detail. We also surveyed 55 consecutive graduates of our Pulmonary and Critical Care Fellowship program located in the United States on the educational value of teaching pathophysiology to preclinical medical students.

RESULTS

Fellows and one or two faculty members co-teach groups of 15 to 25 second-year medical students during 22 h of case-based conferences over a 6-week period. Weekly meetings for teacher assistants foster improved teaching skills and provide insight into the basic tenets of structuring and implementing the course. Fellows receive formal and informal feedback on their performance. Fifty of 55 graduates (91%) returned the survey. Respondents rated the acquisition of teaching skills and knowledge of pulmonary pathophysiology as the most useful aspects of their experience. Eighty-seven percent rated their overall experience favorably, and 90% agreed that teaching the course should remain a curriculum requirement. The responses of physicians in academic and community practice were comparable.

CONCLUSIONS

Graduates of our Pulmonary and Critical Care Fellowship valued their teacher-assistant experience. This article may serve as a model for other subspecialty programs that are interested in formally incorporating teaching into their training program and may provide an impetus for the additional exploration of teacher-assistant experiences at other institutions.

Outcomes of a national faculty development program in teaching skills: prospective follow-up of 110 medicine faculty development teams

BACKGROUND

Awareness of the need for ambulatory care teaching skills training for clinician-educators is increasing. A recent Health Resources and Services Administration (HRSA)-funded national initiative trained 110 teams from U.S. teaching hospitals to implement local faculty development (FD) in teaching skills.

OBJECTIVE

To assess the rate of successful implementation of local FD initiatives by these teams.

METHODS

A prospective observational study followed the 110 teams for up to 24 months. Self-reported implementation, our outcome, was defined as the time from the training conference until the team reported that implementation of their FD project was completely accomplished. Factors associated with success were assessed using Kaplan-Meier analysis.

RESULTS

The median follow-up was 18 months. Fifty-nine of the teams (54%) implemented their local FD project and subsequently trained over 1,400 faculty, of whom over 500 were community based. Teams that implemented their FD projects were more likely than those that did not to have the following attributes: met more frequently (P=.001), had less turnover (P=.01), had protected time (P=.01), rated their likelihood of success high (P=.03), had some project or institutional funding for FD (P=.03), and came from institutions with more than 75 department of medicine faculty (P=.03). The cost to the HRSA was $22,033 per successful team and $533 per faculty member trained.

CONCLUSIONS

This national initiative was able to disseminate teaching skills training to large numbers of faculty at modest cost. Smaller teaching hospitals may have limited success without additional support or targeted funding.

A literature review of "resident-as-teacher" curricula: do teaching courses make a difference?

OBJECTIVES

To examine the evaluation methods of resident teaching courses and to estimate the effectiveness of these teaching courses.

DESIGN

We searched the literature from 1975 to May 2003 using the PubMed MESH terms internship and residency and teaching; 1,436 articles were identified and 77 contained information regarding teaching courses. Fourteen articles contained information regarding outcomes of resident teaching courses and were selected for intensive review.

MAIN RESULTS

Five uncontrolled pre-post studies used resident self-reported teaching skills/behaviors as outcome measures; all reported some improvement in self-reported skills. Three uncontrolled pre-post studies examined live or videotaped resident teaching encounters and all revealed improvement in some teaching skills. One uncontrolled trial and three nonrandomized controlled trials used learner evaluations of resident teaching behaviors as outcomes and all revealed an improvement in ratings of residents after course participation. Four randomized controlled trials of resident teaching curricula are included in this review. One study did not show any quantitative benefit of a resident teaching course on performance on an objective structured teaching evaluation. Two studies assessing resident teaching evaluations before and after course participation showed conflicting results. One study noted improvements in resident teaching skills assessed through videotape analysis.

CONCLUSIONS

Resident teaching courses improve resident self-assessed teaching behaviors and teaching confidence. Teaching courses are linked to improved student evaluations. Further studies must be completed to elucidate the best format, length, timing, and content of resident teaching courses and to determine whether they have an effect on learner performance.

Teaching the teachers: national survey of faculty development in departments of medicine of U.S. teaching hospitals

OBJECTIVE

To determine the prevalence, topics, methods, and intensity of ongoing faculty development (FD) in teaching skills.

DESIGN

Mailed survey.

PARTICIPANTS

Two hundred and seventy-seven of the 386 (72%) U.S. teaching hospitals with internal medicine residency programs.

MEASUREMENTS

Prevalence and characteristics of ongoing FD.

RESULTS

One hundred and eight teaching hospitals (39%) reported ongoing FD. Hospitals with a primary medical school affiliation (university hospitals) were more likely to have ongoing FD than non-university hospitals. For non-university hospitals, funding from the Health Resources Services Administration and >50 house staff were associated with ongoing FD. For university hospitals, >100 department of medicine faculty was associated. Ongoing programs included a mean of 10.4 topics (standard deviation, 5.4). Most offered half-day workshops (80%), but 22% offered > or =1-month programs. Evaluations were predominantly limited to postcourse evaluations forms. Only 14% of the hospitals with ongoing FD (5% of all hospitals) had "advanced" programs, defined as offering > or =10 topics, lasting >2 days, and using > or =3 experiential teaching methods. These were significantly more likely to be university hospitals and to offer salary support and/or protected time to their FD instructors. Generalists and hospital-based faculty were more likely to receive training than subspecialist and community-based faculty. Factors facilitating participation in FD activities were supervisor attitudes, FD expertise, and institutional culture.

CONCLUSIONS

A minority of U.S. teaching hospitals offer ongoing faculty development in teaching skills. Continued progress will likely require increased institutional commitment, improved evaluations, and adequate resources, particularly FD instructors and funding.

Implementation and evaluation of a training program to improve resident teaching skills

OBJECTIVE

This study was undertaken to develop, implement, and evaluate a program designed to improve residents' teaching skills.

STUDY DESIGN

A training program to improve the teaching skills of obstetrics and gynecology residents was developed and implemented. All residents completed a survey of their teaching skills, participated in the program, and then completed a follow-up survey. The surveys were compared using the Wilcoxon signed rank test. Medical students also completed surveys about participating residents' teaching skills either before or after the intervention.

RESULTS

The training program was well received by the residents. Resident self-rating scores significantly improved (P<.05) in a number of areas, such as ability to teach physical examination skills, and explaining topics in a clear and concise manner. Interestingly, student ratings of resident teaching skills did not improve.

CONCLUSION

Although the training program improved resident self-ratings of teaching skills, this was not reflected in student ratings of resident teaching skills. Possible explanations for this difference will be explored.

The pediatric intern retreat: 20-year evolution of a continuing investment

For the past 22 years the interns in pediatrics at the University of Washington and Children's Hospital and Regional Medical Center have been relieved of all clinical duties in order to participate in a five-day retreat. The retreat provides an opportunity for the interns to learn more about their classmates, build stronger bonds, and provide mutual support. This retreat has been supported by the hospital, the department of pediatrics, faculty, fellows, and community physicians. The authors describe the history of the Intern Retreat, present its goals, daily activities, and faculty, and discuss how the retreat is funded and supported by the hospital and the medical community.

Regional teaching improvement programs for community-based teachers

PURPOSE

Community-based clinical teachers provide an important cadre of faculty for medical education. This study was designed to examine the feasibility and value of an American College of Physicians-sponsored regional teaching improvement program for community-based teachers.

SUBJECTS AND METHODS

We conducted five regional (Connecticut, New Hampshire/Vermont, New York, Ohio, and Virginia) 1- to 2-day teaching-improvement workshops for 282 faculty (49% community based, 51% university based). The workshops were conducted by regional facilitators trained by the Stanford Faculty Development Program using large group and small group instructional methods to teach participants a framework for analyzing teaching, to increase their repertoire of teaching behaviors, to define personal teaching goals, and to identify the educational needs of their teaching site. Participants used Likert ratings [1 (low) to 5 (high) scale] to assess workshop quality, facilitator effectiveness, and rewards for and barriers to teaching in their clinics. Using retrospective pre- and postintervention ratings, participants also assessed workshop impacts on teacher knowledge, attitudes, and skills. Finally, participants completed open-ended questions to identify recommended changes to improve their clinic as an educational site for students and residents.

RESULTS

At all sites, participants evaluated the program as highly useful (4.6 +/- 0.6, mean +/- SD). Participants' ratings indicated that the program had a positive effect on their knowledge of teaching principles (4.0 +/- 0.9), an increase in their teaching ability (P <0.001), and an increase in their sense of integration with their affiliated institution (P <0.001).

CONCLUSIONS

Regional training of university and community faculty can be an effective way of promoting the improvement of teaching and the collaboration between community-based teachers and academic centers. National physician organizations and regionally based facilitators can provide important resources for the delivery of such training.