Evaluation of the seminar method to improve clinical teaching

Teacher-led versus student-led seminar blended with portfolio for "assessment of learning": An interventional study

BACKGROUND

Increase in competitive demands has led to the promotion of seminar presentation by the students to increase active learning and for the assessment of learning. Portfolios are an important tool for assessment for learning. The objectives of the study were to compare the gain in knowledge among the conventional and the blended seminar groups (intervention group), analyze the working portfolios to assess for learning, and to gather the perception of students.

MATERIALS AND METHODS

This was an interventional study (2016) with convenience sampling that included 27 students each in the conventional and interventional groups. It was conducted in a medical college in central India. The conventional group was teacher led and the blended seminar group was student led with interaction with the students regarding the contents of the portfolio. Student's unpaired and paired t tests were used for statistical analysis. P < 0.05 was considered as the level of significance. Class average normalized gain (g) was used as a measure of effectiveness of the intervention. Quantitative questions were analyzed using percentages and qualitative data using categorization.

RESULTS

Significant difference was found between the conventional and intervention groups (P < 0.05) with gain "g" being 0.52 for the intervention group. On evaluation of the feedback, students commented regarding its interactive nature and progress during the learning process. The reflections were coded as text as the unit of coding and student as the unit of coding. It was also found that the students who were critical reflectors were the ones who scored > 50% in the posttest scores.

CONCLUSION

The present study showed that seminar when blended with portfolio yielded positive results in the process of learning, and hence was effective in assessment for learning.

Seminar-case learning model improves clinical teaching: a prospective randomized controlled trial

Purpose

The purpose of this research was to assess whether the efficacy of the seminar-case learning model is superior to the traditional lecture-based learning model in the gastroenterology curriculum for first-year graduate students.

Materials & Methods

This research was a prospective randomized controlled trial that enrolled 92 first-year postgraduate students with a rotation internship in the gastroenterology department. The students were randomly divided into 2 groups and then subjected to an identical version of the curriculum for 8 weeks. The experimental group (n = 50) used the seminar-case learning model, while the control group (n = 42) used the traditional lecture-based learning model. Examinations consisted of a theoretical test and a case analysis test, and anonymous questionnaires were used to assess teaching quality.

Results

All participants completed the examinations and questionnaires. The average theoretical test score of the experimental group was no statistical significance with that of the control group (P = 0.17). The average case analysis test score of the experimental group was significantly higher than that of the control group (P < 0.05). The indicators of the experimental group’s feedback were better than those of the control group, such that there were significantly higher learning interest and motivation, a better understanding of diseases and knowledge, improvements in clinical thinking and summary ability, and an active classroom atmosphere in the experimental group (P < 0.05). However, students in the experimental group felt more burdensome.

Conclusion

Compared to the traditional method, the seminar-case learning model showed a higher efficacy. The seminar-case learning model effectively improved students’ outcomes and satisfaction, which helped students narrow the gap between theoretical knowledge and clinical practical application.

A Curriculum for Enhancing Physician Teaching Skills: The Value of Physician-Educator Partnerships

Developing as a physician requires an enormous amount of complex training, and quality of instruction greatly affects training outcomes. But while physicians are expected to teach trainees within the clinic, they often do not receive formal training in effective instructional practices. Providing faculty development programs is one way that institutions can help physicians develop teaching skills, but these programs often are developed without the input of educational specialists and not based in educational theory. In this methodology paper, we describe a 5-module curriculum that was developed in a cross-disciplinary collaboration between instructional designers and physician faculty. By merging educational and medical expertise and using adult learning theory with the Charlotte-Danielson educational framework, an essentials for clinical teaching educational endorsement program (ECTEEP) was created as a feature of the institutional curriculum within a large, urban teaching hospital. Here we describe how the program was developed through a physician-educator partnership, outline the program's key content, and highlight essential aspects of successful implementation. The ECTEEP incorporates active learning approaches within an abbreviated format, distilling 5 critical aspects of effective teaching that are relevant to the clinical environment: cultural humility and safe learning environments, instruction practices for engaging learners, instruction and assessment strategies, receiving and giving feedback, and mentorship and coaching. A central feature of the program is that facilitators actively model the teaching behaviors they are conveying, which underscores the critical importance of facilitator preparation and skill. Our curriculum is offered here as a basic template for institutions that may want to establish a program for enhancing physician teaching skill.

Is Asking Questions on Rounds a Teachable Skill? A Randomized Controlled Trial to Increase Attendings' Asking Questions

BACKGROUND

Morning bedside rounds remain an essential part of Internal Medicine residency education, but rounds vary widely in terms of educational value and learner engagement.

OBJECTIVE

To evaluate the efficacy of an intervention to increase the number and variety of questions asked by attendings at the bedside and assess its impact.

DESIGN

We conducted a randomized, controlled trial to evaluate the efficacy of our intervention.

PARTICIPANTS

Hospitalist attendings on the general medicine service were invited to participate. Twelve hospitalists were randomized to the experimental group and ten hospitalists to the control group.

INTERVENTION

A one-hour interactive session which teaches and models the method of asking questions using a non-medical case, followed by practice using role plays with medical cases.

MAIN MEASURES

Our primary outcome was the number of questions asked by attendings during rounds. We used audio-video recordings of rounds evaluated by blinded reviewers to quantify the number of questions asked, and we also recorded the type of question and the person asked. We assessed whether learners found rounds worthwhile using anonymous surveys of residents, patients, and nurses.

KEY RESULTS

Blinded analysis of the audio-video recordings demonstrated significantly more questions asked by attendings in the experimental group compared to the control group (mean number of questions 23.5 versus 10.8, p< 0.001) with significantly more questions asked of the residents (p<0.003). Residents rated morning bedside rounds with the experimental attendings as significantly more worthwhile compared to rounds with the control group attendings (p=0.009).

CONCLUSION

Our study findings highlight the benefits of a one-hour intervention to teach faculty a method of asking questions during bedside rounds. This educational strategy had the positive outcome of including significantly more resident voices at the bedside. Residents who rounded with attendings in the experimental group were more likely to "strongly agree" that bedside rounds were "worthwhile".

Faculty Development in Basic Science Disciplines for Clinical Teachers: Results of a Cross-Cultural Workshop in the Dominican Republic

BACKGROUND

Faculty development programs, studied both within the USA and internationally, have been shown to be helpful for enhancing scholarly and academic work for academic faculty in teaching institutions. This project investigates the impact of a well-studied faculty development program applied to basic science teachers in an academic medical center in the Dominican Republic.

METHODS

A faculty cohort of physician educators in the Basic Sciences at Universidad Iberoamericana (UNIBE) School of Medicine in the Dominican Republic underwent training in the Stanford Faculty Development Center (SFDC) model of teaching through a sequence of seven workshops that were adapted for basic science content. A validated retrospective pre- and post-test instrument was used to measure study outcomes on specific teaching behaviors at the end of the workshops, at 3-month and at 12-month post-intervention. Thematic analysis of specific teaching techniques and barriers to their teaching were compiled.

RESULTS

Fourteen faculty participants completed the study. All participants found the workshops valuable. Significant improvement in self-reported teaching abilities was seen comparing the mean pre-intervention scores of 106.21 (maximum score = 145, standard deviation [SD] = 12.70) with mean immediate post-intervention scores of 138.28 (SD = 6.12), the 3-month post-intervention scores of 129.79 (SD = 11.12) and the 1-year post-intervention scores of 131.86 (SD = 11.26). Several consistent themes were found among participants.

CONCLUSIONS

Faculty development for improving teaching of basic science concepts by clinicians can be performed across the cultures of the USA and the Dominican Republic.

Peer Observations: Enhancing Bedside Clinical Teaching Behaviors

Background

Medical training relies on direct observations and formative feedback. After residency graduation, opportunities to receive feedback on clinical teaching diminish. Although feedback through learner evaluations is common, these evaluations can be untimely, non-specific, and potentially biased. On the other hand, peer feedback in a small group setting or lecture format has been shown to be beneficial to teaching behaviors, however, little is known if peer observation using a standardized tool followed by feedback results in improved teaching behaviors. Therefore, the objective of this study was to examine if feedback after peer observation results in improved inpatient teaching behaviors.

Methods

This study was conducted at a tertiary care hospital. Academic hospitalists in the Division of Hospital Medicine developed a standardized 28-item peer observation tool based on the Stanford Faculty Development Program to observe their peers during bedside teaching rounds and provide timely feedback after observation. The tool focused on five teaching domains (learning climate, control of session, promotion of understanding and retention, evaluation, and feedback) relevant to the inpatient teaching environment. Teaching hospitalists were observed at the beginning of a two-week teaching rotation, given feedback, and then observed at the end of the rotation. Furthermore, we utilized a post-observation survey to assess the teaching and observing hospitalists’ comfort with observation and the usefulness of the feedback. We used mixed linear models with crossed design to account for correlations between the observations. Models were adjusted for gender, age, and years of experience. We tested the internal validity of the instrument with Cronbach’s alpha.

Results

Seventy (range: one to four observations per faculty) observations were performed involving 27 teaching attendings. A high proportion of teachers were comfortable with the observation (79%) and found the feedback helpful (92%), and useful for their own teaching (88%). Mean scores in teaching behavior domains ranged from 2.1 to 2.7. In unadjusted and adjusted analysis, each teaching observation was followed by higher scores in learning climate (adjusted improvement = 0.09; 95% CI = 0.02-0.15; p = 0.007) and promotion of understanding and retention (adjusted improvement = 0.09; 95% CI = 0.02-0.17; p = 0.01). The standardized observation tool had Cronbach’s alpha of 0.81 showing high internal validity.

Conclusions

Peer observation of bedside teaching followed by feedback using a standardized tool is feasible and results in measured improvements in desirable teaching behaviors. The success of this approach resulted in the expansion of peer observation to other Divisions within the Department of Internal Medicine at our Institution.

Teacher questions and student responses in case-based learning: outcomes of a video study in medical education

BACKGROUND

Case-based learning (CBL) is a highly interactive instructional format widely used in medical education. One goal of CBL is to integrate basic biomedical knowledge and its application to concrete patient cases and their clinical management. In this context, we focus the role of teacher questions as triggers for reproductive vs. elaborative student responses. Specifically, our research questions concern the kinds of questions posed by clinical teachers, the kinds of responses given by students, the prediction of student responses based upon teacher questions, and the differences between the two medical disciplines in focus of our study, internal medicine and surgery.

METHODS

We analyse 19 videotaped seminars (nine internal medicine, ten surgery) taught by clinicians and attended by advanced medical students. Multiple raters performed a low-inference rating process using a theory-based categorical scheme with satisfactory interrater-reliability.

RESULTS

We found that medical teachers mostly posed initial (instead of follow-up) questions and that their questions were more often closed (instead of open). Also, more reasoning (than reproductive) questions were posed. A high rate of student non-response was observed while elaborative and reproductive student responses had a similar prevalence. In the prediction context, follow-up reasoning questions were associated with low non-response and many elaborative answers. In contrast, the highest student non-response rate followed open reproduction questions and initial reasoning questions. Most reproductive statements by students were made following closed reproduction questions.

CONCLUSIONS

These results deepen our understanding of interactive, questions-driven medical teaching and provide an empirical basis for clinical teachers to use questions in didactically fruitful ways.

Singapore Neonatal Resuscitation Guidelines 2016

We present the revised Neonatal Resuscitation Guidelines for Singapore. The 2015 International Liaison Committee on Resuscitation Neonatal Task Force's consensus on science and treatment recommendations (2015), and guidelines from the American Heart Association and European Resuscitation Council were debated and discussed. The final recommendations of the National Resuscitation Council, Singapore, were derived after the task force had carefully reviewed the current available evidence in the literature and addressed their relevance to local clinical practice.

Cross-Cultural Interprofessional Faculty Development in Japan: Results of an Integrated Workshop for Clinical Teachers

BACKGROUND

Faculty development programs, studied both home and abroad, have been shown to be helpful for enhancing the scholarly and academic work of nonacademic clinicians. Interprofessional education and faculty development efforts have been less well studied. This project investigated the effect of a well-studied faculty development program applied in an interprofessional fashion across health profession educators in medicine and nursing.

METHODS

A faculty cohort of nurse and physician educators at The University of Tokyo underwent training in the Stanford Faculty Development Center (SFDC) model of clinical teaching through a sequence of 7 workshops. The workshops were performed in English with all materials translated into Japanese. A validated, retrospective pretest and posttest instrument was used to measure study outcomes on global assessment of teaching abilities and specific teaching behaviors (STBs) at 1 and 12 months after intervention. Successful completion of Commitment to Change statements were also assessed at 12 months.

RESULTS

In total, 19 faculty participants completed the study. All participants found the workshops valuable. For global assessment, significant improvement in self-reported teaching abilities was seen comparing the mean pretest scores of 27.26 (maximum score = 55, standard deviation [SD] = 8.61) with mean scores at both 1 month (36.81, SD = 7.48, P < 0.001) and at 1 year (34.67, SD = 7.32, P < 0.001). For STBs, significant improvement was also seen comparing the mean group pretest score of 82.11 (maximum score = 145, SD = 15.72), to the posttest mean score of 111.11 (SD = 14.48, P < 0.001) and the 1-year mean score of 103.76 (SD = 12.87, P < 0.001). In total, 27/42 Commitment to Change statements were successfully completed at 1 year.

CONCLUSIONS

Faculty development for improving clinical teaching can be performed across the cultures of medicine and nursing, as well as across the cultures of the United States and Japan.

Developing medical education capacity in Russia: twenty years of experience

BACKGROUND

The partnership between Yale University (USA) and Kazan State Medical University (KSMU, Russia) was established in 1996 and transitioned to Western Connecticut Health Network (WCHN)/University of Vermont Robert Larner, M.D. College of Medicine (USA) in 2012 with the goal of modernizing medical education at KSMU primarily through introduction of the American medical education structure, role modeling, and educational capacity building. It was centered on the formation of a select group of Russian junior faculty members familiar with American medical education who would then initiate a gradual change in medical education at KSMU. Here we describe the 20 year partnership, rooted in local capacity building, through which a sustainable, mutually rewarding international collaboration was established. In addition, we evaluate the program's outcomes and impact on medical education at Kazan State Medical University, and assess its influence on Russian program participants.

METHODS

Senior residents and faculty were sent to KSMU to conduct teaching sessions with local faculty and trainees. Their responsibilities included familiarizing Russian colleagues with specific topics in clinical medicine, importing knowledge about the basics of teaching, clinical epidemiology and evidence based medicine, and creating, in consistency with the American model, a "Clinical Teaching Team Structure" that integrates patient care with clinical education. Furthermore, 44 of selected KSMU members, including 13 junior faculty (29.5%), 14 clinical PhD students (31.8%), 12 interns/residents (27.3%), and five medical students (11.4%), were trained at Yale/WCHN or one of their major affiliated community hospitals for a period of 1 to 12 months for a total of 844 participant-weeks of training.

RESULTS

Thirty (68.2%) individuals who were trained in the U.S. are currently working in Kazan primarily as faculty at KSMU. Among them, three trainees (10%) have become heads of their department, eight (26.7%) hold senior faculty positions, and two (6.7%) have clinical and educational administrative leadership positions. Two major clinical departments have adopted the "Clinical Teaching Team Structure." As a result of the collaboration, three teaching courses - Evidence-Based Medicine, Tropical Medicine, and HIV/AIDS Medicine - have been designed and incorporated into the curriculum.

CONCLUSION

This partnership has been instrumental in introducing the American medical education model and expanding the medical knowledge of faculty, residents, and students of KSMU on infectious diseases, HIV/AIDS, tropical medicine, renal diseases, and global health topics. Capacity building through the Yale/WCHN-KSMU exchange program has greatly contributed to the quality of medical education at Kazan State Medical University.

Retrospective Versus Actual Pre-Course Self-Assessments

Self-assessments are frequently used to evaluate the effectiveness of medical education programs. The change in learners' self-ratings preto post-course is attributed to the educational intervention. However, participants may also change their understanding of the dimension being measured and hence, adjust their criteria for self-rating (response shift). This study was designed to compare a conventional prelpost course evaluation design with a retrospective pre/post method used to minimize the effect of the response shift. Participants attending a course in 1987 or 1988 designed to improve interviewing and teaching skills completed a faculty self-rating form (a) one month before the course; (b) after the course, assessing their pre-course skill level (retrospective pre-course); and (c) after the course, assessing their present skill level (post-course). Retrospective pre-course ratings were significantly lower than actual precourse ratings in two out of four factors empirically derived from the assessment instrument in 1987 and three of four in 1988. Greater differences preto post-course were found using the retrospective as opposed to the actual pre-course self-ratings. Our findings indicate that the collection of retrospective pre-course and post-course self-ratings at a single point in time demonstrates greater differences than the conventional pre/post evaluation method. Use of the retrospective precourse method is efficient and may be a more accurate evaluation method than the conventional prelpost evaluation.

Faculty development for hospitalists: structured peer observation of teaching

BACKGROUND

Hospitalists provide much of the clinical teaching in internal medicine, yet formative feedback to improve their teaching is rare.

METHODS

We developed a peer observation, assessment, and feedback program to improve attending hospitalist teaching. Participants were trained to identify 10 optimal teaching behaviors using a structured observation tool that was developed from the validated Stanford Faculty Development Program clinical teaching framework. Participants joined year-long feedback dyads and engaged in peer observation and feedback on teaching. Pre- and post-program surveys assessed confidence in teaching, performance of teaching behaviors, confidence in giving and receiving feedback, attitudes toward peer observation, and overall satisfaction with the program.

RESULTS

Twenty-two attending hospitalists participated, averaging 2.2 years (± 2.1 years standard deviation [SD]) experience; 15 (68%) completed pre- and post-program surveys. Confidence in giving feedback, receiving feedback, and teaching efficacy increased (1 = strongly disagree, 5 = strongly agree, mean ± SD): "I can accurately assess my colleagues' teaching skills," (pre = 3.2 ± 0.9 vs post = 4.1 ± 0.6, P < 0.01), "I can give accurate feedback to my colleagues" (pre = 3.4 ± 0.6 vs post = 4.2 ± 0.6, P < 0.01), and "I am confident in my ability to teach students and residents" (pre = 3.2 ± 0.9 vs post = 3.7 ± 0.8, P = 0.026).

CONCLUSIONS

Peer observation and feedback of teaching increases hospitalist confidence in several domains that are essential for optimizing teaching. Further studies are needed to examine if educational outcomes are improved by this program.

Development and psychometric characteristics of a new domain of the stanford faculty development program instrument

INTRODUCTION

Teacher's attitude domain, a pivotal aspect of clinical teaching, is missing in the Stanford Faculty Development Program Questionnaire (SFDPQ), the most widely used student-based assessment method of clinical teaching skills. This study was conducted to develop and validate the teacher's attitude domain and evaluate the validity and internal consistency reliability of the augmented SFDPQ.

METHODS

Items generated for the new domain included teacher's enthusiasm, sobriety, humility, thoroughness, empathy, and accessibility. The study involved 20 resident doctors assessed once by 64 medical students using the augmented SFDPQ. Construct validity was explored using correlation among the different domains and a global rating scale. Factor analysis was performed.

RESULTS

The response rate was 94%. The new domain had a Cronbach's alpha of 0.89, with 1-factor solution explaining 57.1% of its variance. It showed the strongest correlation to the global rating scale (rho = 0.71). The augmented SFDPQ, which had a Cronbach's alpha of 0.93, correlated better (rho = 0.72, p < 0.00001) to the global rating scale than the original SFDPQ (rho = 0.67, p < 0.00001).

DISCUSSION

The new teacher's attitude domain exhibited good internal consistency and construct and factorial validity. It enhanced the content and construct validity of the SFDPQ. The validated construct of the augmented SFDPQ is recommended for design and evaluation of basic and continuing clinical teaching programs.

Improving clinical teaching in China: initial report of a multihospital pilot faculty development effort

BACKGROUND

The study's purpose was to investigate whether or not a US-based faculty development program could be successfully used to improve the teaching skills of Chinese medical faculty.

DESCRIPTION

The program, based on the Stanford Faculty Development Program (SFDP) model, was presented to 28 faculty teachers affiliated with Zhejiang University School of Medicine, Hangzhou, China. Outcomes included the attendees' satisfaction of the seminars and their ratings of self-reported teaching ability using a previously studied retrospective pre-post questionnaire. Paired mean scores of the retrospective pre-test were statistically compared to the means of the retrospective post-test for all respondents.

EVALUATION

Twenty-eight teachers completed the survey. The seminars were rated highly and summative ratings of both global teaching performance and use of specific teaching behaviors were significantly improved between the retrospective pre- and post-test scores.

CONCLUSION

We were able to demonstrate a positive effect of a Western-based faculty development course on the teaching skills of Chinese clinical medical teachers.

The Questionnaire "SFDP26-German": a reliable tool for evaluation of clinical teaching?

Aims: Evaluation of the effectiveness of clinical teaching is an important contribution for the quality control of medical teaching. This should be evaluated using a reliable instrument in order to be able to both gauge the status quo and the effects of instruction. In the Stanford Faculty Development Program (SFDP), seven categories have proven to be appropriate:

Establishing the Learning Climate,

Controlling a Teaching Session,

Communication of Goals,

Encouraging Understanding and Retention,

Evaluation,

Feedback and

Self-directed Learning.

Since 1998, the SFDP26 questionnaire has established itself as an evaluation tool in English speaking countries. To date there is no equivalent German-language questionnaire available which evaluates the overall effectiveness of teaching.

Question:

Development and theoretical testing of a German-language version of SFDP26 (SFDP26-German),

Check the correlation of subscale of SFDPGerman against overall effectiveness of teaching.

Methods: 19 anaesthetists (7 female, 12 male) from the University of Lübeck were evaluated at the end of a teaching seminar on emergency medical care using SFDP-German. The sample consisted of 173 medical students (119 female (68.8%) and 54 male (31.2%), mostly from the fifth semester (6.6%) and sixth semester (80.3%). The mean age of the students was 23±3 years.

Results: The discriminatory power of all items ranged between good and excellent (r_it=0.48-0.75). All subscales displayed good internal consistency (α=0.69-0.92) and significant positive inter-scale correlations (r=0.40-0.70). The subscales and “overall effectiveness of teaching” showed significant correlation, with the highest correlation for the subscale “communication of goals (p< 0.001; r = 0.61).

Conclusion: The analysis of SFDP26-German confirms high internal consistency. Future research should investigate the effectiveness of the individual categories on the overall effectiveness of teaching and validate according to external criteria.

Short-term outcomes and long-term impact of a programme in medical education for medical students

OBJECTIVES

There is growing appreciation of the value of early preparation of future medical educators. Staff development programmes, conferences and workshops pertaining to the training of educators may be crucial to the pursuit of a school's larger educational mission to educate students, doctors and scholars and to provide comprehensive knowledge, research, patient care and service. This study examined the efficacy of a 1-week educational intervention aimed at preparing medical students to become effective doctor educators by building skills early in their careers. The study asked whether participation in a 5-day teacher training programme led to increased knowledge of instructional methods, more favourable attitudes towards teaching, and the integration of structured instructional design methods in a student-developed teaching project.

METHODS

A mixed methods research design was employed with quantitative data captured through pre- and post-test inventories, qualitative components captured through written comments, and a 2-year post-intervention survey. Quantitative analyses included pre-/post-intervention repeated measures with calculated effect sizes. Qualitative analysis was conducted using constant comparative methods.

RESULTS

Subjects demonstrated improved content knowledge and more positive attitudes towards motivation, teaching confidence, teacher roles, varied pedagogy, and use of assessment, instructional planning, and evaluation. Subjects were able to incorporate the programme's teaching theory and methods into their teaching projects and assessment of peers' and others' teaching in their own institutions 2 years post-training.

CONCLUSIONS

This study demonstrates that a well-designed programme for teacher preparation can be pedagogically effective for training medical students to become better educators and that this learning can be incorporated into long-term practice.

Developing teaching skills for medical educators in Russia: a cross-cultural faculty development project

CONTEXT

Faculty development programmes have proved successful for improving teaching skills. We investigated whether or not a successful US-based faculty development programme for improving the teaching skills of medical faculty could be transported to Russia.

METHODS

Five seminars, based on the 7 categories of the Stanford Faculty Development Program model, were presented to 48 teachers at Kazan State Medical University in Kazan, Russia. The seminars were comprised of mini-lectures, reviews of actual videotaped teaching scenario re-enactments, interactive role plays of teaching situations, and personalised goal setting for future teaching performance. Evaluation was performed through participants' self-reported ratings of teaching ability based on a retrospective pretest/post-test questionnaire and fulfilment of commitment of change (CTC) statements written by workshop participants. Outcomes were measured at both 1 and 12 months post-intervention.

RESULTS

Survey response rates were 98% (47/48) at 1 month and 81% (39/48) at 12 months. Global teaching performance improved (pretest = 38.4, 1 month post-test = 43.7, 12 months post-test = 42.5; P < 0.001) as did ratings of specific teaching behaviours (pretest = 100.2, 1 month post-test = 121.3, 12 months post-test = 116.8; P < 0.001). A total of 127 CTC statements were made and 90 (71%) were successfully instituted.

CONCLUSIONS

Our faculty development intervention demonstrated a positive, lasting effect on the teaching skills of Russian faculty members. This suggests that our efforts in transporting this intervention across the 2 cultures were successful.

A systematic review of faculty development initiatives designed to improve teaching effectiveness in medical education: BEME Guide No. 8

BACKGROUND

Preparing healthcare professionals for teaching is regarded as essential to enhancing teaching effectiveness. Although many reports describe various faculty development interventions, there is a paucity of research demonstrating their effectiveness.

OBJECTIVE

To synthesize the existing evidence that addresses the question: "What are the effects of faculty development interventions on the knowledge, attitudes and skills of teachers in medical education, and on the institutions in which they work?"

METHODS

The search, covering the period 1980-2002, included three databases (Medline, ERIC and EMBASE) and used the keywords: staff development; in-service training; medical faculty; faculty training/development; continuing medical education. Manual searches were also conducted. Articles with a focus on faculty development to improve teaching effectiveness, targeting basic and clinical scientists, were reviewed. All study designs that included outcome data beyond participant satisfaction were accepted. From an initial 2777 abstracts, 53 papers met the review criteria. Data were extracted by six coders, using the standardized BEME coding sheet, adapted for our use. Two reviewers coded each study and coding differences were resolved through discussion. Data were synthesized using Kirkpatrick's four levels of educational outcomes. Findings were grouped by type of intervention and described according to levels of outcome. In addition, 8 high-quality studies were analysed in a 'focused picture'.

RESULTS

The majority of the interventions targeted practicing clinicians. All of the reports focused on teaching improvement and the interventions included workshops, seminar series, short courses, longitudinal programs and 'other interventions'. The study designs included 6 randomized controlled trials and 47 quasi-experimental studies, of which 31 used a pre-test-post-test design.

KEY POINTS

Despite methodological limitations, the faculty development literature tends to support the following outcomes: Overall satisfaction with faculty development programs was high. Participants consistently found programs acceptable, useful and relevant to their objectives. Participants reported positive changes in attitudes toward faculty development and teaching. Participants reported increased knowledge of educational principles and gains in teaching skills. Where formal tests of knowledge were used, significant gains were shown. Changes in teaching behavior were consistently reported by participants and were also detected by students. Changes in organizational practice and student learning were not frequently investigated. However, reported changes included greater educational involvement and establishment of collegiate networks. Key features of effective faculty development contributing to effectiveness included the use of experiential learning, provision of feedback, effective peer and colleague relationships, well-designed interventions following principles of teaching and learning, and the use of a diversity of educational methods within single interventions. Methodological issues: More rigorous designs and a greater use of qualitative and mixed methods are needed to capture the complexity of the interventions. Newer methods of performance-based assessment, utilizing diverse data sources, should be explored, and reliable and valid outcome measures should be developed. The maintenance of change over time should also be considered, as should process-oriented studies comparing different faculty development strategies.

CONCLUSIONS

Faculty development activities appear highly valued by participants, who also report changes in learning and behavior. Notwithstanding the methodological limitations in the literature, certain program characteristics appear to be consistently associated with effectiveness. Further research to explore these associations and document outcomes, at the individual and organizational level, is required.

Preparing fourth-year medical students to teach during internship

Interns are expected to teach medical students, yet there is little formal training in medical school to prepare them for this role. To enhance the teaching skills of our graduating students we initiated a 4-hour "teaching to teach" course as part of the end of the fourth-year curriculum. Course evaluations demonstrate that students strongly support this program (overall ratings 2000 to 2005: mean=4.4 [scale 1 to 5], n=224). When 2004 course participants were surveyed during the last month of their internship, 84%"agree" or "strongly agree" with the statement: "The teaching to teach course helped prepare me for my role as a teacher during internship" (2005: mean 4.2 [scale 1 to 5], n=45, response rate 60%). A course preparing fourth-year students to teach during internship is both feasible and reproducible, with a minimal commitment of faculty and resident time. Participants identify it as an important addition to their education and as useful during internship.

The impact of the Stanford Faculty Development Program on ambulatory teaching behavior

CONTEXT

Faculty development has received considerable investment of resources from medical institutions, though the impact of these efforts has been infrequently studied.

OBJECTIVE

To measure the impact of the Stanford Faculty Development Program in Clinical Teaching on ambulatory teaching behavior.

DESIGN

Pre-post.

SETTING AND PARTICIPANTS

Eight internal medicine faculty participating in local faculty development.

INTERVENTION

Participants received 7 2-hour sessions of faculty development. Each session included didactic, role-play, and videotaped performance evaluation.

MAIN OUTCOME MEASURE

Before and after the intervention, faculty were video-taped during a case presentation from a standardized learner, who had been trained to portray 3 levels of learners: a third-year medical student, an intern, and a senior medical resident. Teacher and learner utterances (i.e, phrases) were blindly and randomly coded, using the Teacher Learner Interaction Analysis System, into categories that capture both the nature and intent of the utterances. We measured change in teaching behavior as detected through analysis of the coded utterances.

RESULTS

Among the 48 videotaped encounters, there were a total of 7,119 utterances, with 3,203 (45%) by the teacher. Examining only the teacher, the total number of questions asked declined (714 vs 426, P=.02) with an increase in the proportion of higher-level, analytic questions (44% vs 55%, P<.0001). The quality of feedback also improved, with less "minimal" feedback (87% vs 76%, P<.0005) and more specific feedback (13% vs 22%) provided.

CONCLUSIONS

Teaching behaviors improved after participation in this faculty development program, specifically in the quality of questions asked and feedback provided.

Faculty Development in Small-Group Teaching Skills Associated with a Training Course on Office-Based Treatment of Opioid Dependence

The Drug Addiction Treatment Act of 2000 (DATA-2000) allows qualified physicians to treat opioid-dependent patients with schedule III-V medications, such as buprenorphine, in practices separate from licensed, accredited opioid treatment programs. Physicians may attain this qualification by completing 8-hours of training in treating opioid dependence. This paper describes the evaluation of a faculty development workshop designed to enhance teaching skills of small-group facilitators involved with DATA-2000 training sessions. This workshop coached the facilitators on their teaching roles in the DATA-2000 session through experiential practice of patient- case discussions related to treatment of opioid-dependence. Descriptive questionnaires evaluated the value of the workshop. Twenty-six facilitators participated in the workshops. Paired mean score responses for specific teaching skill abilities demonstrated statistically significant improvement in all categories. Evaluation of the DATA-2000 training session small-group facilitators was uniformly positive. This faculty development workshop was successful in improving teaching skills for our small-group faculty facilitators.

Long-term follow-up of a longitudinal faculty development program in teaching skills

BACKGROUND

The long-term impact of longitudinal faculty development programs (FDPs) is not well understood.

OBJECTIVE

To follow up past participants in the Johns Hopkins Faculty Development Program in Teaching Skills and members of a comparison group in an effort to describe the long-term impact of the program.

DESIGN AND PARTICIPANTS

In July 2002, we surveyed all 242 participants in the program from 1987 through 2000, and 121 members of a comparison group selected by participants as they entered the program from 1988 through 1995.

MEASUREMENTS

Professional characteristics, scholarly activity, teaching activity, teaching proficiency, and teaching behaviors.

RESULTS

Two hundred participants (83%) and 99 nonparticipants (82%) responded. When participants and nonparticipants from 1988 to 1995 were compared, participants were more likely to have taught medical students and house officers in the last year (both P<.05). Participants rated their proficiency for giving feedback more highly (P<.05). Participants scored higher than nonparticipants for 14 out of 15 behaviors related to being learner centered, building a supportive learning environment, giving and receiving feedback, and being effective leaders, half of which were statistically significant (P<.05). When remote and recent participants from 1987 through 2000 were compared with each other, few differences were found.

CONCLUSIONS

Participation in the longitudinal FDP was associated with continued teaching activities, desirable teaching behaviors, and higher self-assessments related to giving feedback and learner centeredness. Institutions should consider supporting faculty wishing to participate in FDPs in teaching skills.

Faculty development in teaching skills: an intensive longitudinal model

Although reflection contributes to the personal growth of clinician-educators and is important for effective teaching, few teaching skills programs report its use. The Johns Hopkins Faculty Development Program in Teaching Skills, first implemented in 1987 as a theoretically grounded, longitudinal model for faculty development of clinician-educators, comprises a set of conditions intended to promote reflective learning. This paper describes the program and reports evaluation results for 98 participants and a comparison group of 112 nonparticipants between 1988 and 1996. Participants met with facilitators weekly for nine months for 3.5 hours, in stable groups of four to six individuals. Educational methods used across seven content areas emphasized relationships and collaboration, and included information provision, experiential learning with reflection, and personal awareness sessions. A pre-post evaluation design with comparison group measured changes in self-assessed teaching and professional skills, teaching enjoyment, and learning effectiveness. A post-only evaluation design appraised overall program quality, educational methods, facilitation, learning environment, and perceived impact of participation. Program participants had significantly greater pre-post-change scores than nonparticipants for all 14 outcomes (p <.05). Multiple regression modeling indicated that program participation was associated with pre-post improvement in all outcomes except administration skills, controlling for all participant and nonparticipant baseline characteristics (p <.05). All measured programmatic characteristics were highly rated by participants. Experiential methods with reflection were rated significantly higher than information-provision and personal awareness sessions (p <.001). Evaluation results demonstrate a positive impact of this alternative approach to faculty development on clinician-educator perceptions of their attitudes and behaviors towards learners and colleagues.

Cross-cultural faculty development: initial report of an American/Russian experience

BACKGROUND

Faculty development programs can improve the teaching skills of medical teachers. We pilot-tested the transportability of an American-based faculty development seminar program to a Russian academic medical center.

DESCRIPTION

Four seminars, based on the 7 categories of the Stanford Faculty Development Program model, were presented to 14 teachers at Kazan State Medical University. The seminars included minilectures and reviews of actual videotaped teaching scenario reenactments. Evaluation was performed through participants' self-reported ratings of teaching ability based on a validated pretest-posttest questionnaire. Measurements at both 1 month and 12 months postintervention were completed.

EVALUATION

All 14 participants completed the evaluation. At both measured times, statistically significant improvements in the ratings of global teaching performance and specific teaching behaviors were reported.

CONCLUSIONS

Our faculty development intervention was successfully transported across medical systems and demonstrated a positive, lasting affect on the teaching skills of a small number of Russian medical teachers.

Using standardised students in faculty development workshops to improve clinical teaching skills

PURPOSE

We describe the use of standardised students (SSs) in interdisciplinary faculty development programmes to improve clinical teaching skills. Standardised students are actual health professions students who are trained to portray a prototypical teaching challenge consistently across many encounters with different faculty participants.

METHODS

The faculty development programmes described focused on the skills of providing feedback and brief clinical teaching. At the beginning of each session, each participant was videotaped in encounters with 2 different SSs. Using microteaching (an instructional method in which learners view short segments of their own videotaped performance and discuss the tapes with a facilitator, consultant or other workshop participants), each group of participants and instructors reviewed the tapes and reflected on the encounters, providing immediate feedback to participants and modelling different approaches to the same teaching problem. The same process was repeated with more complicated scenarios after 2 weeks and again after 6 months offering reinforcement, further practice and more sophisticated development of the strategies learned. Participants completed post-session evaluations and a follow-up telephone survey.

RESULTS

A total of 36 faculty members from the colleges of medicine, dentistry, pharmacy and nursing participated in workshops in 2000-01. The workshops were rated as highly relevant to participants' teaching, and most participants reported that they had learned a great deal. Participants most appreciated reviewing the videotaped interactions, the feedback they received, the interactions with their colleagues, the interdisciplinary nature of the groups and the practical focus of the workshops.

CONCLUSIONS

Standardised students provide a high fidelity, low risk, simulated environment in which faculty can reflect on and experiment with new teaching behaviours. Such encounters can enhance the effectiveness and impact of faculty development programmes to improve clinical teaching skills.

The effect of video-based interventions on self-care

Caregivers for dependent, elderly community dwellers are frequently themselves at risk for illnesses exacerbated by stress. This study evaluates a videotape scenario intervention designed to increase self-care behaviors in three groups of caregivers randomly assigned to a control or to one of two experimental groups. Within the 6- to 8-week trial, attitudes toward self-care became more positive in the experimental groups, and all groups reported endorsement of behavioral changes related to self-care.

A picture is worth a thousand words: practical use of videotape in teaching

Videotapes, through vividly displayed clinical images and teaching interactions, are valuable tools for both learners and teachers. Visual images in combination with verbal instruction have been shown to significantly increase recall and retention. Many clinicians and medical teachers are aware of videotape resources, but have not had a chance to develop their use in medical education. In this paper, we discuss creative applications of videotapes in three major categories: presenting information, triggering discussion, and as a tool for direct self-observation and feedback. Videotapes may be valuable for presenting information in settings of didactic instruction; for triggering discussion during teaching workshops; and for self-observation of patient-doctor interactions and learner-teacher encounters. The article presents learner-centered approaches to review a videotaped clinical encounter in order to enhance value and comfort for the learner and teacher. Sources of tapes include on-site videotaping, published educational tapes and commercial tapes shown in accordance with fair use guidelines, examples of which are referenced. Videotapes add another dimension to traditional sources of physician education.

Improving residents' teaching skills and attitudes toward teaching

OBJECTIVE

To determine whether a short, 3-hour teaching skills workshop could improve residents' teaching performances and attitudes toward teaching.

DESIGN

Controlled study.

PARTICIPANTS AND SETTING

Forty-four second- and third-year residents in a university-based internal medicine residency program.

INTERVENTIONS

Twenty-two residents were assigned to a nonparticipant (control) group, and 22 residents were assigned to a 3-hour teaching skills workshop designed to help them establish a positive learning climate and provide effective feedback to medical students.

MEASUREMENTS

Questionnaires completed by medical students and residents that measured the residents' abilities to establish a positive learning climate and provide feedback, their overall teaching skills, and their attitudes toward teaching.

RESULTS

Four months after the workshop intervention, workshop participants improved their learning climate and feedback according to student evaluations (p = .02, p = .001, respectively) and resident self-assessments (p = .002, p = .01, respectively) compared with nonparticipants. Overall teaching skills were not significantly changed (p = .20 for student evaluation and p = .09 for self-assessments). Workshop participants also gained more confidence in their teaching (p = .001), and adopted more learner-centered approaches to teaching than did nonparticipants.

CONCLUSIONS

A 3-hour instructional workshop is a feasible and effective method to help residents improve their teaching skills, their confidence in teaching, and the approaches they use to teach medical students on the wards.

Models of faculty development for problem-based learning

Learning an innovative teaching method such as a problem-based learning is difficult for most faculty members because the method is based upon assumptions about learning that are often at variance with their beliefs. Faculty development can challenge assumptions about learning, provide experience with a new technique, and offer specific pedagogical skills that are needed to succeed as a tutor. A comprehensive approach to faculty development, derived from the literature in higher education, would include: instructional development, professional development, leadership development, and organizational development. Research on faculty development indicates positive results of such efforts. Faculty members who choose to learn about problem-based learning appear to progress through predictable stages of development that include: understanding and valuing the rationale for problem-based learning, acquiring general and content-specific tutor knowledge and skills, developing advanced skills in problem-based learning, and developing leadership and scholarship skills. Each of these steps, plus organizational vitality, are described along with recommendations for implementing such programs. Finally, five models of faculty development derived from medical schools with problem-based learning curricula are examined.

Attending rounds on in-patient units: differences between medical and non-medical services

The objective of this cross-sectional observational study was to quantify communication patterns between teachers and trainees on in-patient attending ward rounds and assess trainees' perceptions of the effectiveness of teaching interactions. Sixty-nine in-patient ward rounds on medical and non-medical teaching services at a university hospital and its affiliated VA Medical Center were studied. Teaching rounds were observed and audiotaped, and trained raters coded verbal interchange for its location, speaker identity and topic of the exchange. One to three days following the teaching rounds, residents and students were interviewed and completed a questionnaire concerning recollections of the content of the session. Medical rounds lasted a mean of 90 minutes, while non-medical rounds averaged 38 minutes. Medical teams spent more time than non-medical teams on case presentations and discussions of diseases not directly related to patient care. Both groups averaged approximately 10 minutes directly interacting with patients, and equal times were spent speaking by the teacher and trainees. The role of postgraduate year 1 residents and medical students primarily was to recite details of patients' clinical condition. Twenty-nine per cent of trainees were unable to recall a specific teaching point from rounds when interviewed 1-3 days later. Duration and content of in-patient rounds differed on medical and non-medical services. For both, discourse tended to be hierarchical, with those at different training levels adhering to specific roles. Bedside patient interactions were limited. The content recalled by students and house staff suggests that new, more effective educational paradigms are needed.

Impact of a clinical preventive medicine curriculum for primary care faculty: results of a dissemination model

BACKGROUND

This study was designed to test a dissemination model for providing clinical preventive medicine (CPM) training to general internal medicine faculty across the United States.

METHODS

The model incorporated direct instruction of a few faculty as seminar facilitators who, in turn, taught a CPM curriculum to their faculty colleagues, who then could teach it to housestaff and students. The CPM curriculum consisted of six seminars that focused primarily on the risk factors for chronic diseases and on behavior change methods for modifying smoking, diet, and exercise.

RESULTS

Faculty who participated in the seminars had significant pre- to post-test increase in knowledge and reported self-efficacy to implement CPM strategies with patients, as well as changes in CPM clinical practices. These faculty, in turn, successfully disseminated CPM information to their housestaff, who also had increases in self-efficacy and changed clinical practices regarding CPM topics.

CONCLUSIONS

The successful implementation of the dissemination model attests to its viability as a mechanism for disseminating CPM curricula and increasing the emphasis on CMP issues in both clinical teaching and clinical encounters with patients.

Teaching models in an ambulatory training program

Ambulatory care training is increasingly important in internal medicine. Such training centers on the practice where residents and faculty see their patients; thus, features of the practice model influence what residents learn. A resident-faculty group practice affiliated with a division of general internal medicine has many advantages. In such a practice, learning centers on resident-patient interactions, around which a comprehensive teaching program must be built. Major features of such a program include the mentoring of residents by faculty who work with them longitudinally and the presence of a well-balanced structured curriculum addressing clinical and nonclinical topics related to patient care. Teaching residents to interact and communicate with patients is crucial; approaches include role-modeling by faculty, use of videotaping, and role-playing and other innovative methods. Feedback is integral to learning and helps shape the attitudes and values that permeate residents' practices.

The service/education conflict in residency programs: a model for resolution

Residency programs consist of a range of activities involving service to patients and education of residents. The observation that a conflict exists between the service and education components of residency is widespread and has been used to explain many of the problems afflicting such programs today. The authors believe that the service/education conflict is a significant barrier to change in residency programs. A model is presented for residency education that reorganizes the service and education components. First, they present a broad overview of the conflict. Then they provide a brief historical perspective and comment on some of the current recommendations for residency programs. Next, they discuss how principles of adult learning relate to residency and propose a new model of residency that adheres more closely to these principles. Finally, the proposed model is presented in some detail and its implications are discussed. Only if the service and education components of residency are carefully delineated can residency programs adapt to the changing and growing needs of postgraduate medical education.

The teacher simulation exercise: changes in physician teaching emphasis and strategy. The SGIM Task Force on the Medical Interview

OBJECTIVES

The objective of the study was to determine the effect of a faculty development course in teaching medical interviewing on participants' ability to provide effective feedback to interviewers.

DESIGN

The study used a non-concurrent control group design which randomized subjects into two groups before the intervention. The two groups completed different pre-tests; each group then completed the other group's pre-test as its post-test. The post-course scores of one group were compared with the pre-course scores of the other group to establish differences.

SETTING

The research was conducted at the 1985 faculty development course sponsored by the SGIM Task Force on the Medical Interview.

PARTICIPANTS

49 of 52 teachers of medical interviewing attending the course completed the study.

INTERVENTION

The week-long intervention consisted of a variety of educational activities which assisted the participants in defining and actively pursuing their learning objectives in interviewing, teaching, and self-awareness.

MEASUREMENTS AND MAIN RESULTS

In their assessment of two videotaped segments of initial medical visits, participants were more likely after the course to comment on the interviewer's lack of attention to patient affect (69.0% versus 27.2%, p = 0.005 in one segment) and somewhat more likely to identify teaching strategies that actively involved the interviewer (47.2% vs. 35.0%, p = 0.09 in one segment). Both shifts were congruent with assessments made by course faculty.

CONCLUSIONS

Faculty development can influence teachers to recognize the need to provide feedback on skills that expert teachers would emphasize. The non-concurrent control group design provides an innovative approach to common constraints in evaluating faculty development courses.