A curious case of the phantom professor: mindless teaching evaluations by medical students

CONTEXT

Student evaluations of teaching (SETs) inform faculty promotion decisions and course improvement, a process that is predicated on the assumption that students complete the evaluations with diligence. Anecdotal evidence suggests that this may not be so.

OBJECTIVES

We sought to determine the degree to which medical students complete SETs deliberately in a classroom-style, multi-instructor course.

METHODS

We inserted one fictitious lecturer into each of two pre-clinical courses. Students were required to submit their anonymous ratings of all lecturers, including the fictitious one, within 2 weeks after the course using a 5-point Likert scale, but could choose not to evaluate a lecturer. The following year, we repeated this but included a portrait of the fictitious lecturer. The number of actual lecturers in each course ranged from 23 to 52.

RESULTS

Response rates were 99% and 94%, respectively, in the 2 years of the study. Without a portrait, 66% (183 of 277) of students evaluated the fictitious lecturer, but fewer students (49%, 140 of 285) did so with a portrait (chi-squared test, p < 0.0001).

CONCLUSIONS

These findings suggest that many medical students complete SETs mindlessly, even when a photograph is included, without careful consideration of whom they are evaluating and much less of how that faculty member performed. This hampers programme quality improvement and may harm the academic advancement of faculty members. We present a framework that suggests a fundamentally different approach to SET that involves students prospectively and proactively.

Gender disparities in leadership and scholarly productivity of academic hospitalists

BACKGROUND

Gender disparities still exist for women in academic medicine but may be less evident in younger cohorts. Hospital medicine is a new field, and the majority of hospitalists are <41 years of age.

OBJECTIVE

To determine whether gender disparities exist in leadership and scholarly productivity for academic hospitalists and to compare the findings to academic general internists.

DESIGN

Prospective and retrospective observational study.

SETTING

University programs in the United States.

MEASUREMENTS

Gender distribution of (1) academic hospitalists and general internists, (2) division or section heads for both specialties, (3) speakers at the 2 major national meetings of the 2 specialties, and (4) first and last authors of articles from the specialties' 2 major journals

RESULTS

We found equal gender representation of hospitalists and general internists who worked in university hospitals. Divisions or sections of hospital medicine and general internal medicine were led by women at 11/69 (16%) and 28/80 (35%) of university hospitals, respectively (P = 0.008). Women hospitalists and general internists were listed as speakers on 146/557 (26%) and 291/580 (50%) of the presentations at national meetings, respectively (P < 0.0001), first authors on 153/464 (33%) and 423/895 (47%) publications, respectively (P < 0.0001), and senior authors on 63/305 (21%) and 265/769 (34%) articles, respectively (P < 0.0001).

CONCLUSIONS

Despite hospital medicine being a newer field, gender disparities exist in leadership and scholarly productivity.

Role Modeling in the First 2 Years of Medical School

Role modeling opportunities for osteopathic physician teachers during a student's first 2 years of medical school are emerging as more colleges of osteopathic medicine strive to connect basic science didactics with clinically based learning activities. Examples of positive modeling by physician teachers during the first years of medical school are illustrated by 10 vignettes that can be incorporated into faculty development programs to increase awareness of such opportunities. The physician teacher in each vignette interacts with the student demonstrating desired professional behaviors. These vignettes also illustrate the effect of a positive "hidden curriculum" on a student's professional development. By recognizing these valuable teachable moments, teachers can incorporate role modeling into their daily practice.

How faculty members experience workplace-based assessment rater training: a qualitative study

CONTEXT

Direct observation of clinical skills is a common approach in workplace-based assessment (WBA). Despite widespread use of the mini-clinical evaluation exercise (mini-CEX), faculty development efforts are typically required to improve assessment quality. Little consensus exists regarding the most effective training methods, and few studies explore faculty members' reactions to rater training.

OBJECTIVES

This study was conducted to qualitatively explore the experiences of faculty staff with two rater training approaches - performance dimension training (PDT) and a modified approach to frame of reference training (FoRT) - to elucidate how such faculty development can be optimally designed.

METHODS

In a qualitative study of a multifaceted intervention using complex intervention principles, 45 out-patient resident faculty preceptors from 26 US internal medicine residency programmes participated in a rater training faculty development programme. All participants were interviewed individually and in focus groups during and after the programme to elicit how the training influenced their approach to assessment. A constructivist grounded theory approach was used to analyse the data.

RESULTS

Many participants perceived that rater training positively influenced their approach to direct observation and feedback, their ability to use entrustment as the standard for assessment, and their own clinical skills. However, barriers to implementation and change included: (i) a preference for holistic assessment over frameworks; (ii) challenges in defining competence; (iii) difficulty in changing one's approach to assessment, and (iv) concerns about institutional culture and buy-in.

CONCLUSIONS

Rater training using PDT and a modified approach to FoRT can provide faculty staff with assessment skills that are congruent with principles of criterion-referenced assessment and entrustment, and foundational principles of competency-based education, while providing them with opportunities to reflect on their own clinical skills. However, multiple challenges to incorporating new forms of training exist. Ongoing efforts to improve WBA are needed to address institutional and cultural contexts, and systems of care delivery.

Designing faculty development to support the evaluation of resident competency in the intrinsic CanMEDS roles: practical outcomes of an assessment of program director needs

BACKGROUND

The Royal College of Physicians and Surgeons of Canada and the College of Family Physicians of Canada mandate that faculty members demonstrate they are evaluating residents on all CanMEDS (Canadian Medical Education Directions for Specialists) roles as part of the accreditation process. Postgraduate Medical Education at the University of Ottawa initiated a 5-year project to develop and implement a comprehensive system to assess the full spectrum of CanMEDS roles. This paper presents the findings from a needs assessment with Program Directors, in order to determine how postgraduate medical faculty can be motivated and supported to evaluate residents on the intrinsic CanMEDS roles.

METHODS

Semi-structured individual interviews were conducted with 60 Postgraduate Program Directors in the Faculty of Medicine. Transcribed interviews were analyzed using qualitative analysis. Once the researchers were satisfied the identified themes reflected the views of the participants, the data was assigned to categories to provide rich, detailed, and comprehensive information that would indicate what faculty need in order to effectively evaluate their residents on the intrinsic roles.

RESULTS

Findings indicated faculty members need faculty development and shared point of care resources to support them with how to not only evaluate, but also teach, the intrinsic roles. Program Directors expressed the need to collaborate and share resources across departments and national specialty programs. Based on our findings, we designed and delivered workshops with companion eBooks to teach and evaluate residents at the point of care (Developing the Professional, Health Advocate and Scholar).

CONCLUSIONS

Identifying stakeholder needs is essential for designing effective faculty development. By sharing resources, faculties can prevent 'reinventing the wheel' and collaborate to meet the Colleges' accreditation requirements more efficiently.

Wisdom of the crowd: bright ideas and innovations from the teaching value and choosing wisely challenge

PROBLEM

Medical education has been cited as both part of the problems facing, and part of the solution to reforming, the increasingly challenging U.S. health care system which is fraught with concerns regarding the quality and affordability of care. To teach value in ways that are impactful, sustainable, and scalable, the best and brightest ideas need to be shared such that educators can build on successful existing innovations.

APPROACH

To identify the most promising innovations and bright ideas for teaching value to clinical trainees, the authors hosted the "Teaching Value and Choosing Wisely Challenge." The challenge used crowdsourcing methods to solicit scalable, pedagogical approaches from across North America, and then draw generalizable lessons.

OUTCOMES

The authors received 74 submissions (28 innovations; 46 bright ideas) from 14 students, 20 residents/fellows, 38 faculty members (ranging from instructors to full professors), and 2 nonclinical administrators. Submissions represented 14 clinical disciplines including internal medicine, emergency medicine, surgery, pediatrics, obstetrics-gynecology, laboratory medicine, and pharmacy. Thirty-nine abstracts focused on graduate medical education, 15 addressed undergraduate medical education, and 20 applied to both.

NEXT STEPS

The authors have solicited, shared, and described solutions for teaching high-value care to medical trainees. Challenge participants demonstrated commitment to improving value and ingenuity in addressing professional barriers to change. Further success requires strong local faculty champions and willing trainee participants. Additionally, the use of data to demonstrate the collective positive impact of these ideas and programs will be critical for sustaining pedagogical changes in the health professions.

Brazilian medical students' perceptions of expert versus non-expert facilitators in a (non) problem-based learning environment

BACKGROUND

In problem-based learning (PBL), the facilitator plays an important role in guiding the student learning process. However, although content expertise is generally regarded as a useful but non-essential prerequisite for effective PBL facilitation, the perceived importance of content knowledge may be subject to cultural, contextual, and/or experiential influences.

AIM

We sought to examine medical students' perceptions of subject-matter expertise among PBL facilitators in a region of the world (Brazil) where such active learning pedagogies are not widely used in university or pre-university settings.

RESULTS

Of the 252 Brazilian medical students surveyed, significantly (p≤0.001) greater proportions viewed content expert facilitators to be more effective than their non-expert counterparts at building knowledge (95% vs. 6%), guiding the learning process (93% vs. 7%), achieving cognitive learning (92% vs. 18%), generating learning goals (87% vs. 15%), and motivating self-study (80% vs. 15%).

DISCUSSION/CONCLUSION

According to Brazilian medical students, subject-matter expertise among PBL facilitators is essential to the learning process. We believe this widespread perception is due, in large part, to the relative lack of prior educational exposure to such pedagogies.

Developing a clinical teaching quality questionnaire for use in a university osteopathic pre-registration teaching program

BACKGROUND

Clinical education is an important component of many health professional training programs. There is a range of questionnaires to assess the quality of the clinical educator however none are in student-led clinic environments. The present study developed a questionnaire to assess the quality of the clinical educators in the osteopathy program at Victoria University.

METHODS

A systematic search of the literature was used to identify questionnaires that evaluated the quality of clinical teaching. Eighty-three items were extracted and reviewed for their appropriateness to include in a questionnaire by students, clinical educators and academics. A fifty-six item questionnaire was then trialled with osteopathy students. A variety of statistics were used to determine the number of factors to extract. Exploratory factor analysis (EFA) was used to investigate the factor structure.

RESULTS

The number of factors to extract was calculated to be between 3 and 6. Review of the factor structures suggested the most appropriate fit was four and five factors. The EFA of the four-factor solution collapsed into three factors. The five-factor solution demonstrated the most stable structure. Internal consistency of the five-factor solution was greater than 0.70.

CONCLUSIONS

The five factors were labelled Learning Environment (Factor 1), Reflective Practice (Factor 2), Feedback (Factor 3) and Patient Management (Factor 4) and Modelling (Factor 5). Further research is now required to continue investigating the construct validity and reliability of the questionnaire.

Preventing and managing unprofessionalism in medical school faculties

Professionalism is a required competency for medical students, residents, practicing physicians, and academic faculty. Faculty members must adhere to codes of conduct or risk discipline. The authors describe issues of unprofessionalism that culminate in allegations of faculty misconduct or filing of grievances in academic medicine and outline strategies for early intervention and prevention. The authors, vice and associate deans and executive director of the office of faculty affairs at a large U.S. medical school, have handled many allegations of unprofessional conduct over the past decade. They present case examples based on behaviors such as lack of respect, inappropriate language and behavior, failure to cooperate with members of the health care team, and sexual harassment/discrimination. They discuss factors complicating evaluation of these behaviors, including variable definitions of respect, different cultural norms, and false allegations. The authors make recommendations for prevention and intervention, including early identification, performance management, education about sexual harassment, and referrals to professional coaches, anger management classes, and faculty-staff assistance programs.

Why medical schools are tolerant of unethical behavior

The exposure to unethical and unprofessional behavior is thought to play a major role in the declining empathy experienced by medical students during their training. We reflect on the reasons why medical schools are tolerant of unethical behavior of faculty. First, there are barriers to reporting unprofessional behavior within medical schools including fear of retaliation and lack of mechanisms to ensure anonymity. Second, deans and directors do not want to look for unethical behavior in their colleagues. Third, most of us have learned to take disrespectful circumstances in health care institutions for granted. Fourth, the accreditation of medical schools around the world does not usually cover the processes or outcomes associated with fostering ethical behavior in students. Several initiatives promise to change that picture.

Creation of a metric for faculty effort in teaching in the veterinary curriculum

A teaching-effort metric was generated using information from health sciences literature and self-reported data from faculty members. This metric was used to verify faculty effort based on teaching assignments, to equalize teaching between faculty members within disciplines, and to help faculty members understand the ramifications of new teaching opportunities on their overall effort.

Assessing effective teaching: what medical students value when developing evaluation instruments

PURPOSE

To investigate what criteria medical students would value and use in assessing teaching skills.

METHOD

Fourth-year medical students at the University of Iowa Carver College of Medicine enrolled in a teaching elective course are required to design and use an evaluation instrument to assess effective teaching. Each class uses a similar process in developing their instruments. Since the first class in spring 2007, 193 medical students have created 36 different instruments. Three faculty evaluation experts conducted a thematic analysis of the instruments and coded the information according to what was being evaluated and what types of ratings were indicated. The data were submitted to a fourth faculty reviewer, who synthesized the information and adjusted the codes to better capture the data. Common themes and categories were detected.

RESULTS

Four themes were identified: content (instructor knowledgeable, teaches at level of learner, practical information), learning environment, teacher personal attributes, and teaching methods. Thirty-two descriptors were distinguished across the 36 instruments. Thirteen descriptors were present in 50% or more of the instruments. The most common rating systems were Likert scales and open comments.

CONCLUSIONS

Fourth-year medical students can offer an eclectic resource for evaluating teaching in the classroom and the clinic. Using the descriptors that were identified in greater than 50% of the evaluation instruments will provide effective measures that can be incorporated into medical teacher evaluation instruments.

Validation of the 25-Item Stanford Faculty Development Program Tool on Clinical Teaching Effectiveness

CONSTRUCT: The 25-item Stanford Faculty Development Program Tool on Clinical Teaching Effectiveness assesses clinical teaching effectiveness.

BACKGROUND

Valid and reliable rating of teaching effectiveness is helpful for providing faculty with feedback. The 25-item Stanford Faculty Development Program Tool on Clinical Teaching Effectiveness was intended to evaluate seven dimensions of clinical teaching. Confirmation of the structure of this tool has not been previously performed.

APPROACH

This study sought to validate this tool using a confirmatory factor analysis, testing a 7-factor model and compared its goodness of fit with a modified model. Acceptability of the use of the tool was assessed using a 6-item survey, completed by final year medical students (N = 119 of 156 students; 76%).

RESULTS

The testing of the goodness of fit indicated that the 7-factor model performed poorly, χ(2)(254) = 457.4, p < .001 (root mean square error of approximation [RMSEA] = 0.08, comparative fit index [CFI] = 0.91, non-normed fit index [NNFI] = 0.89). Only standardized root mean square residual (SRMR) indicated acceptable fit (0.06). Further exploratory analysis identified 10 items that cross-loaded on 2 factors. The remainder of the items loaded on factors as originally intended. By removing these 10 items, repeat confirmatory factor analysis on the modified 15-item, 5-factor model demonstrated a better fit than the original model: SRMR = 0.075, NNFI = 0.91, χ(2)(80) = 150.1, p < .001; RMSEA = 0.09; CFI = 0.93. Although 75% of the participants stated they were willing to fill the tool on their preceptors on a biweekly basis, only 25% were willing to do so on a weekly basis.

CONCLUSIONS

Our study failed to confirm factor structure of the 25-item tool. A modified tool with fewer, more conceptually distinct items was best fit by a 5-factor model. Further, the acceptability of use for the 25-item tool may be poor for rotations with a new preceptor weekly. The abbreviated tool may be preferable in that setting.

Mentoring Faculty: A US National Survey of Its Adequacy and Linkage to Culture in Academic Health Centers

INTRODUCTION

The aims of this study were to (1) describe the quantity and quality of mentoring faculty in US academic health centers (AHCs), (2) measure associations between mentoring and 12 dimensions that reflect the culture of AHCs, and (3) assess whether mentoring predicts seriously contemplating leaving one's institution.

METHODS

During 2007-2009, our National Initiative on Gender, Culture and Leadership in Medicine (C - Change) conducted a cross-sectional study of faculty from 26 representative AHCs in the United States using the 74-item C - Change Faculty Survey to assess relationships of faculty characteristics and various aspects of the institutional culture (52% response rate). Among the 2178 eligible respondents (assistant, associate, and full professors), we classified their mentoring experience as either inadequate, neutral, or positive.

RESULTS

In this national sample, 43% of the 2178 respondents had inadequate mentoring; only 30% had a positive assessment of mentoring. There was no statistical difference by sex, minority status, or rank. Inadequate mentoring was most strongly associated with less institutional support, lower self-efficacy in career advancement, and lower scores on the trust/relationship/inclusion scale. The percent of faculty who had seriously considered leaving their institution was highest among those who had inadequate mentoring (58%), compared to those who were neutral (28%) or had positive mentoring (14%) (all paired comparisons, p < .001).

DISCUSSION

In a national survey of faculty of US AHCs, mentoring was frequently inadequate and this was associated with faculty contemplating leaving their institutions. Positive mentoring, although less prevalent, was associated with many other positive dimensions of AHCs.

Dire need of changes in the methods for assessing university faculty productivity in developing countries

Publishing has become a necessity for promotion of faculty in academia. The faculties in developing countries face considerable difficulties publishing due to their prime focus on clinical approaches and resources for proper research. This often leaves no room for the pressured clinicians but to pursue poor quality publications just for the sake of promotion when the time for their promotion comes. The authors suggest establishing separate promotion tracks besides research in these underprivileged areas in order to avoid infestation of original research with poor ones.

Duty hour restrictions: organizational dynamics, systems issues, and the impact on faculty

The potential impact of resident duty hour restrictions on faculty is likely significant; however, the extent of this impact has still not been well documented. We undertook a narrative review of the literature to determine the magnitude of that potential impact and the nature of the evolving discourse related to faculty members as individuals. The literature provides an inconsistent picture of the impact of duty hour restrictions on faculty. While some studies have reported a significant increase in faculty workload, others suggest that the impact of duty hour restrictions has been minimal. Some papers suggest that duty hour restrictions may fundamentally change the nature of resident-teacher interactions and, as a result, will necessitate significant changes to the way education is delivered. Overall, the majority of issues of concern relate to one of the following: volume and composition of work, impact on faculty career choice, evolving perceptions of residents as learners, and the need to find an appropriate balance between learning and the quality and quantity of patient care. In describing these themes we identify some potential solutions and future directions for reconciling duty hour restrictions with faculty perceptions, anxieties, and desired outcomes.

US dermatology residency program rankings

Unlike many other adult specialties, US News & World Report does not rank dermatology residency programs annually. We conducted a study to rank individual US dermatology residency programs based on set criteria. For each residency program, data from 2008 related to a number of factors were collected, including annual amount of National Institutes of Health (NIH) and Dermatology Foundation (DF) funding received; number of publications from full-time faculty members; number of faculty lectures given at 5 annual society meetings; and number of full-time faculty members who were on the editorial boards of 6 dermatology journals with the highest impact factors. Most of the data were obtained through extensive Internet searches, and missing data were obtained by contacting individual residency programs. The programs were ranked based on the prior factors according to a weighted ranking algorithm. A list of overall rankings also was created.

Improvements in teaching behavior at two German medical schools resulting from a modified Flanders interaction analysis feedback intervention process

Abstract Background: Person-centered teachers who are more empathic and "indirect" (accept, encourage, praise and ask questions) tend to be more effective than those who are "direct" (lecture, give directions and criticize) (Amidon & Flanders 1991). The Flanders Interaction Analysis (FIA) is a tool for diagnosing these teaching aspects, though not yet used to improve lecturing in undergraduate medical education.

AIMS

Does structured expert feedback to volunteer lecturers lead to improvement in person-centered teaching behavior as measured by a Modified Flanders Interaction Analysis (MFIA) and student questionnaires?

METHODS

Twenty-one volunteer lecturers from two German medical faculties were stratified by past teaching experience and randomized into two groups. The intervention group received MFIA diagnoses of their lectures plus feedback by an expert observer after winter and summer semester lectures, respectively. The control group was only diagnosed with the MFIA. Teaching behavior changes for both groups were compared and teacher feedback about the intervention process was assessed.

RESULTS

Faculty in the intervention group improved significantly in their summer lectures regarding person-centered teaching behavior while controls did not.

CONCLUSIONS

A structured individual expert feedback intervention using a MFIA as a teaching diagnostic tool is a powerful, cost-effective faculty development process for improving teaching behavior of volunteer lecturers in undergraduate medical education.

Faculty development on professionalism and medical ethics: the design, development and implementation of Objective Structured Teaching Exercises (OSTEs)

BACKGROUND

As students are expected to develop competency in professionalism and medical ethics, faculty are also expected to facilitate medical students' learning and understanding of these areas. One of the main challenges to success in this domain has been uncertainty of whether or not faculty know the content and the methods to teach and assess these competencies.

AIM

We used the Objective Structured Teaching Exercise (OSTE) format as a faculty development tool to train and evaluate faculty on how to teach professionalism and medical ethics to students in clinical settings.

METHODS

The process for the design, development and implementation of OSTEs consisted of five phases: (1) performing a literature review and student needs assessment, (2) developing the OSTE cases and performance checklists, (3) recruiting and training of standardized students, (4) conducting a mock training session and (5) organizing faculty development workshops using OSTEs.

RESULTS

Twenty clinical faculty members participated in one of three half-day OSTE workshops offered. Faculty confidence and attitudes about teaching professionalism increased significantly (p < 0.05) from before participating in the workshop to afterwards.

CONCLUSIONS

Faculty feedback were positive stating that the OSTE scenarios were reflective of issues they generally encounter while teaching medical students, the information and skills they learned from the workshop are important to them as clinical educators, and that the information and skills will likely have an impact on the way they teach professionalism and ethics in the future.

Assessment of faculty productivity in academic departments of medicine in the United States: a national survey

BACKGROUND

Faculty productivity is essential for academic medical centers striving to achieve excellence and national recognition. The objective of this study was to evaluate whether and how academic Departments of Medicine in the United States measure faculty productivity for the purpose of salary compensation.

METHODS

We surveyed the Chairs of academic Departments of Medicine in the United States in 2012. We sent a paper-based questionnaire along with a personalized invitation letter by postal mail. For non-responders, we sent reminder letters, then called them and faxed them the questionnaire. The questionnaire included 8 questions with 23 tabulated close-ended items about the types of productivity measured (clinical, research, teaching, administrative) and the measurement strategies used. We conducted descriptive analyses.

RESULTS

Chairs of 78 of 152 eligible departments responded to the survey (51% response rate). Overall, 82% of respondents reported measuring at least one type of faculty productivity for the purpose of salary compensation. Amongst those measuring faculty productivity, types measured were: clinical (98%), research (61%), teaching (62%), and administrative (64%). Percentages of respondents who reported the use of standardized measurements units (e.g., Relative Value Units (RVUs)) varied from 17% for administrative productivity to 95% for research productivity. Departments reported a wide variation of what exact activities are measured and how they are monetarily compensated. Most compensation plans take into account academic rank (77%). The majority of compensation plans are in the form of a bonus on top of a fixed salary (66%) and/or an adjustment of salary based on previous period productivity (55%).

CONCLUSION

Our survey suggests that most academic Departments of Medicine in the United States measure faculty productivity and convert it into standardized units for the purpose of salary compensation. The exact activities that are measured and how they are monetarily compensated varied substantially across departments.

Investigating teaching performance in seminars; a questionnaire study with a multi-level approach

BACKGROUND

Teachers play an important role in seminars as facilitators and content experts. However, contextual factors like students' preparation, group size, group interaction, and content appear to influence their performance. Understanding the impact of these contextual factors on students' perception of teaching performance may help to further understand seminar teaching. Besides that, it may help curriculum organisers and teachers to get more insight in how to optimise their versatile role in seminars. The aim of this study is to investigate how students' perception of teaching performance in seminars is explained by students' extent of preparation, seminar group size, group interaction, and content.

METHODS

The Utrecht Seminar Evaluation (USEME) questionnaire was used to collect information on teaching performance and the aforementioned explanatory variables. To account for intra-student, intra-seminar, and intra-teacher correlation in the data, multilevel regression was used to analyse 988 completed questionnaires in 80 seminars with 36 different teachers.

RESULTS

Group interaction and seminar content had large (B = 0.418) and medium (B = 0.212) positive effects on perceived teaching performance scores, whereas the effects of students' preparation (B = -0.055) and group size (B = -0.130) were small and negative.

CONCLUSIONS

This study provides curriculum organisers and teachers indications on how to optimise variables that influence perceived teaching performance in seminars. It is suggested that teachers should search for the most appropriate combination of motivating and challenging content and facilitation method within seminars to optimise discussion opportunities between students.

An instrument for evaluating clinical teaching in Japan: content validity and cultural sensitivity

BACKGROUND

Many instruments for evaluating clinical teaching have been developed but almost all in Western countries. None of these instruments have been validated for the Asian culture, and a literature search yielded no instruments that were developed specifically for that culture. A key element that influences content validity in developing instruments for evaluating the quality of teaching is culture. The aim of this study was to develop a culture-specific instrument with strong content validity for evaluating clinical teaching in initial medical postgraduate training in Japan.

METHODS

Based on data from a literature search and an earlier study we prepared a draft evaluation instrument. To ensure a good cultural fit of the instrument with the Asian context we conducted a modified Delphi procedure among three groups of stakeholders (five education experts, twelve clinical teachers and ten residents) to establish content validity, as this factor is particularly susceptible to cultural factors.

RESULTS

Two rounds of Delphi were conducted. Through the procedure, 52 prospective items were reworded, combined or eliminated, resulting in a 25-item instrument validated for the Japanese setting.

CONCLUSIONS

This is the first study describing the development and content validation of an instrument for evaluating clinical teaching specifically tailored to an East Asian setting. The instrument has similarities and differences compared with instruments of Western origin. Our findings suggest that designers of evaluation instruments should consider the probability that the content validity of instruments for evaluating clinical teachers can be influenced by cultural aspects.

Excellence in clinical teaching: knowledge transformation and development required

CONTEXT

Clinical teachers in medicine face the daunting task of mastering the many domains of knowledge needed for practice and teaching. The breadth and complexity of this knowledge continue to increase, as does the difficulty of transforming the knowledge into concepts that are understandable to learners. Properly targeted faculty development has the potential to expedite the knowledge transformation process for clinical teachers.

METHODS

Based on my own research in clinical teaching and faculty development, as well as the work of others, I describe the unique forms of clinical teacher knowledge, the transformation of that knowledge for teaching purposes and implications for faculty development.

RESULTS

The following forms of knowledge for clinical teaching in medicine need to be mastered and transformed: (i) knowledge of medicine and patients; (ii) knowledge of context; (iii) knowledge of pedagogy and learners, and (iv) knowledge integrated into teaching scripts. This knowledge is employed and conveyed through the parallel processes of clinical reasoning and clinical instructional reasoning. Faculty development can facilitate this knowledge transformation process by: (i) examining, deconstructing and practising new teaching scripts; (ii) focusing on foundational concepts; (iii) demonstrating knowledge-in-use, and (iv) creating a supportive organisational climate for clinical teaching.

CONCLUSIONS

To become an excellent clinical teacher in medicine requires the transformation of multiple forms of knowledge for teaching purposes. These domains of knowledge allow clinical teachers to provide tailored instruction to learners at varying levels in the context of fast-paced and demanding clinical practice. Faculty development can facilitate this knowledge transformation process.

Excellence in clinical teaching: knowledge transformation and development required

CONTEXT

Clinical teachers in medicine face the daunting task of mastering the many domains of knowledge needed for practice and teaching. The breadth and complexity of this knowledge continue to increase, as does the difficulty of transforming the knowledge into concepts that are understandable to learners. Properly targeted faculty development has the potential to expedite the knowledge transformation process for clinical teachers.

METHODS

Based on my own research in clinical teaching and faculty development, as well as the work of others, I describe the unique forms of clinical teacher knowledge, the transformation of that knowledge for teaching purposes and implications for faculty development.

RESULTS

The following forms of knowledge for clinical teaching in medicine need to be mastered and transformed: (i) knowledge of medicine and patients; (ii) knowledge of context; (iii) knowledge of pedagogy and learners, and (iv) knowledge integrated into teaching scripts. This knowledge is employed and conveyed through the parallel processes of clinical reasoning and clinical instructional reasoning. Faculty development can facilitate this knowledge transformation process by: (i) examining, deconstructing and practising new teaching scripts; (ii) focusing on foundational concepts; (iii) demonstrating knowledge-in-use, and (iv) creating a supportive organisational climate for clinical teaching.

CONCLUSIONS

To become an excellent clinical teacher in medicine requires the transformation of multiple forms of knowledge for teaching purposes. These domains of knowledge allow clinical teachers to provide tailored instruction to learners at varying levels in the context of fast-paced and demanding clinical practice. Faculty development can facilitate this knowledge transformation process.

Evaluating educators using a novel toolbox: applying rigorous criteria flexibly across institutions

Valuing faculty as educators is essential for medical schools to fulfill their unique mission of educating physicians. The 2006 Consensus Conference on Educational Scholarship, sponsored by the Association of American Medical Colleges (AAMC) Group on Educational Affairs, provided educators seeking academic promotion with a portfolio-based format for documenting activities in five domains, using evidence of quantity, quality, a scholarly approach, and educational scholarship. Yet, the lack of a rigorous, widely accepted system to assess educator portfolio submissions during the promotion and tenure process continues to impede the ability to fully value educators and educational scholars.The AAMC Task Force on Educator Evaluation was formed in 2010 to establish consensus guidelines for use by those responsible for the rigorous evaluation of the educational contributions of faculty. The task force delineated the educational contributions currently valued by institutions and then fulfilled its charge by creating the Toolbox for Evaluating Educators, a resource which contains explicit evidence-based criteria to evaluate faculty in each of the five domains of educator activity. Adoption of such criteria is now the rate-limiting step in using a fair process to recognize educators through academic promotion. To inform institutional review and implementation of these criteria, this article describes the iterative, evidence- and stakeholder-based process to establish the criteria. The authors advocate institutional adoption of these criteria so that faculty seeking academic promotion as educators, like their researcher colleagues, can be judged and valued using established standards for the assessment of their work.

Unprofessional behavior by specialty: a qualitative analysis of six years of student perceptions of medical school faculty

BACKGROUND

Unprofessional behavior has well documented negative effects both on the clinical care environment and on the learning environment. If unprofessional behavior varies by department or specialty, this has implications both for faculty development and for undergraduate and graduate level training.

AIMS

We sought to learn which unprofessional behaviors were endemic in our school, and which were unique to particular departments.

METHODS

Students graduating from medical school between 2007 and 2012 were asked to complete a questionnaire naming the most professional and least professional faculty members they encountered in during school. For the least professional faculty members, they were also asked to provide information about the unprofessional behavior.

RESULTS

Students noted several types of unprofessional behavior regardless of the department faculty were in; however, there were some behaviors only noted in individual departments. The unprofessional behavior profiles for Surgery and Obstetrics/Gynecology were markedly similar, and were substantially different from all other specialties.

CONCLUSION

Undergraduate, graduate, and faculty education focused on unprofessional behavior that may occur in various learning environments may provide a feasible, practical, and an effective approach to creating a culture of professional behavior throughout the organization.

Using standardized videos to validate a measure of handoff quality: the handoff mini-clinical examination exercise

BACKGROUND

The most recent iteration of the Accreditation Council for Graduate Medical Education duty-hour regulations includes language mandating handoff education for trainees and assessments of handoff quality by residency training programs. However, there is a lack of validated tools for the assessment of handoff quality and for use in trainee education.

METHODS

Faculty at 2 sites (University of Chicago and Yale University) were recruited to participate in a workshop on handoff education. Video-based scenarios were developed to represent varying levels of performance in the domains of communication, professionalism, and setting. Videos were shown in a random order, and faculty were instructed to use the Handoff Mini-Clinical Examination Exercise (CEX), a paper-based instrument with qualitative anchors defining each level of performance, to rate the handoffs.

RESULTS

Forty-seven faculty members (14 at site 1; 33 at site 2) participated in the validation workshops, providing a total of 172 observations (of a possible 191 [96%]). Reliability testing revealed a Cronbach α of 0.81 and Kendall coefficient of concordance of 0.59 (>0.6 = high reliability). Faculty were able to reliably distinguish the different levels of performance in each domain in a statistically significant fashion (ie, unsatisfactory professionalism mean 2.42 vs satisfactory professionalism 4.81 vs superior professionalism 6.01, P < 0.001 trend test). Two-way analysis of variance revealed no evidence of rater bias.

CONCLUSIONS

Using standardized video-based scenarios highlighting differing levels of performance, we were able to demonstrate evidence that the Handoff Mini-CEX can draw reliable and valid conclusions regarding handoff performance. Future work to validate the tool in clinical settings is warranted.

Impact of a formal mentoring program on academic promotion of Department of Medicine faculty: a comparative study

PURPOSE

To evaluate the impact of a formal mentoring program on time to academic promotion and differences in gender-based outcomes.

METHODS

Comparisons of time to promotion (i) before and after implementation of a formal mentoring program and (ii) between mentored and non-mentored faculty matched for covariates. Using paired-samples t-testing and mixed repeated measures ANCOVA, we explored the effect of mentor assignment and influence of gender on time to promotion.

RESULTS

Promotional data from 1988 to 2010 for 382 faculty members appointed before 2003 were compared with 229 faculty members appointed in 2003 or later. Faculty appointed in 2003 or later were promoted 1.2 years (mean) sooner versus those appointed before 2003 (3.7 [SD = 1.7] vs. 2.5 [SD = 2], p < 0.0001). Regardless of year of appointment, mentor assignment appears to be significantly associated with a reduction in time to promotion versus non-mentored (3.4 [SD = 2.4] vs. 4.4 [SD = 2.6], p = 0.011). Gender effects were statistically insignificant. Post hoc analyses of time to promotion suggested that observed differences are not attributable to temporal effects, but rather assignment to a mentor.

CONCLUSIONS

Mentoring was a powerful predictor of promotion, regardless of the year of appointment and likely benefited both genders equally. University resource allocation in support of mentoring appears to accelerate faculty advancement.

The potential impact of the revision of the Basic World Federation Medical Education Standards

BACKGROUND

The World Federation of Medical Education has released a revised version of their Basic Medical Education Standards. This paper compares the original and revised versions following an external review of a medical school based on the original version, and indicates potential implications of the changes for making judgements about the quality of medical education.

METHODS

A comparative analysis was conducted of the original and revised standards, producing a list of changes, through either "strengthened" or "new" standards or guidance for their interpretation. The potential impact of the changes on the writing of a report for this external review was then considered.

RESULTS

The revised standards have more specific requirements for curriculum design, clinical placements, research training and the support and participation of students. In order to meet the revised standards, medical schools may need to invest substantially in acquiring in-house medical education expertise and better prepared teaching and research staff.

CONCLUSION

The bar appears to have been raised substantially in the revised standards document. While individually the changes appear sound, the overall impact, particularly in the developing world, may be substantial.

The use of abstract paintings and narratives to foster reflective capacity in medical educators: a multinational faculty development workshop

Reflective capacity is integral to core healthcare professional practice competencies. Reflection plays a central role in teacher education as reflecting on teaching behaviours with critical analysis can potentially improve teaching practice. The humanities including narrative and the visual arts can serve as a valuable tool for fostering reflection. We conducted a multinational faculty development workshop aiming to enhance reflective capacity in medical educators by using a combination of abstract paintings and narratives. Twenty-three family physicians or physicians-in-training from 10 countries participated in the workshop. Qualitative assessment of the workshop showed that the combined use of art and narrative was well received and perceived as contributing to the reflective exercise. Participants generally felt that viewing abstract paintings had facilitated a valuable mood transformation and prepared them emotionally for the reflective writing. Our analysis found that the following themes emerged from participants' responses: (1) narratives from different countries are similar; (2) the use of art helped access feelings; (3) viewing abstract paintings facilitated next steps; (4) writing reflective narratives promoted examination of educational challenges, compassion for self and other, and building an action plan; and (5) sharing of narrative was helpful for fostering active listening and appreciating multiple perspectives. Future research might include comparing outcomes for a group participating in arts-narrative-based workshops with those of a control group using only reflective narrative or in combination with figurative art, and implementing a combination of qualitative and quantitative methods of assessment.

The ADMSEP Education Scholars Program: a novel approach to cultivating scholarship among psychiatry educators

The author describes the Association of Directors of Medical Student Education in Psychiatry's Education Scholars Program, a 2-year longitudinal, guided mentorship program, anchored by didactic instruction in the fundamental concepts of educational scholarship.

Exploring interprofessional education in the family medicine clerkship: a CERA study

BACKGROUND AND OBJECTIVES

The need for interprofessional education (IPE) to improve teamwork skills has been recognized by several national organizations. The purpose of this study was to investigate IPE integration in family medicine clerkships and factors associated with IPE's success.

METHODS

A survey of clerkship directors at US allopathic medical schools was conducted through the Council of Academic Famiily Medicine Educational Research Alliance (CERA). Respondents were asked (1) whether IPE was part of the curriculum, (2) the educational methods used, (3) which health professions students and faculty participated in IPE, (4) what outcomes were measured, (5) the types of faculty development provided, and (6) the barriers encountered when implementing IPE.

RESULTS

The response rate was 66% (88/134), and 38% reported incorporating IPE into the clerkship with most offering IPE in clinical activities. A wide variety of health professions students and faculty participated in clerkship IPE activities. One third of the respondents offered faculty development. Most agreed that third party funding (85%), IPE team training (94%), clearly defined roles (94%), and dedicated time during clinical care for team meetings (93%) were vital for IPE to succeed. Many programs did not measure IPE-specific outcomes (49%). Eighty percent reported at least one barrier to implementing IPE. The most common barriers were scheduling conflicts (46%) and lack of IPE experience (40%). No one reported a lack of institutional support for IPE.

CONCLUSIONS

Few clerkships offered IPE. However, family medicine is in a unique position to highlight the value of interprofessional teamwork for students and should recognize and promote IPE opportunities.

What do trainers think about trainer training courses?

OBJECTIVE

To evaluate the effectiveness of the training-of-trainer courses given to medical school faculty members, and to ensure the standardisation of training..

METHODS

The study comprised faculty members attending seven training-of-trainer courses held at the Ataturk University, Turkey, from November 2010 to May 2011. Tests were administered to the participants evaluating their level of knowledge on course content before and after the five-day course, which covered topics including concepts of teaching/learning, curriculum development, assessment and evaluation, training methods, and training skills. Oral and written feedbacks were obtained from all participants on the last day of each course. Volunteers from among the participants evaluated the impact of training through a questionnaire after at least two months. SPSS 20 was used for statistical analysis.

RESULTS

A total of 136 faculty members participated in the 7 training-of-trainer courses. The mean scores for the pretest and post-test were 6.6 +/- 2.2 vs. 13.9 +/- 3.7, (p < 0.001). Questions with highest percentage of correct answers in the pre-test were those about learning style 78 (70.9%) and the one about presentation skills (66.4%).

CONCLUSION

A structured training-of-trainer course is effective in increasing faculty members' level of knowledge about training. Such programmes are necessary for faculty members who work in the academic field without receiving formal training in teaching as it is the case with medical education in several countries.