Evaluating the performance of medical educators: a novel analysis tool to demonstrate the quality and impact of educational activities

Advocacy in Pediatric Academia: Charting a Path Forward

Pediatricians are effective advocates to improve the health and well-being of children, yet there are limited avenues by which to pursue academic promotion based on these activities. Drawing on an expanded definition of scholarship, pediatric advocates can use the portfolio format to highlight the quantity, quality, and impact of advocacy activities. True congruence with research and education will only be achieved through recognition and value by institutions and organizations.

Advancing the Clinician Educator Career Pathway in Oncology

PURPOSE:

Education is an important component of cancer care; however, most clinician educators (CEs) receive little formal training in this area. Little is known about the factors that influence oncologists to pursue a career as a CE. The primary objective of this study was to determine the current state of oncologists' perceptions regarding the clinician educator role.

MATERIALS AND METHODS:

A one-time cross-sectional survey was administered to program directors/associate program directors (PDs/APDs) and fellows in November 2021. The survey was meant to elicit their perceptions regarding the CE role, training opportunities, and barriers to a career as a CE.

RESULTS:

The surveys were completed by a total of 2,134 oncology fellows and 88 PDs/APDs. Most PDs/APDs were female (52%), were associate professors (42%), and considered themselves a CE (82%). Over one-third of PDs/APDs reported no formal educator training (67%) and did not have a CE track for fellows at their institution (76%). The majority of PDs/APDs (80%) perceived the CE track to be a viable career pathway. Over half of fellows (56%) perceived the CE track to be a viable career pathway. Approximately one-third (62%) reported receiving CE training during their residency/fellowship. The top reported barriers to a career in medical education were a lack of jobs and opportunity for future promotions.

CONCLUSION:

Oncology PDs/APDs and fellows perceive the CE to be a viable career track. Greater advocacy efforts are needed to raise awareness about this career path.

Integrating Graduates of a National Faculty Development Program Into a Community of Practice

PURPOSE

The Educational Scholars Program (ESP) started as a national 3-year faculty development (FD) program for pediatric educators. Fourteen years later, the ESP has grown into a larger community of practice (CoP). The purpose of this study was to identify programmatic processes that serve as bridges or barriers to integrating ESP graduates into the CoP.

METHODS

A cross-sectional survey was sent to 126 ESP graduates. Based on their self-reported level of engagement, graduates were categorized into 4 groups: Core, Active, Peripheral, and Outsider groups. Factors contributing to engagement with the ESP community, and the impact of varying levels of engagement on outcomes related to the organization and the graduate were ascertained.

RESULTS

Half of the ESP graduates completed the survey. Thirty-five percent of the respondents were in the Core or Active groups. Almost all ESP graduates in the Core and Active groups felt a sense of belonging to the ESP community. Opportunities provided to network, collaborate, and receive mentorship were reported as highly significant influences on their decision to remain engaged.

CONCLUSION

Purposeful integration of graduates of an FD program within a CoP, by allowing varying levels of involvement, enhances engagement within the CoP.

Establishing an educational value unit to promote teaching in an academic unit

BACKGROUND

In academic health centers, education remains an incompletely supported and funded mandate. In an attempt to promote education and better support educational endeavors of faculty, some academic health centers and departments have conceived of a metric, the educational value unit (eVU), to begin to "quantify" teaching. What goes into this metric, its intended goals and the logistics of its implementation vary considerably among centers.

LESSONS LEARNED

This practical advice paper highlights the various lessons learned from a review of the limited published literature on eVU systems supplemented with our personal experience in implementing a successful eVU system in the Department of Pediatrics at our institution, to help guide others who may be interested in doing that same. Even in limited-resource settings, our hope is that these lessons can serve as a guide on how to better quantify and reward teaching, whether through monetary or nonfiscal incentives and recognition.

A Systematic Scoping Review on Portfolios of Medical Educators

BACKGROUND

Heralded as a teaching, assessment and reflective tool, and increasingly as a longitudinal and holistic perspective of the educator's development, medical educator's portfolios (MEP)s are increasingly employed to evaluate progress, assess for promotions and career switches, used as a reflective tool and as a means of curating educational activities. However, despite its blossoming role, there is significant dissonance in the content and structure of MEPs. As such, a systematic scoping review (SSR) is proposed to identify what is known of MEPs and its contents.

METHODS

Krishna's Systematic Evidenced Based Approach (SEBA) was adopted to structure this SSR in SEBA of MEPs. SEBA's constructivist approach and relativist lens allow data from a variety of sources to be considered to paint a holistic picture of available information on MEPs.

RESULTS

From the 12 360 abstracts reviewed, 768 full text articles were evaluated, and 79 articles were included. Concurrent thematic and content analysis revealed similar themes and categories including: (1) Definition and Functions of MEPs, (2) Implementing and Assessing MEPs, (3) Strengths and limitations of MEPs and (4) electronic MEPs.

DISCUSSION

This SSR in SEBA proffers a novel 5-staged evidence-based approach to constructing MEPs which allows for consistent application and assessment of MEPs. This 5-stage approach pivots on assessing and verifying the achievement of developmental milestones or 'micro-competencies' that facilitate micro-credentialling and effective evaluation of a medical educator's development and entrust-ability. This allows MEPs to be used as a reflective and collaborative tool and a basis for career planning.

Are Women Disproportionately Represented in Education Compared to Other Roles in Academic Radiology?

RATIONALE AND OBJECTIVES

Women in academic medicine, and radiology specifically, are underrepresented in departmental leadership roles and achieve fewer professional metrics of success. We have observed, however, that women are more broadly represented in medical education leadership. The purpose of this study was to determine if women in academic radiology are overrepresented in educational scholarship and educational leadership compared to general research scholarship and leadership positions, and to determine if there is any difference in the distribution of women in these roles compared to internal medicine.

MATERIALS AND METHODS

We performed a PubMed search of education articles in radiology and internal medicine over the last 5 years, and the gender of the authors was determined. Data on gender for authorship of general research topics, departmental leaders, and society leadership was obtained via literature and internet searches and the data was aggregated. Representation ratios (RR of 1.0 = parity) were obtained via risk ratio calculation to compare education versus general scholarship, and the distribution of leadership roles within and between these fields.

RESULTS

Women make up 28.5% of academic radiologists and 40.1% of academic internists. A higher proportion of education articles were first authored by women than would be expected in both fields with an RR of 1.46 (p < 0.001) in radiology and 1.23 (p < 0.001) in internal medicine. This overrepresentation was significant compared to general research scholarship in both fields (p < 0.001). In both fields, women were overrepresented in the position of Medical Student Director (RR of 1.47 and 1.22, respectively). For Program Directors, women were overrepresented in radiology (RR of 1.12) and underrepresented in internal medicine (RR of 0.69-0.75). Women in radiology were overrepresented in education society leadership (RR = 1.63) compared to general society leadership (0.98, p = 0.001).

CONCLUSION

We found that women in radiology are overrepresented in authorship of education articles, and in departmental and society education leadership roles. This trend was also seen in internal medicine, suggesting that women are more broadly represented in medical education.

An Inter-Institutional External Peer-Review Process to Evaluate Educators at Schools of Veterinary Medicine

Despite its fundamental importance, the educational mission of most schools of veterinary medicine receives far less recognition and support than the missions of research and discovery. This disparity is evident in promotion and tenure processes. Despite the frequent assertion that education is every college's core mission, there is a broad consensus that faculty are promoted primarily on the basis of meeting expectations relative to publications and grant funding. This expectation is evident in the promotion packets faculty are expected to produce and the criteria by which those packets are reviewed. Among the outcomes is increasing difficulty in hiring and retaining faculty, including young clinicians and basic scientists who are drawn to academic institutions because of the opportunity to teach. The Regional Teaching Academy (RTA) of the West Region Consortium of Colleges of Veterinary Medicine initiated an inter-institutional collaboration to address the most important obstacles to recognizing and rewarding teaching in its five member colleges. Working from the medical education literature, the RTA developed an Educator's Promotion Dossier, workshops to train promotion applicants, and an external review process. Initial use has shown that the reviews are efficient and complete. Administrators have expressed strong support for the product, a letter of external review that is returned to a promotion applicant's home institution. The overall result is an evidence-based, structured process by which teaching-intensive faculty can more fully document their achievements in teaching and educational leadership and a more rigorous external review process by which member colleges can assess quality, impact, and scholarly approach.

How are clinician-educators evaluated for educational excellence? A survey of promotion and tenure committee members in the United States

Background: In recent years, educational leaders have proposed domains of educational excellence and corresponding metrics to objectively measure contributions of clinician-educators for promotion and tenure (P&T). The purpose of this study was to explore whether P&T committees in United States (US) have incorporated these recommendations into practice. Method: The authors conducted a survey of P&T leaders across institutions in US. Items included questions related to institutional tracks for P&T, domains included in promotional packets, metrics for their measurement, and use of an Educator's Portfolio (EP). Results: Respondents from 55 institutions completed the survey. The presence of a teaching academy/society was associated with the presence of a promotion track for clinician-educators (p = 0.04). Only teaching activities (91%), assessment of learners (55%), and educational scholarship (51%) were required by a majority of institutions. Few institutions used objective methods for measuring impact and less than half (47%) required an EP. Discussion: These results highlight both progress in the recognition of clinician-educators while also suggesting discordance in the perspective of educational leaders and the practice of P&T committees. The authors advocate for establishing a national community of expert medical educators who may assist P&T committees in adopting consensus-based criteria and metrics to evaluate clinician-educators' contributions.

Transitioning From Radiology Training to Academic Faculty: The Importance of Mentorship

Transitioning from radiology residency to academic faculty presents many challenges. In this review, we discuss the importance of introspection and mentorship to successfully navigate this process. Key points include alignment of goals with those of the institution, formation of a mentorship program, and periodic reassessment of career goals. These tips and tools can help make the transition from residency to academic faculty more seamless.

Blockchain Technology: A Data Framework to Improve Validity, Trust, and Accountability of Information Exchange in Health Professions Education

Health professions educators face multiple challenges, among them the need to adapt educational methods to new technologies. In the last decades, multiple new digital platforms have appeared in the learning arena, including massive open online courses and social-media-based education. The major critique of these novel methods is the lack of the ability to ascertain the origin, validity, and accountability of the knowledge that is created, shared, and acquired. Recently, a novel technology based on secured data storage and transmission, called blockchain, has emerged as a way to generate networks where validity, trust, and accountability can be created. Conceptually, blockchain is an open, public, distributed, and secure digital registry where information transactions are secured and have a clear origin, explicit pathways, and concrete value. Health professions education based on blockchain will potentially allow improved tracking of content and the individuals who create it, quantify educational impact on multiple generations of learners, and build a relative value of educational interventions. Furthermore, institutions adopting blockchain technology would be able to provide certification and credentialing of health care professionals with no intermediaries. There is potential for blockchain to significantly change the future of health professions education and radically transform how patients, professionals, educators, and learners interact around safe, valid, and accountable information.

Using the METRICS model for defining routes to scholarship in healthcare simulation

INTRODUCTION

In this paper, we explored the utility and value of the METRICS model for modeling scholarship in healthcare simulation by: (1) describing the distribution of articles in four healthcare simulation journals across the seven areas of METRICS scholarship; and (2) appraising patterns of scholarship expressed in three programs of simulation scholarship and reflecting on how these patterns potentially influence the pursuit of future scholarly activities.

METHODS

Two raters reviewed abstracts of papers published between January 2015 and August 2017 in four healthcare simulation journals and coded them using METRICS. Descriptive statistics were calculated for scholarship type and distribution across journals. Twenty-eight articles from three scholars were reviewed, with patterns of scholarship within articles mapped to METRICS. Descriptive synthesis was constructed through discussion between two reviewers.

RESULTS

A total of 432 articles from four journals were reviewed. The three most commonly published areas of scholarship were: 32.2% (139/432) evaluation, 18.8% (81/432) innovation, and 15.3% (66/432) conceptual. The METRICS model was able to represent different kinds of scholarship expressed in all of the papers reviewed and across programs of research. Reflecting on patterns of scholarship within their scholarly programs was helpful for research in planning future directions.

CONCLUSIONS

The METRICS model for scholarship can describe a wide range of patterns of simulation scholarship within individual articles, programs of research, or across journals.

The Advocacy Portfolio: A Standardized Tool for Documenting Physician Advocacy

Recent changes in health care delivery systems and in medical training have primed academia for a paradigm shift, with strengthened support for an expanded definition of scholarship. Physicians who consider advocacy to be relevant to their scholarly endeavors need a standardized format to display activities and measure the value of health outcomes to which their work can be attributed. Similar to the Educator Portfolio, the authors here propose the Advocacy Portfolio (AP) to document a scholarly approach to advocacy.Despite common challenges faced in the arguments for both education and advocacy to be viewed as scholarship, the authors highlight inherent differences between the two fields. On the basis of prior literature, the authors propose a broad yet comprehensive set of domains to categorize advocacy activities, including advocacy engagement, knowledge dissemination, community outreach, advocacy teaching/mentoring, and advocacy leadership/administration. Documenting quality, quantity, and a scholarly approach to advocacy within each domain is the first of many steps to establish congruence between advocacy and scholarship for physicians using the AP format.This standardized format can be applied in a variety of settings, from medical training to academic promotion. Such documentation will encourage institutional buy-in by aligning measured outcomes with institutional missions. The AP will also provide physician-advocates with a method to display the impact of advocacy projects on health outcomes for patients and populations. Future challenges to broad application include establishing institutional support and developing consensus regarding criteria by which to evaluate the contributions of advocacy activities to scholarship.

Twelve tips for creating an academic teaching portfolio

An academic teaching portfolio is not only a requirement at many academic teaching institutions, but it is also important in a medical educator's growth and development through documentation, reflection, evaluation, and change. Creating an academic portfolio may appear daunting at first but with careful advanced preparation, organized evidence collection of your educational work, proof of scholarship, and thorough documentation of self-reflection and change, you can produce a successful product that accurately represents your educational beliefs, accomplishments, and growth throughout your career. This article provides medical educators with twelve steps for creating a successful academic teaching portfolio.

Educational Added Value Unit: Development and Testing of a Measure for Educational Activities

OBJECTIVES

University-based hospitalists educate health care professionals as an expectation, often lacking time and support for these activities. The purpose of this study was to (1) develop a tracking tool to record educational activities, (2) demonstrate its applicability and ease of completion for faculty members in different divisions, and (3) compare educational efforts of individuals from different professional pathways and divisions by using the educational added value unit (EAVU).

METHODS

Educational activities were selected and ranked according to preparation effort, presentation time, and impact to calculate the EAVU. Faculty participants from 5 divisions at 1 institution (hospital medicine, general and community pediatrics, emergency medicine, behavior medicine and clinical psychology, and biostatistics and epidemiology) completed the retrospective, self-report tracking tool.

RESULTS

A total of 62% (74 of 119) of invited faculty members participated. All faculty earned some EAVUs; however, there was a wide distribution range. The median EAVU varied by division (hospital medicine [21.7], general and community pediatrics [20.6], emergency medicine [26.1], behavior medicine and clinical psychology [18.3], and biostatistics and epidemiology [8.2]). Faculty on the educator pathway had a higher median EAVU compared with clinical or research pathways.

CONCLUSIONS

The EAVU tracking tool holds promise as a mechanism to track educational activities of different faculty pathways. EAVU collection could be of particular benefit to hospitalists, who often perform unsupported teaching activities. Additional studies are needed to determine how to apply a similar process in different institutions and to determine how EAVUs could be used for additional support for teaching, curriculum development, and educational scholarship.

The Impact of a National Faculty Development Program Embedded Within an Academic Professional Organization

A sizeable literature describes the effectiveness of institution-based faculty development programs in nurturing faculty educators as scholars, but national programs are less common and seldom evaluated. To fill this role, the Educational Scholars Program (ESP) was created within the Academic Pediatric Association (APA) in 2006. It is a national, three-year, cohort-based certification program focused on fostering educational scholarship. This article describes the development and outcomes of an innovative program embedded within the framework of a national professional organization, and offers a model for potential adaptation by similar organizations to enhance their support of educators.After 10 years, 171 scholars have enrolled in the ESP, and 50 faculty have participated. Scholars are assigned a faculty advisor and participate in three full-day sessions at a national meeting; online, interactive learning modules; and a mentored, scholarly project. The program receives support from the APA in four organizational frames: structural, human resource, political, and symbolic. The self-perceived scholarly proficiency of the scholars in Cohort 1 increased significantly over time, and their productivity and collaborations increased during and after the program. Scholars wrote enthusiastically about their experience in yearly and postprogram evaluations. In interviews, eight past APA presidents explained that the ESP strengthened the APA's mission, created new leaders, and provided a new model for other APA programs. Outcomes of the ESP suggest that a longitudinal faculty development program embedded within a national professional organization can create a social enterprise not only within the organization but also within the broader national community of educator-scholars.

Influences of faculty evaluating system on educational performance of medical school faculty

PURPOSE

The promotion of educators is challenged by the lack of accepted standards to evaluate the quality and impact of educational activities. Traditionally, promotion is related to research productivity. This study developed an evaluation tool for educational performance of medical school faculty using educator portfolios (EPs).

METHODS

Design principles and quantitative items for EPs were developed in a consensus workshop. These principles were tested in a simulation and revised based on feedback. The changes of total educational activities following introduction of the system were analyzed.

RESULTS

A total of 71% faculty members answered the simulation of the system and the score distributed widely (mean±standard deviation, 65.43±68.64). The introduction of new system significantly increased the total educational activities, especially in assistant professors.

CONCLUSION

The authors offer comprehensive and practical tool for enhancing educational participation of faculty members. Further research for development of qualitative evaluation systems is needed.

Education scholarship in emergency medicine part 2: supporting and developing scholars

Emergency medicine (EM) is defined, in part, by clinical excellence across an immense breadth of content and the provision of exemplary bedside teaching to a wide variety of learners. The specialty is also well-suited to a number of emerging areas of education scholarship, particularly in relation to team-based learning, clinical reasoning, acute care response, and simulation-based teaching. The success of EM education scholarship will be predicated on systematic, collective attention to providing the infrastructure for this to occur. Specifically, as a new generation of emergency physicians prepares for education careers, academic organizations need to develop means not only to identify potential scholars but also to mentor, support, and encourage their careers. This paper summarizes the supporting literature and presents related recommendations from a 2013 consensus conference on EM education scholarship led by the Academic Section of the Canadian Association of Emergency Physicians.

Toward a common understanding: supporting and promoting education scholarship for medical school faculty

CONTEXT

Education scholarship (ES) is integral to the transformation of medical education. Faculty members who engage in ES need encouragement and recognition of this work. Beginning with the definition of ES as 'an umbrella term which can encompass both research and innovation in health professions education', and which as such represents an activity that is separate and distinct from teaching and leadership, the purpose of our study was to explore how promotion policies and processes are used in Canadian medical schools to support and promote ES.

METHODS

We conducted an analysis of the promotion policies of 17 Canadian medical schools and interviews with a key informant at each institution. We drew on an interpretive approach to policy analysis to analyse the data and to understand explicit messages about how ES was represented and supported.

RESULTS

Of the 17 schools' promotion documents, only nine contained specific reference to ES. There was wide variation in focus and level of detail. All key informants indicated that ES is recognised and considered for academic promotion. Barriers to the support and recognition of ES included a lack of understanding of ES and its relationship to teaching and leadership. This was manifest in the variability in promotion policies and processes, support systems, and career planning and pathways for ES.

CONCLUSIONS

This lack of clarity may make it challenging for medical school faculty members to make sense of how they might successfully align ES within an academic career. There is a need therefore to better articulate ES in promotion policies and support systems. Creating a common understanding of ES, developing guidelines to assess the impact of all forms of ES, developing an informed leadership and system of mentors, and creating explicit role descriptions and guidelines are identified as potential strategies to ensure that ES is appropriately valued.

Medical education scholarship: an introductory guide: AMEE Guide No. 89

Abstract This AMEE Guide provides an overview of medical education scholarship for early career scholars, based upon a summary of the existing literature and pragmatic advice derived from the experience of its authors. After providing an introduction to the principles of scholarship and describing questions that the Guide addresses, the authors offer a conceptual description of the complementary traditions of teaching and educational discovery, and advocate for the development of educational scholars with both traditions. They then describe the attributes of effective mentor-mentee relationships and how early career scholars can identify potential mentors who can fulfill this role. In the subsequent sections, they describe the appropriate development of scholarly questions and other components of a complete scholarly plan, including how to use conceptual frameworks in guiding such plans. From here, they describe methods that align with both the teaching and discovery traditions and provide concrete examples of each. They then provide guidelines for assessing the impact of scholarship, identify the various opportunities for sharing it, and how to effectively interpret and describe it. Additionally, they provide practical advice on how appropriately to demonstrate the scholarship in a promotional packet, including the principle of reflectivity in scholarship. Finally, they address the principles of applied research ethics for educational scholarship and when to consider soliciting approval for scholarly activities by a human research board.

Pediatric hospitalists and medical education

Pediatric hospital medicine (PHM) is moving toward becoming an American Board of Pediatrics (ABP) subspecialty, roughly a decade after its formal inception in 2003. Education has played a central role as the field has evolved. Hospitalists are needed to educate trainees, medical students, residents, fellows, and nurse practitioner and physician assistant students in inpatient pediatric practice. Continuous professional development is needed for hospitalists currently in practice to augment clinical skills, such as providing sedation and placing peripherally inserted central catheter lines, and nonclinical skills in areas such as quality improvement methodology, hospital administration, and health service research. To address the educational needs of the current and future state of PHM, additional training is now needed beyond residency training. Fellowship training will be essential to continue to advance the field of PHM as well as to petition the ABP for specialty accreditation. Training in using adult educational theory, curriculum, and assessment design are critical for pediatric hospitalists choosing to advance their careers as clinician-educators. Several venues are available for gaining advanced knowledge and skill as an educator. PHM clinician-educators are advancing the field of pediatric education as well as their own academic careers by virtue of the scholarly approach they have taken to designing and implementing curricula for unique PHM teaching situations. PHM educators are changing the educational paradigm to address challenges to traditional education strategies posed by duty hour restrictions and the increasing drive to shorten the duration of the hospitalization. By embracing learning with technology, such as simulation and e-learning with mobile devices, PHM educators can address these challenges as well as respond to learning preferences of millennial learners. The future for PHM education is bright.

Further dissemination of medical education projects after presentation at a pediatric national meeting (1998-2008)

BACKGROUND

Further dissemination of medical education work presented at national meetings is limited.

PURPOSES

The purpose of this study was to explore dissemination outcomes of scholarly work in pediatric medical education.

METHODS

Council on Medical Student Education in Pediatrics (COMSEP) members who presented at COMSEP national meetings from 1998 to 2008 received a questionnaire about scholarly dissemination outcomes. Descriptive statistics and chi-square analysis explored variables related to dissemination. Qualitative analysis of free text comments explored barriers to dissemination.

RESULTS

Outcomes were determined for 81% of presentations (138/171). The dissemination rate was 67% (92/138 presentations), with 47 publications (34%). Dissemination rates did not vary by presentation type (poster vs. oral) or project type. There was no relationship between presentation type, project type, and dissemination method. Barriers included perceived inadequate time, mentorship, and methodological skills for scholarly work.

CONCLUSIONS

Most projects were further disseminated. Additional resources including mentoring and protected time for scholarly work are needed by educators to optimize dissemination.

Keynote address: the focus on competencies and individual learner assessment as emerging themes in medical education research

This article describes opportunities for scholarship in medical education, based on a brief overview of recent changes in medical education. The implications arising from these changes are discussed, with recommendations for focus, and suggestions and examples for making progress in this field. The author discusses 1) the historical context of the current shift toward competency-based medical education, 2) the potential contribution of social and behavioral sciences to medical education scholarship, 3) methods and approaches for supporting scholarship in medical education, and very briefly 4) trends in simulation. The author concludes with a call for quality in medical education scholarship and argues that the most promising and fruitful area of medical education scholarship for the future lies in the field of assessment of individual competence.

Academic advancement of clinician educators: why is it so difficult?

The role of clinician educators (CEs) in institutions and medical centres continues to be vital without any doubt. Although there has been more than a century since Sir William Osler established the role of the CE and the tradition by encouraging bed-side teaching, there is still a lack of consensus on the attributes that define a 'clinician-educator'. The concept of a superior clinician who is also a dedicated teacher seems to fit the description of a CE but most often seems insufficient to support the CE's academic advancement.

To the point: medical education reviews-ongoing call for faculty development

This article in the To the Point series will focus on best practices regarding faculty development in medical education in the field of obstetrics and gynecology. Faculty development is an essential component in achieving teacher and learner satisfaction as well as improving learner outcomes. The Liaison Committee on Medical Education requires medical school faculty to have the capability and longitudinal commitment to be effective teachers. Although many programs have been created to address faculty development, there remains a paucity of literature documenting the impact of these programs on learner outcomes. We reviewed the qualities of an excellent medical educator, expectations regarding medical school teaching faculty, elements of comprehensive faculty development programs, and outcome measures for evaluating the effectiveness of these programs.

Guidelines for evaluating the educational performance of medical school faculty: priming a national conversation

BACKGROUND

The academic community needs a sound framework for the promotion and advancement of educators. The Group on Educational Affairs of the Association of American Medical Colleges organized a consensus conference that affirmed the use of five domains for documenting the quantity and quality of scholarly engagement in educational activities: teaching, curriculum, advising/mentoring, educational leadership/administration, and learner assessment.

SUMMARY

In this article, we offer detailed guidelines to evaluate these five domains of educator performance and the essential elements of scholarly activity. The guidelines are adapted from our developmental educator portfolio template and educator portfolio analysis tool, previously published in MedEdPORTAL. A short tool for educator performance evaluation that summarizes items in the guidelines is proposed for discussion.

CONCLUSIONS

Our goal in this article is to itemize criteria for systematic faculty evaluation that can be applied in any institutional setting to assist promotion decision makers in their task of evaluating medical school faculty.

Tracking the careers of academic general pediatric fellowship program graduates: academic productivity and leadership roles

OBJECTIVE

Little is known about the careers of graduates of academic general pediatric (AGP) fellowship programs. We evaluated the careers of 2 cohorts of AGP fellowship graduates: an early cohort trained during 1978 to 1988, and a later cohort trained during 1989 to 1999.

METHODS

We surveyed all known AGP fellowship graduates in both cohorts by using a confidential mailed survey. We assessed graduates' current professional work and analyzed curricula vitae for principal investigator (PI) grants; first-authored, peer-reviewed publications; and leadership positions.

RESULTS

From the early cohort, 95 of 131 eligible graduates (73%) responded; from the later cohort, 93 of 133 (70%) responded. Two thirds of each cohort remain in academics; of these, nearly half are on tenure tracks and over half have major educational roles within their university. The percentage in the early cohort who have been PI on a research grant by 5, 10, and 15 years postfellowship was 44%, 53%, and 54%, respectively; in the later cohort, it was 62%, 75%, and 75%, respectively (P = .004 vs early cohort). During the 10 years postfellowship, the early and later cohorts averaged 5.5 and 7.4 first-authored, peer-reviewed papers, respectively (P = .4). By 10 years, a high proportion of both cohorts had become division chief (19% vs 16%), had other academic leadership positions (43% vs 59%), or were leaders in professional organizations (20% vs 30%; all P = NS).

CONCLUSIONS

Graduates of AGP fellowship programs have achieved considerable academic success. Recently trained fellows appear even more successful. The academic outcomes of these AGP fellows bode well for the future of AGP.

Developing and testing an evaluation tool to measure clinical teaching

PURPOSE

Teaching evaluation tools can be used to provide feedback to medical educators and help them improve their teaching skills. The purpose of this study was to develop a clinical teaching evaluation tool and test its efficacy.

METHODS

The draft tool was developed based on clinical education literature and was modified by the stakeholder groups. To examine the quality of the instrument, content and construct validity, as well as reliability and fitness were analyzed. Content validity was tested by the stakeholder groups using a survey, and construct validity was verified by confirmatory factor analysis using LISREL 8.8. Internal consistency of items was assessed thorough Cronbach alpha estimation. Rasch analysis using Winstep 3.65 was performed to estimate the fitness of the tool.

RESULTS

The resulting tool consisted of 4 large categories, 25 small categories, and 43 items. According to the test results, the average importance of all 43 items was 4.03 (3.63 to 4.29). Cronbach alpha was 0.9689, and the correlation coefficients between the items were high. With regard to construct validity, 10 items needed minor modifications in the category setting. As the infit (0.76 to 1.23) and outfit (0.75 to 1.40) indices show, 42 items were fit to the item response theory.

CONCLUSION

The clinical teaching evaluation tool that has been developed in this study is valid and reliable and fits the item response theory. It can be used as an evaluation method in a variety of clinical teaching settings.