Critical Thoughts About the Core Entrustable Professional Activities in Undergraduate Medical Education

Identifying emergency medicine program directors' expectations of competence upon entry into residency: Bridging the distance from the Association of American Medical Colleges Core Entrustable Professional Activities

Background

Residency program directors (PDs) frequently describe students as unprepared for the patient care responsibilities expected of them upon entry into residency. The Association of American Medical Colleges (AAMC) developed the Core Entrustable Professional Activities (Core EPAs) to address this concern by defining 13 tasks students should be able to do with minimal supervision upon graduation. However, the Core EPAs remain difficult for PDs to use due to their breadth and lack of granularity.

Methods

Using Delphi consensus methodology, we identified granular observable practice activities (OPAs) that PDs expect of entering interns derived from the Core EPAs. Twelve emergency medicine education experts drafted OPAs based on the Core EPAs and their associated core functions described in one-page schematics. A separate group of 12 PDs underwent three rounds of voting, and consensus for inclusion was set at 70%. Thematic analysis of comments discussing votes was performed using an inductive approach.

Results

A total of 321 OPAs were drafted and 127 adopted as expectations for entering interns based on the Core EPAs. The adopted OPAs were all general expectations; none were specialty-specific. Four main themes were identified from the comments: Schools are not responsible for specialty-specific training, PDs do not fully trust schools' assessments, supervision expectations of graduates should be lowered for higher-order EPAs, and the context in which the student performs a task and its associated complexity matter greatly in entrustment decisions.

Conclusions

PDs agree with the generalist focus of the AAMC Core EPAs and feel strongly that specialty training should be left to residency programs. They also have mechanisms in place to verify entrustment within their local context. Transparency in assessment and summative entrustment processes in UME may unify expectations. Finally, the granularity of OPAs may aid in a post-Match handover to further operationalize the EPAs and optimize the UME-to-GME transition.

Domains Influencing Faculty Decisions on the Level of Supervision Required for Anesthesiology EPAs with Analysis of Feedback Comments

OBJECTIVE

The purpose of this qualitative study was to examine responses related to entrustment and feedback comments from an assessment tool.

DESIGN

Qualitative analyses using semi-structured interviews and analysis of narrative comments.

SETTING

Main hospital OR suite at a large academic medical center.

PARTICIPANTS

faculty, and residents who work in the OR suite.

RESULTS

Seven of the 14 theoretical domains from the Theoretical Domains Framework were identified as influencing faculty decision on entrustment: knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. The majority (651/1116 (58.4%)) of faculty comments were critical/modest praise and relevant, consistent across all 6 EPAs. The written in feedback comments for all 1,116 Web App EPA assessments yielded a total of 1,599 sub-competency specific responses. These responses were mapped to core competencies, and at least once to 13 of the 23 ACGME subcompetencies.

CONCLUSIONS

Domains identified as influencing faculty decision on entrustment were knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. Most narrative feedback comments were critical/modest praise and relevant, consistent across each of the EPAs.

Factors associated with academic performance among medical students at a medical school in South Korea: A retrospective cohort study

Longitudinal research has provided systematic empirical data on the short- and long-term outcomes of admissions policies, curricular innovations, and complex decisions on students' academic progress. This study aimed to investigate the academic performance of medical students and related factors using cohort database collected from a medical school. The study participants included 134 medical students who graduated from Chonnam National University Medical School in 2022. The medical school's cohort database was used to collect data on demographics, admission, academic performance, extracurricular activities, and performance on the National Korean Medical Licensing Examination (KMLE). Participating in club activities had a significant association with medical students' academic advancement delay or leave of absence during the entire course of medical school (P = 0.007). Logistic regression analysis indicated that the nationwide clinical knowledge mock examination during the fourth year of medical school was significantly associated with passing the KMLE (adjusted odds ratio 1.12, 95% confidence interval 1.02-1.22; P = 0.014). Extracurricular school activities (a non-cognitive student attribute) and a wide range of cognitive student attributes captured from the cohort database were associated with medical students' academic performance. In conclusion, this study can reinforce a strong emphasis on the inclusion of cognitive and non-cognitive information in medical school curricula and assessments in order to improve medical education programs and future postgraduate performance.

Capturing growth curves of medical students' clinical skills performance

INTRODUCTION

A benefit of a milestone or Entrustable Professional Activity (EPA) assessment framework is the ability to capture longitudinal performance with growth curves using multi-level modelling (MLM). Growth curves can inform curriculum design and individualised learning. Residency programmes have found growth curves to vary by resident and by milestone. Only one study has analysed medical students' growth curves for EPAs. Analysis of EPA growth curves is critical because no change in performance raises concerns for EPAs as an assessment framework.

METHODS

Spencer Fox Eccles School of Medicine-University of Utah students' workplace-based assessment ratings for 7 EPAs were captured at 3 time-points in years 3-4 of AY2017-2018 to AY2020-2021. MLM was used to capture EPA growth curves and determine if variation in growth curves was explained by internal medicine (IM) clerkship order.

FINDINGS

A curvilinear slope significantly captured 256 students' average ratings overtime for EPA1a-history-taking, EPA2-clinical reasoning, EPA3-diagnostics, EPA5-documentation and EPA6-presentation, and a linear slope significantly captured EPA9-teamwork ratings, p ≤ 0.001. Growth curves were steepest for EPA2-clinical reasoning and EPA3-diagnostics. Growth curves varied by students, p < 0.05 for all EPA ratings, but IM clerkship rotation order did not significantly explain the variance, p > 0.05.

DISCUSSION

The increase in ratings from Year 3 to Year 4 provides validity evidence for use of EPAs in an assessment framework. Students may benefit from more curriculum/skills practice for EPA2-clinical reasoning and EPA3-diagnostics prior to year 3. Variation in student's growth curves is important for coaching and skill development; a one size fits all approach may not suffice.

Twelve tips for developing a systematic acute care curriculum for medical students

There are inadequacies in the practice-readiness of junior doctors for providing acute care in areas of clinical deterioration. In addition, the existing undergraduate curricula are fragmented in how acute care is taught in medical schools. We propose twelve tips for developing a systematic acute care curriculum, including what to teach, how to teach it and, how to assess. Furthermore, we propose and incorporate an acute care learning dashboard as an assessment tool which collates and demonstrates the occurrence of learning, faculty feedback, and students' reflection. We also summarise the existing online resources available for acute care training. We hope to address the existing issues and improve acute care training to prepare the graduates to become practice-ready professionals.

Comparing Entrustment Decision-Making Outcomes of the Core Entrustable Professional Activities Pilot, 2019-2020

IMPORTANCE

Gaps in readiness for indirect supervision have been identified for essential responsibilities encountered early in residency, presenting risks to patient safety. Core Entrustable Professional Activities (EPAs) for entering residency have been proposed as a framework to address these gaps and strengthen the transition from medical school to residency.

OBJECTIVE

To assess progress in developing an entrustment process in the Core EPAs framework.

DESIGN, SETTING, AND PARTICIPANTS

In this quality improvement study in the Core EPAs for Entering Residency Pilot, trained faculty made theoretical entrustment determinations and recorded the number of workplace-based assessments (WBAs) available for each determination in 2019 and 2020. Four participating schools attempted entrustment decision-making for all graduating students or a randomly selected subset of students. Deidentified, individual-level data were merged into a multischool database.

INTERVENTIONS

Schools implemented EPA-related curriculum, WBAs, and faculty development; developed systems to compile and display data; and convened groups to make theoretical summative entrustment determinations.

MAIN OUTCOMES AND MEASURES

On an EPA-specific basis, the percentage of students for whom an entrustment determination could be made, the percentage of students ready for indirect supervision, and the volume of WBAs available were recorded.

RESULTS

Four participating schools made 4525 EPA-specific readiness determinations (2296 determinations in 2019 and 2229 determinations in 2020) for 732 graduating students (349 students in 2019 and 383 students in 2020). Across all EPAs, the proportion of determinations of "ready for indirect supervision" increased from 2019 to 2020 (997 determinations [43.4%] vs 1340 determinations [60.1%]; 16.7 percentage point increase; 95% CI, 13.8-19.6 percentage points; P < .001), as did the proportion of determinations for which there were 4 or more WBAs (456 of 2295 determinations with WBA data [19.9%] vs 938 [42.1%]; 22.2 percentage point increase; 95% CI, 19.6-24.8 percentage points; P < .001). The proportion of EPA-specific data sets considered for which an entrustment determination could be made increased from 1731 determinations (75.4%) in 2019 to 2010 determinations (90.2%) in 2020 (14.8 percentage point increase; 95% CI, 12.6-16.9 percentage points; P < .001). On an EPA-specific basis, there were 5 EPAs (EPA 4 [orders], EPA 8 [handovers], EPA 10 [urgent care], EPA 11 [informed consent], and EPA 13 [patient safety]) for which few students were deemed ready for indirect supervision and for which there were few WBAs available per student in either year. For example, for EPA 13, 0 of 125 students were deemed ready in 2019 and 0 of 127 students were deemed ready in 2020, while 0 determinations in either year included 4 or more WBAs.

CONCLUSIONS AND RELEVANCE

These findings suggest that there was progress in WBA data collected, the extent to which entrustment determinations could be made, and proportions of entrustment determinations reported as ready for indirect supervision. However, important gaps remained, particularly for a subset of Core EPAs.

Telehealth Competencies: Leveled for Continuous Advanced Practice Nurse Development

BACKGROUND

The COVID-19 pandemic spurred a rapid uptake of telehealth utilization, with advanced practice registered nurses (APRNs) at the forefront of telehealth care delivery. To advance training of nurse practitioners and support curricular development, essential APRN student competencies in telehealth were developed.

PROBLEM

Although telehealth competencies have been developed, little is understood about their evaluation across the curricula. Moving to competency-based nursing education involves leveling broad competencies into subcompetencies, including those for telehealth. Subcompetencies support frequent, multimodal evaluation of student progress across APRN curricula.

APPROACH

Adapting Benner's Novice to Expert Theory, faculty experts in telehealth and graduate nursing education used an iterative process to develop and level subcompetencies aligned with the Four Ps of Telehealth framework.

OUTCOMES

Telehealth subcompetencies were leveled for preclinical and clinical rotations and for readiness for practice.

CONCLUSIONS

The leveled subcompetencies, aligned with the Four Ps of Telehealth framework, will support APRN faculty in diverse programs as they implement competency-based education in telehealth.

WBAs in UME-How Many Are Needed? A Reliability Analysis of 5 AAMC Core EPAs Implemented in the Internal Medicine Clerkship

BACKGROUND

Reliable assessments of clinical skills are important for undergraduate medical education, trustworthy handoffs to graduate medical programs, and safe, effective patient care. Entrustable professional activities (EPAs) for entering residency have been developed; research is needed to assess reliability of such assessments in authentic clinical workspaces.

DESIGN

A student-driven mobile assessment platform was developed and used for clinical supervisors to record ad hoc entrustment decisions using the modified Ottawa scale on 5 core EPAs in an 8-week internal medicine (IM) clerkship. After a 12-month period, generalizability (G) theory analysis was performed to estimate the reliability of entrustment scores and determine the proportion of variance attributable to the student and the other facets, including particular EPA, evaluator type (attending versus resident), or case complexity. Decision (D) theory analysis determined the expected reliability based on the number of hypothetical observations. A g-coefficient of 0.7 was used as a generally agreed upon minimum reliability threshold.

KEY RESULTS

A total of 1368 ratings over the 5 EPAs were completed on 94 students. Variance attributed to person (true variance) was high for all EPAs; EPA-5 had the lowest person variance (9.8% across cases and four blocks). Across cases, reliability ranged from 0.02 to 0.60. Applying this to the Decision study, the estimated number of observations needed to reach a reliability index of 0.7 ranged between 9 and 11 for all EPAs except EPA5 which was sensitive to case complexity.

CONCLUSIONS

Work place-based clinical skills in IM clerkship students were assessed and logged using a convenient mobile platform. Our analysis suggests that 9-11 observations are needed for these EPA workplace-based assessments (WBAs) to achieve a reliability index of 0.7. Note writing was very sensitive to case complexity. Further reliability analyses of core EPAs are needed before US medical schools consider wider adoption into summative entrustment processes and GME handoffs.

Competency frameworks, nursing perspectives, and interdisciplinary collaborations for good patient care: Delineating boundaries

To enhance patient care in the inevitable conditions of complexity that exist in contemporary healthcare, collaboration among healthcare professions is critical. While each profession necessarily has its own primary focus and perspective on the nature of human healthcare needs, these alone are insufficient for meeting the complex needs of patients (and potential patients). Persons are inevitably contextual entities, inseparable from their environments, and are subject to institutional and social barriers that can detract from good care or from accessing healthcare. These are some of the reasons behind current movements to develop competency frameworks that can enhance cross-disciplinary communication and collaboration. No single profession can claim the big picture. Effective teamwork is essential and requires members of diverse professions to understand the nature of each other's knowledge, skills, roles, perspectives, and perceived responsibilities so that they are optimally utilized on behalf of patients and their families. Interdisciplinary approaches to care permit different aspects of a person's needs to be addressed seamlessly and facilitate the removal of obstacles by engaging the range of resources exemplified by the different professions. Additionally, collaborative efforts are needed to influence policy changes on behalf of individual and social good and to address root causes of poor health especially as these impact society's most vulnerable. Here, we explore both the benefits and the risks of an uncritical acceptance of competency frameworks as a way to enhance interdisciplinary communication. We highlight the importance of anchoring proposed competency domains in the reason for being of a given profession and exemplify one way this has been accomplished for advanced practice nursing. Additionally, we argue that having this mooring, permits integration of the various competencies that both enhances professional moral agency and facilitates interdisciplinary collaboration to further the mutual goals of the healthcare professions on behalf of quality patient care.

Pre-clerkship EPA assessments: a thematic analysis of rater cognition

BACKGROUND

Entrustable Professional Activities (EPAs) assessments measure learners' competence with an entrustment or supervisory scale. Designed for workplace-based assessment EPA assessments have also been proposed for undergraduate medical education (UME), where assessments frequently occur outside the workplace and may be less intuitive, raising validity concerns. This study explored how assessors make entrustment determinations in UME, with additional specific comparison based on familiarity with prior performance in the context of longitudinal student-assessor relationships.

METHODS

A qualitative approach using think-alouds was employed. Assessors assessed two students (familiar and unfamiliar) completing a history and physical examination using a supervisory scale and then thought-aloud after each assessment. We conducted a thematic analysis of assessors' response processes and compared them based on their familiarity with a student.

RESULTS

Four themes and fifteen subthemes were identified. The most prevalent theme related to "student performance." The other three themes included "frame of reference," "assessor uncertainty," and "the patient." "Previous student performance" and "affective reactions" were subthemes more likely to inform scoring when faculty were familiar with a student, while unfamiliar faculty were more likely to reference "self" and "lack confidence in their ability to assess."

CONCLUSIONS

Student performance appears to be assessors' main consideration for all students, providing some validity evidence for the response process in EPA assessments. Several problematic themes could be addressed with faculty development while others appear to be inherent to entrustment and may be more challenging to mitigate. Differences based on assessor familiarity with student merits further research on how trust develops over time.

A Pilot Study of the Generalizability of Preclinical Entrustment Assessments in Undergraduate Medical Education

PURPOSE

The reproducibility and consistency of assessments of entrustable professional activities (EPAs) in undergraduate medical education (UME) have been identified as potential areas of concern. EPAs were designed to facilitate workplace-based assessments by faculty with a shared mental model of a task who could observe a trainee complete the task multiple times. In UME, trainees are frequently assessed outside the workplace by faculty who only observe a task once.

METHOD

In November 2019, the authors conducted a generalizability study (G-study) to examine the impact of student, faculty, case, and faculty familiarity with the student on the reliability of 162 entrustment assessments completed in a preclerkship environment. Three faculty were recruited to evaluate 18 students completing 3 standardized patient (SP) cases. Faculty familiarity with each student was determined. Decision studies were also completed. Secondary analysis of the relationship between student performance and entrustment (scoring inference) compared average SP checklist scores and entrustment scores.

RESULTS

G-study analysis revealed that entrustment assessments struggled to achieve moderate reliability. The student accounted for 30.1% of the variance in entrustment scores with minimal influence from faculty and case, while the relationship between student and faculty accounted for 26.1% of the variance. G-study analysis also revealed a difference in generalizability between assessments by unfamiliar (φ = 0.75) and familiar (φ = 0.27) faculty. Subanalyses showed that entrustment assessments by familiar faculty were moderately correlated to average SP checklist scores (r = 0.44, P < .001), while those by unfamiliar faculty were weakly correlated (r = 0.16, P = .13).

CONCLUSIONS

While faculty and case had a limited impact on the generalizability of entrustment assessments made outside the workplace in UME, faculty who were familiar with a student's ability had a notable impact on generalizability and potentially on the scoring validity of entrustment assessments, which warrants further study.

It does not have to be either or! Assessing competence in medicine should be a continuum between an analytic and a holistic approach

Assessing competence is a tremendous challenge in medical education. There are two contrasting approaches in competence assessment: an analytic approach that aims to precisely measure observable constituents and facets of competence and a holistic approach that focuses on a comprehensive assessment of competences in complex real situations reflecting actual performance. We would like to contribute to the existing discourse about medical competence and its assessment by proposing an approach that can provide orientation for the development of competence-based assessment concepts in undergraduate and postgraduate medical education. The approach follows Kane's framework of an "argument-based approach" to validity and is based on insights into task complexity, testing and learning theories as well as the importance of the learning environment. It describes a continuum from analytic to holistic approaches to assess the constituents and facets of competence to performance. We conclude that the complexity of a task should determine the selection of the assessment and suggest to use this approach to reorganize and adapt competence assessment.

Entrustable Professional Activity-Based Summative Performance Assessment in the Surgery Clerkship

OBJECTIVE

The objectives of this study were to 1) assess the performance Entrustable Professional Activities (EPAs) when integrated into the summative assessment of third-year medical students on the surgery clerkship and 2) to compare EPAs to traditional clinical performance assessment tools.

DESIGN

EPA assessments were collected prospectively from a minimum of 4 evaluators at the completion of each surgical clerkship rotation from November 2019 to June 2019. Overall EPA-based clinical performance scores were calculated as the sum of the mean EPA score from each evaluator. A rating of overall clinical performance called the clinical performance appraisal (CPA) was also collected. EPA ratings were compared to the CPA score, National Board of Medical Examiners exam score, objective structured clinical exam scores, and final clerkship grade.

SETTING

Northwestern Memorial Hospital, a tertiary care teaching institution in Chicago, IL.

RESULTS

Overall, 446 evaluations (111 students) were included in the analysis. The aggregate EPA scores ranged from 11.6-24.0 (mean 19.9 ± 2.0), and the CPA scores ranged from 4.4-9.0 (mean 7.6 ± 0.7). The variance among learners in EPA scores was significantly higher than CPA scores (p < 0.001). The aggregate EPA scores correlated well with CPA scores (Spearman's rho 0.803) but had lesser, positive correlations with the objective structured clinical exam (rho 0.153) and National Board of Medical Examiners (rho 0.265) scores. When all EPA scores were included in ordinal logistic regression, only EPA 6, oral presentation of patients, was independently associated with students' final grades (OR: 10.05, 95%CI 1.41-71.80; p = 0.02).

CONCLUSION

Integration of EPAs for use in clinical performance assessment of medical students is feasible within a surgery clerkship. Compared to a global clinical performance assessment, EPA-based assessment provided better discrimination of clinical performance among learners. Use of EPAs may better identify advanced learners and those that need additional time.

Entrustable Professional Activities and Entrustment Decision Making: A Development and Research Agenda for the Next Decade

To establish a research and development agenda for Entrustable Professional Activities (EPAs) for the coming decade, the authors, all active in this area of investigation, reviewed recent research papers, seeking recommendations for future research. They pooled their knowledge and experience to identify 3 levels of potential research and development: the micro level of learning and teaching; the meso level of institutions, programs, and specialty domains; and the macro level of regional, national, and international dynamics. Within these levels, the authors categorized their recommendations for research and development. The authors identified 14 discrete themes, each including multiple questions or issues for potential exploration, that range from foundational and conceptual to practical. Much research to date has focused on a variety of issues regarding development and early implementation of EPAs. Future research should focus on large-scale implementation of EPAs to support competency-based medical education (CBME) and on its consequences at the 3 levels. In addition, emerging from the implementation phase, the authors call for rigorous studies focusing on conceptual issues. These issues include the nature of entrustment decisions and their relationship with education and learner progress and the use of EPAs across boundaries of training phases, disciplines and professions, including continuing professional development. International studies evaluating the value of EPAs across countries are another important consideration. Future studies should also remain alert for unintended consequences of the use of EPAs. EPAs were conceptualized to support CBME in its endeavor to improve outcomes of education and patient care, prompting creation of this agenda.

Integrative medicine and health in undergraduate and postgraduate medical education

Background and objective: Integrative Medicine and Health (IMH) is a theory-based paradigm shift for health, disease and health care, which can probably only be achieved by supplementing medical roles and competences. Definition of IMH: The definitions of the Academic Consortium for Integrative Medicineand Health 2015 and the so-called Berlin Agreement: Self-Responsibility and Social Action in Practicing and Fostering Integrative Medicine and Health Globally are used. The basic features of evidence-based Integrative Medicine and Health (EB-IMH) are based on the recommendations on EBM by David L. Sackett. Global State of Undergraduate and Postgraduate Medical Education (UG-PGME) for IMH: The USA and Canada are most advanced in the development of IMH regarding practice, teaching and research worldwide. Despite socio-cultural peculiarities, they can provide guidance for Europe and especially for Germany. Of interest here are competences for UG-PGME in IMH in primary care and in some specialist disciplines (e.g. internal medicine, gynecology, pediatrics, geriatrics, oncology, palliative care). For these specialties, the need for an interprofessional UG-PGME for IMH was shown in the early stages of development. UG-PGME for IMH in Germany: In the course of the development of the new Medical Licensure Act in Germany (ÄApprO), based on a revision of the National Competence-based Catalogue of Learning Objectives for Medicine (NKLM 2.0) and new regulations for Postgraduate Medical Education in Germany, suggestions for an extension of UG-PGME are particularly topical. To some extent there are already approaches to IMH. Old and new regulations are set out and are partly compared. As a result, some essential elements of IMH are mapped in the new ÄApprO. The new regulations for Postgraduate Medical Education do not mention IMH. Conclusion: The development of medical competences for IMH in the continuum of the UG-PGME could be supported by the coordinated introduction of appropriate entrustable professional activities (EPA) and IMH sub-competences combined with appropriate assessment.

Assessing Trainees and Making Entrustment Decisions: On the Nature and Use of Entrustment-Supervision Scales

Clinical teachers are continuously entrusting trainees with care responsibilities in health care settings. Entrustable professional activities employ entrustment decision making as an approach to assessment in the workplace.Various scales have been created to measure "entrustment," all basically expressing the level or type of supervision a trainee requires for safe and high-quality care. However, some of these scales are only weakly related to the purpose of making decisions about the autonomy trainees will be granted. The authors aim to increase understanding about the nature, purpose, and practice of supervision scales aimed at entrustment.After arguing for entrustment as a component of workplace-based assessment, the distinction between ad hoc entrustment decisions (daily decisions in health care settings) and summative entrustment decisions (with a certifying nature) is clarified. Next, the noncontinuous nature of entrustment-supervision (ES) scales, as opposed to most workplace-based assessment scales, is explained. ES scales have ordinal, rather than interval, properties and focus on discrete decisions. Finally, some scales are retrospective ("how much supervision was provided?"), and others are prospective ("how much supervision will be needed in the near future?"). Although retrospective scales reflect observed behavior, prospective scales truly focus on entrustment and ask for more holistic judgment, as they include a broader evaluation and a risk estimation to enable a decision about increase of autonomy.The analysis concludes with a discussion about entrustment for unsupervised practice and supervision of others, as well as the program, context, and specialty specificity of scales.

EQual Rubric Evaluation of the Association of American Medical Colleges' Core Entrustable Professional Activities for Entering Residency

PURPOSE

To have subject matter experts evaluate the Core Entrustable Professional Activities for Entering Residency (Core EPAs) with the EQual rubric to determine if revisions were required and, if applicable, how to focus revision efforts.

METHOD

Ten entrustable professional activity (EPA) experts were invited to evaluate the 13 Core EPAs. Experts had a 6-month window (December 2018-May 2019) to complete the evaluation, which contained the complete EQual rubric and 3 additional prompts, one of which-"Do you think this EPA requires revision?"-was limited to a "yes/no" response. Descriptive statistics for overall and domain-specific EQual rubric scores for each of the 13 Core EPAs were calculated. Free-text responses to why and/or how a Core EPA should be revised were summarized for any Core EPA that scored below a cutoff or for which the majority of experts recommended revision.

RESULTS

Six experts completed the evaluation. Most Core EPAs' (9/13) overall score was above the cutoff, indicating that they align with the key domains of the EPA construct. The remaining 4 Core EPAs (2, 7, 9, and 13) scored below the overall cutoff, suggesting that they may require revision. A majority of experts felt that Core EPAs 6, 7, 9, and 13 required revision. With regard to domain-specific scores, Core EPAs 2, 3, 7, 9, and 13 were below the discrete units of work cutoff; Core EPAs 7, 9, and 13 were below the entrustable, essential, and important tasks of the profession of medicine cutoff; and Core EPA 9 was below the curricular role cutoff.

CONCLUSIONS

The Core EPAs represent a promising initial framework of EPAs for undergraduate medical education. Some Core EPAs might benefit from revision. The process of improving the Core EPAs must continue if they are to standardize outcomes for medical school graduates.

Development of core entrustable professional activities linked to a competency-based veterinary education framework

Purpose: Despite the adoption of competency-based education in some veterinary schools over the past 15 years, only recently has a concerted effort been directed toward this in veterinary education internationally.Methods: In 2015, educational leaders from the Association of American Veterinary Medical Colleges (AAVMC) member schools came together with a strong call to action to create shared tools for clinical competency assessment.Results: This resulted in the formation of the AAVMC Competency-Based Veterinary Education (CBVE) Working Group, which then embarked on the creation of a shared competency framework and the development of eight core entrustable professional activities (EPAs) linked to this framework.Conclusions: This paper will report on the development of these EPAs and their integration with the concurrently-developed CBVE Framework.

The relationship between students' perception of the educational environment and their subjective happiness

BACKGROUND

Happiness, a subjective judgment about one's quality of life, is influenced by environmental factors and should be considered as an important goal of medical education, which should support each learner's development as a person as well as a professional. However, although several studies have reported on the correlation between Dundee Ready Educational Environment Measure (DREEM) scores and students' academic achievement, few have investigated the relationship between DREEM scores and students' subjective happiness. This study examined different perceptions of the educational environment between phases of the curriculum and determined which DREEM subscales affect the overall level of happiness.

METHODS

We used the Korean version of the DREEM questionnaire and a single item measure of happiness on a scale of 0 to 10. First we analyzed student perceptions of the educational environment according to their demographic characteristics using independent sample t-tests and one-way analysis of variance. A multiple regression analysis was performed to reveal which subscales affect the overall level of happiness while controlling for grade point average (GPA) and other demographic characteristics.

RESULTS

The subjects were 239 medical school students across all stages of the curriculum. The students' overall perception was more positive for the educational environment during Phase 3 (clerkship) than Phase 1 (pre-medical). Among the DREEM subscales, this difference was especially prominent in Students' Perception of Learning and Students' Academic Self-Perceptions. In contrast, no difference in the subjective perception of happiness was found between phases. The effect of GPA on happiness became insignificant under the control of other variables, but the influence of the Students' Social Self-Perceptions (SSSP) subscale remained significant.

CONCLUSIONS

The students' overall perception of the educational environment was more positive during the clerkship period than in the pre-medical period. Based on our finding that the SSSP correlates significantly with subjective happiness, we suggest that institutions promote not only students' academic development but also their happiness by fostering an appropriate educational environment.

Can Teachers Distinguish Competencies From Entrustable Professional Activities?

Introduction

There has been a recent transition from the use of "competencies" to "entrustable professional activities" (EPAs) in medical education assessment paradigms. Although this transition proceeds apace, few studies have examined these concepts in a practical context. Our study sought to examine how distinct the concepts of competencies and EPAs were to front-line clinical educators.

Methods

A 20-item survey tool was developed based on the University of Calgary Department of Family Medicine's publicly available lists of competencies and EPAs. This tool required participants to identify given items as either a competency or an EPA, after reading a description of each. The tool was administered to a convenience sample of consenting clinical educators at 5 of the 14 training sites at the University of Toronto Department of Family and Community Medicine in 2018. We also collected information on years in practice, hours spent supervising per week, and direct involvement in medical education.

Results

We analyzed a total of 60 surveys. The mean rate of correct responses was 45.3% (+/- 21.8%). Subgroup analysis failed to reveal any correlation between any of the secondary characteristics and correct responses.

Conclusion

Clinical educators in our study were not able to distinguish between competencies and EPAs. Further research is recommended prior to intensive curricular changes.

To What Degree Are the 13 Entrustable Professional Activities Already Incorporated Into Physicians' Performance Schemas for Medical Students?

Phenomenon: There is an abundance of literature on Entrustable Professional Activities (EPAs) in theory, but there are few studies on the EPAs in practice for undergraduate clinical education. In addition, little is known about the degree to which the EPAs are or are not aligned with physician assessors' performance schemas of the clerkship student. Investigating the degree to which physician assessors' performance schemas are already aligned with the activities described by the EPAs is critical for effective workplace assessment design. Approach: We sampled 1,032 areas of strength (strength) and areas for improvement (improvement) written evaluation comments by 423 physician assessors for clerkship students' performance in academic years 2014-15 and 2015-16 at the University of Utah School of Medicine. Two researchers independently categorized each comment by EPA and/or coded by non-EPA topic. The proportion of comment types was compared between strength comments and improvement comments with the Wilcoxon Signed-Rank Test. Findings: The most frequently mentioned EPAs in comments were about history gathering/physical exam, differential diagnosis, documentation, presentation, and interprofessional collaboration; few mentioned diagnostic tests, patient handovers, recognition of urgent patient care, and patient safety, and none mentioned orders/prescriptions and informed consent. The most frequent non-EPA topics were about medical knowledge, need to read more, learning attitude, work ethic, professionalism/maturity, and receptiveness to feedback. The proportion of comments aligned with an EPA only, a non-EPA topic only, or both an EPA and non-EPA topic was significantly different for clerkship students' strength compared to improvement. Insights: Physician assessors' performance schemas for clerkship students were aligned with EPAs to varying degrees depending on the specific EPA and whether describing strength or improvement. Of interest, the frequently mentioned non-EPA comments represented some of the competencies that contribute to effectively performing particular EPAs and are Accreditation Council for Graduate Medical Education (ACGME) core competencies (e.g., medical knowledge, professionalism), used in residency programs. Because physician assessors for undergraduate medical education often also participate in graduate medical education, the frequency of non-EPA topics aligned to ACGME competencies may suggest influence of graduate medical education evaluative frameworks on performance schemas for clerkship students; this could be important when considering implementation of EPAs in undergraduate medical education.

Entrustable professional activities in health care education: a scoping review

CONTEXT

The shift in medical education from time-based learning to outcome-based learning has drawn much attention to entrustable professional activities (EPAs) as an ideal assessment framework to translate competencies into clinical practice. Given the relative novelty of EPAs, this review aims to highlight research gaps and explore and consolidate available evidence pertaining to the development and implementation of EPAs in health care.

METHOD

Arksey and O'Malley's scoping review framework was used to present the findings. The authors performed a systematic search of PubMed, Embase, CINAHL, Scopus, MedNar, OpenGrey and ProQuest Dissertation and Theses for English articles published from the inception of each database to May 2018. A manual search of the reference lists of the included studies was conducted and an expert panel was consulted. Two reviewers screened the articles for eligibility using the inclusion criteria. All authors extracted key data and analysed the data descriptively. Thematic analysis was used to categorise the results into themes.

RESULTS

Eighty articles were included in the review. All articles were published between 2010 and 2018. Three major themes and eight sub-themes were generated: (i) development of EPAs (frameworks for EPA development and implementation, identifying core or specialty-specific EPAs, and EPAs for faculty development), (ii) evaluation of EPAs and EPA entrustment factors (revised curriculum, entrustment decisions for professional activities, and feedback on implemented EPAs and the development process), and (iii) future directions and recommendations for EPAs (implementation of EPAs in undergraduate medical education and specific disciplines, and other criticisms and recommendations for EPAs).

CONCLUSIONS

Entrustable professional activities are an essential means to translate competencies into observable and measurable clinical practice. However, high-level evidence-based research on the efficacy, development and implementation of EPAs for specific target groups (i.e. undergraduates and staff) and geographical regions (i.e. Asia and Africa) is still lacking, which suggests a direction for future research.

Scoping Review of Entrustable Professional Activities in Undergraduate Medical Education

PURPOSE

Entrustable professional activities (EPAs) are a hot topic in undergraduate medical education (UME); however, the usefulness of EPAs as an assessment approach remains unclear. The authors sought to better understand the literature on EPAs in UME through the lens of the 2010 Ottawa Conference Criteria for Good Assessment.

METHOD

The authors conducted a scoping review of the health professions literature (search updated February 2018), mapping publications to the Ottawa Criteria using a collaboratively designed charting tool.

RESULTS

Of the 1,089 publications found, 71 (6.5%) met inclusion criteria. All were published after 2013. Forty-five (63.4%) referenced the 13 Core Entrustable Professional Activities for Entering Residency developed by the Association of American Medical Colleges (AAMC). Forty (56.3%) were perspectives, 5 (7.0%) were reviews, and 26 (36.6%) were prospective empirical studies. The publications mapped to the Ottawa Criteria 158 times. Perspectives mapped more positively (83.7%) than empirical studies (76.7%). Reproducibility did not appear to be a strength of EPAs in UME; however, reproducibility, equivalence, educational effect, and catalytic effect all require further study. Inconsistent use of the term "EPA" and conflation of concepts (activity vs assessment vs advancement decision vs curricular framework) limited interpretation of published results. Overgeneralization of the AAMC's work on EPAs has influenced the literature.

CONCLUSIONS

Much has been published on EPAs in UME in a short time. Now is the time to move beyond opinion, clarify terms, and delineate topics so that well-designed empirical studies can demonstrate if and how EPAs should be implemented in UME.

Entrustable Professional Activities in final year undergraduate medical training - advancement of the final year training logbook in Germany

Objective: Training in the final year (FY) of undergraduate medical training currently does not adequately prepare students for the independent performance of medical professional activities after graduation. The concept of Entrustable Professional Activities (EPA) offers the opportunity for a competency-based FY training with the focus on medical professional activities. Methodology: In regular meetings, the FY sub-working group of the German Medical Faculty Association (MFT), which includes representatives with clinical and didactic expertise of the Associations of Internal Medicine, Surgery and General Medicine, developed a concept for the competecy-orientated, EPA-based, FY model logbook 2.0. The selection of the units of practice was made in a cross-disciplinary, consensus-orientated discussion process based on the question which medical professional activities a young professional has to master in the inpatient or outpatient working environment. Results: For the FY electives internal medicine, surgery and general medicine, a blueprint of a total of 18 comprehensive, partially interdisciplinary EPAs relating to inpatient and outpatient care contexts were developed. Each EPA was operationalised by a short description, supervision levels were attributed, and the process of transparent entrustment was determined. Conclusions: The concept for a new FY model logbook 2.0 focuses on the interdisciplinary core medical professional activities in an inpatient and outpatient care context, in order to facilitate transition from undergraduate training to professional practice, and to help avoid overload, thus increasing patient safety.

Managing tensions in assessment: moving beyond either-or thinking

CONTEXT

In health professions education, assessment systems are bound to be rife with tensions as they must fulfil formative and summative assessment purposes, be efficient and effective, and meet the needs of learners and education institutes, as well as those of patients and health care organisations. The way we respond to these tensions determines the fate of assessment practices and reform. In this study, we argue that traditional 'fix-the-problem' approaches (i.e. either-or solutions) are generally inadequate and that we need alternative strategies to help us further understand, accept and actually engage with the multiple recurring tensions in assessment programmes.

METHODS

Drawing from research in organisation science and health care, we outline how the Polarity Thinking™ model and its 'both-and' approach offer ways to systematically leverage assessment tensions as opportunities to drive improvement, rather than as intractable problems. In reviewing the assessment literature, we highlight and discuss exemplars of specific assessment polarities and tensions in educational settings. Using key concepts and principles of the Polarity Thinking™ model, and two examples of common tensions in assessment design, we describe how the model can be applied in a stepwise approach to the management of key polarities in assessment.

DISCUSSION

Assessment polarities and tensions are likely to surface with the continued rise of complexity and change in education and health care organisations. With increasing pressures of accountability in times of stretched resources, assessment tensions and dilemmas will become more pronounced. We propose to add to our repertoire of strategies for managing key dilemmas in education and assessment design through the adoption of the polarity framework. Its 'both-and' approach may advance our efforts to transform assessment systems to meet complex 21st century education, health and health care needs.

Competency-Based Medical Education and the Ghost of Kuhn: Reflections on the Messy and Meaningful Work of Transformation

The transition, if not transformation, to outcomes-based medical education likely represents a paradigm shift struggling to be realized. Paradigm shifts are messy and difficult but ultimately meaningful if done successfully. This struggle has engen dered tension and disagreements, with many of these disagreements cast as either-or polarities. There is little disagreement, however, that the health care system is not effectively achieving the triple aim for all patients. Much of the tension and polarity revolve around how more effectively to prepare students and residents to work in and help change a complex health care system.Competencies were an initial attempt to facilitate this shift by creating frameworks of essential abilities needed by physicians. However, implementation of competencies has proven to be difficult. Entrustable professional activities (EPAs) in undergraduate and graduate medical education and Milestones in graduate medical education are recent concepts being tried and studied as approaches to guide the shift to outcomes. Their primary purpose is to help facilitate implementation of an outcomes-based approach by creating shared mental models of the competencies, which in turn can help to improve curricula and assessment. Understanding whether and how EPAs and Milestones effectively facilitate the shift to outcomes has been and will continue to be an iterative and ongoing reflective process across the entire medical education community using lessons from implementation and complexity science. In this Invited Commentary, the author reflects on what got the community to this point and some sources of tension involved in the struggle to move to outcomes-based education.

Competency-Based, Time-Variable Education in the Health Professions: Crossroads

Health care systems around the world are transforming to align with the needs of 21st-century patients and populations. Transformation must also occur in the educational systems that prepare the health professionals who deliver care, advance discovery, and educate the next generation of physicians in these evolving systems. Competency-based, time-variable education, a comprehensive educational strategy guided by the roles and responsibilities that health professionals must assume to meet the needs of contemporary patients and communities, has the potential to catalyze optimization of educational and health care delivery systems. By designing educational and assessment programs that require learners to meet specific competencies before transitioning between the stages of formal education and into practice, this framework assures the public that every physician is capable of providing high-quality care. By engaging learners as partners in assessment, competency-based, time-variable education prepares graduates for careers as lifelong learners. While the medical education community has embraced the notion of competencies as a guiding framework for educational institutions, the structure and conduct of formal educational programs remain more aligned with a time-based, competency-variable paradigm.The authors outline the rationale behind this recommended shift to a competency-based, time-variable education system. They then introduce the other articles included in this supplement to Academic Medicine, which summarize the history of, theories behind, examples demonstrating, and challenges associated with competency-based, time-variable education in the health professions.