EQual Rubric Evaluation of the Association of American Medical Colleges' Core Entrustable Professional Activities for Entering Residency

International Delphi study on developing entrustable professional activities (EPAs) for master's learners in health professions education

BACKGROUND

Competencies-based education has gained global recognition, emphasizing the need for educators to align educational outcomes with healthcare system requirements. However, limited literature exists on competency frameworks and Entrustable Professional Activities (EPAs) specific to health professions educationists, hindering the development of tailored Master's programs.

AIM

This study aimed to develop a competency framework outlining the roles and functions for Master's learners in Health Professions Education (MHPE) and identify the key EPAs that form the foundation of a task-based MHPE curriculum, along with the expected levels of entrustment.

METHODS

An international Delphi study was conducted involving three rounds of surveys and qualitative discussions with an expert panel of health professions educationists from diverse contexts (n = 29). The Delphi technique, including open-ended and quantitative rating scales, was employed to reach consensus on the EPAs and their levels of entrustment. The final list of EPAs was validated using the EQual rubric.

RESULTS

A total of 16 EPAs were identified and validated, mapped to core and potential roles and functions of health professions educationists. There is less agreement on the level of entrustment that should be attained at the master level, resulting in EPAs that must be fully entrusted before graduation and others which need further development afterwards.

CONCLUSION

The study presents a comprehensive competency framework and a set of EPAs tailored for MHPE programs, providing a structured approach to curriculum design and learner assessment. The findings underscore the importance of incorporating context-specific considerations and aligning educational objectives with the evolving roles and responsibilities of health professions educationists.

Development of entrustable professional activities for novice nurses: A modified Delphi study

AIM

This study aimed to develop a tailored EPA framework for novice nurses to support their integration and performance in hospital settings.

BACKGROUND

Novice nurses often struggle with the transition from education to practice due to the gap between theoretical knowledge and practical skills. EPAs help bridge this gap by defining and assessing key competencies, but a framework specific to novice nurses is needed to address their unique challenges.

DESIGN

A modified Delphi method was used.

METHODS

Conducted from October 2023 to March 2024, the study followed a four-stage process: team formation, identification, revision and evaluation of EPAs. Initial EPAs were identified through literature reviews, clinical guidelines and focus groups, then refined using the EQual rubric. Two Delphi rounds were conducted for validation. Experts rated the importance of EPAs in the first round, followed by online discussions, with the second round focusing on dimensions including Focus, Observable, Realistic, Generalizable and Multiple competencies.

RESULTS

The final framework included 12 EPAs: 1) Manage Admission and Discharge, 2) Provide Basic Care, 3) Collect Specimens, 4) Administer Medications, 5) Perform Non- Medication Therapies, 6) Monitor Conditions, 7) Respond to Emergencies, 8) Educate Patients, 9) Prevent Adverse Events, 10) Conduct Handovers, 11) Document Care and 12) Operate Equipment.

CONCLUSION

This EPA framework integrates theory and practice to improve training and assessment. Future research should focus on its implementation and refinement in diverse clinical settings to ensure broader applicability.

Development of the core competency-based entrustable professional activities for Master of Nursing Specialist (MNS) graduates in China

BACKGROUND

Advanced nursing practice and education with a Master's degree as the necessary preparation, is viewed as a major strategy to cultivate senior nursing talents. Competencies are central to advanced nursing practice and education, but how can competencies be measured? Entrustable professional activities (EPAs) have been used widely in medicine as a practical approach for bridging the gaps between competency and clinical practice. Considering the paucity of research in EPAs for nursing graduates in China, it is needed to develop EPAs specifically for Chinese Master of Nursing Specialist (MNS) graduates to improve patient safety and quality patient care.

OBJECTIVES

To develop and evaluate a core competency-based EPAs framework for Chinese MNS graduates.

METHODS

A four-stage approach was adopted for the EPAs development, including: (1) forming a research team, (2) drafting an initial EPAs framework, (3) reviewing EPAs framework, and (4) conducting EPAs consensus assessment.

RESULTS

A framework containing twelve EPAs was developed, including: 1) perform health assessments, 2) identify and prioritize nursing diagnoses, 3) formulate and implement care plan, 4) perform basic and specialized care operations, 5) recognize and manage medication needs of patients, 6) assess and manage patients with mental health problems, 7) recognize and assist in rescuing critically ill patients, 8) perform transition and handover, 9) participate in multidisciplinary team collaborative care, 10) provide health education and nursing consultation, 11) formulate and implement discharge plans, and 12) instruct nursing students in a clinical setting. The I-CVI score for the two rounds of Delphi ranged from 0.92 ∼ 1.00 and 0.96 ∼ 1.00, respectively. The mean of Equal's score for the three domains ranged from 4.20 ∼ 4.47, 4.25 ∼ 4.51, and 4.23 ∼ 4.37, respectively.

CONCLUSION

The developed EPAs framework in this study is a reliable tool to assess the core competencies of Chinese MNS graduates in clinical practice and assist with their curricula design.

Developing an entrustable professional activity for providing health education and consultation in occupational therapy and examining its validity

BACKGROUND

Entrustable Professional Activities (EPA)-based assessment is easily and intuitively used in evaluating the learning outcomes of competency-based medical education (CBME). This study aimed to develop an EPA for occupational therapy focused on providing health education and consultation (TP-EPA3) and examine its validity.

METHODS

Nineteen occupational therapists who had completed online training on the EQual rubric evaluation participated in this study. An expert committee identified six core EPAs for pediatric occupational therapy. TP-EPA3 was developed following the EPA template and refined through consensus meetings. The EQual rubric, a 14-item, five-point criterion-based anchor system, encompassing discrete units of work (DU), entrustable, essential, and important tasks of the profession (EEIT), and curricular role (CR), was used to evaluate the quality of TP-EPA3. Overall scores below 4.07, or scores for DU, EEIT, and CR domains below 4.17. 4.00, and 4.00, respectively, indicate the need for modifications.

RESULTS

The TP-EPA3 demonstrated good validity, surpassing the required cut-off score with an average overall EQual score of 4.21 (SD = 0.41). Specific domain scores for DU, EEIT, and CR were 3.90 (SD = 0.69), 4.46 (SD = 0.44), and 4.42 (SD = 0.45), respectively. Subsequent revisions clarified observation contexts, enhancing specificity and focus. Further validation of the revised TP-EPA3 and a thorough examination of its reliability and validity are needed.

CONCLUSION

The successful validation of TP-EPA3 suggests its potential as a valid assessment tool in occupational therapy education, offering a structured approach for developing competency in providing health education and consultation. This process model for EPA development and validation can guide occupational therapists in creating tailored EPAs for diverse specialties and settings.

Development of entrustable professional activities for emergency medicine medical education fellowships: A modified Delphi study

Background

It is essential that medical education (MedEd) fellows achieve desired outcomes prior to graduation. Despite the increase in postgraduate MedEd fellowships in emergency medicine (EM), there is no consistently applied competency framework. We sought to develop entrustable professional activities (EPAs) for EM MedEd fellows.

Methods

From 2021 to 2022, we used a modified Delphi method to achieve consensus for EPAs. EM education experts generated an initial list of 173 EPAs after literature review. In each Delphi round, panelists were asked to make a binary choice of whether to include the EPA. We determined an inclusion threshold of 70% agreement a priori. After the first round, given the large number of EPAs meeting inclusion threshold, panelists were instructed to vote whether each EPA should be included in the "20 most important" EPAs for a MedEd fellowship. Modifications were made between rounds based on expert feedback. We calculated descriptive statistics.

Results

Seventeen experts completed four Delphi rounds each with 100% response. After Round 1, 87 EPAs were eliminated and two were combined. Following Round 2, 46 EPAs were eliminated, seven were combined, and three were included in the final list. After the third round, one EPA was eliminated and 13 were included. After the fourth round, 11 EPAs were eliminated. The final list consisted of 16 EPAs in domains of career development, education theory and methods, research and scholarship, and educational program administration.

Conclusions

We developed a list of 16 EPAs for EM MedEd fellowships, the first step in implementing competency-based MedEd.

Development, implementation, and evaluation of entrustable professional activities (EPAs) for medical radiation technologists in Taiwan: a nationwide experience

BACKGROUND

Competency-based medical education (CBME) is an outcomes-oriented approach focused on developing competencies that translate into clinical practice. Entrustable professional activities (EPAs) bridge competency assessment and clinical performance by delineating essential day-to-day activities that can be entrusted to trainees. EPAs have been widely adopted internationally, but not yet implemented for medical radiation professionals in Taiwan.

MATERIALS AND METHODS

A nationwide consensus process engaged 97 experts in radiation technology education representing diagnostic radiography, radiation therapy, and nuclear medicine. Preliminary EPAs were developed through the focus group discussion and the modified Delphi method. The validity of these EPAs was evaluated using the QUEPA and EQual tools.

RESULTS

Through iterative consensus building, six core EPAs with 18 component observable practice activities (OPAs) in total were developed, encompassing routines specific to each radiation technology specialty. QUEPA and EQual questionnaire data verified these EPAs were valid, and of high quality for clinical teaching and evaluation.

CONCLUSION

The consensus development of tailored EPAs enables rigorous competency assessment during medical radiation technology education in Taiwan. Further expansion of EPAs and training of clinical staff could potentially enhance care quality by producing competent professionals.

Entrustable professional activities (EPAs) for undergraduate medical education - development and exploration of social validity

BACKGROUND

The development of entrustable professional activities (EPAs) as a framework for work-based training and assessment in undergraduate medical education has become popular. EPAs are defined as units of a professional activity requiring adequate knowledge, skills, and attitudes, with a recognized output of professional labor, independently executable within a time frame, observable and measurable in its process and outcome, and reflecting one or more competencies. Before a new framework is implemented in a specific context, it is valuable to explore social validity, that is, the acceptability by relevant stakeholders.

AIM

The aim of our work was to define Core EPAs for undergraduate medical education and further explore the social validity of the constructs.

METHOD AND MATERIAL

In a nationwide collaboration, EPAs were developed using a modified Delphi procedure and validated according to EQual by a group consisting of teachers nominated from each of the seven Swedish medical schools, two student representatives, and an educational developer (n = 16). In the next step, social validity was explored in a nationwide survey. The survey introduced the suggested EPAs. For each EPA, the importance of the EPA was rated, as was the rater's perception of the present graduates' required level of supervision when performing the activity. Free-text comments were also included and analyzed.

RESULTS

Ten Core EPAs were defined and validated. The validation scores for EQual ranged from 4.1 to 4.9. The nationwide survey had 473 responders. All activities were rated as "important" by most responders, ranging from 54 to 96%. When asked how independent current graduates were in performing the ten activities, 6 to 35% reported "independent". The three themes of the free text comments were: 'relevant target areas and content'; 'definition of the activities'; and 'clinical practice and learning'.

CONCLUSION

Ten Core EPAs were defined and assessed as relevant for Swedish undergraduate medical education. There was a consistent gap between the perceived importance and the certainty that the students could perform these professional activities independently at the time of graduation. These results indicate that the ten EPAs may have a role in undergraduate education by creating clarity for all stakeholders.

Development and Validation of Entrustable Professional Activities for Provisionally Registered (Intern) Pharmacists in Australia

OBJECTIVE

The purpose of this study was to describe the development and validation of entrustable professional activities (EPAs) for a pharmacy intern training program in Australia.

METHODS

Performance outcomes with EPAs suggested as an assessment in the intern year were identified and EPAs were developed. The EQual rubric, designed for EPAs in graduate medical education settings, was used to validate the EPAs. Eight subject matter experts were recruited to validate 14 EPAs using prespecified cutoffs in the EQual rubric to determine the degree of alignment with established key domains of the EPA construct. Descriptive statistics for the EQual rubric scores were calculated for each of the 14 EPAs overall and for each of the 3 domains. Any EPA falling below the prespecified cut score required revisions and a second round of validation.

RESULTS

Eight responses were recorded for the first round of validation. Seven EPAs were below the discrete units of work cutoff (4.17); 2 of these EPAs were also below the EPA curricular role cutoff (4.00). All scores described in the EPAs as entrustable, essential, and important tasks of the profession were above the cutoff (4.00). Seven subject matter experts responded to the second round of validation. All 7 revised EPAs scored above the EQual rubric's overall cutoff of 4.07.

CONCLUSION

Fourteen EPAs were validated for use with provisionally registered (intern) pharmacists. To our knowledge, this is the first time that the EQual rubric has been used in the evaluation and validation of pharmacy EPAs.

Evaluating the Quality of the Core Entrustable Professional Activities for New Pharmacy Graduates

This study aimed to evaluate the quality of the American Association of Colleges of Pharmacy Core Entrustable Professional Activities (Core EPAs) for New Pharmacy Graduates according to standards outlined in competency-based education literature utilizing the Queen's EPA Quality (EQual) rubric. A cohort of pharmacists with EPA expertise rated Core EPA quality with the EQual rubric and provided recommendations for revisions. A generalizability study determined the reliability of the EQual ratings with pharmacist users. Nine pharmacists responded (4.4%). Most EPAs (9/15) did not reach the overall cut-off score, indicating low quality. EPAs 1 through 5 and EPA 14 (fulfill a medication order) were deemed high quality. EPA 12 (use evidence-based information to advance patient care) scored the lowest at 3.47 (SEM 0.29). EPA 14 scored the highest at 4.60 (SEM 0.14). EPA 15 (create a written plan for continuous professional development) was the only EPA to fail to reach the cut-off across all EQual domains. EPAs in the Patient Care Provider Domain received significantly higher ratings than other EPAs. On average, three respondents recommended revision for each. Most comments aligned with the EPA's EQual rubric performance. The generalizability study analysis revealed excellent reliability (G = 0.80). Determining EPA quality utilizing objective measurement tools should drive EPA development and revisions to more accurately reflect the roles, responsibilities, and expectations of pharmacists on the healthcare team.

EPMO: A novel medical student assessment tool that integrates entrustable professional activities, prime, and the modified Ottawa coactivity scale

PURPOSE

Alignment of workplace-based assessments (WPBA) with core entrustable professional activities (EPAs) for entering residency may provide opportunities to monitor student progress across the continuum of undergraduate medical education. Core EPAs, however, reflect tasks of varying degrees of difficulty and faculty assessors are not accustomed to rating students based on entrustability. Expectations of student progress should vary depending on the complexity of the tasks associated with the EPAs. An assessment tool that orients evaluators to the developmental progression of specific EPA tasks will be critical to fairly evaluate learners.

METHODS

The authors developed an EPA assessment tool combining the frameworks of Professionalism, Reporter, Interpreter, Manager, Educator (PRIME), and Modified Ottawa coactivity scales. Only those EPAs that could be repeatedly observed and assessed across clinical clerkships were included. From July 2019 to March 2020, third-year medical students across multiple clerkships were assessed using this tool. The authors hypothesized that if the tool was applied correctly, ratings of learner independence would be lower with higher complexity tasks and that such ratings would increase over the course of year with ongoing clinical learning.

RESULTS

Assessment data for 247 medical students were similar across clerkships suggesting that evaluators in diverse clinical contexts were able to use this tool to assign scores reflective of developing entrustability in the workplace. Faculty rated student entrustability highest in skills emphasized in the pre-clerkship curriculum (professionalism and reporter) and progressively lower in more advanced skills (interpreter and manager). Students' ratings increased over time with more clinical exposure.

CONCLUSIONS

The authors developed a composite WBPA tool that combines the frameworks of EPAs, PRIME, and Modified Ottawa Co- Activity and demonstrated the usability of applying it for learner assessments in clinical settings. Further multicenter studies with cohorts of pre- and post-clerkship students may provide additional validity evidence for the tool.

The Need for Quality Assessment of Entrustable Professional Activities in Pharmacy Education

Entrustable professional activities (EPAs) are specific tasks that a professional is entrusted to perform autonomously and, together, they define the scope of a profession. There are specific attributes that a task must possess both structurally and conceptually to be classified as an EPA. A high-quality EPA must be an observable, measurable, and professional task that requires training to execute, is fit for entrustment, and can be performed independently. In 2017, the American Association of Colleges of Pharmacy (AACP) defined 15 core EPAs expected of a pharmacy learner upon graduation (Core EPAs). Despite acceptance and implementation by pharmacy schools across the country, the Core EPAs have not been evaluated using an objective assessment tool to ensure that they meet specific EPA quality standards outlined in the literature. This article describes existing objective assessment tools for EPA quality and highlights the importance of ensuring high-quality EPAs in pharmacy education, which would be an important step for the AACP Academic Affairs Committee to take to further develop EPAs for implementation.

Multifaceted Approach to Evaluation in a Pediatric and Adolescent Gynecology Rotation for Medical Students

STUDY OBJECTIVE

We evaluated whether and to what extent a novel medical student rotation in pediatric and adolescent gynecology (PAG) increases clinical knowledge and skills and meets student needs and expectations.

DESIGN

Constructivist prospective pre-post study and post-rotation student survey SETTING: Academic medical center PARTICIPANTS: Pilot study of 9 medical students, which represents the entire population of those who completed the rotation.

INTERVENTIONS

Four-week clinical rotation in PAG MAIN OUTCOME MEASURES: Changes in clinical knowledge were measured by a pre- and post-intervention multiple-choice assessment, and clinical skills were assessed before and after the intervention using entrustable professional activities (EPAs); these data were analyzed with paired Student's t tests. Student evaluations of the rotation were measured through an anonymous, end-of-rotation, closed- and open-ended survey and were analyzed using descriptive statistics.

RESULTS

A statistically significant increase in clinical knowledge was observed post-rotation, with a mean pretest score of 67.0% (standard deviation [SD] 1.7%) and a mean posttest score of 75.2% (SD 3.2%, P = 0.02). Statistically significant increases were observed for all EPAs between the first and final day of the rotation. Eight students who completed the post-rotation survey rated the rotation favorably (5 on a scale from 1 to 5).

CONCLUSION

A multipronged evaluation showed that a new PAG clinical rotation significantly increased medical students' clinical skills and knowledge. This multifaceted evaluation method provides valuable insights to educators on how best to tailor a rotation to individual learners' levels of clinical skills and knowledge. If comparable rotations could be instituted and similarly evaluated in other medical schools, a noticeable knowledge/skill gap among trainees might be addressed.

Pre-clerkship EPA assessments: a thematic analysis of rater cognition

BACKGROUND

Entrustable Professional Activities (EPAs) assessments measure learners' competence with an entrustment or supervisory scale. Designed for workplace-based assessment EPA assessments have also been proposed for undergraduate medical education (UME), where assessments frequently occur outside the workplace and may be less intuitive, raising validity concerns. This study explored how assessors make entrustment determinations in UME, with additional specific comparison based on familiarity with prior performance in the context of longitudinal student-assessor relationships.

METHODS

A qualitative approach using think-alouds was employed. Assessors assessed two students (familiar and unfamiliar) completing a history and physical examination using a supervisory scale and then thought-aloud after each assessment. We conducted a thematic analysis of assessors' response processes and compared them based on their familiarity with a student.

RESULTS

Four themes and fifteen subthemes were identified. The most prevalent theme related to "student performance." The other three themes included "frame of reference," "assessor uncertainty," and "the patient." "Previous student performance" and "affective reactions" were subthemes more likely to inform scoring when faculty were familiar with a student, while unfamiliar faculty were more likely to reference "self" and "lack confidence in their ability to assess."

CONCLUSIONS

Student performance appears to be assessors' main consideration for all students, providing some validity evidence for the response process in EPA assessments. Several problematic themes could be addressed with faculty development while others appear to be inherent to entrustment and may be more challenging to mitigate. Differences based on assessor familiarity with student merits further research on how trust develops over time.

Entrustable professional activities versus competencies and skills: Exploring why different concepts are often conflated

Despite explanations in the literature, a returning question in the use of entrustable professional activities (EPAs) is how to distinguish them from competencies and skills. In this article, we attempt to analyze the causes of the frequent confusion and conflation of EPAs with competencies and skills, and argue why the distinction is important for education, qualification and patient safety. 'Tracheotomy', 'lumbar puncture', 'interprofessional collaboration' for example are colloquially called 'skills', but its is a person's ability to perform these activities that is the actual skill; the EPA is simply the activity itself. We identify two possible causes for the confusion. One is a tendency to frame all educational objectives as EPAs. Many objectives of medical training can be conceptualized as EPAs, if 'the ability to do X' is the corresponding competency; but that does not work for all. We offer ways to deal with objectives of training that are not usefully conceptualized as EPAs. A more fundamental cause relates to entrustment decisions. The permission to contribute to health care reflects entrustment. Entrustment decisions are the links or pivots between a person's readiness for the task and the actual task execution. However, if entrustment decisions do not lead to increased autonomy in the practice of health care, but only serve to decide upon the advancement to a next stage of training, EPAs can become the tick boxes learners feel they need to collect to 'pass'. Gradually, then, EPAs can loose their original meaning of units of practice for which one becomes qualified.

A Pilot Study of the Generalizability of Preclinical Entrustment Assessments in Undergraduate Medical Education

PURPOSE

The reproducibility and consistency of assessments of entrustable professional activities (EPAs) in undergraduate medical education (UME) have been identified as potential areas of concern. EPAs were designed to facilitate workplace-based assessments by faculty with a shared mental model of a task who could observe a trainee complete the task multiple times. In UME, trainees are frequently assessed outside the workplace by faculty who only observe a task once.

METHOD

In November 2019, the authors conducted a generalizability study (G-study) to examine the impact of student, faculty, case, and faculty familiarity with the student on the reliability of 162 entrustment assessments completed in a preclerkship environment. Three faculty were recruited to evaluate 18 students completing 3 standardized patient (SP) cases. Faculty familiarity with each student was determined. Decision studies were also completed. Secondary analysis of the relationship between student performance and entrustment (scoring inference) compared average SP checklist scores and entrustment scores.

RESULTS

G-study analysis revealed that entrustment assessments struggled to achieve moderate reliability. The student accounted for 30.1% of the variance in entrustment scores with minimal influence from faculty and case, while the relationship between student and faculty accounted for 26.1% of the variance. G-study analysis also revealed a difference in generalizability between assessments by unfamiliar (φ = 0.75) and familiar (φ = 0.27) faculty. Subanalyses showed that entrustment assessments by familiar faculty were moderately correlated to average SP checklist scores (r = 0.44, P < .001), while those by unfamiliar faculty were weakly correlated (r = 0.16, P = .13).

CONCLUSIONS

While faculty and case had a limited impact on the generalizability of entrustment assessments made outside the workplace in UME, faculty who were familiar with a student's ability had a notable impact on generalizability and potentially on the scoring validity of entrustment assessments, which warrants further study.

Revisiting the Core Entrustable Professional Activities for Entering Residency

The Core EPAs for Entering Residency Pilot project aimed to test the feasibility of implementing 13 entrustable professional activities (EPAs) at 10 U.S. medical schools and to gauge whether the use of the Core EPAs could improve graduates' performance early in residency. In this manuscript, the authors (members of the pilot institutions and Association of American Medical Colleges staff supporting the project evaluation) describe the schools' capacity to collect multimodal evidence about their students' performance in each of the Core EPAs and the ability of faculty committees to use those data to make decisions regarding learners' readiness for entrustment. In reviewing data for each of the Core EPAs, the authors reflected on how each activity performed as an EPA informed by how well it could be assessed and entrusted. For EPAs that did not perform well, the authors examined whether there are underlying practical and/or theoretical issues limiting its utility as a measure of student performance in medical school.

Working with entrustable professional activities in clinical education in undergraduate medical education: a scoping review

BACKGROUND

Entrustable professional activities (EPAs) are increasingly used in undergraduate medical education (UME). We conducted a scoping review to summarize the evidence for the use of EPAs in clinical rotations in UME.

METHODS

We searched multiple databases for scoping reviews based on the PRISMA guidelines for articles reporting qualitative and quantitative research, as well as conceptual and curriculum development reports, on EPAs in UME clinical rotations.

RESULTS

We identified 3309 records by searching through multiple databases. After the removal of duplicates, 1858 reports were screened. A total of 36 articles were used for data extraction. Of these, 47% reported on EPA and EPA-based curriculum development for clerkships, 50% reported on implementation strategies, and 53% reported on assessment methods and tools used in clerkships. Validity frameworks for developing EPAs in the context of clerkships were inconsistent. Several specialties reported feasible implementation strategies for EPA-based clerkship curricula, however, these required additional faculty time and resources. Limited exposure to clinical activities was identified as a barrier to relevant learning experiences. Educators used nationally defined, or specialty-specific EPAs, and a range of entrustability and supervision scales. We found only one study that used an empirical research approach for EPA assessment. One article reported on the earlier advancement of trainees from UME to graduate medical education based on summative entrustment decisions.

CONCLUSIONS

There is emerging evidence concerning how EPAs can be effectively introduced to clinical training in UME. Specialty-specific, nested EPAs with context-adapted, entrustment-supervision scales might be helpful in better leveraging their formative assessment potential.

An analysis of core EPAs reveals a gap between curricular expectations and medical school graduates' self-perceived level of competence

BACKGROUND

Entrustable Professional Activities (EPAs) are being implemented worldwide as a means to promote competency-based medical education. In Switzerland, the new EPA-based curriculum for undergraduate medical education will be implemented in 2021. The aim of our study was to analyze the perceived, self-reported competence of graduates in 2019. The data represent a pre-implementation baseline and will provide guidance for curriculum developers.

METHODS

Two hundred eighty-one graduates of the Master of Human Medicine program of the University of Zurich who had passed the Federal Licensing Exam in September 2019 were invited to complete an online survey. They were asked to rate their needed level of supervision ("observe only", "direct, proactive supervision", "indirect, reactive supervision") for 46 selected EPAs. We compared the perceived competence with the expected competence of the new curriculum.

RESULTS

The response rate was 54%. The need for supervision expressed by graduates varied considerably by EPA. The proportion of graduates rating themselves at expected level was high for "history taking", "physical examination" "and documentation"; medium for "prioritizing differential diagnoses", "interpreting results" and "developing and communicating a management plan"; low for "practical skills"; and very low for EPAs related to "urgent and emergency care".

CONCLUSIONS

Currently, there are significant gaps between the expectations of curriculum developers and the perceived competences of students. This is most obvious for practical skills and emergency situations. The new curriculum will either need to fill this gap or expectations might need to be revised.

Constructing Approaches to Entrustable Professional Activity Development that Deliver Valid Descriptions of Professional Practice

Issue: Entrustable Professional Activities (EPAs) describe the core tasks health professionals must be competent performing prior to promotion and/or moving into unsupervised practice. When used for learner assessment, they serve as gateways to increased responsibility and autonomy. It follows that identifying and describing EPAs is a high-stakes form of work analysis aiming to describe the core work of a profession. However, hasty creation and adoption of EPAs without rigorous attention to content threatens the quality of judgments subsequently made from using EPA-based assessment tools. There is a clear need for approaches to identify validity evidence for EPAs themselves prior to their deployment in workplace-based assessment. Evidence: For EPAs to realize their potential in health professions education, they must first be constructed to reflect accurately the work of that profession or specialty. If the EPAs fail to do so, they cannot predict a graduate's readiness for or future performance in professional practice. Evaluating the methods used for identification, description, and adoption of EPAs through a construct validity lens helps give leaders and stakeholders of EPA development confidence that the EPAs constructed are, in fact, an accurate representation of the profession's work. Implications: Application of a construct validity lens to EPA development impacts all five commonly followed steps in EPA development: selection of experts; identification of candidate EPAs; iterative revisions; evaluation of proposed EPAs; and formal adoption of EPAs into curricula. It allows curricular developers to avoid pitfalls, bias, and common mistakes. Further, construct validity evidence for EPA development provides assurance that the EPAs adopted are appropriate for use in workplace-based assessment and entrustment decision-making.