Entrustable Professional Activities and Entrustment Decision Making: A Development and Research Agenda for the Next Decade

Graduating Residents' Readiness for Unsupervised Practice

OBJECTIVE

This study sought to determine whether graduating residents were deemed ready for unsupervised practice for each of the 17 general pediatrics entrustable professional activities (EPAs).

METHODS

At the end of the 2021-22, 2022-23, and 2023-24 academic years, the authors collected entrustment-supervision levels assigned by clinical competency committees for graduating residents from pediatric and medicine/pediatrics residency training programs to determine readiness for unsupervised practice at the time of graduation. They did this for each of the general pediatrics EPAs and examined the levels reported to determine the proportion of residents ready for unsupervised practice on each EPA and on all EPAs. They compared rates of readiness by academic year using a mixed-effects logistic regression model.

RESULTS

Across all EPAs, 33 190 total entrustment-supervision levels were reported for 2276 graduating pediatrics residents, and 2607 entrustment-supervision levels were reported for 168 graduating medicine/pediatrics residents. There were no EPAs in which programs reported more than 89% of residents as ready for unsupervised practice at the time of graduation. Only 31.3% of graduating residents with observations on all EPAs (414/1322) were deemed ready for unsupervised practice for all EPAs. Graduating medicine/pediatrics residents were more likely than pediatrics residents to be deemed ready for all EPAs (P = .002). Across study years, the rates of readiness at graduation for all EPAs rose from 18.0% to 38.5% (linear contrast P < .001).

CONCLUSION

While there are reasons beyond actual resident readiness that may contribute, this study highlights a gap in readiness for unsupervised practice at the time of graduation.

EMYWAY Workplace-Based Entrustable Professional Activities Assessments in Otolaryngology Residency Training: A Nationwide Experience

OBJECTIVE

To present workplace-based entrustable professional activities (EPAs) assessment data from the first 2 years of the EMYWAY platform in otolaryngology residency training in Taiwan.

STUDY DESIGN

Two-year cross-sectional study.

SETTING

Otolaryngology training programs.

METHODS

In 2020, the Taiwan Society of Otorhinolaryngology-Head and Neck Surgery (TSO-HNS) developed a workplace-based assessment (WBA) framework with 11 EPAs, integrating milestones to evaluate resident competency. In 2021, TSO-HNS piloted the EMYWAY platform for WBAs, which includes an EPA-based assessment workflow, coaching feedback, and a dashboard displaying residents' entrustment-supervision levels. Data are analyzed annually for accreditation and curriculum enhancement. This study reports on the pilot year and the first full-scale year of implementation.

RESULTS

Eleven programs participated in the pilot year. Subsequently, 362 faculty members and 274 resident physicians from 34 programs nationwide engaged with EMYWAY. In the full-scale year from August 2022 to July 2023, 9805 responses were recorded, primarily from surgical theaters (45.9%; 4502/9805) and third-year residents (23.8%; 2331/9805). The most frequently evaluated EPAs were "head and neck" (17.5%; 1716/9805), "sinonasal" (13.5%; 1324/9805), and "ear" (12.2%; 1193/9805), with task complexity increasing with resident seniority (P < .0001). A positive correlation was found between residents' self-assessments and faculty members' ratings (r = 0.531; P < .001). Over 98.2% of residents and 88.4% of faculty members provided substantial feedback (>10 words). Analysis of WBAs reported by training programs identified faculty development targets and teaching-intensive tasks.

CONCLUSION

EMYWAY effectively documents workplace learning and tracks resident competency progression. Continuous improvement of WBA quality is essential for advancing the competency-based medical education ecosystem.

Factors influencing the incorporation of Entrustable Professional Activities into assessment in nutrition and dietetics education: A systematic review

AIMS

This study aimed to identify and synthesise the evidence on factors influencing the incorporation of Entrustable Professional Activities into assessment in nutrition and dietetics education.

METHODS

A systematic review was conducted with a narrative synthesis and was undertaken and reported in accordance with the PRISMA guidelines. Six electronic databases were searched (MEDLINE, CINAHL, SCOPUS, PsycINFO, Web of Science, and EMBASE) on 9 September 2024. Methodological quality was assessed using the Critical Appraisal Skills Program checklists. Key patterns identified from the narrative synthesis of the included manuscripts were labelled as themes and represented in a figure.

RESULTS

Across the international literature, six articles were identified revealing six main and interconnected themes related to factors influencing the incorporation of Entrustable Professional Activities in nutrition and dietetics. In summary, development and review processes for Entrustable Professional Activities require key stakeholder engagement in addition to ensuring they are linked to assessment structures and existing frameworks. Furthermore, technology platforms and applications appeared to support Entrustable Professional Activity incorporation, and training is an important part of integration.

CONCLUSIONS

Further research on factors influencing incorporation is occurring and is suggested to continue, especially given Entrustable Professional Activities seemingly offer a tangible option to simplify the intricacy of competency-based assessment in work-based practice. However, further research to enhance understanding of whether Entrustable Professional Activities support nutrition and dietetics learners and assessors in undertaking high-quality assessment with utility is warranted.

How to Teach Clinical Reasoning in the Context of Rare Diseases in Undergraduate Medical Education

While rare diseases are individually rare, they are collectively common. Physicians are likely to see patients presenting with rare diseases during their medical practice. Despite the fact that rare diseases present diagnostic and therapeutic challenges, they are not sufficiently addressed by undergraduate medical curricula. The need to train medical students has been expressed by stakeholders in several countries. Moreover, among the objectives of the World Health Organization are to reduce a diagnostic time, decrease diagnostic errors and ensure the provision of multidisciplinary care. We have initiated an educational module aspiring to enhance understanding of rare diseases among undergraduate medical students. Based on our experience, on the literature data and related learning theories, this document presents some tips on how to foster training of undergraduate medical students in the context of rare diseases.

Developing and Validating Entrustable Professional Activities (EPAs) for Rheumatology Fellowship Training Programs in Saudi Arabia: A Delphi Study

Background

Entrustable professional activities (EPAs) define the core tasks that a graduating rheumatologist needs to perform independently in practice. The objective of this study was to develop and validate EPAs for rheumatology fellowship training programs in Saudi Arabia.

Methods

Experts met to develop an initial set of potential end-of-training EPAs by conducting a comprehensive literature review of EPAs and studying the Saudi rheumatology fellowship curriculum. Then, to validate the EPAs, we conducted two rounds of the modified Delphi technique among rheumatology experts in Saudi Arabia. A response rate of 80% was considered and the minimum number of experts needed to be 25 to 30. Descriptive statistics were utilized to describe participants' demographic characteristics and group responses to each statement in all rounds. The experts were asked to rate the relevancy of each EPA using a 5-point Likert scale in both Delphi rounds.

Results

In the preliminary phase, four rheumatologists developed an initial set of 36 core EPAs for rheumatology training program in Saudi Arabia. For the two-rounds Delphi techniques, 32 experts were invited to complete the study. The response rate of the first and second round were, 78.12% (25) and 93.75% (30), respectively. The first-round Delphi resulted in a robust consensus on 31 EPAs for rheumatology training. Five EPAs were excluded, and one new EPA was proposed. In the subsequent round, all 32 EPAs achieved strong consensus. The eliminated EPAs likely fell short in one or more of the following areas: relevance to rheumatology practice in Saudi Arabia, overlapping with other EPAs, or practical challenges in the implementation.

Conclusion

We have developed and validated a core set of EPAs for rheumatology fellowship training programs in Saudi Arabia. Mapping and identifying milestones for these EPAs are essential steps to follow to enhance workplace curriculum development.

Does a service provide safe, effective rehabilitation? An evaluation method for providers and purchasers

BACKGROUND

Independent organisations monitor the safety and governance of clinical services but do not assess specialist expertise. Peer review can assess service capability but is resource-intense and infeasible.

THE PROBLEM

How can you ensure a service provides safe, effective rehabilitation? You ask them to provide data as evidence that they can be trusted to do so. This article suggests a structured approach to providing data on entrustability.

AN ANALOGY

How is the specialist skill of a doctor in training established? They provide evidence about high-level outcomes (capabilities in practice) related to their speciality. An educational supervisor assesses whether they can be trusted to perform safely and effectively without supervision. The capabilities in practice define their expertise.

THE SOLUTION

A service can use seven high-level rehabilitation service capabilities, based on the clinical capabilities associated with medical training, with observable indicative descriptors, to collect evidence of their rehabilitation approach. A service must also select four to eight high-level competencies indicating they can rehabilitate their patient caseload safely and effectively. These competencies also need indicative descriptors as evidence of their performance in the service; 11 examples are given.

CAPABILITIES.

The seven rehabilitation capabilities are: using the biopsychosocial model, having a multi-professional team, making a person-centred rehabilitation plan, working collaboratively across all boundaries, tailoring treatments to the patient's needs, ensuring staff have specific competencies required for their caseload, and acknowledging and managing uncertainty and complexity.

CONCLUSION.

Service providers could use this structured approach to develop and provide users with evidence of their rehabilitation expertise.

Multifaceted Strategy That Improves Students' Achievement of Entrustable Professional Activities Across Advanced Pharmacy Practice Experiences

OBJECTIVE

To outline an approach to help students achieve Entrustable Professional Activities (EPAs) entrustment during a sequence of Advanced Pharmacy Practice Experiences (APPEs) by implementing longitudinal monitoring and individualized intervention and remediation strategies.

METHODS

Using the recommended EPAs within the core APPEs (acute care, ambulatory care, community, institutional), students were expected to achieve entrustment on all EPAs by graduation. A longitudinal monitoring approach, using an "EPA report card," was implemented to continuously identify students at risk of not meeting the EPA requirement of "Level 3" entrustment (perform with reactive supervision). Individualized interventions, including proactive outreach and in-sequence remediation, were incorporated into the APPE core and elective sequence to help ensure all students were entrusted by the end of APPEs without requiring further end-of-year remediation to graduate.

RESULTS

For the graduating classes of 2023 and 2024, 12% (8 of 69) and 16% (12 of 75) students, respectively, were identified as at risk of not meeting EPA entrustment. Proactive outreach, in-sequence remediation, or a combination of both strategies, were used to enhance learning and EPA performance. As a result, all students achieved "Level 3" entrustment on the deficient EPA(s) by the end of the APPE sequence. No student required further end-of-year remediation to graduate.

CONCLUSION

Utilizing a multifaceted strategy provided timely, real-world practice opportunities to improve the students' achievement of EPAs across the APPE curriculum and decreased the need for end-of-year remediation and potential graduation delays.

A realist synthesis of prospective entrustment decision making by entrustment or clinical competency committees

INTRODUCTION

The real-world mechanisms underlying prospective entrustment decision making (PEDM) by entrustment or clinical competency committees (E/CCCs) are poorly understood. To advance understanding in this area, the authors conducted a realist synthesis of the published literature to address the following research question: In E/CCC efforts to make defensible prospective entrustment decisions (PEDs), what works, for whom, under what circumstances and why?

METHODS

Realist work seeks to understand the contexts (C), mechanisms (M) and outcomes (O) that explain how and why things work (or do not). In the authors' study, contexts included individual E/CCC members, E/CCC structures and processes, and training programmes. The outcome (i.e. desired outcome) was a PED. Mechanisms were a substantial focus of the analysis and informed the core findings. To define a final corpus of 52 included papers, the authors searched four databases, screened all results from those searches and performed a full-text review of a subset of screened papers. Data extraction focused on developing context-mechanism-outcome configurations from the papers, which were used to create a theory for how PEDM leads to PEDs.

RESULTS

PEDM is often driven by default (non-deliberate) decision making rather than a deliberate process of deciding whether a trainee should be entrusted or not. When defaulting, some E/CCCs find red flags that sometimes lead to being more deliberate with decision making. E/CCCs that seek to be deliberate describe PEDM that can be effortful (when data are insufficient or incongruent) or effortless (when data are robust and tell a congruent story about a trainee). Both information about trainee trustworthiness and the sufficiency of data about trainee performance influence PEDM. Several moderators influence what is considered to be sufficient data, how trustworthiness data are viewed and how PEDM is carried out. These include perceived consequences and associated risks, E/CCC member trust propensity, E/CCC member personal knowledge of and experience with trainees and E/CCC structures and processes.

DISCUSSION

PEDM is rarely deliberate but should be. Data about trainee trustworthiness are foundational to making PEDs. Bias, equity and fairness are nearly absent from the papers in this synthesis, and future efforts must seek to advance understanding and practice regarding the roles of bias, equity and fairness in PEDM.

Assessment of Entrustable Professional Activities Among Dutch Endocrine Supervisors

Entrustable Professional Activities (EPAs) are an important tool to support individualisation of medical training in a competency-based setting and are increasingly implemented in the clinical speciality training for endocrinologist. This study aims to assess interrater agreement and factors that potentially impact EPA scores. Five known factors that affect entrustment decisions in health profesions training (capability, integrity, reliability, humility, agency) were used in this study. A case-vignette study using standardised written cases. Case vignettes (n = 6) on the topics thyroid disease, pituitary disease, adrenal disease, calcium and bone disorders, diabetes mellitus, and gonadal disorders were written by two endocrinologists and a medical education expert and assessed by endocrinologists experienced in the supervision of residents in training. Primary outcome is the inter-rater agreement of entrustment decisions for endocrine EPAs among raters. Secondary outcomes included the dichotomous interrater agreement (entrusted vs. non-entrusted), and an exploration of factors that impact decision-making. The study protocol was registered and approved by the Ethical Review Board of the Netherlands Association for Medical Education (NVMO-ERB # 2020.2.5). Nine endocrinologists from six different academic regions participated. Overall, the Fleiss Kappa measure of agreement for the EPA level was 0.11 (95% CI: 0.03-0.22) and for the entrustment decision 0.24 (95% CI 0.11-0.37). Of the five features that impacted the entrustment decision, capability was ranked as the most important by a majority of raters (56%-67%) in every case. There is a considerable discrepancy between the EPA levels assigned by different raters. These findings emphasise the need to base entrustment decisions on multiple observations, made by a team of supervisors and enriched with factors other than direct medical competence.

Domains Influencing Faculty Decisions on the Level of Supervision Required for Anesthesiology EPAs with Analysis of Feedback Comments

OBJECTIVE

The purpose of this qualitative study was to examine responses related to entrustment and feedback comments from an assessment tool.

DESIGN

Qualitative analyses using semi-structured interviews and analysis of narrative comments.

SETTING

Main hospital OR suite at a large academic medical center.

PARTICIPANTS

faculty, and residents who work in the OR suite.

RESULTS

Seven of the 14 theoretical domains from the Theoretical Domains Framework were identified as influencing faculty decision on entrustment: knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. The majority (651/1116 (58.4%)) of faculty comments were critical/modest praise and relevant, consistent across all 6 EPAs. The written in feedback comments for all 1,116 Web App EPA assessments yielded a total of 1,599 sub-competency specific responses. These responses were mapped to core competencies, and at least once to 13 of the 23 ACGME subcompetencies.

CONCLUSIONS

Domains identified as influencing faculty decision on entrustment were knowledge, skills, intention, memory/attention/decision processes, environmental context, and resources, beliefs of capabilities, and reinforcement. Most narrative feedback comments were critical/modest praise and relevant, consistent across each of the EPAs.

Clinical supervisors' experience of a first-time application of entrustable professional activities in clinical supervision of medical students: findings from a Swedish pilot study

BACKGROUND

Recently, all medical universities in Sweden jointly developed a framework for Entrustable Professional Activities (EPAs) for work-based training and assessment. This framework is now being introduced nationally in the new 6-year undergraduate medical programme that directly lead to a licence to practise. When EPAs are introduced, it is of central importance to gain clinical supervisors' acceptance to apply the framework in their supervision of students. The aim of this study was therefore to investigate how clinical supervisors, not familiar with EPAs, experience clinical supervision using the framework for EPAs.

METHODS

We used a purposive sampling to recruit clinical supervisors. They were given written information on EPAs with a selection of suitable EPAs and the Swedish observation rating scale for assessment of autonomy, and they were offered to attend a 30-minute introductory web course. The participants were informed that EPAs were to be tested, and the students were asked to participate. After the study period the clinical supervisors participated in semi-structured interviews. Inductive qualitative content analysis was used to analyse the transcribed interviews.

RESULTS

Three general themes emerged in the qualitative analysis: Promoting Feedback, Trusting Assessments and Engaging Stakeholders. The participants described benefits from using EPAs, but pointed out a need for preparation and adaptation to facilitate implementation. The structure was perceived to provide structured support for feedback, student involvement, entrustment decisions, enabling supervisors to allow the students to do more things independently, although some expressed caution to rely on others' assessments. Another concern was whether assessments of EPAs would be perceived as a form of examination, steeling focus from formative feedback. To understand the concept of EPA, the short web-based course and written information was regarded as sufficient. However, concern was expressed whether EPA could be applied by all clinical supervisors. Involvement and adaption of the workplace was pointed out as important since more frequent observation and feedback, with documentation requirements, increase the time required for supervision.

CONCLUSIONS

EPAs were accepted as beneficial, promoting structured feedback and assessments of the students' autonomy. Preparation of supervisors and students as well as involvement and adaptation of the workplace was pointed out as important.

Assessment for learning of paediatric cardiology trainees in 41 centres from 19 European countries

BACKGROUND

Limited data exist on how trainees in paediatric cardiology are assessed among countries affiliated with the Association of European Paediatric and Congenital Cardiology.

METHODS

A structured and approved questionnaire was circulated to educationalists/trainers in 95 Association for European Paediatric and Congenital Cardiology training centres.

RESULTS

Trainers from 46 centres responded with complete data in 41 centres. Instructional design included bedside teaching (41/41), didactic teaching (38/41), problem-based learning (28/41), cardiac catheterisation calculations (34/41), journal club (31/41), fellows presenting in the multidisciplinary meeting (41/41), fellows reporting on echocardiograms (34/41), clinical simulation (17/41), echocardiography simulation (10/41), and catheterisation simulation (3/41). Assessment included case-based discussion (n = 27), mini-clinical evaluation exercise (mini-CEX) (n = 12), directly observed procedures (n = 12), oral examination (n = 16), long cases (n = 11), written essay questions (n = 6), multiple choice questions (n = 5), and objective structured clinical examination (n = 2). Entrustable professional activities were utilised in 10 (24%) centres. Feedback was summative only in 17/41 (41%) centres, formative only in 12/41 (29%) centres and a combination of formative and summative feedback in 10/41 (24%) centres. Written feedback was provided in 10/41 (24%) centres. Verbal feedback was most common in 37/41 (90 %) centres.

CONCLUSION

There is a marked variation in instructional design and assessment across European paediatric cardiac centres. A wide mix of assessment tools are used. Feedback is provided by the majority of centres, mostly verbal summative feedback. Adopting a programmatic assessment focusing on competency/capability using multiple assessment tools with regular formative multisource feedback may promote assessment for learning of paediatric cardiology trainees.

Pedagogic strategies of supervisors in healthcare placements

PURPOSE

Supervisors are responsible to train students in healthcare placements. Although there is knowledge about workplace learning and supervision in general, little is known about supervisors' pedagogic strategies in specific healthcare placements. In this study, we identify how supervisors' reasoning and interrelated actions manifest in physiotherapy and nursing work settings.

METHODS

Following the stimulating recall approach, we conducted 16 interviews with supervisors at seven work settings. Using a theoretical framework of workplace supervision, we performed a deductive template analysis.

RESULTS

Four configurations of pedagogic strategies reveal how supervision manifests in healthcare placements. The results provide unique insights into specific supervision moments, and elucidate the situatedness of the supervisors' strategies.

CONCLUSIONS

The present study illustrates the variation in aims and focus of supervisors in placements. Supervisors' pedagogic strategies were found to be mainly based on (A) role modelling, (B) overall support, (C) trust, and (D) letting go. Further research is needed to investigate the interplay between supervisors and students in learning situations within work settings.

Multi-stakeholder validation of entrustable professional activities for a family medicine care of the elderly residency program: A focus group study

Entrustable Professional Activities (EPAs) have become widely used within Competency-Based Medical Education (CBME) for the training and evaluation of residents. Little is known about the effectiveness of incorporating multiple stakeholder groups in the validation of EPAs. Here, we seek to validate an EPA framework developed for the University of Manitoba Care of the Elderly Enhanced Skills program using online focus groups consisting of five stakeholder groups. Participants were recruited to take part in one of five online focus groups, one for each stakeholder group (physician faculty, residents, non-physician healthcare professionals, administrators/managers, and patients). Each group met one time for 90 minutes over ZOOM®. The themes arising from stakeholder feedback suggest that successful EPAs must neither be too specific nor too expansive in scope, clearly delineate appropriate means of evaluation, and indicate specific clinical settings in which each EPA should be evaluated. Cross-cutting themes included requiring trainees to collaborate with other professionals when it would optimize patient care, and preparing trainees to advocate for their patients' health (Advocacy). The present study demonstrates that multi-stakeholder analysis yields diverse feedback that can help make EPAs more clear, easier to use in evaluation, and more socially accountable.

Exploring Factors for Implementation of EPAs in Pediatric Subspecialty Fellowships: A Qualitative Study of Program Directors

OBJECTIVE

To understand fellowship program directors' (FPDs) perspectives on facilitators and barriers to using entrustable professional activities (EPAs) in pediatric subspecialty training.

METHODS

We performed a qualitative study of FPDs, balancing subspecialty, program size, geographic region and current uses of EPAs. A study coordinator conducted 1-on-1 interviews using a semistructured approach to explore EPA use or nonuse and factors supporting or preventing their use. Investigators independently coded transcribed interviews using an inductive approach and the constant comparative method. Group discussion informed code structure development and refinement. Iterative data collection and analysis continued until theoretical sufficiency was achieved, yielding a thematic analysis.

RESULTS

Twenty-eight FPDs representing 11 pediatric subspecialties were interviewed, of whom 16 (57%) reported current EPA use. Five major themes emerged: (1) facilitators including the intuitive nature and simple wording of EPAs; (2) barriers such as workload burden and lack of a regulatory requirement; (2) variable knowledge and training surrounding EPAs, leading to differing levels of understanding; (3) limited current use of EPAs, even among self-reported users; and (4) complementary nature of EPAs and milestones. FPDs acknowledged the differing strengths of both EPAs and milestones but sought additional knowledge about the value added by EPAs for assessing trainees, including the impact on outcomes.

CONCLUSIONS

Identified themes can inform effective and meaningful EPA implementation strategies: Supporting and educating FPDs, ongoing assessment of the value of EPAs in training, and practical integration with current workflow. Generating additional data and engaging stakeholders is critical for successful implementation for the pediatric subspecialties.

Measuring the Learning Orientation Fostered by Pediatric Residency Programs With the Graduate Medical Education Learning Environment Inventory Instrument

INTRODUCTION

Mastery learning orientation, conceptualized as a growth mindset, can be beneficial to learners in medical education and is supported by a program...s learning environment. Currently, there are no instruments which reliably assess the learning orientation of a graduate medical education program...s learning environment.

OBJECTIVE

To explore the reliability and validity of the Graduate Medical Education Learning Environment Inventory (GME-LEI).

METHODS

Leaders of the Education in Pediatrics Across the Continuum (EPAC) project revised Krupat...s Educational Climate Inventory to create the GME-LEI. We investigated the GME-LEI...s reliability and validity through confirmatory factor and parallel factor analyses and calculated Cronbach...s alpha for each subscale. We compared mean subscale scores between residents in traditional programs and the EPAC project. As EPAC is known to foster a mastery-focused learning orientation, we hypothesized differences detected between resident groups would strengthen the instrument...s validity.

RESULTS

One hundred and twenty-seven pediatric residents completed the GME-LEI. The final 3-factor model was an acceptable fit to the data, and Cronbach...s alpha for each subscale was acceptable (Centrality: 0.87; Stress: 0.73; Support: 0.77). Mean scores on each subscale varied by program type (EPAC vs traditional) with EPAC residents reporting statistically significant higher scores in the Centrality of Learning subscale (2.03, SD 0.30, vs 1.79, SD 0.42; P.ß=.ß.023; scale of 1...4).

CONCLUSIONS

The GME-LEI reliably measures 3 distinct aspects of the GME learning environment with respect to learning orientation. The GME-LEI may be used to help programs better monitor the learning environment and make changes to support mastery-oriented learning.

The Urgency of Now: Rethinking and Improving Assessment Practices in Medical Education Programs

Assessment is essential to professional development. Assessment provides the information needed to give feedback, support coaching and the creation of individualized learning plans, inform progress decisions, determine appropriate supervision levels, and, most importantly, help ensure patients and families receive high-quality, safe care in the training environment. While the introduction of competency-based medical education has catalyzed advances in assessment, much work remains to be done. First, becoming a physician (or other health professional) is primarily a developmental process, and assessment programs must be designed using a developmental and growth mindset. Second, medical education programs must have integrated programs of assessment that address the interconnected domains of implicit, explicit and structural bias. Third, improving programs of assessment will require a systems-thinking approach. In this paper, the authors first address these overarching issues as key principles that must be embraced so that training programs may optimize assessment to ensure all learners achieve desired medical education outcomes. The authors then explore specific needs in assessment and provide suggestions to improve assessment practices. This paper is by no means inclusive of all medical education assessment challenges or possible solutions. However, there is a wealth of current assessment research and practice that medical education programs can use to improve educational outcomes and help reduce the harmful effects of bias. The authors' goal is to help improve and guide innovation in assessment by catalyzing further conversations.

"A whole other competence story": exploring faculty perspectives on the process of workplace-based assessment of entrustable professional activities

The centrality of entrustable professional activities (EPAs) in competency-based medical education (CBME) is predicated on the assumption that low-stakes, high-frequency workplace-based assessments used in a programmatic approach will result in accurate and defensible judgments of competence. While there have been conversations in the literature regarding the potential of this approach, only recently has the conversation begun to explore the actual experiences of clinical faculty in this process. The purpose of this qualitative study was to explore the process of EPA assessment for faculty in everyday practice. We conducted 18 semi-structured interviews with Anesthesia faculty at a Canadian academic center. Participants were asked to describe how they engage in EPA assessment in daily practice and the factors they considered. Interviews were audio-recorded, transcribed, and analysed using the constant comparative method of grounded theory. Participants in this study perceived two sources of tension in the EPA assessment process that influenced their scoring on official forms: the potential constraints of the assessment forms and the potential consequences of their assessment outcome. This was particularly salient in circumstances of uncertainty regarding the learner's level of competence. Ultimately, EPA assessment in CBME may be experienced as higher-stakes by faculty than officially recognized due to these tensions, suggesting a layer of discomfort and burden in the process that may potentially interfere with the goal of assessment for learning. Acknowledging and understanding the nature of this burden and identifying strategies to mitigate it are critical to achieving the assessment goals of CBME.

"What kind of doctor do you want to become?": Clinical supervisors' perceptions of their roles in the professional identity formation of General Practice residents

PURPOSE

Supporting the development of a professional identity is a primary objective in postgraduate education. Few empirical studies have explored professional identity formation (PIF) in residency, and little is known about supervisors' perceptions of their roles in residents' PIF. In this study, we sought to understand how supervisors perceive their roles in the PIF of General Practice (GP) residents.

MATERIALS AND METHODS

Guided by principles of qualitative description, we conducted eight focus groups with 55 supervisors at four General Practice training institutes across the Netherlands. Informed by a conceptual framework of PIF, we performed a thematic analysis of focus group transcripts.

RESULTS

Three themes related to how GP supervisors described their roles in supporting residents' PIF: supervising with the desired goal of GP training in mind; role modeling and mentoring as key strategies to achieve that goal; and the value of developing bonds of trust to support the process.

CONCLUSIONS

To our knowledge, this study is the first to explore PIF in GP training from the perspective of clinical supervisors. The identified themes mirror the components of the therapeutic alliance between doctors and patients from a supervisor's perspective and highlight the pivotal roles of the supervisor in a resident's PIF.

Examining enablers and barriers to entrustable professional activity acquisition using the theoretical domains framework: A qualitative framework analysis study

Background

Without a clear understanding of the factors contributing to the effective acquisition of high-quality entrustable professional activity (EPA) assessments, trainees, supervising faculty, and training programs may lack appropriate strategies for successful EPA implementation and utilization. The purpose of this study was to identify barriers and facilitators to acquiring high-quality EPA assessments in Canadian emergency medicine (EM) training programs.

Methods

We conducted a qualitative framework analysis study utilizing the Theoretical Domains Framework (TDF). Semistructured interviews of EM resident and faculty participants underwent audio recording, deidentification, and line-by-line coding by two authors, being coded to extract themes and subthemes across the domains of the TDF.

Results

From 14 interviews (eight faculty and six residents) we identified, within the 14 TDF domains, major themes and subthemes for barriers and facilitators to EPA acquisition for both faculty and residents. The two most cited domains (and their frequencies) among residents and faculty were environmental context and resources (56) and behavioral regulation (48). Example strategies to improving EPA acquisition include orienting residents to the competency-based medical education (CBME) paradigm, recalibrating expectations relating to "low ratings" on EPAs, engaging in continuous faculty development to ensure familiarity and fluency with EPAs, and implementing longitudinal coaching programs between residents and faculty to encourage repetitive longitudinal interactions and high-quality specific feedback.

Conclusions

We identified key strategies to support residents, faculty, programs, and institutions in overcoming barriers and improving EPA assessment processes. This is an important step toward ensuring the successful implementation of CBME and the effective operationalization of EPAs within EM training programs.

The Fate of Entrustable Professional Activities After Graduation: A Survey Study Among Graduated Physician Assistants

INTRODUCTION

Entrustable professional activities (EPAs) are used during training but may also have significance after graduation. This domain has not yet been much explored. We investigated the use of EPAs among alumni of Master Physician Assistant programs in the Netherlands, where EPAs have been used since 2010. We aim to describe if and how EPAs have a role after graduation. Are EPAs used for continuous entrustment decisions or when shifting work settings?

METHODS

In September 2020, all registered Dutch physician assistants (PAs) ( n = 1441) were invited to participate in a voluntary, anonymous survey focusing on the use of EPAs after graduation. Survey items included questions regarding the use and usefulness of EPAs, views on continuous entrustment decisions, and how PAs document their competence.

RESULTS

A total of 421 PAs (response rate 29%) yielded information about the significance of EPAs after training. Among the respondents, 60% ( n = 252) reported adding new competencies and skills after graduation. One-third ( n = 120) of the respondents were trained in EPA-based programs. Almost all EPA-trained PAs (96%; n = 103) considered the EPA structure suitable to maintain and renew entrustment. Furthermore, PAs reported continued use of the EPA framework to seek formal qualifications through entrustment decisions.

CONCLUSION

This study shows that EPAs can play an ongoing role for PAs after graduation. EPA-trained PAs overwhelmingly support the continuous use of the EPA framework for entrustment decisions from graduation until retirement.

Entrustable Professional Activities and Their Relevance to Pediatric Cardiology Training

Entrustable professional activities (EPAs) have become a popular framework for medical trainee assessment and a supplemental component for milestone and competency assessment. EPAs were developed to facilitate assessment of competencies and furthermore to facilitate translation into clinical practice. In this review, we explore the rationale for the introduction of EPAs, examine whether they fulfill the promise expected of them, and contemplate further developments in their application with specific reference to training in pediatric cardiology.

Which Attributes of Credibility Matter for Quality Improvement Projects in Hospital Care-A Multiple Case Study among Hospitalists in Training

Healthcare professionals have to give substance to the role of a champion in order to successfully lead quality improvement (QI) initiatives. This study aims to unravel how hospitalists in training shape their role as a champion within the context of QI projects in hospital care and why some are more effective in leading a QI project than others. We focus on the role of credibility, as it is a prerequisite for fulfilling the role of champion. This multiple-case study builds upon 23 semi-structured interviews with hospitalists in training: quality officers and medical specialists. We first coded data for each case and then described the different contexts of each case in detail to enable comparison across settings. We then compared the cases and contrasted the attributes of credibility. Four attributes of credibility emerged and were identified as essential for the hospitalist in training to succeed as a champion: (1) being convincing about the need for change by providing supportive clinical evidence, (2) displaying competence in their clinical work and commitment to their tasks, (3) generating shared ownership of the QI project with other healthcare professionals, and (4) acting as a team player to foster collaboration during the QI project. We also identified two contextual factors that supported the credibility of the hospitalist in training: (1) choosing a subject for the QI project that was perceived as urgently required by the group of stakeholders involved, and (2) being supported by the board of directors and other formal and informal leaders as the leader of a QI project. Further research is needed to gain a deeper understanding of the relationship between credibility and sustainability of change.

Entrustable Professional Activities in Clinical Education: A Practical Approach for Advanced Nursing Education

BACKGROUND

Competency attainment is foremost in the ethos of this project that provides a practical guide to implementing entrustable professional activities (EPAs) as one approach to competency acquisition in the clinical education component of a health care program. EPAs are units of work that provide evidence of required competencies for the clinical education component of a program of study for educators, preceptors, and students.

PROBLEM

The American Association of Colleges of Nursing has raised the importance of competency-based education and need for a practical approach to assess clinical competency challenges (eg, EPAs) in nursing education.

APPROACH

The implementation of EPAs in the clinical component of advanced nursing education is detailed as an example in this article. Prioritizing a systematic approach, the Knowledge to Action framework, was chosen to guide the process.

CONCLUSION

Nursing must act now to put in place a robust competency acquisition and validation system.

Concordance of Narrative Comments with Supervision Ratings Provided During Entrustable Professional Activity Assessments

BACKGROUND

Use of EPA-based entrustment-supervision ratings to determine a learner's readiness to assume patient care responsibilities is expanding.

OBJECTIVE

In this study, we investigate the correlation between narrative comments and supervision ratings assigned during ad hoc assessments of medical students' performance of EPA tasks.

DESIGN

Data from assessments completed for students enrolled in the clerkship phase over 2 academic years were used to extract a stratified random sample of 100 narrative comments for review by an expert panel.

PARTICIPANTS

A review panel, comprised of faculty with specific expertise related to their roles within the EPA program, provided a "gold standard" supervision rating using the comments provided by the original assessor.

MAIN MEASURES

Interrater reliability (IRR) between members of review panel and correlation coefficients (CC) between expert ratings and supervision ratings from original assessors.

KEY RESULTS

IRR among members of the expert panel ranged from .536 for comments associated with focused history taking to .833 for complete physical exam. CC (Kendall's correlation coefficient W) between panel members' assignment of supervision ratings and the ratings provided by the original assessors for history taking, physical examination, and oral presentation comments were .668, .697, and .735 respectively. The supervision ratings of the expert panel had the highest degree of correlation with ratings provided during assessments done by master assessors, faculty trained to assess students across clinical contexts. Correlation between supervision ratings provided with the narrative comments at the time of observation and supervision ratings assigned by the expert panel differed by clinical discipline, perhaps reflecting the value placed on, and perhaps the comfort level with, assessment of the task in a given specialty.

CONCLUSIONS

To realize the full educational and catalytic effect of EPA assessments, assessors must apply established performance expectations and provide high-quality narrative comments aligned with the criteria.

Design, implementation and evaluation of novel work-based clinical assessment tool: An e-portfolio with embedded Entrustable Professional Activities

BACKGROUND

A new concept in work-based assessment, Entrustable Professional Activities (EPAs) describe key activities a work-ready practitioner can perform independently. EPAs are widely used in medicine to support competency-based education and are gaining acceptance in other professions such as nursing, pharmacy and dietetics. There is currently no widely accepted work-based assessment tool for student clinical placement assessment in dietetics and EPAs offer potential to address this gap.

OBJECTIVES

To design, implement and evaluate a work-based assessment tool using EPAs embedded in an e-portfolio for use in clinical dietetics.

SETTING

An accredited dietetic training program in Australian university and affiliated metropolitan and rural hospitals.

PARTICIPANTS

Three consecutive cohorts of final year dietetic students (n = 126) and their professional placement supervisors (n = 101).

METHODS

A working party consisting of dietetic academics with experience in clinical dietetics and education developed clinical dietetic EPAs and milestones that were mapped to the National Competency Standards for Dietitians in Australia. A design model incorporating multiple rounds of user feedback was used to create an e-portfolio with the EPAs embedded. Students and supervisors were invited to complete anonymous on-line surveys with Likert responses (1 = highly dissatisfied, 5 = highly satisfied) to evaluate the face validity, acceptability and feasibility of the tool.

RESULTS

A total of 37 EPAs were developed with an accompanying four-point entrustment scale to measure performance. Evaluation survey response rates were high and ranged from 35 to 45% for students and 57-83% for supervisors across the study period. Both students and supervisors evaluated the tool positively with a mean overall satisfaction of 4.12 ± 0.69 and 4.03 ± 0.68, respectively. Survey items evaluating face validity, feasibility and acceptability scored >3.75 for both groups of users.

CONCLUSIONS

The EPA based e-portfolio demonstrated face validity, feasibility and acceptability and offers potential for a widely accepted tool for work-based assessment of clinical dietetic students.

From Utopia Through Dystopia: Charting a Course for Learning Analytics in Competency-Based Medical Education

The transition to the assessment of entrustable professional activities as part of competency-based medical education (CBME) has substantially increased the number of assessments completed on each trainee. Many CBME programs are having difficulty synthesizing the increased amount of assessment data. Learning analytics are a way of addressing this by systematically drawing inferences from large datasets to support trainee learning, faculty development, and program evaluation. Early work in this field has tended to emphasize the significant potential of analytics in medical education. However, concerns have been raised regarding data security, data ownership, validity, and other issues that could transform these dreams into nightmares. In this paper, the authors explore these contrasting perspectives by alternately describing utopian and dystopian futures for learning analytics within CBME. Seeing learning analytics as an important way to maximize the value of CBME assessment data for organizational development, they argue that their implementation should continue within the guidance of an ethical framework.