Entrustment Decisions: Bringing the Patient Into the Assessment Equation

EMYWAY Workplace-Based Entrustable Professional Activities Assessments in Otolaryngology Residency Training: A Nationwide Experience

OBJECTIVE

To present workplace-based entrustable professional activities (EPAs) assessment data from the first 2 years of the EMYWAY platform in otolaryngology residency training in Taiwan.

STUDY DESIGN

Two-year cross-sectional study.

SETTING

Otolaryngology training programs.

METHODS

In 2020, the Taiwan Society of Otorhinolaryngology-Head and Neck Surgery (TSO-HNS) developed a workplace-based assessment (WBA) framework with 11 EPAs, integrating milestones to evaluate resident competency. In 2021, TSO-HNS piloted the EMYWAY platform for WBAs, which includes an EPA-based assessment workflow, coaching feedback, and a dashboard displaying residents' entrustment-supervision levels. Data are analyzed annually for accreditation and curriculum enhancement. This study reports on the pilot year and the first full-scale year of implementation.

RESULTS

Eleven programs participated in the pilot year. Subsequently, 362 faculty members and 274 resident physicians from 34 programs nationwide engaged with EMYWAY. In the full-scale year from August 2022 to July 2023, 9805 responses were recorded, primarily from surgical theaters (45.9%; 4502/9805) and third-year residents (23.8%; 2331/9805). The most frequently evaluated EPAs were "head and neck" (17.5%; 1716/9805), "sinonasal" (13.5%; 1324/9805), and "ear" (12.2%; 1193/9805), with task complexity increasing with resident seniority (P < .0001). A positive correlation was found between residents' self-assessments and faculty members' ratings (r = 0.531; P < .001). Over 98.2% of residents and 88.4% of faculty members provided substantial feedback (>10 words). Analysis of WBAs reported by training programs identified faculty development targets and teaching-intensive tasks.

CONCLUSION

EMYWAY effectively documents workplace learning and tracks resident competency progression. Continuous improvement of WBA quality is essential for advancing the competency-based medical education ecosystem.

Competency-based medical education: Connecting training outcomes to patient care

Competency-based medical education (CBME) is a patient-centered and learner-focused approach to education where curricula are delivered in a manner tailored to the individuals' learning needs, and assessment focuses on ensuring trainees achieve requisite and clearly specified learning outcomes. Despite calls to focus assessment on what matters for patients. In this article, the authors explore one aspect of this next era: the use of electronic health record clinical performance indicators, such as Resident-Sensitive Quality Measures (RSQMs) and TRainee Attributable and Automatable Care Evaluations in Real-time (TRACERs), for learner assessment. They elaborate on both the promise and the potential limitations of using such measures in a program of learner assessment.

Factors necessary for entrustment decision-making in surgical operating rooms: A modified Delphi study

Objective

Resident autonomy in an operation theatre has been directly linked with patient safety in healthcare. The objective of this study was to identify the factors necessary for making resident-entrustment decisions from the perspective of a supervisor/consultant viewpoint. The second objective was to develop a checklist for assessing resident readiness for independent work.

Materials and methods

This study employed a mixed-method Delphi approach. In the first stage, a comprehensive literature review and a qualitative exploratory study produced a list of factors related to residents. The second phase involved content validation by a panel of experts, followed by a two-round Delphi study with 20 expert panelists.

Results

A total of 49 resident-related factors for entrustment were identified, which were reduced to 46 after content validation. During Delphi Round I, 17 factors were fully accepted, 7 were rejected and 22 items were partially accepted. Out of the 39 items sent to Delphi Round II, 23 items were accepted and 16 were rejected. A final 23-item checklist was formed based on the following factors; 6 Cognitive (knowledgeable, risk manager, safe doctor, general manager, field of interest, communicator), 5 Psychomotor (past performer, competent, ability to pick critical findings, ability to act situationally, decision maker) and 12 affective (responsible, leader, honest, empathetic, ethical, receptive, humble, emotionally intelligent, motivated, accountable, team player, disciplined) factors.

Conclusion

The study resulted in the formation of a checklist based on the factors necessary for entrustment decision-making in surgical operating rooms. Some of the novel contextual factors were 'general manager', 'field of interest', 'ability to pick critical findings', 'accountable', 'risk manager', and 'past performer'. This framework offers a guideline for supervisors and residents to evaluate progress throughout the residency program. The developed tool demonstrates good content validity and is suitable for entrustment assessment following construct validation.

American Dental Education Association Compendium Entrustable Professional Activities Workgroup report

PURPOSE

Entrustable professional activities (EPAs) are discrete clinical tasks that can be evaluated to help define readiness for independent practice in the health professions and are intended to increase trust in the dental graduate. EPAs provide a framework that bridges competencies to clinical practice. This report describes the work of the American Dental Education Association (ADEA) Compendium EPA Workgroup to develop a list of EPAs for dental education and supportive resources, including specifications and a glossary.

METHODS

Preliminary work including literature and resource review, mapping of existing competencies, and review of other health professions' EPAs informed the development of our EPAs list. Workgroup members achieved consensus using a modified Delphi process. A Qualtrics survey using a validated rubric for the assessment of EPAs as described in peer-reviewed literature was used. Dental educators, including academic deans, were surveyed for feedback on the content and format of the EPAs.

RESULTS

Based on findings in the literature analysis of existing EPAs and competencies in health professions, a list of EPAs was developed along with a description of specifications. The EPA workgroup (nine members from multiple institutions) used the Delphi process in receiving feedback from various experts. A list of 11 core EPAs was vetted by dental educators including academic deans (n = ∼23), and the process of development was reviewed by EPAs experts outside dental education. A glossary was developed to align language.

CONCLUSION

These EPAs define the scope of dental practice. This report represents Phase 1 of the EPA framework development and vetting process. Future directions will include a broader vetting of the EPA list, faculty development, and national standardized technology that support this work to optimize implementation.

Exploring Interprofessional Development of Entrustable Professional Activities For Pediatric Intensive Care Fellows: A Proof-of-Concept Study

Phenomenon: Entrustable professional activities (EPAs) delineate major professional activities that an individual in a given specialty must be "entrusted" to perform, ultimately without supervision, to provide quality patient care. Until now, most EPA frameworks have been developed by professionals within the same specialty. As safe, effective, and sustainable health care ultimately depends on interprofessional collaboration, we hypothesized that members of interprofessional teams might have clear and possibly additional insight into which activities are essential to the professional work of a medical specialist. Approach: We recently employed a national modified Delphi study to develop and validate a set of EPAs for Dutch pediatric intensive care fellows. In this proof-of-concept study, we explored what pediatric intensive care physicians' non-physician team members (physician assistants, nurse practitioners, and nurses) constitute as essential professional activities for PICU physicians and how they regarded the newly developed set of nine EPAs. We compared their judgments with the PICU physicians' opinions. Findings: This study shows that non-physician team members share a mental model with physicians about which EPAs are indispensable for pediatric intensive care physicians. Despite this agreement however, descriptions of EPAs are not always clear for non-physician team members who have to work with them on a daily basis. Insights: Ambiguity as to what an EPA entails when qualifying a trainee can have implications for patient safety and trainees themselves. Input from non-physician team members may add to the clarity of EPA descriptions. This finding supports the involvement of non-physician team members in the developmental process of EPAs for (sub)specialty training programs.

Competency-based medical education: The spark to ignite healthcare's escape fire

High-value care is what patients deserve and what healthcare professionals should deliver. However, it is not what happens much of the time. Quality improvement master Dr. Don Berwick argued more than two decades ago that American healthcare needs an escape fire, which is a new way of seeing and acting in a crisis situation. While coined in the U.S. context, the analogy applies in other Western healthcare contexts as well. Therefore, in this paper, the authors revisit Berwick's analogy, arguing that medical education can, and should, provide the spark for such an escape fire across the globe. They assert that medical education can achieve this by fully embracing competency-based medical education (CBME) as a way to place medicine's focus on the patient. CBME targets training outcomes that prepare graduates to optimize patient care. The authors use the escape fire analogy to argue that medical educators must drop long-held approaches and tools; treat CBME implementation as an adaptive challenge rather than a technical fix; demand genuine, rich discussions and engagement about the path forward; and, above all, center the patient in all they do.

Medical competence as a multilayered construct

BACKGROUND

The conceptualisation of medical competence is central to its use in competency-based medical education. Calls for 'fixed standards' with 'flexible pathways', recommended in recent reports, require competence to be well defined. Making competence explicit and measurable has, however, been difficult, in part due to a tension between the need for standardisation and the acknowledgment that medical professionals must also be valued as unique individuals. To address these conflicting demands, a multilayered conceptualisation of competence is proposed, with implications for the definition of standards and approaches to assessment.

THE MODEL

Three layers are elaborated. This first is a core layer of canonical knowledge and skill, 'that, which every professional should possess', independent of the context of practice. The second layer is context-dependent knowledge, skill, and attitude, visible through practice in health care. The third layer of personalised competence includes personal skills, interests, habits and convictions, integrated with one's personality. This layer, discussed with reference to Vygotsky's concept of Perezhivanie, cognitive load theory, self-determination theory and Maslow's 'self-actualisation', may be regarded as the art of medicine. We propose that fully matured professional competence requires all three layers, but that the assessment of each layer is different.

IMPLICATIONS

The assessment of canonical knowledge and skills (Layer 1) can be approached with classical psychometric conditions, that is, similar tests, circumstances and criteria for all. Context-dependent medical competence (Layer 2) must be assessed differently, because conditions of assessment across candidates cannot be standardised. Here, multiple sources of information must be merged and intersubjective expert agreement should ground decisions about progression and level of clinical autonomy of trainees. Competence as the art of medicine (Layer 3) cannot be standardised and should not be assessed with the purpose of permission to practice. The pursuit of personal excellence in this level, however, can be recognised and rewarded.

Family-Educator Partnership in the Development of Entrustable Professional Activities in Complex Care

PROBLEM

Incorporating patient and family voices in the development of entrustable professional activities (EPAs) is not standard practice. Care of children with medical complexity (CMC) is an area of pediatrics that relies on family partnership, and families of CMC are ideal partners in EPA development given their expertise in their child's care and experience interacting with the health care system. The authors describe their model for partnering with families to develop EPAs and reflect on the unique contributions of family leaders to the process.

APPROACH

After recruitment of family leaders from a national organization of families and friends of children with special health care needs, the authors used a multistage process for EPA development from June 2019 to February 2021. Family leaders were integrated throughout the process, including creating EPA descriptions, revising content across all EPAs, appraising EPAs through virtual focus groups with other key stakeholders, and finalizing and publishing EPAs. The authors used content analysis to identify recommendations for patient- and family-integrated EPA development.

OUTCOMES

Family leaders and educators partnered in every phase of developing EPAs for the care of CMC, including as content experts, editors, focus group facilitators, and coauthors. Family leaders recommended substantive changes to all EPAs, including revising language, augmenting content, and modifying scope of practice. In addition, content analysis of family leaders' revisions yielded 10 recommendations to ensure that written EPA descriptions are patient- and family-centered.

NEXT STEPS

The described process of EPA development for the care of CMC models how families can be integrated into competency framework development and highlights their contributions. Family leader recommendations for embedding patient and family voices in EPA descriptions can serve as a guide for EPA development in other specialties.

How can Entrustable Professional Activities serve the quality of health care provision through licensing and certification?

This paper about Entrustable Professional Activities (EPAs) was solicited to support the discussion about the future of licensing within the Medical Council of Canada. EPAs, units of professional practice to be entrusted to learners or professionals once they have shown to possess sufficient competence, were proposed in 2005 to operationalize competency-based postgraduate medical education and have become widely popular for various health professions education programs in many countries. EPAs break the breadth of competence for license down to units of practice that can be overseen, assessed, monitored, documented, and entrusted. EPAs together may constitute an individual's portfolio of qualifications, and define a scope of practice. A medical license and a specialty certification can then be defined as the required combination of EPAs for which one is qualified at any specific moment in time. That 'snapshot' could change over time and reflect the professional development of the individual, both in their competence and in their privileges to practice. Micro-credentialing and digital badges might become an adequate option to show-case one's scope of practice at any time and operationalize the idea of a dynamic portfolio of EPAs.

Competency-Based Education and Practice in Physical Therapy: It's Time to Act!

Competency-based education (CBE) is a concept, a philosophy, and an approach to educational design where learner progression occurs when competency is demonstrated. It assumes a set of standard defined performance outcomes for any level of professional practice-students, residents, or practicing physical therapists. Those outcomes are based on the health needs of society and guide the curricular design, implementation, and evaluation of health professions education programs. Lack of a CBE framework-with no required demonstration of competence throughout one's career-has the potential to lead to variation in physical therapists' skills and to unwarranted variation in practice, potentially hindering delivery of the highest quality of patient care. CBE requires a framework that includes a commonly understood language; standardized, defined performance outcomes at various stages of learner development; and a process to assess whether competence has been demonstrated. The purpose of this perspective article is to (1) highlight the need for a shared language, (2) provide an overview of CBE and the impetus for the change, (3) propose a shift toward CBE in physical therapy, and (4) discuss the need for the profession to adopt a mindset requiring purposeful practice across one's career to safely and most efficiently practice in a given area. Utilizing a CBE philosophy throughout one's career should ensure high-quality and safe patient care to all-patient care that can adapt to the changing scope of physical therapist practice as well as the health care needs of society. The physical therapy profession is at a point at which we must step up the transition to a competency-based system of physical therapist education.

A Pilot Study of the Generalizability of Preclinical Entrustment Assessments in Undergraduate Medical Education

PURPOSE

The reproducibility and consistency of assessments of entrustable professional activities (EPAs) in undergraduate medical education (UME) have been identified as potential areas of concern. EPAs were designed to facilitate workplace-based assessments by faculty with a shared mental model of a task who could observe a trainee complete the task multiple times. In UME, trainees are frequently assessed outside the workplace by faculty who only observe a task once.

METHOD

In November 2019, the authors conducted a generalizability study (G-study) to examine the impact of student, faculty, case, and faculty familiarity with the student on the reliability of 162 entrustment assessments completed in a preclerkship environment. Three faculty were recruited to evaluate 18 students completing 3 standardized patient (SP) cases. Faculty familiarity with each student was determined. Decision studies were also completed. Secondary analysis of the relationship between student performance and entrustment (scoring inference) compared average SP checklist scores and entrustment scores.

RESULTS

G-study analysis revealed that entrustment assessments struggled to achieve moderate reliability. The student accounted for 30.1% of the variance in entrustment scores with minimal influence from faculty and case, while the relationship between student and faculty accounted for 26.1% of the variance. G-study analysis also revealed a difference in generalizability between assessments by unfamiliar (φ = 0.75) and familiar (φ = 0.27) faculty. Subanalyses showed that entrustment assessments by familiar faculty were moderately correlated to average SP checklist scores (r = 0.44, P < .001), while those by unfamiliar faculty were weakly correlated (r = 0.16, P = .13).

CONCLUSIONS

While faculty and case had a limited impact on the generalizability of entrustment assessments made outside the workplace in UME, faculty who were familiar with a student's ability had a notable impact on generalizability and potentially on the scoring validity of entrustment assessments, which warrants further study.

Implementation and Use of Workplace-Based Assessment in Clinical Learning Environments: A Scoping Review

PURPOSE

Workplace-based assessment (WBA) serves a critical role in supporting competency-based medical education (CBME) by providing assessment data to inform competency decisions and support learning. Many WBA systems have been developed, but little is known about how to effectively implement WBA. Filling this gap is important for creating suitable and beneficial assessment processes that support large-scale use of CBME. As a step toward filling this gap, the authors describe what is known about WBA implementation and use to identify knowledge gaps and future directions.

METHOD

The authors used Arksey and O'Malley's 6-stage scoping review framework to conduct the review, including: (1) identifying the research question; (2) identifying relevant studies; (3) study selection; (4) charting the data; (5) collating, summarizing, and reporting the results; and (6) consulting with relevant stakeholders.

RESULTS

In 2019-2020, the authors searched and screened 726 papers for eligibility using defined inclusion and exclusion criteria. One hundred sixty-three met inclusion criteria. The authors identified 5 themes in their analysis: (1) Many WBA tools and programs have been implemented, and barriers are common across fields and specialties; (2) Theoretical perspectives emphasize the need for data-driven implementation strategies; (3) User perceptions of WBA vary and are often dependent on implementation factors; (4) Technology solutions could provide useful tools to support WBA; and (5) Many areas of future research and innovation remain.

CONCLUSIONS

Knowledge of WBA as an implemented practice to support CBME remains constrained. To remove these constraints, future research should aim to generate generalizable knowledge on WBA implementation and use, address implementation factors, and investigate remaining knowledge gaps.

Feasibility, and validity aspects of Entrustable Professional Activity (EPA)-based assessment in general practice training

INTRODUCTION

Entrustable Professional Activities (EPAs) are developed to support the practical application of competency frameworks in postgraduate medical education (PGME) programmes. EPAs are used for the assessment of the trainees' competence development, which takes place by means of an entrustment decision, aiming to stimulate learning and independent practice in trainees. In this pilot study, we explore the feasibility and validity of EPA-based assessment in a General Practice (GP) training programme.

METHODS

We used questionnaires to evaluate trainers' and trainees' experiences with the use of six EPAs for trainee learning, assessment and independent practice at the Out-of-Hours GP Center. Data were analysed quantitatively and qualitatively. Additionally, we examined the inter-item correlation between scores on EPA-based assessment and competency-based assessment using Spearman's Rho.

RESULTS

EPA-based assessment provided opportunities for giving concrete feedback and substantiating competency-based assessment. No consistent correlation between EPA-based assessment and competency-based assessment could be detected. Only later in the course of the training programme a correlation was found between the EPA scores and the degree of independence of trainees.

DISCUSSION

Results of this pilot study confirm the theories behind EPAs, as well as earlier research on EPAs in the workplace regarding trainee learning, assessment and independent practice. An important limitation of this study was the COVID-19 pandemic, as it influenced the results through reduced inclusion and follow-up, and through the impact on the workplace and trainee learning possibilities. Further research is needed to determine how EPAs support independent practice of trainees, as well as the assessment of trainee competency development.

Distant and Hidden Figures: Foregrounding Patients in the Development, Content, and Implementation of Entrustable Professional Activities

Entrustable professional activities (EPAs) describe activities that qualified professionals must be able to perform to deliver safe and effective care to patients. The entrustable aspect of EPAs can be used to assess learners through documentation of entrustment decisions, while the professional activity aspect can be used to map curricula. When used as an assessment framework, the entrustment decisions reflect supervisory judgments that combine trainees' relational autonomy and patient safety considerations. Thus, the design of EPAs incorporates the supervisor, trainee, and patient in a way that uniquely offers a link between educational outcomes and patient outcomes. However, achieving a patient-centered approach to education amidst both curricular and assessment obligations, educational and patient outcomes, and a supervisor-trainee-patient triad is not simple nor guaranteed. As medical educators continue to advance EPAs as part of their approach to competency-based medical education, the authors share a critical discussion of how patients are currently positioned in EPAs. In this article, the authors examine EPAs and discuss how their development, content, and implementation can result in emphasizing the trainee and/or supervisor while unintentionally distancing or hiding the patient. They consider creative possibilities for how EPAs might better integrate the patient as finding ways to better foreground the patient in EPAs holds promise for aligning educational outcomes and patient outcomes.

Entrustable Professional Activities and Entrustment Decision Making: A Development and Research Agenda for the Next Decade

To establish a research and development agenda for Entrustable Professional Activities (EPAs) for the coming decade, the authors, all active in this area of investigation, reviewed recent research papers, seeking recommendations for future research. They pooled their knowledge and experience to identify 3 levels of potential research and development: the micro level of learning and teaching; the meso level of institutions, programs, and specialty domains; and the macro level of regional, national, and international dynamics. Within these levels, the authors categorized their recommendations for research and development. The authors identified 14 discrete themes, each including multiple questions or issues for potential exploration, that range from foundational and conceptual to practical. Much research to date has focused on a variety of issues regarding development and early implementation of EPAs. Future research should focus on large-scale implementation of EPAs to support competency-based medical education (CBME) and on its consequences at the 3 levels. In addition, emerging from the implementation phase, the authors call for rigorous studies focusing on conceptual issues. These issues include the nature of entrustment decisions and their relationship with education and learner progress and the use of EPAs across boundaries of training phases, disciplines and professions, including continuing professional development. International studies evaluating the value of EPAs across countries are another important consideration. Future studies should also remain alert for unintended consequences of the use of EPAs. EPAs were conceptualized to support CBME in its endeavor to improve outcomes of education and patient care, prompting creation of this agenda.

On the validity of summative entrustment decisions

Health care revolves around trust. Patients are often in a position that gives them no other choice than to trust the people taking care of them. Educational programs thus have the responsibility to develop physicians who can be trusted to deliver safe and effective care, ultimately making a final decision to entrust trainees to graduate to unsupervised practice. Such entrustment decisions deserve to be scrutinized for their validity. This end-of-training entrustment decision is arguably the most important one, although earlier entrustment decisions, for smaller units of professional practice, should also be scrutinized for their validity. Validity of entrustment decisions implies a defensible argument that can be analyzed in components that together support the decision. According to Kane, building a validity argument is a process designed to support inferences of scoring, generalization across observations, extrapolation to new instances, and implications of the decision. A lack of validity can be caused by inadequate evidence in terms of, according to Messick, content, response process, internal structure (coherence) and relationship to other variables, and in misinterpreted consequences. These two leading frameworks (Kane and Messick) in educational and psychological testing can be well applied to summative entrustment decision-making. The authors elaborate the types of questions that need to be answered to arrive at defensible, well-argued summative decisions regarding performance to provide a grounding for high-quality safe patient care.

Entrustment Unpacked: Aligning Purposes, Stakes, and Processes to Enhance Learner Assessment

Educators use entrustment, a common framework in competency-based medical education, in multiple ways, including frontline assessment instruments, learner feedback tools, and group decision making within promotions or competence committees. Within these multiple contexts, entrustment decisions can vary in purpose (i.e., intended use), stakes (i.e., perceived risk or consequences), and process (i.e., how entrustment is rendered). Each of these characteristics can be conceptualized as having 2 distinct poles: (1) purpose has formative and summative, (2) stakes has low and high, and (3) process has ad hoc and structured. For each characteristic, entrustment decisions often do not fall squarely at one pole or the other, but rather lie somewhere along a spectrum. While distinct, these continua can, and sometimes should, influence one another, and can be manipulated to optimally integrate entrustment within a program of assessment. In this article, the authors describe each of these continua and depict how key alignments between them can help optimize value when using entrustment in programmatic assessment within competency-based medical education. As they think through these continua, the authors will begin and end with a case study to demonstrate the practical application as it might occur in the clinical learning environment.

True Entrustment Decisions Regarding Entrustable Professional Activities Happen in the Workplace, not in the Classroom Setting

Entrustable Professional Activities (EPAs) are workplace responsibilities that directly impact patient care. The use of EPAs allows pharmacy faculty and preceptors to provide learners with feedback and assessment in the clinical setting. Because they focus assessment on a learner's execution of professional activities which requires integration of the respective competencies, EPAs help provide a more holistic picture of a learner's performance. Using EPAs to backwards design classroom learning for those competencies is highly encouraged, but instructors cannot or should not assess performance and make entrustment decisions using EPAs in the classroom setting for several reasons: a learner's classroom performance usually does not predict clinical performance very well, assessment of EPAs require direct observation of the learner performing the EPAs, EPA assessment requires multiple observations of the learner with different patients with varying level of acuity, and most importantly, EPA assessment must result in a decision to trust the learner to perform the clinical activity with limited supervision. By ensuring all entrustment decisions are made in a clinical or experiential setting, students will receive an accurate assessment and benchmark of their performance that will lead them one step closer to becoming independent practitioners.

Entrustment decisions and the clinical team: A case study of early clinical students

PURPOSE

Clinical learning contexts influence how medical students engage with entrustment decisions. However, it is unclear how students and health care team members perceive the entrustment decision process. This study explored which factors students and team members consider relevant to entrustment decisions in early clinical rotations.

METHODS

The authors conducted a case study at an academic teaching hospital, interviewing 28 medical students and four health care team members during the clerkship year. Within a social constructivist epistemology, we explored students' and health care team members' perceptions of ad hoc entrustment decisions using semi-structured interviews. Transcripts from the interviews and notes from feedback rounds with students were used for analysis.

RESULTS

Medical students in their core clerkship year perceived clinical residents as critical educational gatekeepers and key facilitators of entrustment decisions. Another important theme emerged around students' motivation, initiative and willingness to engage with the health care team and patients. Students actively engaged in trust formation processes with different health care team members. The entrustment decision process was perceived as multilateral and dynamic, involving all health care team members and patients. Multiple entrusting supervisors for clerkship students, including nurses and psychologists, emerged from our interview data. They assumed an active role in negotiating entrustment decisions both with and for clerkship students, either facilitating or hindering opportunities. The entrustment decisions emerged as a result of a multifaceted supervisor network interaction.

CONCLUSIONS

Supervising residents' ability to integrate students into clinical teams seems to be a critical factor in facilitating entrustment opportunities for clinical activities. Students' active management of informal supervisor networks of health care team members and these team members' willingness to assume responsibility for the students' education emerged as relevant aspects for ad hoc entrustment. Our data suggest that supervision from different health professionals is beneficial for clinical education of medical students and merits further exploration.

Medical Students' Views on Implementing the Core EPAs: Recommendations From Student Leaders at the Core EPAs Pilot Institutions

In 2014, the Association of American Medical Colleges recruited 10 institutions across the United States to pilot the 13 Core Entrustable Professional Activities for Entering Residency (Core EPAs). The goal was to establish a competency-based framework to prepare graduating medical students for the transition to residency. Within the Core EPAs pilot, medical students play an influential role in the development and implementation of EPA-related curricula. Student engagement was a priority for the Core EPAs institutions given students' roles as the end users of the curriculum, thus they may offer valuable insight into its design and implementation. Here, the authors provide the perspective of medical students who serve as leaders in the Core EPAs pilot at their respective institutions. They describe student leadership models across the pilot institutions as well as 6 key challenges to implementation of the Core EPAs: (1) How and when should the Core EPAs be introduced? (2) Who is responsible for driving the assessment process? (3) What feedback mechanisms are required? (4) What systems are required for advising, mentoring, or coaching students? (5) Should EPA performance contribute to students' grades? and (6) Should entrustment decisions be tied to graduation requirements? Using a polarity management framework to address each challenge, the authors describe inherent tensions, approaches used by the Core EPAs pilot institutions, and student-centered recommendations for resolving each tension. By sharing the experiences and perspectives of students engaged in the Core EPAs pilot, the authors hope to inform implementation of EPA-oriented assessment practices and feedback across institutions in the United States.

Entrustment Decision Making: Extending Miller's Pyramid

The iconic Miller's pyramid, proposed in 1989, characterizes 4 levels of assessment in medical education ("knows," "knows how," "shows how," "does"). The frame work has created a worldwide awareness of the need to have different assessment approaches for different expected outcomes of education and training. At the time, Miller stressed the innovative use of simulation techniques, geared at the third level ("shows how"); however, the "does" level, assessment in the workplace, remained a largely uncharted area. In the 30 years since Miller's conference address and seminal paper, much attention has been devoted to procedures and instrument development for workplace-based assessment. With the rise of competency-based medical education (CBME), the need for approaches to determine the competence of learners in the clinical workplace has intensified. The proposal to use entrustable professional activities as a framework of assessment and the related entrustment decision making for clinical responsibilities at designated levels of supervision of learners (e.g., direct, indirect, and no supervision) has become a recent critical innovation of CBME at the "does" level. Analysis of the entrustment concept reveals that trust in a learner to work without assistance or supervision encompasses more than the observation of "doing" in practice (the "does" level). It implies the readiness of educators to accept the inherent risks involved in health care tasks and the judgment that the learner has enough experience to act appropriately when facing unexpected challenges. Earning this qualification requires qualities beyond observed proficiency, which led the authors to propose adding the level "trusted" to the apex of Miller's pyramid.

The ingredients of a rich entrustment decision

Entrustment decision-making has become a topic of interest in workplace-based assessment in the health professions and is germane to the use of entrustable professional activities. Entrustment decisions stem from judgments of a trainee's competence and include the permission to act with a higher level of responsibility or autonomy and a lower level of supervision. Making entrustment decisions differs from regular assessment of trainees, which usually has no consequences beyond marking trainee progress. Studies show that clinicians generally weigh more factors in making an entrustment decision than when merely assessing trainee competence or performance without direct consequences for patient care. To synthesize the varying factors reported in literature, the authors performed a thematic analysis of key qualitative studies that investigated trainee features clinical supervisors find important when making entrustment decisions. Five themes emerged from the 13 publications: Capability (specific knowledge, skills, experience, situational awareness), Integrity (truthful, benevolent, patient-centered), Reliability (conscientious, predictable, accountable, responsible), Humility (recognizes limits, asks for help, receptive to feedback), Agency (proactive toward work, team, safety, personal development). Thoughtful entrustment decisions, made either by individual clinical supervisors or by clinical competency committees, may be enriched by taking into account these five features.

Living in a World of Change: Bridging the Gap From Competency-Based Medical Education Theory to Practice in Canada

Within graduate medical education, many educators are experiencing a climate of significant change. One transformation, competency-based medical education (CBME), is occurring simultaneously across much of the world, and implementation will require navigating numerous tensions and paradoxes. Successful transformation requires many types of power and is most likely to happen when the medical education community of professionals is engaged in designing, experimenting, acting, and sensemaking together.In this complex climate, the craft of change facilitators and community leaders is needed more than ever. National top-down policies and structures, while important, are not sufficient. The operationalization of new advances is best done when local leaders are afforded room to shape their local context. An evidence-based approach to thinking about the transformative change associated with CBME needs to be adopted. In this age of entrustment, 3 priorities are paramount: (1) engage, entrust, and empower professionals with increasing shared ownership of the innovation; (2) better prepare education professionals in leadership and transformational change techniques in the complex system of medical education; and (3) leverage the wider community of practice to maximize local CBME customization. These recommendations, although based largely on the Canadian experience, are intended to inform CBME transformation in any context.

Assessing Trainees and Making Entrustment Decisions: On the Nature and Use of Entrustment-Supervision Scales

Clinical teachers are continuously entrusting trainees with care responsibilities in health care settings. Entrustable professional activities employ entrustment decision making as an approach to assessment in the workplace.Various scales have been created to measure "entrustment," all basically expressing the level or type of supervision a trainee requires for safe and high-quality care. However, some of these scales are only weakly related to the purpose of making decisions about the autonomy trainees will be granted. The authors aim to increase understanding about the nature, purpose, and practice of supervision scales aimed at entrustment.After arguing for entrustment as a component of workplace-based assessment, the distinction between ad hoc entrustment decisions (daily decisions in health care settings) and summative entrustment decisions (with a certifying nature) is clarified. Next, the noncontinuous nature of entrustment-supervision (ES) scales, as opposed to most workplace-based assessment scales, is explained. ES scales have ordinal, rather than interval, properties and focus on discrete decisions. Finally, some scales are retrospective ("how much supervision was provided?"), and others are prospective ("how much supervision will be needed in the near future?"). Although retrospective scales reflect observed behavior, prospective scales truly focus on entrustment and ask for more holistic judgment, as they include a broader evaluation and a risk estimation to enable a decision about increase of autonomy.The analysis concludes with a discussion about entrustment for unsupervised practice and supervision of others, as well as the program, context, and specialty specificity of scales.

Entrustment decisions: Implications for curriculum development and assessment

With increased interest in the use of entrustable professional activities (EPAs) in undergraduate and postgraduate medical education, comes questions about their implications for curriculum development and assessment. This paper addresses some of those questions, discussed at a symposium of the 2017 conference of AMEE, by presenting the components of an EPA, describing their importance and application, identifying their implications for assessment, and pinpointing some of challenges they pose in undergraduate and postgraduate settings. It defines entrustment, describes the three levels of trust, and presents trainee and supervisor factors that influence it as well as perceived benefits, and risks. Two aspects of EPAs have implications for assessment: units of professional practice and decisions based on entrustment, which impact an assessment's blueprint, test methods, scores, and standards. In an undergraduate setting EPAs have great appeal, but work is needed to identify and develop a robust assessment system for core EPAs. At the postgraduate level, there is tension between the granularity of the competencies and the integrated nature of the EPAs. Even though work remains, EPAs offer an important step in the evolution of competency-based education.

What attributes should a specialist in rehabilitation have? Seven suggested specialist Capabilities in Practice

PROBLEM

Many services and professionals refer to themselves as providing rehabilitation. There is no agreed method for determining whether someone has specific expertise in rehabilitation. This makes it difficult for patients and payers to know whether professionals who claim to provide rehabilitation are specifically expert in rehabilitation.

CONTEXT

Doctors have a medical speciality of rehabilitation. The medical training curriculum gives attributes that differentiate a rehabilitation specialist from other doctors. Until recently, these attributes were competencies to undertake activities associated with specialization. Apart from nurses, who have at least one, unofficial, curriculum identifying specific competencies, other professions involved in rehabilitation do not have any way to show specialization in rehabilitation.

CAPABILITIES IN PRACTICE

The U.K. General Medical Council accredits specialist medical training. It has moved from specifying multiple practical clinical competencies to specifying fewer high-level 'Capabilities in Practice'. Six are generic to all doctors, eight identify the trained doctor as having specialist rehabilitation skills. This article adopts this approach to put forward seven generic and seven specialist capabilities to identify any professional as having special expertise in rehabilitation. The seven specialist capabilities centre on the biopsychosocial model of illness and multidisciplinary teamwork. Four of them could be used to define a specialist rehabilitation team.

CONCLUSION

Seven capabilities identifying specialization in rehabilitation are put forward for discussion. They could form the basis of a formal recognition that any professional has additional expertise in rehabilitation. A validating authority would be needed to provide oversight and governance.

What works best in a general practice specific OSCE for medical students: Mini-CEX or content-related checklists?

Aim: To develop and pilot a General Practice (GPr) OSCE assessing medical students dealing with patient encounters, which are typical for GPr and to compare different measurement instruments (global ratings, content-specific checklists).Methods: A blueprint based on Entrusted Professional Activities was used to develop prototypical OSCE stations. Four stations were tested with voluntary medical students. Students were videotaped and assessed with self-developed content-specific checklists, a global rating for communication skills, and mini-CEX. Results were compared according to students' phases of studies.Results: All three measurements were able to discriminate between clinical and pre-clinical students. Clearest results were achieved by using mini-CEX. Content-specific checklists were not able to differentiate between those groups for the more difficult stations. Inter-station reliability for the global ratings was sufficient for high-stakes exams. Students enjoyed the OSCE-setting simulating GPr consultation hours. They would prefer feedback from GPs after the OSCE and from simulated patients after each encounter.Discussion and conclusion: Although the OSCE was short, results indicate advantages for using a global rating instead of checklists. Further research should include validating these results with a larger group of students and to find the threshold during the phases of education for switching from checklists to global ratings.

Development and inclusion of an entrustable professional activity (EPA) scale in a simulation-based medicine dispensing assessment

BACKGROUND AND PURPOSE

Effective, safe, and patient-centred dispensing is a core task of community pharmacists. Entrustable professional activities (EPAs) offer a way of defining and assessing these daily practice activities. Although EPAs have become popular within competency-based medical education programs, their use is new to pharmacy education and assessment.

EDUCATIONAL ACTIVITY AND SETTING

A simulation-based assessment framework containing a scale of entrustment was developed to evaluate the readiness of Year 4 undergraduate pharmacy students to safely manage the supply of prescribed medicine(s) in a community pharmacy. The assessment framework was piloted in a fourth year "Transition to Practice" course with 28 simulation-based assessments conducted.

FINDINGS

An entrustment framework was developed and implemented successfully with Year 4 undergraduate pharmacy students. The EPA for medicine dispensing integrates competency domains that include information gathering, providing patient-centred care, clinical reasoning, medicine dispensing, and professional communications. On a scale ranging from level 1 to level 5, the majority (73%) of entrustment ratings were level 2 or level 3; and of the students who achieved different ratings between clinical scenarios, 75% of students improved on their second simulation attempt. There was a strong correlation between the global EPA ratings with the total score achieved across the domains.

SUMMARY

Using simulation-based assessment, entrustment decision making can be incorporated in "entry to profession" undergraduate and postgraduate pharmacy courses to assess students' readiness to transition between learning and professional practice.

Entrustable Professional Activities in Psychiatry: A Systematic Review

OBJECTIVE

Entrustable professional activities (EPAs) represent discrete clinical tasks that can be entrusted to trainees in psychiatry. They are increasingly being used as educational framework in several countries. However, the empirical evidence available has not been synthesized in the field of psychiatry. Therefore, the authors conducted a systematic review in order to summarize and evaluate the available evidence in the field of EPAs in undergraduate and graduate medical education in psychiatry.

METHODS

The authors searched PubMed, Cochrane Library, ERIC, Embase, PsycINFO, all Ovid journals, Scopus, Web of Science, MedEdPORTAL, and the archives of Academic Psychiatry for articles reporting quantitative and qualitative research as well as educational case reports on EPAs in undergraduate and graduate psychiatry education published since 2005. All included articles were assessed for content (development, implementation, and assessment of EPAs) and quality using the Quality Assessment Tool for Studies with Diverse Designs.

RESULTS

The authors screened 2807 records and included a total of 20 articles in the final data extraction. Most studies were expert consensus reports (n = 6, 30%) and predominantly conducted in English-speaking countries (n = 17, 85%). Papers reported mainly EPA development and/or EPA implementation studies (n = 14, 70%), whereas EPA assessment studies were less frequent (n = 6, 30%). Publications per year showed an increasing trend both in quantity (from 1 in 2011 to 7 in 2018) and quality (from a QATSDD score of 27 in 2011 to an average score of 39 in 2018). The main focus of the articles was the development of individual EPAs for different levels of training for psychiatry or on curricular frameworks based on EPAs in psychiatry (n = 10, 50%). The lack of empirical controlled studies does currently not allow for meta-analyses of educational outcomes.

CONCLUSIONS

The concept of EPA-based curricula seems to become increasingly present, a focus in the specialty of psychiatry both in UME and GME. The lack of empirical research in this context is an important limitation for educational practice recommendations. Currently there is only preliminary but promising data available for using EPAs with regard to educational outcomes. EPAs seem to be effectively used from a curriculum design perspective for UME and GME in psychiatry.

Entrustable professional activities in health care education: a scoping review

CONTEXT

The shift in medical education from time-based learning to outcome-based learning has drawn much attention to entrustable professional activities (EPAs) as an ideal assessment framework to translate competencies into clinical practice. Given the relative novelty of EPAs, this review aims to highlight research gaps and explore and consolidate available evidence pertaining to the development and implementation of EPAs in health care.

METHOD

Arksey and O'Malley's scoping review framework was used to present the findings. The authors performed a systematic search of PubMed, Embase, CINAHL, Scopus, MedNar, OpenGrey and ProQuest Dissertation and Theses for English articles published from the inception of each database to May 2018. A manual search of the reference lists of the included studies was conducted and an expert panel was consulted. Two reviewers screened the articles for eligibility using the inclusion criteria. All authors extracted key data and analysed the data descriptively. Thematic analysis was used to categorise the results into themes.

RESULTS

Eighty articles were included in the review. All articles were published between 2010 and 2018. Three major themes and eight sub-themes were generated: (i) development of EPAs (frameworks for EPA development and implementation, identifying core or specialty-specific EPAs, and EPAs for faculty development), (ii) evaluation of EPAs and EPA entrustment factors (revised curriculum, entrustment decisions for professional activities, and feedback on implemented EPAs and the development process), and (iii) future directions and recommendations for EPAs (implementation of EPAs in undergraduate medical education and specific disciplines, and other criticisms and recommendations for EPAs).

CONCLUSIONS

Entrustable professional activities are an essential means to translate competencies into observable and measurable clinical practice. However, high-level evidence-based research on the efficacy, development and implementation of EPAs for specific target groups (i.e. undergraduates and staff) and geographical regions (i.e. Asia and Africa) is still lacking, which suggests a direction for future research.

Scoping Review of Entrustable Professional Activities in Undergraduate Medical Education

PURPOSE

Entrustable professional activities (EPAs) are a hot topic in undergraduate medical education (UME); however, the usefulness of EPAs as an assessment approach remains unclear. The authors sought to better understand the literature on EPAs in UME through the lens of the 2010 Ottawa Conference Criteria for Good Assessment.

METHOD

The authors conducted a scoping review of the health professions literature (search updated February 2018), mapping publications to the Ottawa Criteria using a collaboratively designed charting tool.

RESULTS

Of the 1,089 publications found, 71 (6.5%) met inclusion criteria. All were published after 2013. Forty-five (63.4%) referenced the 13 Core Entrustable Professional Activities for Entering Residency developed by the Association of American Medical Colleges (AAMC). Forty (56.3%) were perspectives, 5 (7.0%) were reviews, and 26 (36.6%) were prospective empirical studies. The publications mapped to the Ottawa Criteria 158 times. Perspectives mapped more positively (83.7%) than empirical studies (76.7%). Reproducibility did not appear to be a strength of EPAs in UME; however, reproducibility, equivalence, educational effect, and catalytic effect all require further study. Inconsistent use of the term "EPA" and conflation of concepts (activity vs assessment vs advancement decision vs curricular framework) limited interpretation of published results. Overgeneralization of the AAMC's work on EPAs has influenced the literature.

CONCLUSIONS

Much has been published on EPAs in UME in a short time. Now is the time to move beyond opinion, clarify terms, and delineate topics so that well-designed empirical studies can demonstrate if and how EPAs should be implemented in UME.

Current Trends and Opportunities for Competency Assessment in Pharmacy Education-A Literature Review

An increasing emphasis on health professional competency in recent times has been matched by an increased prevalence of competency-based education models. Assessments can generate information on competence, and authentic, practice-based assessment methods are critical. Assessment reform has emerged as an academic response to the demands of the pharmacy profession and the need to equip graduates with the necessary knowledge, skills and attributes to face the challenges of the modern workforce. The objective of this review was to identify and appraise the range of assessment methods used in entry-level pharmacy education and examine current trends in health professional assessment. The initial search located 2854 articles. After screening, 36 sources were included in the review, 13 primary research studies, 12 non-experimental pharmacy research papers, and 11 standards and guidelines from the grey literature. Primary research studies were critically appraised using the Medical Education Research Study Quality Instrument (MERSQI). This review identified three areas in pharmacy practice assessment which provide opportunities for expansion and improvement of assessment approaches: (1) integrated approaches to performance assessment; (2) simulation-based assessment approaches, and; (3) collection of validity evidence to support assessment decisions. Competency-based assessment shows great potential for expanded use in pharmacy, but there is a need for further research and development to ensure its appropriate and effective use.

Introducing an assessment tool based on a full set of end-of-training EPAs to capture the workplace performance of final-year medical students

BACKGROUND

While literature on the theoretical value of entrustable professional activities (EPAs) for assessment is rapidly expanding, little experience exists on its application. The aims of this study are to develop and explore the utility of an EPA-based assessment tool for capturing the workplace performance of final-year medical students based on a full set of end-of-training EPAs.

METHODS

The tool was developed in a systematic iterative process. Twelve 12 end-of-undergraduate medical training EPAs were nested into 72 smaller EPAs and cross-mapped onto a 6-point supervision level scale, both adjusted to the context of final-year clerkships. One version was created for students' self-assessment of their ability to carry out tasks and their history of carrying out tasks, and another version was created for supervisors' assessment of students' ability to carry out tasks. The tool was administered to final-year clerkship students and their clinical supervisors to explore its utility as an assessment approach. The results were analysed using descriptive and interferential statistics.

RESULTS

We enrolled a total of 60 final-year medical students. For 33 students, ratings were provided from one supervisor and for 27 students from two supervisors. With regard to the reliability and validity of the tool, students' and supervisors' ratings showed an overall good internal consistency as well as variability between and within the EPAs. Over the full EPA range, students rated their ability to perform a task slightly higher than their task performance history and slightly lower than the supervisors' ratings. Students' self-ratings of their ability to perform a task correlated with their history in performing the task. Supervisors' ratings correlated among supervisors and not with students' ratings. Concerning educational outcomes, supervisors' average rating of students' ability to perform the EPAs without direct supervision was 64%, and key findings being double-checked.

CONCLUSIONS

This study introduces a tool that is adjusted to the final-year clerkship context and can assess the workplace performance of trainees based on a full set of end-of-training EPAs. Its utility characteristics suggest that the tool may be employed as a formative and outcome-aligned approach to the assessment of final-year students before entering into residency.

The Power of Subjectivity in the Assessment of Medical Trainees

Objectivity in the assessment of students and trainees has been a hallmark of quality since the introduction of multiple-choice items in the 1960s. In medical education, this has extended to the structured examination of clinical skills and workplace-based assessment. Competency-based medical education, a pervasive movement that started roughly around the turn of the century, similarly calls for rigorous, objective assessment to ensure that all medical trainees meet standards to assure quality of health care. At the same time, measures of objectivity, such as reliability, have consistently shown disappointing results. This raises questions about the extent to which objectivity in such assessments can be ensured.In fact, the legitimacy of "objective" assessment of individual trainees, particularly in the clinical workplace, may be questioned. Workplaces are highly dynamic and ratings by observers are inherently subjective, as they are based on expert judgment, and experts do not always agree-for good, idiosyncratic, reasons. Thus, efforts to "objectify" these assessments may be problematically distorting the assessment process itself. In addition, "competence" must meet standards, but it is also context dependent.Educators are now arriving at the insight that subjective expert judgments by medical professionals are not only unavoidable but actually should be embraced as the core of assessment of medical trainees. This paper elaborates on the case for subjectivity in assessment.

Patient safety during final-year clerkships: A qualitative study of possible error sources and of the potential of Entrustable Professional Activities

Aim: In final-year clerkships, such as the Practical Year in Germany, students' workplace learning has to be balanced with the ensuring of patient safety. In this qualitative study, we investigated problems concerning patient safety as perceived from the perspective of supervising physicians, and whether and to what extent Entrustable Professional Activities (EPAs) can lead to an improvement in patient safety. Method: Data was collected through focus groups. Participants were specialist physicians with experience of final-year clerkship training (n=11). The analysis of problems influencing patient safety was carried out deductively with an existing system of categories (error factors in the clinic). To identify potential improvements through EPAs, an inductively developed category system on the influence of EPAs in final-year clerkships was used. Results: Supervising physicians perceive a variety of problems which affect patient safety. These can be found in the categories organization and management, individual factors, task factors and work environment. The physicians feel that EPAs may lead to an improvement in training and subsequently in patient safety. Their comments can be collated to the categories improvement in training, performance levels and supporting learning processes, transparency and minimizing uncertainty. Conclusions: Statements by supervising physicians indicate a variety of problems in patient safety during the training of final-year clerkship students, for instance the lack of structure to the training. In their view, the implementation of EPAs can substantially reduce such risks, as they provide better content and organizational structure to the final-year clerkship.

The Development of Entrustable Professional Activities for Competency-Based Veterinary Education in Farm Animal Health

Entrustable professional activities (EPAs) are professional tasks that can be entrusted to a student under a given level of supervision once he or she has demonstrated competence in these tasks. The EPA construct was conceived to increase transparency in objectives for clinical workplace learning and to help ensure patient safety and the quality of care. A first step in implementing EPAs in a veterinary curriculum is to identify the core EPAs of the profession. The aim of this study was to develop EPAs for farm animal health. An initial set of 36 EPAs for farm animal health was prepared by a team of six veterinarians and curriculum developers and used in a modified Delphi study. In this iterative process, the EPAs were evaluated until higher than 80% agreement was reached. Of 83 veterinarians who participated, 39 (47%) completed the Delphi procedure. After two rounds, the panel reached consensus. A small expert group further refined and reorganized the EPAs for educational purposes into seven core EPAs for farm animal health and 29 sub-EPAs. This study is an important step in optimizing competency-based training in veterinary medicine. Future steps are to implement EPAs in the curriculum and train supervisors to assess students' ability to perform EPAs with increasing levels of independence.

Factors Affecting Entrustment and Autonomy in Emergency Medicine: "How much rope do I give them?"

Introduction

During residency, the faculty’s role is to provide supervision while granting the trainee autonomy. This concept is termed entrustment. The goal is appropriate progression from supervision to autonomy while decreasing oversight as residents train. The objective of this study was to better understand the factors affecting the degree of autonomy or supervision faculty choose to provide residents.

Methods

This was a qualitative study of resident and faculty perceptions. We conducted two faculty and two resident focus groups. We then transcribed the transcripts of the audiotaped discussions and coded them using grounded theory.

Results

Analysis of the transcripts yielded four major factors affecting entrustment of residents.

Patient Factors included the acuity of the patient, sociomedical issues of patient/family, and complexity of risk with patient or procedure. For example, “sometimes there are families and patients who are exceedingly difficult that immediately sort of force me [to allow less autonomy].”

Environmental Factors included patient volume and systems protocols (i.e., trauma). “If you’re very busy and you have a resident that you already trust, you will give them more rope because you’re trying to juggle more balls.”

Resident Factors included the year of training, resident performance, clinical direct observation, and patient presentations. “But if you have a resident that you do not trust […] I tell them you’re going to do this, this, this, this, this.”

Faculty Factors included confidence in his/her own practice, risk-averse attitude, degree of ownership of the patient, commitment to education, and personality (e.g., micro-manager). Significant variability in entrustment by faculty existed, from being “micromanagers” to not seeing the patients. One resident noted: “There are some attendings, no matter how much they like you and how much you’ve worked with them, they’re always going to be in your face in the trauma bay. And there’s some attendings that are going to be ghosts.”

Conclusion

Multiple factors affect the amount of autonomy and entrustment given to residents and their level of supervision by faculty, leading to wide variability in entrustment. In the end, regardless of resident, patient, or environment, some faculty are more likely to entrust than others.

The Status of Interdisciplinary Education in Advanced Education Programs at U.S. Dental Schools

Interdisciplinary education (IDE) during dental residencies can help produce dentists who work more efficiently to provide continuous and reliable patient care. The aim of this study was to assess the extent and type of interdisciplinary education in dental residency programs at U.S. dental schools. A 24-item survey addressing didactic courses and patient care was sent to academic and/or clinical deans at 65 U.S. dental schools in fall 2016; they were asked to forward the survey to the appropriate person in their school. The questions addressed IDE characteristics such as the academic home for IDE, focus areas, defined outcomes, program objectives, and perceived institutional barriers. Of the 65 schools invited to participate, 31 responded to the survey for an overall response rate of 48%. Of those schools, 23 (74%) reported having IDE for their advanced/postgraduate students. Among the schools with IDE, their IDE learning experiences primarily involved residents in different disciplines participating in clinical case group projects and small group workshops, and 77% of IDE faculty taught in department-specific clinical spaces as opposed to teaching together in a single clinic. The respondents identified barriers to implementing IDE such as a lack of clearly defined competencies, logistical challenges including academic calendars and scheduling, and faculty resistance. Also, 43% reported that their institutions did not have a program to support IDE faculty development. At the time of this survey, most of the respondents did not have a clearly defined IDE model consisting of competencies, defined assessments, and milestones.

Bridging the gap: The intersection of entrustability and perceived autonomy for surgical residents in the OR

BACKGROUND

Faculty entrustment decisions affect resident entrustability behaviors and surgical autonomy. The relationship between entrustability and autonomy is not well understood. This pilot study explores that relationship.

METHODS

108 case observations were completed. Entrustment behaviors were rated using OpTrust. Residents completed a Zwisch self-assessment to measure surgical autonomy. Resident perceived autonomy was collected for 67 cases used for this pilot study.

RESULTS

Full entrustability was observed in 5 of the 108 observed cases. Residents in our study did not report full autonomy. Spearman's rank correlation coefficient identified that resident entrustability was positively correlated with perceived resident autonomy (ρ = 0.66, p < 0.05). Ordinal logistic regression assessed the relationship between resident entrustability and autonomy. The relationship persisted while controlling for PGY level, gender, and case complexity (OR = 8.42, SEM = 4.54, p < 0.000).

CONCLUSIONS

Resident entrustability is positively associated with perceived autonomy, yet full entrustability is not translating to the perception of full autonomy for residents.

A competency-based approach to critical care education

The medical education environment is rapidly changing. Competency-based medical education (CBME) is a great advance, but operationalizing competencies for teaching and assessment is problematic. Entrustable professional activities (EPAs) can revitalize CBME by connecting competencies to practice, creating flexibility in programs. CBME requires and deepens the nature of workplace-based assessments. It is important to use EPAs to verify residents' ability to care for critically ill patients unsupervised in simulation education.

When to trust our learners? Clinical teachers' perceptions of decision variables in the entrustment process

INTRODUCTION

Clinical training programs increasingly use entrustable professional activities (EPAs) as focus of assessment. However, questions remain about which information should ground decisions to trust learners. This qualitative study aimed to identify decision variables in the workplace that clinical teachers find relevant in the elaboration of the entrustment decision processes. The findings can substantiate entrustment decision-making in the clinical workplace.

METHODS

Focus groups were conducted with medical and veterinary clinical teachers, using the structured consensus method of the Nominal Group Technique to generate decision variables. A ranking was made based on a relevance score assigned by the clinical teachers to the different decision variables. Field notes, audio recordings and flip chart lists were analyzed and subsequently translated and, as a form of axial coding, merged into one list, combining the decision variables that were similar in their meaning.

RESULTS

A list of 11 and 17 decision variables were acknowledged as relevant by the medical and veterinary teacher groups, respectively. The focus groups yielded 21 unique decision variables that were considered relevant to inform readiness to perform a clinical task on a designated level of supervision. The decision variables consisted of skills, generic qualities, characteristics, previous performance or other information. We were able to group the decision variables into five categories: ability, humility, integrity, reliability and adequate exposure.

DISCUSSION

To entrust a learner to perform a task at a specific level of supervision, a supervisor needs information to support such a judgement. This trust cannot be credited on a single case at a single moment of assessment, but requires different variables and multiple sources of information. This study provides an overview of decision variables giving evidence to justify the multifactorial process of making an entrustment decision.

Enhanced Requirements for Assessment in a Competency-Based, Time-Variable Medical Education System

Competency-based, time-variable medical education has reshaped the perceptions and practices of teachers, curriculum designers, faculty developers, clinician educators, and program administrators. This increasingly popular approach highlights the fact that learning among different individuals varies in duration, foundation, and goal. Time variability places particular demands on the assessment data that are so necessary for making decisions about learner progress. These decisions may be formative (e.g., feedback for improvement) or summative (e.g., decisions about advancing a student). This article identifies challenges to collecting assessment data and to making assessment decisions in a time-variable system. These challenges include managing assessment data, defining and making valid assessment decisions, innovating in assessment, and modeling the considerable complexity of assessment in real-world settings and richly interconnected social systems. There are hopeful signs of creativity in assessment both from researchers and practitioners, but the transition from a traditional to a competency-based medical education system will likely continue to create much controversy and offer opportunities for originality and innovation in assessment.

Choosing entrustable professional activities for neonatology: a Delphi study

OBJECTIVE

To assess candidate neonatology EPAs taken from separate but overlapping sets from two organizations.

STUDY DESIGN

Using a Delphi process, we asked that neonatology fellowship directors (1) assess importance and scope of 19 candidate EPAs, and (2) propose additional EPAs if necessary. In round 2, we sought clarification of first round responses and evaluated proposed additional EPAs.

RESULTS

Twenty program directors participated. In round 1, all EPAs were scored as important, but four were overly broad. In round 2, respondents rejected proposed subdivisions of one overly broad EPA, retaining it as originally proposed. Specification of entrustment criteria improved the scope of the three other broad EPAs. However, after specification, they were re-rated as insufficiently important and therefore rejected. Neither newly proposed EPA from round 1 was rated as sufficiently important.

CONCLUSION

The Delphi process yielded 13 EPAs with which to assess capability to practice clinical neonatology.