Revisiting the Core Entrustable Professional Activities for Entering Residency

Disclosure of Adverse Events in Pediatrics: Policy Statement

Disclosure of adverse events has become the expectation in medicine and is widely regarded as the appropriate path when medical errors occur. Although data are limited on adverse events in pediatrics, that they occur frequently is uncontested. Types and rates of errors vary depending on the care setting and patient population. Patients with complex medical conditions or from historically marginalized groups or minoritized communities likely suffer disparate health and safety outcomes. Systemic factors, including nonpunitive safety cultures and supportive environments within institutions, are essential to promoting disclosure. State laws protecting apologies from use in legal proceedings can also help to encourage open communication. Some states have adopted laws to advance disclosure, and governmental agencies provide materials encouraging open communication and early resolution after adverse events occur. Many programs emphasize the importance of supporting health care workers involved in adverse events. Shame, fear of professional and legal repercussions, and lack of training remain barriers to disclosure. Education for health care clinicians, support in health care settings, additional research on programs and disparities, and governmental and regulatory initiatives can support disclosure of adverse events.

Factors influencing the incorporation of Entrustable Professional Activities into assessment in nutrition and dietetics education: A systematic review

AIMS

This study aimed to identify and synthesise the evidence on factors influencing the incorporation of Entrustable Professional Activities into assessment in nutrition and dietetics education.

METHODS

A systematic review was conducted with a narrative synthesis and was undertaken and reported in accordance with the PRISMA guidelines. Six electronic databases were searched (MEDLINE, CINAHL, SCOPUS, PsycINFO, Web of Science, and EMBASE) on 9 September 2024. Methodological quality was assessed using the Critical Appraisal Skills Program checklists. Key patterns identified from the narrative synthesis of the included manuscripts were labelled as themes and represented in a figure.

RESULTS

Across the international literature, six articles were identified revealing six main and interconnected themes related to factors influencing the incorporation of Entrustable Professional Activities in nutrition and dietetics. In summary, development and review processes for Entrustable Professional Activities require key stakeholder engagement in addition to ensuring they are linked to assessment structures and existing frameworks. Furthermore, technology platforms and applications appeared to support Entrustable Professional Activity incorporation, and training is an important part of integration.

CONCLUSIONS

Further research on factors influencing incorporation is occurring and is suggested to continue, especially given Entrustable Professional Activities seemingly offer a tangible option to simplify the intricacy of competency-based assessment in work-based practice. However, further research to enhance understanding of whether Entrustable Professional Activities support nutrition and dietetics learners and assessors in undertaking high-quality assessment with utility is warranted.

Effectiveness of educational interventions for improving healthcare professionals' information literacy: A systematic review

BACKGROUND

It is unclear which educational interventions effectively improve healthcare professionals' information literacy.

OBJECTIVES

We aimed to evaluate the effectiveness of educational interventions for improving the formulation of answerable clinical questions and the search skills of healthcare professionals.

METHODS

We followed the Cochrane methodology and reported according to the PRISMA statement. The following databases from inception to November 2022: MEDLINE, Cochrane CENTRAL, EMBASE, Web of Science, CINAHL, and Google Scholar search engine, were searched. Randomised controlled trials and crossover trials on any educational interventions were included. Studies on search tools that are obsolete were excluded.

RESULTS

Ten studies that mainly compared the effectiveness of lectures and bedside education to lectures or no intervention for searching of PubMed and/or MEDLINE, were included. There was evidence for improved attitude towards the intervention favouring lecture with self-directed learning over lecture, bedside education, and computer-assisted self-directed learning (RR: 1.14; 95% CI 1.06-1.23; N = 2 studies; 1064 participants; I2 = 0%; moderate certainty evidence). There were limited findings on the knowledge, skills, satisfaction, and behaviour outcomes.

CONCLUSION

Future research should include a wider set of outcomes, be reported better and explore the use of digital technology for delivery of educational interventions. Further research should entail well-designed trials with relevant outcomes evaluating novel digital-based educational interventions.

A Rapid Response Mobile Application Improves First-Year Resident Clinical Performance During Simulated Care Events: A Randomized Controlled Trial

BACKGROUND

Rapid response teams (RRTs) are critical to the timely and appropriate management of acutely decompensating patients. In the academic setting, the vital role of RRT leader is often filled by a junior resident physician who may lack the necessary medical knowledge and experience. Cognitive aids help improve guideline adherence and may support resident performance as they transition into leadership roles.

OBJECTIVE

This study evaluated the impact of a rapid response mobile application on intern performance during simulated rapid response events.

DESIGN

This randomized controlled trial compared the performance of interns in two simulated rapid response scenarios with and without access to the rapid response mobile application. The scenarios included anaphylaxis and supraventricular tachycardia (SVT). Simulations were video recorded and coded by trained raters.

PARTICIPANTS

Interns in all specialties at our institution.

MAIN MEASURES

Outcomes included (1) time to ordering critical medications (epinephrine and adenosine), (2) overall clinical performance using a checklist-based performance measure, and (3) usability of the mobile application. Enrollment and data collection occurred between November 2022 and February 2023.

KEY RESULTS

Forty-four interns from 12 specialties were randomized to the intervention group (N = 22) and the control group (N = 22). Time to order critical medications was significantly reduced in the intervention group compared to control for anaphylaxis (P < 0.005) and SVT (P < 0.005). The intervention group had significantly higher performance scores compared to the control group for the anaphylaxis portion (P < 0.006). Usability scores for the rapid response toolkit were good.

CONCLUSIONS

Access to a rapid response mobile application improved the quality of care administered by interns during two simulated rapid response scenarios as determined by a decrease in time to ordering critical medications and improved performance scores. The intervention group found the mobile application to be usable. This work adds to existing literature supporting the use of technology-based cognitive aids to improve patient care.

Assessing readiness: the impact of an experiential learning entrustable professional activity-based residency preparatory course

As medical schools move to integrate the Core Entrustable Professional Activities for Entering Residency (EPAs) into curricula and address the transition from student to resident, residency preparatory courses have become more prevalent. The authors developed an experiential learning EPA-based capstone course for assessment to determine impact on learner self-assessed ratings of readiness for residency and acquisition of medical knowledge. All fourth-year students from the classes of 2018-2020 completed a required course in the spring for assessment of multiple EPAs, including managing core complaints, performing basic procedures, obtaining informed consent, and providing patient handoffs. Learners selected between three specialty-based parallel tracks - adult medicine, surgery, or pediatrics. Students completed a retrospective pre-post questionnaire to provide self-assessed ratings of residency preparedness and comfort in performing EPAs. Finally, the authors studied the impact of the course on knowledge acquisition by comparing student performance in the adult medicine track on multiple choice pre- and post-tests. Four hundred and eighty-one students were eligible for the study and 452 (94%) completed the questionnaire. For all three tracks, there was a statistically significant change in learner self-assessed ratings of preparedness for residency from pre- to post-course (moderately or very prepared: adult medicine 61.4% to 88.6% [p-value < 0.001]; surgery 56.8% to 81.1% [p-value < 0.001]; pediatrics 32.6% to 83.7% [p-value 0.02]). A similar change was noted in all tracks in learner self-assessed ratings of comfort from pre- to post-course for all studied EPAs. Of the 203 students who participated in the adult medicine track from 2019-2020, 200 (99%) completed both the pre- and post-test knowledge assessments. The mean performance improved from 65.0% to 77.5% (p-value < 0.001). An experiential capstone course for the assessment of EPAs can be effective to improve learner self-assessed ratings of readiness for residency training and acquisition of medical knowledge.

Developing competency-based medical education for dental education in Taiwan: A pilot study of tooth extraction entrustable professional activities

Background/purpose

This study aimed to evaluate the initial implementation of competency-based medical education (CBME) through entrustable professional activities (EPAs) in Taiwan dental education, focusing on tooth extraction EPAs across undergraduate year (UGY), postgraduate year (PGY), and oral and maxillofacial surgery-residency (OS-R) levels.

Materials and methods

Using the Delphi method, an advisory team developed and validated three levels of trial EPAs, which were implemented through the Emyway platform. A structured questionnaire was used to evaluate teachers' and students' experiences and satisfaction with Emyway and the EPAs. Data were collected from the participants at two university-affiliated hospitals in Taiwan. Statistical analyses comparing teachers' and students' experiences and satisfaction were conducted.

Results

The EQual rubric scores for the EPAs were high across all levels (UGY: 4.80, PGY: 4.78, R: 4.79) with no significant differences. The demographic characteristics and Emyway experiences were similar among 7 teachers and 17 students. No significant difference was observed in terms of satisfaction with EPA design and usage between the teachers and students. For the teachers and students, the overall Emyway satisfaction scores were 3.86 and 3.76 and the EPA content satisfaction scores were 4.00 and 3.71, respectively, with no significant differences.

Conclusion

The study successfully introduced EPAs into Taiwan dental education, demonstrating high EQual rubric scores and moderate satisfaction. Emyway is a convenient and effective platform for EPA implementation. Future efforts should focus on the expansion of EPAs to other dental specialties and integration of CBME into Taiwan dental education to align with international standards.

Reimagining Undergraduate Critical Care Medical Education: A Path for the Next Decade

Foundational training in critical care medicine is an integral part of both undergraduate and graduate medical education. Yet, many medical school graduates enter residency underprepared to care for critically ill patients because of a lack of ubiquity of undergraduate critical care education and the heterogeneity of existing didactic and clinical experiences. This Perspective explores the importance of undergraduate critical care education, the current national and international landscape, innovative educational strategies and exemplar curricula, and recent advances in assessment that may better reflect learner-centered educational outcomes. As broad curricular reforms push medical education toward a more innovative, interactive, and collaborative future, now is the time to rethink and reimagine undergraduate critical care education.

Integration of Entrustable Professional Activities in a Competency-Based Clinical Assessment Tool in a Nurse Practitioner Program

BACKGROUND

Entrustable professional activities (EPAs) are used in competency-based clinical assessment.

PROBLEM

Faculty in a Family Nurse Practitioner program aimed to improve their clinical assessment tool to more effectively measure students' clinical performance, outline expectations, track progress, and document evidence related to clinical competence.

APPROACH

EPAs that aligned with the American Association of Colleges of Nursing Essentials and National Organization of Nurse Practitioner (NP) Faculties Core NP Competencies were integrated into a clinical assessment tool. Student performance expectations at different points in the program were outlined. Students documented each time they performed an EPA during clinical experiences.

OUTCOME

The EPA-based clinical assessment tool captured valuable information about student performance during clinical experiences and contributed to evidence related to competencies.

CONCLUSIONS

EPAs can be used to assess clinical performance and document evidence of competence and readiness for practice. Tracking the frequency of EPA performance was essential to ensure students had sufficient practice.

Identifying primary care clinicians' preferences for, barriers to, and facilitators of information-seeking in clinical practice in Singapore: a qualitative study

BACKGROUND

The growth of medical knowledge and patient care complexity calls for improved clinician access to evidence-based resources. This study aimed to explore the primary care clinicians' preferences for, barriers to, and facilitators of information-seeking in clinical practice in Singapore.

METHODS

A convenience sample of ten doctors and ten nurses was recruited. We conducted semi-structured face-to-face in-depth interviews. The interviews were recorded, transcribed verbatim, and analysed using thematic content analysis.

RESULTS

Of the 20 participants, eight doctors and ten nurses worked at government-funded polyclinics and two doctors worked in private practice. Most clinicians sought clinical information daily at the point-of-care. The most searched-for information by clinicians in practice was less common conditions. Clinicians preferred evidence-based resources such as clinical practice guidelines and UpToDate®. Clinical practice guidelines were mostly used when they were updated or based on memory. Clinicians also commonly sought answers from their peers. Furthermore, clinicians frequently use smartphones to access the Google search engine and UpToDate® app. The barriers to accessing clinical information included the lack of time, internet surfing separation of work computers, limited search functions in the organisation's server, and limited access to medical literature databases. The facilitators of accessing clinical information included convenience, easy access, and trustworthiness of information sources.

CONCLUSION

Most primary care clinicians in our study sought clinical information at the point-of-care daily and reported increasing use of smartphones for information-seeking. Future research focusing on interventions to improve access to credible clinical information for primary care clinicians at the point-of-care is recommended.

TRIAL REGISTRATION

This study has been reviewed by NHG Domain Specific Review Board (NHG DSRB) (the central ethics committee) for ethics approval. NHG DSRB Reference Number: 2018/01355 (31/07/2019).

Using Virtual Reality to Teach Medical Students Cross-Coverage Skills

BACKGROUND

Recognizing and attempting management of patients with urgent or emergent conditions is one of the Association of American Medical Colleges Core Entrustable Professional Activities (#10) and a skill desired of new interns. However, given the acuity of these patient conditions, medical students often struggle to gain experience in these situations. Virtual reality could help fill this void while lowering costs and resources compared with high-fidelity simulation.

METHODS

We converted a high-fidelity chest pain simulation case to virtual reality format utilizing short video clips filmed with a 360-degree camera and superimposed menus of options at decision points. This virtual reality simulation was offered to fourth-year medical students during their transition to residency course in the spring of 2023. Students were offered a post-survey on the simulation.

RESULTS

There were 47 fourth-year students that completed the virtual reality simulation; 41 completed the post-survey (response rate 87.2%). Over 90% of the students agreed or strongly agreed with the following statements: the virtual reality simulation was a valuable part of the transition to residency course, the virtual reality case was similar to what they will face as an intern, and they would like to have more virtual reality simulations earlier in the fourth year; 85.4% agreed or strongly agreed that the virtual reality simulation helped prepare them for the first few days of intern year.

CONCLUSIONS

We demonstrated that virtual reality is an acceptable, cost-effective, and feasible modality to teach medical students how to recognize and attempt management of urgent clinical situations (Core Entrustable Professional Activity 10).

Using ACGME milestones as a formative assessment for the internal medicine clerkship: a consecutive two-year outcome and follow-up after graduation

BACKGROUND

This study evaluated the utility of using Accreditation Council for Graduate Medical Education (ACGME) Milestones as a formative assessment tool for the fifth- and sixth-grade medical students' performance in their internal medicine (IM) clerkship and the same students' performance in their post-graduate year (PGY) IM training.

METHODS

Retrospective data were collected from 65 medical students completing the two-year IM clerkship in the academic years 2019 and 2020 and 26 of the above students completing their PGY-1 training at the same university hospital in the academic year 2021. Data included the assessment results of 7 of the ACGME IM Milestones, information on admitted patients assigned to the students, and surveys of the students' satisfaction.

RESULTS

The analysis included 390 assessment results during the IM clerkship and 78 assessment results during the PGY-1 training. Clinical teachers commonly rated level 3 to medical students in the IM clerkship, with PC-2 subcompetency receiving the lowest rating among seven subcompetencies. The levels of most subcompetencies showed stationary in the two-year IM clerkship. Significant improvement was observed in all subcompetencies during the PGY-1 training. The medical students in the second-year IM clerkship expressed higher satisfaction with implementing Milestones than in their first-year IM clerkship and perceived Milestones assessments' usefulness as learning feedback.

CONCLUSIONS

Using ACGME Milestones as a formative assessment tool in the IM clerkship yielded promising outcomes. Longitudinal follow-up of subcompetencies facilitated tracking students' development and providing constructive feedback.

Medical Error Disclosure: An Entrustable Professional Activity During an Objective Standardized Clinical Examination for Clerkship Students

Introduction

Most health care providers will be involved in a medical error during their careers. It is critical that future physicians receive formal training on error disclosure.

Methods

We designed a formative skills-based objective standardized clinical exam (OSCE) for fourth-year medical students to assess competence in disclosing an error during a required entrustable professional activity. Faculty observed the encounter and completed a checklist evaluating students' performance in communication skills and content knowledge. Students received immediate formative feedback. They then participated in a facilitated case-based experience, discussed the critical elements of disclosure, utilized role-play to reinforce skills, and reflected on self-care practices. Finally, students completed a survey evaluating their perception of the OSCE's impact on their disclosure knowledge, skills, and attitudes.

Results

Ninety-two students participated in the OSCE. Of those, 67 (73%) completed a retrospective pre/post survey assessing their disclosure knowledge, skills, and attitudes. Forty-one (62%) did not identify the error. Students who identified the error (26, 39%) were more likely to use the two-patient identifier than students who did not identify the error, χ2(1) = 13.3, p < .001. Self-reported comfort and confidence in disclosure improved, as did self-care practices (ps ≤ .005).

Discussion

Students agreed that health care providers should disclose an error and know how to do so. Student self-reported comfort in disclosure and knowledge of how to disclose and how to report an error all improved following the OSCE and structured debrief. The OSCE and case-based experience can be adapted for implementation in curricula about error disclosure.

Harnessing the potential of large language models in medical education: promise and pitfalls

OBJECTIVES

To provide balanced consideration of the opportunities and challenges associated with integrating Large Language Models (LLMs) throughout the medical school continuum.

PROCESS

Narrative review of published literature contextualized by current reports of LLM application in medical education.

CONCLUSIONS

LLMs like OpenAI's ChatGPT can potentially revolutionize traditional teaching methodologies. LLMs offer several potential advantages to students, including direct access to vast information, facilitation of personalized learning experiences, and enhancement of clinical skills development. For faculty and instructors, LLMs can facilitate innovative approaches to teaching complex medical concepts and fostering student engagement. Notable challenges of LLMs integration include the risk of fostering academic misconduct, inadvertent overreliance on AI, potential dilution of critical thinking skills, concerns regarding the accuracy and reliability of LLM-generated content, and the possible implications on teaching staff.

Competency-based medical education: The spark to ignite healthcare's escape fire

High-value care is what patients deserve and what healthcare professionals should deliver. However, it is not what happens much of the time. Quality improvement master Dr. Don Berwick argued more than two decades ago that American healthcare needs an escape fire, which is a new way of seeing and acting in a crisis situation. While coined in the U.S. context, the analogy applies in other Western healthcare contexts as well. Therefore, in this paper, the authors revisit Berwick's analogy, arguing that medical education can, and should, provide the spark for such an escape fire across the globe. They assert that medical education can achieve this by fully embracing competency-based medical education (CBME) as a way to place medicine's focus on the patient. CBME targets training outcomes that prepare graduates to optimize patient care. The authors use the escape fire analogy to argue that medical educators must drop long-held approaches and tools; treat CBME implementation as an adaptive challenge rather than a technical fix; demand genuine, rich discussions and engagement about the path forward; and, above all, center the patient in all they do.

Reimagining Core Entrustable Professional Activities for Undergraduate Medical Education in the Era of Artificial Intelligence

The proliferation of generative artificial intelligence (AI) and its extensive potential for integration into many aspects of health care signal a transformational shift within the health care environment. In this context, medical education must evolve to ensure that medical trainees are adequately prepared to navigate the rapidly changing health care landscape. Medical education has moved toward a competency-based education paradigm, leading the Association of American Medical Colleges (AAMC) to define a set of Entrustable Professional Activities (EPAs) as its practical operational framework in undergraduate medical education. The AAMC's 13 core EPAs for entering residencies have been implemented with varying levels of success across medical schools. In this paper, we critically assess the existing core EPAs in the context of rapid AI integration in medicine. We identify EPAs that require refinement, redefinition, or comprehensive change to align with the emerging trends in health care. Moreover, this perspective proposes a set of "emerging" EPAs, informed by the changing landscape and capabilities presented by generative AI technologies. We provide a practical evaluation of the EPAs, alongside actionable recommendations on how medical education, viewed through the lens of the AAMC EPAs, can adapt and remain relevant amid rapid technological advancements. By leveraging the transformative potential of AI, we can reshape medical education to align with an AI-integrated future of medicine. This approach will help equip future health care professionals with technological competence and adaptive skills to meet the dynamic and evolving demands in health care.

Competency-Based Medical Education in a Norm-Referenced World: A Root Cause Analysis of Challenges to the Competency-Based Paradigm in Medical School

Competency-based medical education (CBME) requires a criterion-referenced approach to assessment. However, despite best efforts to advance CBME, there remains an implicit, and at times, explicit, demand for norm-referencing, particularly at the junction of undergraduate medical education (UME) and graduate medical education (GME). In this manuscript, the authors perform a root cause analysis to determine the underlying reasons for continued norm-referencing in the context of the movement toward CBME. The root cause analysis consisted of 2 processes: (1) identification of potential causes and effects organized into a fishbone diagram and (2) identification of the 5 whys. The fishbone diagram identified 2 primary drivers: the false notion that measures such as grades are truly objective and the importance of different incentives for different key constituents. From these drivers, the importance of norm-referencing for residency selection was identified as a critical component. Exploration of the 5 whys further detailed the reasons for continuation of norm-referenced grading to facilitate selection, including the need for efficient screening in residency selection, dependence upon rank-order lists, perception that there is a best outcome to the match, lack of trust between residency programs and medical schools, and inadequate resources to support progression of trainees. Based on these findings, the authors argue that the implied purpose of assessment in UME is primarily stratification for residency selection. Because stratification requires comparison, a norm-referenced approach is needed. To advance CBME, the authors recommend reconsideration of the approach to assessment in UME to maintain the purpose of selection while also advancing the purpose of rendering a competency decision. Changing the approach will require a collaboration between national organizations, accrediting bodies, GME programs, UME programs, students, and patients/societies. Details are provided regarding the specific approaches required of each key constituent group.

Entrustable professional activities (EPAs) for undergraduate medical education - development and exploration of social validity

BACKGROUND

The development of entrustable professional activities (EPAs) as a framework for work-based training and assessment in undergraduate medical education has become popular. EPAs are defined as units of a professional activity requiring adequate knowledge, skills, and attitudes, with a recognized output of professional labor, independently executable within a time frame, observable and measurable in its process and outcome, and reflecting one or more competencies. Before a new framework is implemented in a specific context, it is valuable to explore social validity, that is, the acceptability by relevant stakeholders.

AIM

The aim of our work was to define Core EPAs for undergraduate medical education and further explore the social validity of the constructs.

METHOD AND MATERIAL

In a nationwide collaboration, EPAs were developed using a modified Delphi procedure and validated according to EQual by a group consisting of teachers nominated from each of the seven Swedish medical schools, two student representatives, and an educational developer (n = 16). In the next step, social validity was explored in a nationwide survey. The survey introduced the suggested EPAs. For each EPA, the importance of the EPA was rated, as was the rater's perception of the present graduates' required level of supervision when performing the activity. Free-text comments were also included and analyzed.

RESULTS

Ten Core EPAs were defined and validated. The validation scores for EQual ranged from 4.1 to 4.9. The nationwide survey had 473 responders. All activities were rated as "important" by most responders, ranging from 54 to 96%. When asked how independent current graduates were in performing the ten activities, 6 to 35% reported "independent". The three themes of the free text comments were: 'relevant target areas and content'; 'definition of the activities'; and 'clinical practice and learning'.

CONCLUSION

Ten Core EPAs were defined and assessed as relevant for Swedish undergraduate medical education. There was a consistent gap between the perceived importance and the certainty that the students could perform these professional activities independently at the time of graduation. These results indicate that the ten EPAs may have a role in undergraduate education by creating clarity for all stakeholders.

Fixing a Broken Clerkship Assessment Process: Reflections on Objectivity and Equity Following the USMLE Step 1 Change to Pass/Fail

Clerkship grading is a core feature of evaluation for medical students' skills as physicians and is considered by most residency program directors to be an indicator of future performance and success. With the transition of the U.S. Medical Licensing Examination Step 1 score to pass/fail, there will likely be even greater reliance on clerkship grades, which raises several important issues that need to be urgently addressed. This article details the current landscape of clerkship grading and the systemic discrepancies in assessment and allocation of honors. The authors examine not only objectivity and fairness in clerkship grading but also the reliability of clerkship grading in predicting residency performance and the potential benefits and drawbacks to adoption of a pass/fail clinical clerkship grading system. In the promotion of a more fair and equitable residency selection process, there must be standardization of grading systems with consideration of explicit grading criteria, grading committees, and/or structured education of evaluators and assessors regarding implicit bias. In addition, greater adherence and enforcement of transparency in grade distributions in the Medical Student Performance Evaluation is needed. These changes have the potential to level the playing field, foster equitable comparisons, and ultimately add more fairness to the residency selection process.

EPMO: A novel medical student assessment tool that integrates entrustable professional activities, prime, and the modified Ottawa coactivity scale

PURPOSE

Alignment of workplace-based assessments (WPBA) with core entrustable professional activities (EPAs) for entering residency may provide opportunities to monitor student progress across the continuum of undergraduate medical education. Core EPAs, however, reflect tasks of varying degrees of difficulty and faculty assessors are not accustomed to rating students based on entrustability. Expectations of student progress should vary depending on the complexity of the tasks associated with the EPAs. An assessment tool that orients evaluators to the developmental progression of specific EPA tasks will be critical to fairly evaluate learners.

METHODS

The authors developed an EPA assessment tool combining the frameworks of Professionalism, Reporter, Interpreter, Manager, Educator (PRIME), and Modified Ottawa coactivity scales. Only those EPAs that could be repeatedly observed and assessed across clinical clerkships were included. From July 2019 to March 2020, third-year medical students across multiple clerkships were assessed using this tool. The authors hypothesized that if the tool was applied correctly, ratings of learner independence would be lower with higher complexity tasks and that such ratings would increase over the course of year with ongoing clinical learning.

RESULTS

Assessment data for 247 medical students were similar across clerkships suggesting that evaluators in diverse clinical contexts were able to use this tool to assign scores reflective of developing entrustability in the workplace. Faculty rated student entrustability highest in skills emphasized in the pre-clerkship curriculum (professionalism and reporter) and progressively lower in more advanced skills (interpreter and manager). Students' ratings increased over time with more clinical exposure.

CONCLUSIONS

The authors developed a composite WBPA tool that combines the frameworks of EPAs, PRIME, and Modified Ottawa Co- Activity and demonstrated the usability of applying it for learner assessments in clinical settings. Further multicenter studies with cohorts of pre- and post-clerkship students may provide additional validity evidence for the tool.

Neurology Education in 2035: The Neurology Future Forecasting Series

In the past decade, there have been dramatic changes in all aspects of neurologic care, and along with this, neurology education has transformed. These changes have affected all aspects of education across the educational continuum, including learners, teachers, educators, content, delivery methods, assessments, and outcomes. Health systems science, health humanities, diversity, equity, and inclusion and health disparities are becoming core components of neurology curricula, and, in the future, will be integrated into every aspect of our educational mission. The ways in which material is taught and learned have been influenced by technologic innovations and a growing understanding of the science of learning. We forecast that this trend will continue, with learners choosing from an array of electronic resources to engage with fundamental topics, allowing front-line clinical teachers to spend more time supporting critical reasoning and teaching students how to learn. There has been a growing differentiation of educational roles (i.e., teachers, educators, and scholars). We forecast that these roles will become more distinct, each with an individualized pattern of support and expectations. Assessment has become more aligned with the work of the learners, and there are growing calls to focus more on the impact of educational programs on patient care. We forecast that there will be an increased emphasis on educational outcomes and public accountability for training programs. In this article, we reflect on the history of medical education in neurology and explore the current state to forecast the future of neurology education and discuss ways in which we can prepare.

Oral examination of fourth-year medical students on surgery rotations allows faculty to assess Core Entrustable Professional Activities for entering Residency (CEPAR): Proof-of-concept and analysis of student submissions

BACKGROUND

As a community-based medical school which recruited faculty preceptors new to teaching, we sought to create objective assessments for fourth-year surgery experiences via administration of an oral exam. Students provided three authentic cases, which faculty used as a springboard to ascertain student proficiency in five entrustable professional activities: 1-oral presentation, 2-recognition of urgency/instability, 3-calling consults, 4-transitions of care, 5-informed consent. We present proof-of-concept and analysis of student case submissions.

METHODS

Twenty-seven student submissions (79 cases in total) were evaluated for case complexity, level-appropriateness, and an estimation of the ability to conduct a quality exam based on the information provided (subjective measures). Objective metrics included word count, instruction adherence, inclusion of figures/captions. A resident-in-training rated cases via the same metrics. In-examination data was separately culled.

RESULTS

The average word count was 281.70 (SD 140.23; range 40-743). Figures were included in 26.1% of cases. Faculty raters scored 29.0% as low-complexity, 37.7% medium-complexity, and 33.3% high-complexity. Raters felt 62.3% of cases provided enough information to conduct a quality exam. The majority of cases submitted (65.2%) were level-appropriate or higher. The resident rater scored cases more favorably than surgeons (Cohen's kappa of -0.5), suggesting low inter-rater agreement between those of differing experience levels.

CONCLUSION

Student's case submissions lessened faculty burden and provided assessors with adequate information to deliver a quality exam to assess proficiency in clinical skills essential for residency. Cases demonstrated sufficient complexity and level-appropriateness. The request to correlate case rating with exam performance is under review by our institution's assessment office. Near-peer tutoring by resident alumni is a program under development.

Filling the Core EPA 10 assessment void: A framework for individual assessment of Core Entrustable professional activity 10 competencies in medical students

Objectives

The goal of this study was to develop and evaluate a novel curriculum and assessment tool for Core Entrustable Professional Activity (EPA) 10 competencies and entrustment scoring in a cohort of medical students in their emergency medicine (EM) clerkship using a framework of individualized, ad hoc, formative assessment. Core EPA 10 is an observable workplace-based activity for graduating medical students to recognize a patient requiring urgent or emergent care and initiate evaluation and management.

Methods

This is a prospective, pretest-posttest study of medical students during their EM clerkship. Using the Thomas and Kern framework, we created a curriculum of simulation cases about chest pain/cardiac arrest and respiratory distress, which included novel assessment checklists, and instructional videos about recognizing and managing emergencies. Students were individually pretested on EPA 10 competencies using the simulation cases. Two raters scored students using standardized checklists. Students then watched instructional videos, underwent a posttest with the simulation cases, and were scored again by the two raters using the checklists. Differences between pretest and posttest scores were analyzed using paired t-tests and Wilcoxon signed-rank tests.

Results

Seventy-three out of 85 (86%) students completed the curriculum. Mean scores from pretest to final posttest in the chest pain/cardiac arrest and respiratory distress cases significantly improved from 14.8/19 (SD 1.91), to 17.1/19 (SD = 1.00), t(68) = 10.56, p < 0.001, and 8.5/13 (SD 1.79), to 11.1/13(SD 0.89), t(67) = 11.15, p < 0.001, respectively. The kappa coefficients were 0.909 (n = 2698, p < 0.001) and 0.931 (n = 1872, p < 0.001). Median modified Chen entrustment scores improved from 1b (i.e., "Watch me do this") to 2b (i.e., "I'll watch you") for the chest pain/cardiac arrest case (p < 0.001) and 1b/2a (i.e., "Watch me do this"/ "Let's do this together") to 3a (i.e. "You go ahead, and I'll double-check all of your findings") for the respiratory distress case (p < 0.001).

Conclusion

A new directed curriculum of standardized simulation cases and asynchronous instructional videos improved medical student performance in EPA 10 competencies and entrustment scores. This study provides a curricular framework to support formative individualized assessments for EPA 10.

Core Entrustable Professional Activities for Entering Residency: A National Survey of Graduating Medical Students' Self-Assessed Skills by Specialty

BACKGROUND

The Association of American Medical Colleges described 13 Core Entrustable Professional Activities (EPAs) that graduating students should be prepared to perform under indirect supervision on day one of residency. Surgery program directors recently recommended entrustability in these Core EPAs for incoming surgery interns. We sought to determine if graduating students intending to enter surgery agreed they had the skills to perform these Core EPAs.

STUDY DESIGN

Using de-identified, individual-level data collected from and about 2019 Association of American Medical Colleges Graduation Questionnaire respondents, latent profile analysis was used to group respondents based on their self-assessed Core EPAs skills' response patterns. Associations between intended specialty, among other variables, and latent profile analysis group were assessed using independent sample t -tests and chi-square tests and multivariable logistic regression methods.

RESULTS

Among 12,308 Graduation Questionnaire respondents, latent profile analysis identified 2 respondent groups: 7,863 (63.9%) in a high skill acquisition agreement (SAA) group and 4,445 (36.1%) in a moderate SAA group. Specialty was associated with SAA group membership (p < 0.001), with general surgery, orthopaedic surgery, and emergency medicine respondents (among others) overrepresented in the high SAA group. In the multivariable logistic regression models, each of anesthesiology, ophthalmology, pediatrics, psychiatry, and radiology (vs general surgery) specialty intention was associated with a lower odds of high SAA group membership.

CONCLUSION

Graduating students' self-assessed Core EPAs skills were higher for those intending general surgery than for those intending some other specialties. Our findings can inform collaborative efforts to ensure graduates' acquisition of the skills expected of them at the start of residency.

Representational scaffolding in digital simulations – learning professional practices in higher education

Purpose

To advance the learning of professional practices in teacher education and medical education, this conceptual paper aims to introduce the idea of representational scaffolding for digital simulations in higher education.

Design/methodology/approach

This study outlines the ideas of core practices in two important fields of higher education, namely, teacher and medical education. To facilitate future professionals’ learning of relevant practices, using digital simulations for the approximation of practice offers multiple options for selecting and adjusting representations of practice situations. Adjusting the demands of the learning task in simulations by selecting and modifying representations of practice to match relevant learner characteristics can be characterized as representational scaffolding. Building on research on problem-solving and scientific reasoning, this article identifies leverage points for employing representational scaffolding.

Findings

The four suggested sets of representational scaffolds that target relevant features of practice situations in simulations are: informational complexity, typicality, required agency and situation dynamics. Representational scaffolds might be implemented in a strategy for approximating practice that involves the media design, sequencing and adaptation of representational scaffolding.

Originality/value

The outlined conceptualization of representational scaffolding can systematize the design and adaptation of digital simulations in higher education and might contribute to the advancement of future professionals’ learning to further engage in professional practices. This conceptual paper offers a necessary foundation and terminology for approaching related future research.

Fidelity to Best Practices in EPA Implementation: Outcomes Supporting Use of the Core Components Framework From the University of Virginia Entrustable Professional Activity Program

PROBLEM

The rapid expansion of entrustable professional activity (EPA) assessment programs has led to calls to ensure fidelity in implementation and integrity in meeting the goals of competency-based medical education. Initiated in July 2017, in advance of the articulated core components of EPA implementation, this article describes the structure and outcomes of the University of Virginia (UVA) EPA Program and provides support for the identified essential components.

APPROACH

The UVA EPA Program includes workplace assessments by residents/fellows, attending faculty, and master assessors (MAs), experienced clinicians who assess students across disciplines and clinical settings. All assessors participate in formal professional development and provide verbal and written comments to support their supervision ratings. The Entrustment Committee, composed of 12 MAs, uses a shared mental model and aggregates all assessor data to make a high-stakes summative entrustment decision about students' readiness to assume the role of an acting intern.

OUTCOMES

Since 2017, over 2,000 assessors have completed 56,969 EPA assessments for 1,479 students. Ninety-four percent of assessments have been done during the clerkship phase. Residents/fellows have completed a mean of 18 assessments, attending faculty a mean of 27, and MAs a mean of 882. Seventy-four percent of observed encounters involved patients with acute concerns with or without a co-morbid condition. Fifty percent of assessments occurred in inpatient and 32% in ambulatory settings. Eighty-seven percent of assessments contained narrative comments with more than 100 characters.

NEXT STEPS

Planned next steps will include earlier identification of students who require individualized learning to promote the development of skills related to EPAs, expansion of the remediation program to enable more students to engage in a clinical performance mastery elective, and creation of targeted professional development for assessors to reinforce the tenets of the EPA program.

Comparing Entrustment Decision-Making Outcomes of the Core Entrustable Professional Activities Pilot, 2019-2020

IMPORTANCE

Gaps in readiness for indirect supervision have been identified for essential responsibilities encountered early in residency, presenting risks to patient safety. Core Entrustable Professional Activities (EPAs) for entering residency have been proposed as a framework to address these gaps and strengthen the transition from medical school to residency.

OBJECTIVE

To assess progress in developing an entrustment process in the Core EPAs framework.

DESIGN, SETTING, AND PARTICIPANTS

In this quality improvement study in the Core EPAs for Entering Residency Pilot, trained faculty made theoretical entrustment determinations and recorded the number of workplace-based assessments (WBAs) available for each determination in 2019 and 2020. Four participating schools attempted entrustment decision-making for all graduating students or a randomly selected subset of students. Deidentified, individual-level data were merged into a multischool database.

INTERVENTIONS

Schools implemented EPA-related curriculum, WBAs, and faculty development; developed systems to compile and display data; and convened groups to make theoretical summative entrustment determinations.

MAIN OUTCOMES AND MEASURES

On an EPA-specific basis, the percentage of students for whom an entrustment determination could be made, the percentage of students ready for indirect supervision, and the volume of WBAs available were recorded.

RESULTS

Four participating schools made 4525 EPA-specific readiness determinations (2296 determinations in 2019 and 2229 determinations in 2020) for 732 graduating students (349 students in 2019 and 383 students in 2020). Across all EPAs, the proportion of determinations of "ready for indirect supervision" increased from 2019 to 2020 (997 determinations [43.4%] vs 1340 determinations [60.1%]; 16.7 percentage point increase; 95% CI, 13.8-19.6 percentage points; P < .001), as did the proportion of determinations for which there were 4 or more WBAs (456 of 2295 determinations with WBA data [19.9%] vs 938 [42.1%]; 22.2 percentage point increase; 95% CI, 19.6-24.8 percentage points; P < .001). The proportion of EPA-specific data sets considered for which an entrustment determination could be made increased from 1731 determinations (75.4%) in 2019 to 2010 determinations (90.2%) in 2020 (14.8 percentage point increase; 95% CI, 12.6-16.9 percentage points; P < .001). On an EPA-specific basis, there were 5 EPAs (EPA 4 [orders], EPA 8 [handovers], EPA 10 [urgent care], EPA 11 [informed consent], and EPA 13 [patient safety]) for which few students were deemed ready for indirect supervision and for which there were few WBAs available per student in either year. For example, for EPA 13, 0 of 125 students were deemed ready in 2019 and 0 of 127 students were deemed ready in 2020, while 0 determinations in either year included 4 or more WBAs.

CONCLUSIONS AND RELEVANCE

These findings suggest that there was progress in WBA data collected, the extent to which entrustment determinations could be made, and proportions of entrustment determinations reported as ready for indirect supervision. However, important gaps remained, particularly for a subset of Core EPAs.

Design and implementation of a core EPA-based acting internship curriculum

PURPOSE

The Association of American Medical Colleges published Core Entrustable Professional Activities (Core EPAs), expected independent clinical skills for first-day interns. We sought to determine whether a required acting internship (AI) in the fourth-year curriculum could be used for summative assessment of students' mastery of Core EPAs to a predefined level that would readily generalize across disciplines and campuses.

METHODS

The University of North Dakota School of Medicine and Health Sciences MD Program created a standardized, required Core EPA-based AI curriculum for multiple specialties at multiple geographic sites providing a final entrustability assessment for 10 EPAs in a single course.

RESULTS

The course was successfully designed and launched for all students in a single class. During the AI, students functioned at the level of an acting intern, rated the courses as superior, and performed at satisfactory exit-level competence for 10 Core EPAs.

CONCLUSIONS

A standardized, EPA-based AI curriculum can provide an opportunity for exit level EPA assessment in the medical curriculum. This model functions well within multiple specialties and at diverse community-based, volunteer faculty teaching sites.

Concordance of Narrative Comments with Supervision Ratings Provided During Entrustable Professional Activity Assessments

BACKGROUND

Use of EPA-based entrustment-supervision ratings to determine a learner's readiness to assume patient care responsibilities is expanding.

OBJECTIVE

In this study, we investigate the correlation between narrative comments and supervision ratings assigned during ad hoc assessments of medical students' performance of EPA tasks.

DESIGN

Data from assessments completed for students enrolled in the clerkship phase over 2 academic years were used to extract a stratified random sample of 100 narrative comments for review by an expert panel.

PARTICIPANTS

A review panel, comprised of faculty with specific expertise related to their roles within the EPA program, provided a "gold standard" supervision rating using the comments provided by the original assessor.

MAIN MEASURES

Interrater reliability (IRR) between members of review panel and correlation coefficients (CC) between expert ratings and supervision ratings from original assessors.

KEY RESULTS

IRR among members of the expert panel ranged from .536 for comments associated with focused history taking to .833 for complete physical exam. CC (Kendall's correlation coefficient W) between panel members' assignment of supervision ratings and the ratings provided by the original assessors for history taking, physical examination, and oral presentation comments were .668, .697, and .735 respectively. The supervision ratings of the expert panel had the highest degree of correlation with ratings provided during assessments done by master assessors, faculty trained to assess students across clinical contexts. Correlation between supervision ratings provided with the narrative comments at the time of observation and supervision ratings assigned by the expert panel differed by clinical discipline, perhaps reflecting the value placed on, and perhaps the comfort level with, assessment of the task in a given specialty.

CONCLUSIONS

To realize the full educational and catalytic effect of EPA assessments, assessors must apply established performance expectations and provide high-quality narrative comments aligned with the criteria.

Using the EPAs to Evaluate the Clinical Experience of Medical Students

INTRODUCTION

Although there are standardized assessments of medical students throughout medical school, the clinical experience of each student may vary widely. Currently, medical schools do not have a systematic method to ensure that students perform clinical skills during clinical clerkships. Our study used the Entrustable Professional Activities for Entering Residency (EPAs) framework to assess the ways in which medical students meaningfully participate in patient care at various clinical sites in each required clerkship and over time.

METHODS

Over the 2020-2021 academic year, 170 third-year medical students were asked to report the number of times they performed EPA-1 (gather history and perform physical examination), EPA-5 (document clinical encounter), and EPA-6 (provide oral presentation of clinical encounter) at the end of each required clinical rotation (emergency medicine, internal medicine, surgery, obstetrics/gynecology, pediatrics, neurology, psychiatry) at a single medical school. We used descriptive statistics and t tests to compare frequency of these EPAs by campus type, site type, clerkship, and time.

RESULTS

One thousand, two hundred sixty-one surveys met inclusion criteria. Students performed EPA-5 more often at an academic medical center, and EPA-1 more frequently in the outpatient setting. Students performed EPA-1 and EPA-6 most often during emergency medicine and EPA-5 most often during internal medicine. Performance of all three EPAs increased over time.

CONCLUSION

This reporting system produced a robust data set that allowed for EPA performance comparisons by campus, site type, clerkship, and time. EPA performance varied by rotation, site type, clerkship, and time.

Competency-based medical education across the continuum: How well aligned are medical school EPAs to residency milestones?

INTRODUCTION

Competency-based medical education (CBME) provides a framework for describing learner progression throughout training. However, specific approaches to CBME implementation vary widely across educational settings. Alignment between various methods used across the continuum is critical to support transitions and assess learner performance. The purpose of this study was to investigate alignment between CBME frameworks used in undergraduate medical education (UME) and graduate medical education (GME) settings using the US context as a model.

METHOD

The authors analyzed content from the core entrustable professional activities for entering residency (Core EPAs; UME model) and residency milestones (GME model). From that analysis, they performed a series of cross-walk activities to investigate alignment between frameworks. After independent review, authors discussed findings until consensus was reached.

RESULTS

Some alignment was found for activities associated with history taking, physical examination, differential diagnosis, patient safety, and interprofessional care; however, there were far more examples of misalignment.

CONCLUSIONS

These findings highlight challenges creating alignment of assessment frameworks across the continuum of training. The importance of these findings includes implications for assessment and persistence of the educational gap across UME and GME. The authors provide four next steps to improve upon the continuum of education.

An analysis of student essays on medical leadership and its educational implications in South Korea

To examine medical students' perceptions of leadership and explore their implications for medical leadership education. We conducted a qualitative analysis of the essays submitted by students in the medical leadership course from 2015 to 2019. We categorised the essays by the characteristics of the selected model leaders (N = 563) and types of leadership (N = 605). A statistically significant proportion of students selected leaders who were of the same gender as themselves (P < 0.001), graduate track students chose leaders in science (P = 0.005), while; military track students chose leaders in the military (P < 0.001). Although the highest proportion of students chose politicians as their model leaders (22.7%), this number decreased over time (P < 0.001), and a wider range of occupational groups were represented between 2015 and 2019. Charismatic leadership was the most frequently selected (31.9%), and over time there was a statistically significant (P = 0.004) increase in the selection of transformational leadership. Students tended to choose individuals whose acts of leadership could be seen and applied. Medical leadership education should account for students' changing perceptions and present a feasible leadership model, introducing specific examples to illustrate these leadership skills.