Entrustable professional activities

Identifying emergency medicine program directors' expectations of competence upon entry into residency: Bridging the distance from the Association of American Medical Colleges Core Entrustable Professional Activities

Background

Residency program directors (PDs) frequently describe students as unprepared for the patient care responsibilities expected of them upon entry into residency. The Association of American Medical Colleges (AAMC) developed the Core Entrustable Professional Activities (Core EPAs) to address this concern by defining 13 tasks students should be able to do with minimal supervision upon graduation. However, the Core EPAs remain difficult for PDs to use due to their breadth and lack of granularity.

Methods

Using Delphi consensus methodology, we identified granular observable practice activities (OPAs) that PDs expect of entering interns derived from the Core EPAs. Twelve emergency medicine education experts drafted OPAs based on the Core EPAs and their associated core functions described in one-page schematics. A separate group of 12 PDs underwent three rounds of voting, and consensus for inclusion was set at 70%. Thematic analysis of comments discussing votes was performed using an inductive approach.

Results

A total of 321 OPAs were drafted and 127 adopted as expectations for entering interns based on the Core EPAs. The adopted OPAs were all general expectations; none were specialty-specific. Four main themes were identified from the comments: Schools are not responsible for specialty-specific training, PDs do not fully trust schools' assessments, supervision expectations of graduates should be lowered for higher-order EPAs, and the context in which the student performs a task and its associated complexity matter greatly in entrustment decisions.

Conclusions

PDs agree with the generalist focus of the AAMC Core EPAs and feel strongly that specialty training should be left to residency programs. They also have mechanisms in place to verify entrustment within their local context. Transparency in assessment and summative entrustment processes in UME may unify expectations. Finally, the granularity of OPAs may aid in a post-Match handover to further operationalize the EPAs and optimize the UME-to-GME transition.

Neurology Undergraduate Medical Education: A Scoping Review

PURPOSE

To map the current literature on undergraduate neurology medical education and research. Recommendations for future undergraduate neurology education and research are described.

METHOD

PRISMA-Scoping Review guidelines and Arksey and O'Malley's methodological framework are followed. Four databases and gray literature was searched with Oxford Evidence-Based Medicine level of evidence applied. A thematic framework was used to identify the main study outcomes. A narrative description and quantitative frequency analysis were used for results.

RESULTS

Nine-hundred and twenty-two articles were retrieved, 102 studies met the inclusion criteria. We identified four main study outcomes using a thematic framework. Our review found that (1) the main undergraduate neurology teaching styles are didactic and experiential teaching methods. (2) Research design of undergraduate neurology teaching is heterogenous. (3) The outcome measures most frequently used in undergraduate neurology research are student perception and knowledge.

CONCLUSION

Undergraduate neurology education research is challenging due to the heterogeneity in research design and teaching methodology. Evidence-based guidelines are limited. This gap in the literature represents an opportunity to develop tailored guidelines for undergraduate neurology education and research.

Comparison of the effects of different teaching methods on the effectiveness of teaching neurology in China: a bayesian network meta-analysis and systematic review

BACKGROUND

In China, investigations into the efficacy of neurological clinical teaching try to ascertain the impacts of various teaching methods on intervention outcomes. However, these studies often suffer from limited sample sizes, single-center studies and low quality, compounded by the lack of direct comparative analyses between teaching methods, thereby leaving the identification of the most effective method unresolved. This study aims to compare the effectiveness of various teaching methods in the standardized training of Chinese neurology clinicians to inform an optimal teaching model utilizing a Bayesian network meta-analysis (NMA) approach.

METHODS

A comprehensive computer search was conducted to identify randomized controlled trials (RCTs) assessing the efficacy of 7 teaching methods: problem-based learning (PBL), case-based learning (CBL), flipped classroom model (FCM), evidence-based medicine (EBM), clinical practice (CP), team-based learning (TBL), and lecture-based learning (LBL). The search, which spanned databases including the China National Knowledge Infrastructure (CNKI), Chinese Citation Database (CDD), China Science Periodical Database (CSPD), Chinese BioMedical Literature Database (CBM), PubMed, Web of Science, and the Cochrane Library, covered the period from the inception of these databases to April 1, 2023. The quality of the included studies was evaluated, and the data were analyzed in R 4.3.2 and Stata 17.0 software.

RESULTS

From the 31 studies included, comprising 2124 subjects, significant findings emerged. In theoretical examinations, a statistically significant difference was noted among the teaching methods, with CBL, PBL, TBL, FCM, and EBM showing superior performance over the LBL method. The effectiveness ranking of these methods was as follows: CBL > PBL > TBL > FCM > EBM > CP > LBL. In terms of practical skills examinations, a similar pattern of effectiveness was observed. Here, the order of effectiveness was CBL > EBM > PBL > TBL > FCM > CP > LBL.

CONCLUSIONS

This NMA indicated that the modern teaching pedagogies, particularly CBL, could be effective in neurology education, and might help improve the theoretical examinations and practical skills of neurology clinicians. Fully tapping into the strengths of modern teaching methods in neurology teaching will require additional work and advancing research.

Improving Electroencephalography Screening via an Online Module for Neurology Trainees: A Multicenter Study

PURPOSE

To investigate the utility of a 15-minute online module to improve the self-confidence and knowledge of neurology trainees when screening an EEG.

METHODS

We developed a fast, convenient, and accessible 15-minute online module to teach basic concepts of EEG screening using a five-step approach. To assess the efficacy of the module among neurology trainees, three surveys were developed. The EEG module and surveys were distributed to neurology trainees at multiple institutions in the United States. Associations between continuous variables were evaluated using t-test and ANOVA test.

RESULTS

A total of 199 residents from 7 different adult neurology residency programs participated in the study. We obtained a response rate of 38% (76/199) for EEG survey 1 and 25% (49/199) for the demographic survey, among junior and senior residents; 22 senior residents completed EEG survey 2. The online EEG module improved confidence when interpreting an EEG among junior residents (1.67 vs. 2.56, p < 0.0001). Senior residents improved their EEG survey score after completing the online module (53.9 vs. 68.6%, p < 0.001). Most of the trainees would recommend the online module to other trainees (93.9%) and would consider including it in an introductory course (93.9%).

CONCLUSIONS

A brief and accessible online EEG module was easily implemented as an early introduction to EEG screening for junior neurology trainees and to improve EEG screening skills for senior trainees. These findings clarify specific areas where EEG learning may be optimized and reinforce the importance of implementing innovative curricula that are accessible and efficient for all neurology trainees.

A Modified Delphi Consensus Approach to Define Entrustable Professional Activities for Neurocritical Care Advanced Practice Providers

OBJECTIVES

To define consensus entrustable professional activities (EPAs) for neurocritical care (NCC) advanced practice providers (APPs), establish validity evidence for the EPAs, and evaluate factors that inform entrustment expectations of NCC APP supervisors.

DESIGN

A three-round modified Delphi consensus process followed by application of the EQual rubric and assessment of generalizability by clinicians not affiliated with academic medical centers.

SETTING

Electronic surveys.

SUBJECTS

NCC APPs ( n = 18) and physicians ( n = 12) in the United States with experience in education scholarship or APP program leadership.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

The steering committee generated an initial list of 61 possible EPAs. The panel proposed 30 additional EPAs. A total of 47 unique nested EPAs were retained by consensus opinion. The steering committee defined six core EPAs addressing medical knowledge, procedural competencies, and communication proficiency which encompassed the nested EPAs. All core EPAs were retained and subsequently met the previously described cut score for quality and structure using the EQual rubric. Most clinicians who were not affiliated with academic medical centers rated each of the six core EPAs as very important or mandatory. Entrustment expectations did not vary by prespecified groups.

CONCLUSIONS

Expert consensus was used to create EPAs for NCC APPs that reached a predefined quality standard and were important to most clinicians in different practice settings. We did not identify variables that significantly predicted entrustment expectations. These EPAs may aid in curricular design for an EPA-based assessment of new NCC APPs and may inform the development of EPAs for APPs in other critical care subspecialties.

Building a pediatric neurocritical care program: The role of the clinical pharmacist practitioner on clinical practice and education. A curriculum for neuropharmacology training

Clinical pharmacists are a part of the integrated health care team and provide valuable input on medication management for patients with acute and chronic disease states. Using epilepsy as a model, pharmacist involvement in patient care has been associated with significant reductions in monthly seizure frequency. Given differences in etiology, pediatric patients with epilepsy are likely to have higher number of treatments, with additional pharmacodynamic and pharmacokinetic differences, adding to the importance of utilizing a pediatric clinical pharmacist practitioner with neuropharmacology expertise. There is an increasing exposure to critically ill patients with epilepsy and other neurological disorders in the pediatric intensive care unit (PICU). These patients are more medically complex, increasing the risk for medication errors and increased health care costs. Emphasis on neurocritical care education is a vital component to improving patient outcomes. Inclusion of a clinical pharmacist practitioner in these settings yields a positive impact on major health outcomes. In 2018, the Neurocritical Care Society developed consensus recommendations on the standards for the development of adult neurocritical care units. A pharmacist-delivered pediatric critical care neuropharmacology rotation represents a novel approach to expanding physician education to improve patient outcomes. While there are sparse publications highlighting the importance of adult critical care and NCC pharmacists, no such literature exists describing the benefits of pediatric neurocritical care (PNCC) pharmacists. To the best of our knowledge, this is the first manuscript describing the role of clinical pharmacist practitioners in the development of PNCC program and the benefits they provide to patient care and education.

Study on a job competence evaluation system for resident physicians (including integrated postgraduates) receiving standardized training

BACKGROUND

Standardized training for resident physicians is the primary form of postgraduate medical education, and it plays a pivotal role in healthcare safety and industry stability. Currently, it has garnered significant attention from healthcare institutions.

METHODS

By conducting a comprehensive literature review and a Delphi consultation in June 2022 for which 40 experts in clinical medicine, public health, and other related fields in China were invited. The indicators were adjusted according to the results of the consultation, and the final indicator weights were determined through an analytic hierarchy process.

RESULTS

The response rate was 100%, and the expert authority coefficient was 0.879. The consistency among the experts on the tertiary indicators, as measured by Kendall's W, was 0.675 (χ2 = 42.516, p < 0.001). Based on the results of the expert consultation, a job competence evaluation system for resident physicians (including integrated postgraduates) receiving standardized training was established, which included 6 primary indicators, 18 secondary indicators, and 116 tertiary indicators. The weights for the primary indicators, namely professional quality, skills and knowledge, patient care, communication and collaboration, teaching skills, and lifelong learning, were 0.313, 0.248, 0.181, 0.083, 0.066, and 0.110, respectively. The top three secondary indicators in terms of combined weights were clinical skills (0.122), professional ethics (0.120), and professional dedication (0.109). The three tertiary indicators with the highest scores were "maintains collaboration with peers and colleagues in patient treatment," "has clinical thinking skills, makes diagnosis and treatment decisions based on analysis of evidence, and has the ability to administer suitable treatments," and "abides by laws and discipline and refuses to seek personal gains in medical practice"; their combined weights were 0.035, 0.028, and 0.027, respectively.

CONCLUSION

This study has established a concrete, objective, and quantifiable competency assessment index system for standardized training of clinical resident physicians (including integrated postgraduates). This system provides a foundation for the quantitative evaluation of the competency of clinical resident physicians (including integrated postgraduates) undergoing standardized training.

Aligning Medical Student Workplace-Based Assessments with Entrustable Professional Activities and the RIME Model in a Psychiatry Clerkship

OBJECTIVE

The authors piloted use of workplace-based assessments of students during the psychiatry clerkship utilizing both entrustable professional activities (EPAs) and the reporter, interpreter, manager, and educator (RIME) model.

METHODS

After supervising clinicians conducted assessments of medical students (N=109) during the psychiatry clerkship using a supervisory scale aligned with both EPA and RIME models, each student received individualized formative feedback. Students were then surveyed on the usefulness of this feedback, and participating faculty/residents were surveyed on the ease of completion of the supervisory scale.

RESULTS

Students' mean skill profile suggested they no longer needed direct supervision on EPA1 and EPA6. Mean scores on other studied EPAs suggested students were well on their way toward performing these EPAs without direct supervision. Students had mean RIME scores that exceeded the suggested levels identified for a Reporter to start clerkships, for an Interpreter to start clerkships, and for a Manager to transition to the fourth year. Close to half of the students found the feedback helpful in their development as a clinician but most felt their performance should not be shared with residency program directors, either before or after the Match. Almost all responding preceptors felt the supervisory ratings were easy to complete.

CONCLUSIONS

This pilot RIME/EPA framework served as a successful step toward a more competency-based medical education in the psychiatry clerkship with relatively little additional faculty time commitment by using workplace-based assessments already in place and a supervisory scale based on EPAs and RIME.

Working with entrustable professional activities in clinical education in undergraduate medical education: a scoping review

BACKGROUND

Entrustable professional activities (EPAs) are increasingly used in undergraduate medical education (UME). We conducted a scoping review to summarize the evidence for the use of EPAs in clinical rotations in UME.

METHODS

We searched multiple databases for scoping reviews based on the PRISMA guidelines for articles reporting qualitative and quantitative research, as well as conceptual and curriculum development reports, on EPAs in UME clinical rotations.

RESULTS

We identified 3309 records by searching through multiple databases. After the removal of duplicates, 1858 reports were screened. A total of 36 articles were used for data extraction. Of these, 47% reported on EPA and EPA-based curriculum development for clerkships, 50% reported on implementation strategies, and 53% reported on assessment methods and tools used in clerkships. Validity frameworks for developing EPAs in the context of clerkships were inconsistent. Several specialties reported feasible implementation strategies for EPA-based clerkship curricula, however, these required additional faculty time and resources. Limited exposure to clinical activities was identified as a barrier to relevant learning experiences. Educators used nationally defined, or specialty-specific EPAs, and a range of entrustability and supervision scales. We found only one study that used an empirical research approach for EPA assessment. One article reported on the earlier advancement of trainees from UME to graduate medical education based on summative entrustment decisions.

CONCLUSIONS

There is emerging evidence concerning how EPAs can be effectively introduced to clinical training in UME. Specialty-specific, nested EPAs with context-adapted, entrustment-supervision scales might be helpful in better leveraging their formative assessment potential.

Developing a New Set of ACGME Milestones for Child Neurology Residency

BACKGROUND

The Educational Milestones developed by the Accreditation Council for Graduate Medical Education (ACGME) are a construct used to evaluate the development of core competencies during residency and fellowship training. The milestones were developed to create a framework for professional development during graduate medical education. The first iteration of milestones for the child neurology residency was implemented in 2015. In the years that followed, the ACGME received and reviewed feedback about the milestones and set out to revise them.

METHODS

A committee was assembled to review the original milestones and develop a new set of milestones. The group was also encouraged to not only consider the child neurology residency graduate of today but also the graduate of tomorrow, taking into account growing fields such as genetics and technology.

RESULTS

A diverse group of 12 individuals, including 10 child neurologists (all of whom were current or previous program directors or associate program directors), one child neurology resident, and one non-physician program coordinator, were recruited from programs of varying size across the country.

CONCLUSIONS

The committee developed a revision to the child neurology milestones. All changes made were with a focus on how the milestones can be useful to trainees, program directors, and clinical competency committee members. Implementation and further feedback should help guide future revisions. These changes should help trainees, clinical competency committee members, and program directors find more meaning from their use.

A novel longitudinal framework aimed at improving the teaching of the neurologic examination

Objective

To develop an educational framework basis for improving the teaching of the neurologic examination (NE) by asking German neurologists to (1) identify the basic elements of the screening NE and (2) nominate the steps they would deem mandatory for medical students to master.

Methods

We conducted a questionnaire-based survey among neurologists working in a hospital or ambulatory setting in southwest Germany. To define the screening NE, neurologists were asked to list the NE components they normally use in clinical encounters with patients in whom neurologic findings are unlikely. Furthermore, they were asked to identify additional elements of the NE which they would consider mandatory for students to master.

Results

Our neurologists nominated a set of 23 elements as being essential for a screening NE. There was high consensus among the 2 groups, and the results were concordant with international data. Furthermore, nearly 60 additional maneuvers of the NE were deemed obligatory for students to master.

Conclusion

Our results reinforce the international consensus for screening NE components and confirm a large set of additional examination steps that medical students should master, thereby indicating the need for an educational NE teaching concept. To solve this educational challenge, we propose a longitudinal curriculum that incorporates the “core + clusters” framework, thus combining the screening NE (core) with hypothesis-driven sets of maneuvers (clusters). Based on our data, we provide an initial proposal for the core and neurologic diagnostic clusters which is applicable to both novice and advanced learners across the continuum of training.

Entrustable professional activities in health care education: a scoping review

CONTEXT

The shift in medical education from time-based learning to outcome-based learning has drawn much attention to entrustable professional activities (EPAs) as an ideal assessment framework to translate competencies into clinical practice. Given the relative novelty of EPAs, this review aims to highlight research gaps and explore and consolidate available evidence pertaining to the development and implementation of EPAs in health care.

METHOD

Arksey and O'Malley's scoping review framework was used to present the findings. The authors performed a systematic search of PubMed, Embase, CINAHL, Scopus, MedNar, OpenGrey and ProQuest Dissertation and Theses for English articles published from the inception of each database to May 2018. A manual search of the reference lists of the included studies was conducted and an expert panel was consulted. Two reviewers screened the articles for eligibility using the inclusion criteria. All authors extracted key data and analysed the data descriptively. Thematic analysis was used to categorise the results into themes.

RESULTS

Eighty articles were included in the review. All articles were published between 2010 and 2018. Three major themes and eight sub-themes were generated: (i) development of EPAs (frameworks for EPA development and implementation, identifying core or specialty-specific EPAs, and EPAs for faculty development), (ii) evaluation of EPAs and EPA entrustment factors (revised curriculum, entrustment decisions for professional activities, and feedback on implemented EPAs and the development process), and (iii) future directions and recommendations for EPAs (implementation of EPAs in undergraduate medical education and specific disciplines, and other criticisms and recommendations for EPAs).

CONCLUSIONS

Entrustable professional activities are an essential means to translate competencies into observable and measurable clinical practice. However, high-level evidence-based research on the efficacy, development and implementation of EPAs for specific target groups (i.e. undergraduates and staff) and geographical regions (i.e. Asia and Africa) is still lacking, which suggests a direction for future research.