Entrustable professional activities (EPAs) for postgraduate competency based intensive care medicine training in the Netherlands: The next step towards excellence in intensive care medicine training

Fit for Service: Preparing Residents for Neurointensive Care with Entrustable Professional Activities: A Delphi Study

BACKGROUND

Although the relevance of neurointensive medicine and high-quality training of corresponding physicians is increasingly recognized, there is high heterogeneity in the nature, duration, and quality of neurointensive care curricula around the world. Thus, we aimed to identify, define, and establish validity evidence for entrustable professional activities (EPAs) for postgraduate training in neurointensive care to determine trainees' readiness for being on-call.

METHODS

After defining EPAs through an iterative process by an expert group, we used a modified Delphi approach with a single-center development process followed by a national consensus and a single-center validation step. EPAs were evaluated by using the EQual rubric (Queen's EPA Quality Rubric). Interrater reliability was measured with Krippendorff's α.

RESULTS

The expert group defined seven preliminary EPAs for neurointensive care. In two consecutive Delphi rounds, EPAs were adapted, and consensus was reached for level of entrustment and time of expiration. Ultimately, EPAs reached a high EQual score of 4.5 of 5 and above. Interrater reliability for the EQual scoring was 0.8.

CONCLUSIONS

Using a multistep Delphi process, we defined and established validity evidence for seven EPAs for neurointensive medicine with a high degree of consensus to objectively describe readiness for on-call duty in neurointensive care. This operationalization of pivotal clinical tasks may help to better train clinical residents in neurointensive care across sites and health care systems and has the potential to serve as a blueprint for training in general intensive care medicine. It also represents a starting point for further research and development of medical curricula.

Trainee-supervisor collaboration, progress-visualisation, and coaching: a survey on challenges in assessment of ICU trainees

BACKGROUND

Assessing trainees is crucial for development of their competence, yet it remains a challenging endeavour. Identifying contributing and influencing factors affecting this process is imperative for improvement.

METHODS

We surveyed residents, fellows, and intensivists working in an intensive care unit (ICU) at a large non-university hospital in Switzerland to investigate the challenges in assessing ICU trainees. Thematic analysis revealed three major themes.

RESULTS

Among 45 physicians, 37(82%) responded. The first theme identified is trainee-intensivist collaboration discontinuity. The limited duration of trainees' ICU rotations, large team size operating in a discordant three-shift system, and busy and unpredictable day-planning hinder sustained collaboration. Potential solutions include a concise pre-collaboration briefing, shared bedside care, and post-collaboration debriefing involving formative assessment and reflection on collaboration. The second theme is the lack of trainees' progress visualisation, which is caused by unsatisfactory familiarisation with the trainees' development. The lack of an overview of a trainee's previous achievements, activities, strengths, weaknesses, and goals may result in inappropriate assessments. Participants suggested implementing digital assessment tools, a competence committee, and dashboards to facilitate progress visualisation. The third theme we identified is insufficient coaching and feedback. Factors like personality traits, hierarchy, and competing interests can impede coaching, while high-quality feedback is essential for correct assessment. Skilled coaches can define short-term goals and may optimise trainee assessment by seeking feedback from multiple supervisors and assisting in both formative and summative assessment. Based on these three themes and the suggested solutions, we developed the acronym "ICU-STAR" representing a potentially powerful framework to enhance short-term trainee-supervisor collaboration in the workplace and to co-scaffold the principles of adequate assessment.

CONCLUSIONS

According to ICU physicians, trainee-supervisor collaboration discontinuity, the lack of visualisation of trainee's development, and insufficient coaching and feedback skills of supervisors are the major factors hampering trainees' assessment in the workplace. Based on suggestions by the survey participants, we propose the acronym "ICU-STAR" as a framework including briefing, shared bedside care, and debriefing of the trainee-supervisor collaboration at the workplace as its core components. With the attending intensivists acting as coaches, progress visualisation can be enhanced by actively collecting more data points.

TRIAL REGISTRATION

N/A.

Evaluating the Impact of Supervision on Surgical Trainees Stress Response During Simulated Surgical Procedures; A Crossover Randomized Trial

OBJECTIVE

The aim of this study was to evaluate the cumulative impact of supervision on technical skills and surrogate stress markers in surgical trainees.

DESIGN

This was a quasi-experimental crossover study to evaluate the impact of attending supervision on orthopedic trainee stress response during a simulated surgical procedure. Enrolled residents performed a proximal femoral nail module with the Precision OS system twice; once independently, and once under direct attending supervision, whilst wearing a heart rate monitor. Mean and maximum heart rates were recorded. Simulated performance was assessed using validated simulator-based metrics. Student's t-test was used to evaluate the impact of supervision on trainee heart rate, and performance ranking.

SETTING

Tertiary trauma center in a Regional Orthopedic Unit PARTICIPANTS: Orthopedic interns and residents within our institution were invited to participate, with 20 participants included for analysis.

RESULTS

Both supervised and unsupervised mean heart rate was significantly higher (p = 0.001) than baseline recorded heart rates. Supervised mean and maximum HR were significantly higher than unsupervised HR during module completion (p = 0.015; p = 0.001). Calories burned demonstrated correlation to surrogate stress markers, significantly higher in supervised sessions (p = 0.004). Performance metrics demonstrated superior performance in senior-level participants, with a decrement in performance during supervision, failing to reach significance.

CONCLUSION

The development of accretion of technical and non-technical skills required in surgical training pathways may derive benefit from the use of simulation-based training in surgical residents with both supervised and unsupervised sessions.

The logic behind entrustable professional activity frameworks: A scoping review of the literature

INTRODUCTION

Entrustable professional activities (EPAs), discrete profession-specific tasks requiring integration of multiple competencies, are increasingly used to help define and inform curricula of specialty training programmes. Although guidelines exist to help guide the developmental process, deciding what logic to use to draft a preliminary EPA framework poses a crucial but often difficult first step. The logic of an EPA framework can be defined as the perspective used by its developers to break down the practice of a profession into units of professional work. This study aimed to map dominant logics and their rationales across postgraduate medical education and fellowship programmes.

METHODS

A scoping review using systematic searches within five electronic databases (Medline, Embase, Google Scholar, Scopus and Web of Science) was performed. Dominant logics of included papers were identified using inductive coding and iterative analysis.

RESULTS

In total, 42 studies were included. Most studies were conducted in the United States (n = 22; 52%), Canada (n = 6; 14%) and the Netherlands (n = 4; 10%). Across the reported range of specialties, family medicine (n = 4; 10%), internal medicine (n = 4; 10%), paediatrics (n = 3; 7%) and psychiatry (n = 3; 7%) were the most common. Three dominant logics could be identified, namely, 'service provision', 'procedures' and/or 'disease or patient categories'. The majority of papers (n = 37; 88%) used two or more logics when developing EPA frameworks (median = 3, range = 1-4). Disease or patient groups and service provision were the most common logics used (39% and 37%, respectively).

CONCLUSIONS

Most programmes used a combination of logics when trying to capture the essential tasks of a profession in EPAs. For each of the three dominant logics, the authors arrived at a definition and identified benefits, limitations and examples. These findings may potentially inform best practice guidelines for EPA development.

Key dimensions of innovations in workplace-based assessment for postgraduate medical education: a scoping review

BACKGROUND

Specialist training bodies continue to devise innovative methods of gathering information on trainee workplace performance to meet the requirements of competency-based medical education. We reviewed recent innovations in workplace-based assessment (WBA) tools to identify strengths, weaknesses, and trade-offs inherent in their design and use.

METHODS

In this scoping review, using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines, we systematically searched databases between 2009 and 2019 for WBA tools with novel characteristics not typically seen in traditional WBAs. These included innovations in rating scales, ways of collecting information, technological innovations, ways of triggering WBAs, and approaches to compiling and using information.

RESULTS

We identified 30 innovative WBA tools whose characteristics could be categorised into seven dimensions: frequency of assessment, granularity (unit of performance assessed), coverage of the curriculum, rating method, initiation of the WBA, information use, and incentives. These dimensions had multiple interdependencies and trade-offs, often balancing generating assessment data with available resources. Philosophical stance on assessment also influenced WBA choice, for example prioritising trainee-centred learning (i.e. initiation of WBA and transparency of assessment data), perceptions of assessment and feedback as burdensome or beneficial, and holistic vs reductionist views on assessment of performance.

CONCLUSIONS

Our synthesis of the literature on innovative WBAs provides a framework for categorising tool characteristics across seven dimensions, systematically teasing apart the considerations in design and use of workplace assessments. It also draws attention to the trade-offs inherent in tool design and selection, and enables a more deliberate consideration of the tool characteristics most appropriate to the local context.

A Guide to Performance Evaluation for the Intensivist: Ongoing Professional Practice Evaluation and Focused Professional Practice Evaluation in the ICU

OBJECTIVES

In 2008, The Joint Commission implemented a new standard mandating a detailed evaluation of a provider's performance. The Ongoing Professional Practice Evaluation was designed to provide ongoing performance evaluation as opposed to periodic evaluation. The Focused Professional Practice Evaluation was designed to evaluate the performance of providers new to the medical staff or providers who are requesting new privileges. To date, we are unable to find critical care specific literature on the implementation of Ongoing Professional Practice Evaluation/Focused Professional Practice Evaluation. The purpose of this concise definitive review is to familiarize the reader with The Joint Commission standards and their application to Ongoing Professional Practice Evaluation/Focused Professional Practice Evaluation design and implementation, literature review in the noncritical care setting, and future process optimization and automation.

DATA SOURCES

Studies were identified through MEDLINE search using a variety of search phrases related to Ongoing Professional Practice Evaluation, Focused Professional Practice Evaluation, critical care medicine, healthcare quality, and The Joint Commission. Additional articles were identified through a review of the reference lists of identified articles.

STUDY SELECTION

Original articles, review articles, and systematic reviews were considered.

DATA EXTRACTION

Manuscripts were selected for inclusion based on expert opinion of well-designed or key studies and review articles.

DATA SYNTHESIS

There is limited data for the process of Ongoing Professional Practice Evaluation and Focused Professional Practice Evaluation implementation in critical care medicine. Key recommendations exist from The Joint Commission but leave it up to healthcare institutions to realize these. The process and metrics can be tailored to specific institutions and departments.

CONCLUSIONS

Currently, there is no standard process to develop Ongoing Professional Practice Evaluation and Focused Professional Practice Evaluation processes in critical care medicine. Departments and institutions can tailor metrics and processes but it might be useful to standardize some metrics to assure the overall quality of care. In the future utilization of newer technologies like applications might make this process less time-intensive.

Development of entrustable professional activities for paediatric intensive care fellows: A national modified Delphi study

Entrustable professional activities (EPAs), as a focus of learner assessment, are supported by validity evidence. An EPA is a unit of professional practice requiring proficiency in multiple competencies simultaneously, that can be entrusted to a sufficiently competent learner. Taken collectively, a set of EPAs define and inform the curriculum of a specialty training. The goal of this study was to develop a set of EPAs for Dutch PICU fellows. A multistage methodology was employed incorporating sequential input from task force members, a medical education expert, PICU fellowship program directors, and PICU physicians and fellows via a modified three-round Delphi study. In the first modified Delphi round, experts rated indispensability and clarity of preliminary EPAs. In the subsequent rounds, aggregated scores for each EPA and group comments were provided. In round two, respondents rated indispensability and clarity of revised EPAs. Round three was used to gain explicit confirmation of suitability to implement these EPAs. Based on median ratings and content validity index (CVI) analysis for indispensability in the first two rounds, all nine preliminary EPAs covered activities that were deemed essential to the clinical practice of PICU physicians. Based on median ratings and CVI analysis for clarity however, four EPAs needed revision. With an agreement percentage of 93–100% for all individual EPAs as well as the set as a whole, a high degree of consensus among experts was reached in the third round. The resulting nine PICU EPAs provide a succinct overview of the core tasks of Dutch PICU physicians. These EPAs were created as an essential first step towards developing an assessment system for PICU fellows, grounded in core professional activities. The robust methodology used, may have broad applicability for other (sub)specialty training programs aiming to develop specialty specific EPAs.