Achieving Entrustable Professional Activities During Fellowship

Entrustable professional activities for pediatric and subspecialties residency training in China

BACKGROUND AND OBJECTIVES

Outcome-based medical education is the latest focus in the past decade, and Entrustable Professional Activities(EPAs) have emerged as efficient vehicles to assess physicians. However, few studies have discussed the use of EPAs for residency training in pediatric medicine and its subspecialties. We conducted a pilot study to examine the feasibility of EPAs as a component of the clinical program of assessment in pediatric standardized residency training.

METHODS

We conducted a cross-sectional study for standardized residency training in different subspecialties within pediatric medicine at Qilu Hospital of Shandong University. Totally 65 residents and 35 directors joined in this study. An electronic EPA survey using 8 scales composed of 15 categories was distributed among residents and directors. Kruskal-Wallis test and Mann-Whitney U test were applied for comparing the self-assessments and director-assessments. Generalized estimated equation (GEE) was used to analyze the effect of postgraduate year(PGY), gender, and position on the EPA scores of director assessments.

RESULTS

A total of 401 director-assessment and 65 residents' self-assessment ( response rate 100%) questionnaires were collected, both demonstrating rising trends in scores across PGYs. Significant differences were found between PGY1 and PGY2 (p < 0.01) and between PGY1 and PGY3 (p < 0.01), but not between PGY2 and PGY3 (p > 0.01). With an effect analysis of PGY, gender, and position on EPA scores performed, PGY had a significant effect on 13 out of 15 EPA scores, while gender affected only four EPA scores significantly, and position affected only three EPA categories. Meanwhile, some EPA categories revealed significant differences across various pediatric subspecialties (p < 0.01).

CONCLUSIONS

The study findings suggest that EPA assessments is feasible among different PGYs in standardized Chinese residency training in pediatric medicine and its subspecialties. Postgraduate year had a significant impact on EPA scores, while gender and resident position also affected EPA scores to a certain extent. Improved stratified teaching programs are required for better subspecialty consistency.

Many Pediatric Subspecialty Fellows Are Not Ready to Graduate From Fellowship in 2 Years

BACKGROUND AND OBJECTIVES

The American Board of Pediatrics requires that proposed changes to the duration of pediatric subspecialty training must include a framework for competency assessment with a measurement component. We analyzed the clinical Entrustable Professional Activity (EPA) level of supervision ratings across 3-year pediatric fellowships to determine if trainees met the minimum thresholds for graduation after 2 years of fellowship training.

METHODS

From spring 2019 through spring 2022, Clinical Competency Committees (CCCs) reported fellow supervision level ratings for all clinical EPAs, fellowship program directors (FPDs) assessed the scholarship EPA supervision level, and fellows self-reported their required level of supervision for all EPAs. Ratings were compared with minimum supervision level thresholds for fellow graduation previously identified by FPDs. We analyzed the proportion of fellows achieving these EPA supervision level thresholds after 2 and 3 years of training.

RESULTS

CCCs reported ratings for 1538 second-year and 1505 third-year fellows. Fewer than 50% of fellows met clinical EPA supervision level thresholds for graduation after 2 years of training, increasing to 86%-100% across subspecialties at 3 years. Fellow self-assessment aligned well with CCC ratings. FPDs reported that 64%-68% of fellows across subspecialties met the scholarship EPA supervision level threshold for graduation after 2 years compared with 99%-100% at 3 years.

CONCLUSIONS

As pediatric fellowships are currently structured and using an EPA assessment framework, many trainees are not ready to graduate after 2 years.

Projecting the Future Pediatric Subspecialty Workforce: Summary and Recommendations

This article summarizes the findings of a Pediatrics supplement addressing the United States workforce for 15 pediatric subspecialties. It includes results from a microsimulation model projecting supply through 2040; growth is forecasted to be uneven across the subspecialties with worsening geographic maldistribution. Although each subspecialty has unique characteristics, commonalities include (1) the changing demographics and healthcare needs of children, including mental health; (2) poor outcomes for children experiencing adverse social drivers of health, including racism; and (3) dependence on other subspecialties. Common healthcare delivery challenges include (1) physician shortages for some subspecialties; (2) misalignment between locations of training programs and subspecialists and areas of projected child population growth; (3) tension between increasing subsubspecialization to address rare diseases and general subspecialty care; (4) the need to expand clinical reach through collaboration with other physicians and advanced practice providers; (5) the lack of parity between Medicare, which funds much of adult care, and Medicaid, which funds over half of pediatric subspecialty care; and (6) low compensation of pediatric subspecialists compared with adult subspecialists. Overall, subspecialists identified the lack of a central authority to monitor and inform child healthcare provided by pediatric subspecialists as a challenge. Future research on the pediatric subspecialty workforce and the children it serves will be necessary to ensure these children's needs are met. Together, these articles provide overarching and subspecialty-specific recommendations to improve training, recruitment, and retention of a diverse workforce, implement innovative models of care, drive policy changes, and advise future research.

Relationship between epa level of supervision with their associated subcompetency milestone levels in pediatric fellow assessment

BACKGROUND

Entrustable Professional Activities (EPA) and competencies represent components of a competency-based education framework. EPAs are assessed based on the level of supervision (LOS) necessary to perform the activity safely and effectively. The broad competencies, broken down into narrower subcompetencies, are assessed using milestones, observable behaviors of one's abilities along a developmental spectrum. Integration of the two methods, accomplished by mapping the most relevant subcompetencies to each EPA, may provide a cross check between the two forms of assessment and uncover those subcompetencies that have the greatest influence on the EPA assessment.

OBJECTIVES

We hypothesized that 1) there would be a strong correlation between EPA LOS ratings with the milestone levels for the subcompetencies mapped to the EPA; 2) some subcompetencies would be more critical in determining entrustment decisions than others, and 3) the correlation would be weaker if the analysis included only milestones reported to the Accreditation Council for Graduate Medical Education (ACGME).

METHODS

In fall 2014 and spring 2015, the Subspecialty Pediatrics Investigator Network asked Clinical Competency Committees to assign milestone levels to each trainee enrolled in a pediatric fellowship for all subcompetencies mapped to 6 Common Pediatric Subspecialty EPAs as well as provide a rating for each EPA based upon a 5-point LOS scale.

RESULTS

One-thousand forty fellows were assessed in fall and 1048 in spring, representing about 27% of all fellows. For each EPA and in both periods, the average milestone level was highly correlated with LOS (rho range 0.59-0.74; p < 0.001). Correlations were similar when using a weighted versus unweighted milestone score or using only the ACGME reported milestones (p > 0.05).

CONCLUSIONS

We found a strong relationship between milestone level and EPA LOS rating but no difference if the subcompetencies were weighted, or if only milestones reported to the ACGME were used. Our results suggest that representative behaviors needed to effectively perform the EPA, such as key subcompetencies and milestones, allow for future language adaptations while still supporting the current model of assessment. In addition, these data provide additional validity evidence for using these complementary tools in building a program of assessment.

Pediatric Emergency Medicine Fellowship Program Directors' Viewpoint: Minimum Levels of Entrustment for Graduating Fellows and Practicing Physicians to Perform the Subspecialty's Professional Activities

OBJECTIVES

Entrustable Professional Activities (EPAs) are essential tasks physicians perform within their professions. Entrustment levels that pediatric emergency medicine (PEM) fellowship program directors (FPDs) expect graduating fellows to achieve for PEM-specific and common pediatric subspecialty EPAs remain unreported. This study aims to determine minimum entrustment levels FPDs require fellows to achieve to graduate from fellowship and to compare FPD expectations for fellows versus practicing PEM physicians.

METHODS

Secondary analysis of PEM-specific data from a national multispecialty cross-sectional survey of pediatric subspecialty FPDs. For 6 PEM-specific and 7 common pediatric subspecialty EPAs, PEM FPDs indicated (1) minimum entrustment levels fellows should achieve by training completion, (2) whether they would allow a fellow to graduate below these minimum levels, and (3) minimum levels for safe and effective practice by PEM physicians. Minimum levels were defined as the level that more than 80% of FPDs would not drop below.

RESULTS

Sixty of 77 PEM FPDs (78%) completed the survey. Most respondents did not require fellows to achieve the highest level (level 5-no supervision) by graduation for any PEM-specific EPAs. The median level FPDs expected for practicing PEM physicians was 5 (trusted to perform without supervision) for EPAs 1 and 4 and level 4 (indirect supervision for complex cases) for the remaining PEM-specific EPAs. Minimum levels expected by FPDs for common subspecialty pediatric EPAs were lower for both groups.

CONCLUSIONS

Most PEM FPDs indicated that they would graduate fellows before their achievement of the highest entrustment level for all EPAs. Most also indicated that they do not expect practicing PEM physicians to perform all EPAs without supervision. These findings indicate need for stakeholders to evaluate current structure and outcomes of PEM fellowship programs and for institutions and organizations to ensure adequate support in time and resources for ongoing learning for practicing PEM physicians.

Clinical competency committee perceptions of entrustable professional activities and their value in assessing fellows: A qualitative study of pediatric subspecialty program directors

OBJECTIVES

To examine the composition and processes of Clinical Competency Committees (CCCs) assigning entrustable professional activity (EPA) levels of supervision for pediatric subspecialty fellows and to examine fellowship program director (FPD) perspectives about using EPAs to determine fellows' graduation readiness.

METHODS

A qualitative study was performed using one-on-one interviews with a purposeful sample of pediatric subspecialty FPDs to yield a thematic analysis. Semi-structured interview guides were used for participants who self-identified as EPA users or non-users. Inductive analysis and coding were performed on transcripts until theoretical sufficiency was attained.

RESULTS

Twenty-eight FPDs were interviewed. There was significant variability in the composition and processes of CCCs across subspecialties. FPDs felt that CCCs intuitively understand what entrustment means, allowing for ease of application of level of supervision (LOS) scales and consensus. FPDs perceived that EPAs provided a global assessment of fellows and are one tool to determine graduation readiness.

CONCLUSIONS

Although there was variability in the makeup and processes of CCCs across subspecialties, FPDs believe EPAs are intuitive and relatively easy to implement. Consensus can be reached easily using EPA-specific LOS scales focusing on entrustment. FPDs desire a better understanding of how EPAs should be used for graduation.

Providers' Attitudes to Proposed Changes in the Critical Congenital Heart Disease Screening Algorithm

Screening for critical congenital heart disease (CCHD) in newborns based on a 2009 Swedish study has become the standard of care despite the complexity of the algorithm. A simplified algorithm which might increase the false-positive rate was proposed in 2020 but not formally endorsed by the American Academy of Pediatrics. We sought to determine how the current protocol and the proposed changes are perceived by clinicians. We performed an anonymous survey of professionals involved in the care of newborns regarding their perception of the current and proposed CCHD screening algorithms. 335 responses were evaluated. Less than 2% of respondents were dissatisfied with the existing screening algorithm or felt it was difficult to perform. 47% endorsed and 12% opposed the adoption of the proposed modifications with those most familiar with the proposed changes more likely to endorse them. Although many providers would accept a higher false-positive rate in CCHD screening, those who would have to transfer a baby from the birth site for assessment after a failed CCHD screening were less tolerant of an increased false-positive rate. Although the existing CCHD screening mechanisms appear to be very well received, the proposed changes to the CCHD algorithm were viewed positively by many respondents. Changes in this algorithm would likely be better tolerated in those setting where the consequences of a failed CCHD screening are more easily addressed.

Multifaceted Approach to Evaluation in a Pediatric and Adolescent Gynecology Rotation for Medical Students

STUDY OBJECTIVE

We evaluated whether and to what extent a novel medical student rotation in pediatric and adolescent gynecology (PAG) increases clinical knowledge and skills and meets student needs and expectations.

DESIGN

Constructivist prospective pre-post study and post-rotation student survey SETTING: Academic medical center PARTICIPANTS: Pilot study of 9 medical students, which represents the entire population of those who completed the rotation.

INTERVENTIONS

Four-week clinical rotation in PAG MAIN OUTCOME MEASURES: Changes in clinical knowledge were measured by a pre- and post-intervention multiple-choice assessment, and clinical skills were assessed before and after the intervention using entrustable professional activities (EPAs); these data were analyzed with paired Student's t tests. Student evaluations of the rotation were measured through an anonymous, end-of-rotation, closed- and open-ended survey and were analyzed using descriptive statistics.

RESULTS

A statistically significant increase in clinical knowledge was observed post-rotation, with a mean pretest score of 67.0% (standard deviation [SD] 1.7%) and a mean posttest score of 75.2% (SD 3.2%, P = 0.02). Statistically significant increases were observed for all EPAs between the first and final day of the rotation. Eight students who completed the post-rotation survey rated the rotation favorably (5 on a scale from 1 to 5).

CONCLUSION

A multipronged evaluation showed that a new PAG clinical rotation significantly increased medical students' clinical skills and knowledge. This multifaceted evaluation method provides valuable insights to educators on how best to tailor a rotation to individual learners' levels of clinical skills and knowledge. If comparable rotations could be instituted and similarly evaluated in other medical schools, a noticeable knowledge/skill gap among trainees might be addressed.