Longitudinal Milestone Assessment Extending Through Subspecialty Training: The Relationship Between ACGME Internal Medicine Residency Milestones and Subsequent Pulmonary and Critical Care Fellowship Milestones

Are Final Residency Milestones Correlated With Early Fellowship Performance in Pediatrics?

PURPOSE

Milestones have been used to assess trainees across graduate medical education programs and reflect a developmental continuum from novice to expert. This study examined whether residency milestones are correlated with initial fellowship milestone performance in pediatrics.

METHOD

This retrospective cohort study used descriptive statistics to assess milestone scores from pediatric fellows who began fellowship training between July 2017 and July 2020. Milestone scores were obtained at the end of residency (R), middle of the first fellowship year (F1), and end of the first fellowship year (F2).

RESULTS

Data represent 3,592 unique trainees. High composite R scores, much lower F1 scores, and slightly higher F2 scores were found over time for all pediatric subspecialities. R scores were positively correlated with F1 scores (Spearman ρ = 0.12, P < .001) and F2 scores (Spearman ρ = 0.15, P < .001). Although scores are negligibly different when trainees graduate from residency, there were differences in F1 and F2 scores among fellows in different specialties. Those who trained at the same institution for residency and fellowship had higher composite milestone F1 and F2 scores compared with those who trained at different institutions ( P < .001). The strongest associations were between R and F2 scores for the professionalism and communication milestones, although associations were still relatively weak overall (r s = 0.13-0.20).

CONCLUSIONS

This study found high R scores and low F1 and F2 scores across all shared milestones, with weak association of scores within competencies, indicating that milestones are context dependent. Although professionalism and communication milestones had a higher correlation compared with the other competencies, the association was still weak. Residency milestones may be useful for individualized education in early fellowship, but fellowship programs should be cautious about overreliance on R scores due to the weak correlation with F1 and F2 scores.

Best Practices for Remediation in Pulmonary and Critical Care Medicine Fellowship Training

Background

Remediation of struggling learners in pulmonary and critical care fellowship programs is a challenge, even for experienced medical educators.

Objective

This evidence-based narrative review provides a framework program leaders may use to address fellows having difficulty achieving competency during fellowship training.

Methods

The relevant evidence for approaches on the basis of each learner’s needs is reviewed and interpreted in the context of fellowship training in pulmonary medicine and critical care. Issues addressed include bias in fellow assessments and remediation, the impacts of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic, the specific challenges of pulmonary and critical care fellowship programs, a brief review of relevant legal issues, guidance on building and leveraging program resources, and a discussion of learner outcomes.

Results

This results in a concise, evidence-based toolkit for program leaders based around four pillars: early identification, fellow assessment, collaborative intervention, and reassessment. Important concepts also include the need for documentation, clear and written communication, and fellow-directed approaches to the creation of achievable goals.

Conclusion

Evidence-based remediation helps struggling learners in pulmonary and critical care fellowship to improve their ability to meet Accreditation Council for Graduate Medical Education (ACGME) milestones.

The education passport: connecting programmatic assessment across learning and practice

Competency-based medical education (CBME) shifts us from static assessment of learning to developmental assessment for learning. However, implementation challenges associated with CBME remain a major hurdle, especially after training and into practice. The full benefit of developmental assessment for learning over time requires collaboration, cooperation, and trust among learners, regulators, and the public that transcends each individual phase. The authors introduce the concept of an "Education Passport" that provides evidence of readiness to travel across the boundaries between undergraduate medical education, graduate medical education, and the expanse of practice. The Education Passport uses programmatic assessment, a process of collecting numerous low stakes assessments from multiple sources over time, judging these data using criterion-referencing, and enhancing this with coaching and competency committees to understand, process, and accelerate growth without end. Information in the Passport is housed on a cloud-based server controlled by the student/physician over the course of training and practice. These data are mapped to various educational frameworks such Entrustable Professional Activities or milestones for ease of longitudinal performance tracking. At each stage of education and practice the student/physician grants Passport access to all entities that can provide data on performance. Database managers use learning analytics to connect and display information over time that are then used by the student/physician, their assigned or chosen coaches, and review committees to maintain or improve performance. Global information is also collected and analyzed to improve the entire system of learning and care. Developing a true continuum that embraces performance and growth will be a long-term adaptive challenge across many organizations and jurisdictions and will require coordination from regulatory and national agencies. An Education Passport could also serve as an organizing tool and will require research and high-value communication strategies to maximize public trust in the work.

Longitudinal assessment of milestone development among internal medicine residents in Taiwan

BACKGROUND/PURPOSE

The Accreditation Council for Graduate Medical Education (ACGME) milestones have been implemented in residency training worldwide. We investigated the development of individual competency in first-year residents (R1) and second-year postgraduate students (PGY2) who received internal medicine training in Taiwan.

METHODS

A multicenter observational cohort study was conducted to evaluate the competency-based milestone evaluation designed by the Taiwan Society of Internal Medicine in 2019. The evaluation was based on the ACGME-accredited milestone ratings. Periodic evaluation of milestone achievements of R1 and PGY2, who entered the internal medicine residency training at six medical centers, was performed. Each resident was evaluated every 3 months.

RESULTS

Among the 98 R1 enrolled in 2019, substantial improvement in sub-competencies, including skill in performing procedures (Patient Care 4), clinical knowledge (Medical Knowledge 1), knowledge of diagnostic testing and procedures (Medical Knowledge 2), and identify impact the cost of health care and practices cost-effective care (Systems Based Practice 3) during the two years of training. Among the 107 R1 and 46 PGY2 enrolled in 2020, no significant difference in baseline milestone ratings was observed. However, the milestone assessments of R1 in 2020 showed improvement in nearly all sub-competencies compared with the stationary status of PGY2 in 2020.

CONCLUSION

We demonstrate the application of ACGME-based accredited milestone ratings to target the educational goals of internal medicine residency training in Taiwan. Differences in milestone ratings between different PGY training systems exist. The long-term impact of performance among different PGY training systems requires further investigation.