The End of Step 2 CS Should Be the Beginning of a New Approach to Clinical Skills Assessment

Integrating Formative and Summative Clinical Skills Examinations to Promote Learning for Early Medical Students: A Mixed Methods Study

Background

Medical educators aim to train students with high-quality clinical skills through the promotion of self-regulated learning and the development of lifelong learning skills. Formative clinical skills examinations (FCSE) allow for real-time learner-centered feedback and coaching which are key in promoting the development of expertise in early learners. This study assessed the impact of the integration of FCSE with learner-centered, real-time feedback and coaching based on an "educational plan-do-study-act" (PDSA) cycle on early medical students' experience and performance.

Methods

A mixed methods study was designed to assess the integration of FCSE for first- and second-year medical students. FCSE consisted of linked stations: students gathered a history from a standardized patient (SP), performed a hypothesis-driven physical examination followed by real-time learner-centered feedback and coaching. Each student met with one faculty and one SP to reflect on their performance, identify areas for improvement, re-practice of skills, and identify a plan for ongoing practice improvement. Students were surveyed upon completion of formative and summative clinical skills examinations. Student communication and clinical reasoning performance were compared to historical controls.

Results

Students reported that FSCE improved the learning environment and helped prepare them for subsequent summative clinical skills examinations. Students appreciated the opportunity for practice and real-time feedback and reported applying their take-home points on subsequent exams. Student longitudinal performance was not impacted by the transition to FSCE.

Conclusion

While labor-intensive, FCSE with real-time feedback and coaching are an effective means of promoting learning and should be considered for integration early in medical school curricula.

The effect of specific learning difficulties on general practice written and clinical assessments

BACKGROUND

Substantial numbers of medical students and doctors have specific learning difficulties (SpLDs) and failure to accommodate their needs can disadvantage them academically. Evidence about how SpLDs affect performance during postgraduate general practice (GP) specialty training across the different licencing assessments is lacking. We aimed to investigate the performance of doctors with SpLDs across the range of licencing assessments.

METHODS

We adopted the social model of disability as a conceptual framework arguing that problems of disability are societal and that barriers that restrict life choices for people with disabilities need to be addressed. We used a longitudinal design linking Multi-Specialty Assessment (MSRA) records from 2016 and 2017 with their Applied Knowledge Test (AKT), Clinical Skills Assessment (CSA), Recorded Consultation Assessment (RCA) and Workplace Based Assessment (WPBA) outcomes up to 2021. Multivariable logistic regression models accounting for prior attainment and demographics were used to determine the SpLD doctors' likelihood of passing licencing assessments.

RESULTS

The sample included 2070 doctors, with 214 (10.34%) declaring a SpLD. Candidates declaring a SpLD were significantly less likely to pass the CSA (OR 0.43, 95% CI 0.26, 0.71, p = 0.001) but not the AKT (OR 0.96, 95% CI 0.44, 2.09, p = 0.913) or RCA (OR 0.81, 95% CI 0.35, 1.85, p = 0.615). Importantly, they were significantly more likely to have difficulties with WPBA (OR 0.28, 95% CI 0.20, 0.40, p < 0.001). When looking at licencing tests subdomains, doctors with SpLD performed significantly less well on the CSA Interpersonal Skills (B = -0.70, 95% CI -1.2, -0.19, p = 0.007) and the RCA Clinical Management Skills (B = -1.68, 95% CI -3.24, -0.13, p = 0.034).

CONCLUSIONS

Candidates with SpLDs encounter difficulties in multiple domains of the licencing tests and during their training. More adjustments tailored to their needs should be put in place for the applied clinical skills tests and during their training.

After the Discontinuation of Step 2 CS: A Collaborative Statement from the Directors of Clinical Skills Education (DOCS)

Issue: The United States Medical Licensing Exam (USMLE) Step 2 Clinical Skills Examination (Step 2 CS), the only clinical skills competency testing required for licensure in the United States, has been discontinued. Evidence: This exam, though controversial, propelled a movement emphasizing the value of clinical skills instruction and assessment in undergraduate medical education. While disappointed by the loss of this national driver that facilitated standardization of clinical skills education, the Directors of Clinical Skills Education (DOCS) see prospects for educational innovation and growth. DOCS is a national organization and inclusive community of clinical skills education leaders. This statement from DOCS regarding the discontinuation of USMLE Step 2 CS has been informed by DOCS meetings, listserv discussions, an internal survey, and a review of recent literature. Implications: Rigorous clinical skills assessment remains central to effective and patient-centered healthcare. DOCS shares specific concerns as well as potential solutions. Now free from the external pressure to prepare students for success on Step 2 CS, clinical skills educators can reprioritize content and restructure clinical skills programs to best meet the needs of learners and the ever-evolving healthcare landscape. DOCS, as an organization of clinical skills leaders, makes the following recommendations: 1) Collaboration amongst institutions must be prioritized; clinical skills assessment consortia should be expanded. 2) Governing, accrediting, and licensing organizations should leverage their influence to support and require high quality clinical skills assessments. 3) UME clinical skills leaders should develop ways to identify students who perform with exceptional, borderline, and poor clinical skills at their local institutions. 4) UME leadership should fully commit resources and curricular time to graduate students with excellent clinical skills.

Quality assurance in health professions education: Role of accreditation and licensure

OBJECTIVE

The aim of this paper is to provide an overview of the major quality assurance strategies, accreditation and licensure, in health professions education. It explores the nature of these regulatory processes using Brazil and the United States as examples because these large systems are at different ends of the developmental continuum. For each, it describes the tensions that arise, offers a critical synthesis of the evidence and maps out future directions.

RESULTS

Given wide variability among operating medical schools in curricular design, length of study, resources and facilities for clinical training and supervision, the nature of regulatory bodies varies considerably. Nonetheless, they share tensions related purpose and process including quality assurance versus quality improvement, outcomes versus process and continuous versus episodic evaluations and assessments. Clear evidence of effectiveness, especially for accreditation, is scarce and difficult to obtain, particularly as it relates to health outcomes.

CONCLUSIONS

Regulatory processes need to be built around clear definitions of the goals for each stage of professional development, the current movement towards competency-based education and the variable durations of medical education. These changes must motivate revisions in the content and process of programmes for accreditation and licensure, complimentary efforts towards quality of care, and stimulate a significant research effort.

The Florida Clinical Skills Collaborative: A New Regional Consortium for the Assessment of Clinical Skills

Discontinuation of the United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills (CS) exam and Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 2 Performance Evaluation (2-PE) raised questions about the ability of medical schools to ensure the clinical skills competence of graduating students. In February 2021, representatives from all Florida, United States, allopathic and osteopathic schools initiated a collaboration to address this critically important issue in the evolving landscape of medical education. A 5-point Likert scale survey of all members (n=18/20 individuals representing 10/10 institutions) reveals that initial interest in joining the collaboration was high among both individuals (mean 4.78, SD 0.43) and institutions (mean 4.69, SD 0.48). Most individuals (mean 4.78, SD 0.55) and institutions (mean 4.53, SD 0.72) are highly satisfied with their decision to join. Members most commonly cited a "desire to establish a shared assessment in place of Step 2 CS/2-PE" as their most important reason for joining. Experienced benefits of membership were ranked as the following: 1) Networking, 2) Shared resources for curriculum implementation, 3) Scholarship, and 4) Work towards a shared assessment in place of Step 2 CS/2-PE. Challenges of membership were ranked as the following: 1) Logistics such as scheduling and technology, 2) Agreement on common goals, 3) Total time commitment, and 4) Large group size. Members cited the "administration of a joint assessment pilot" as the highest priority for the coming year. Florida has successfully launched a regional consortium for the assessment of clinical skills competency with high levels of member satisfaction which may serve as a model for future regional consortia.

The Past, Present, and Future of the United States Medical Licensing Examination Step 2 Clinical Skills Examination

In January 2021, the United States Medical Licensing Examination (USMLE) announced the permanent suspension of their Step 2 Clinical Skills (CS) examination. Launched in 2004, the Step 2 CS examination was intended to ensure that physicians entering graduate medical education possess the necessary information gathering, clinical reasoning, and communication skills necessary to provide patient care. Although the requirement that doctors pass a clinical skills examination as a condition of licensure likely improved some elements of medical education and physician practice, the Step 2 CS examination was deeply unpopular among many medical students since its inception. The demise of USMLE Step 2 CS provides an opportunity to re-examine the test’s value and incorporate improvements in future iterations. However, doing so requires a clear understanding of why the test was so vigorously challenged. Here, we review the history of clinical skills examinations used for medical licensure in the United States and highlight the persistent concerns regarding Step 2 CS’s cost, value, validity, and lack of examinee feedback before proposing future improvements to address each concern.