Step Up-Not On-The Step 2 Clinical Skills Exam: Directors of Clinical Skills Courses (DOCS) Oppose Ending Step 2 CS

Integrating Formative and Summative Clinical Skills Examinations to Promote Learning for Early Medical Students: A Mixed Methods Study

Background

Medical educators aim to train students with high-quality clinical skills through the promotion of self-regulated learning and the development of lifelong learning skills. Formative clinical skills examinations (FCSE) allow for real-time learner-centered feedback and coaching which are key in promoting the development of expertise in early learners. This study assessed the impact of the integration of FCSE with learner-centered, real-time feedback and coaching based on an "educational plan-do-study-act" (PDSA) cycle on early medical students' experience and performance.

Methods

A mixed methods study was designed to assess the integration of FCSE for first- and second-year medical students. FCSE consisted of linked stations: students gathered a history from a standardized patient (SP), performed a hypothesis-driven physical examination followed by real-time learner-centered feedback and coaching. Each student met with one faculty and one SP to reflect on their performance, identify areas for improvement, re-practice of skills, and identify a plan for ongoing practice improvement. Students were surveyed upon completion of formative and summative clinical skills examinations. Student communication and clinical reasoning performance were compared to historical controls.

Results

Students reported that FSCE improved the learning environment and helped prepare them for subsequent summative clinical skills examinations. Students appreciated the opportunity for practice and real-time feedback and reported applying their take-home points on subsequent exams. Student longitudinal performance was not impacted by the transition to FSCE.

Conclusion

While labor-intensive, FCSE with real-time feedback and coaching are an effective means of promoting learning and should be considered for integration early in medical school curricula.

Demise of the USMLE Step-2 CS exam: Rationalizing a way forward

The COVID-19 pandemic has compelled rethinking and changes in medical education, the most controversial perhaps being the cancelation of USMLE Step-2 Clinical Skills exam (Step-2 CS). What started in March of 2020 as suspension of this professional licensure exam, because of concerns about infection risk for examinees, standardized patients (SPs), and administrators, soon became permanent cancelation in January 2021. Expectedly, it triggered debate in medical education circles. Positively, however, the USMLE regulatory agencies (NBME and FSMB) saw an opportunity to innovate an exam tainted with perceptions of validity deficits, cost, examinee inconvenience, and worries about future pandemics; they therefore called for a public debate to fashion a way forward. We have approached the issue by defining Clinical Skills (CS), exploring its epistemology and historic evolution, including assessment modalities from Hippocratic times to the modern era. We defined CS as the art of medicine manifest in the physician-patient encounter as history taking (driven by communication skills and cultural competence) and physical examination. We classified CS components into knowledge and psychomotor skill domains, established their relative importance in the physician process (clinical reasoning) of diagnosis, thus establishing a theoretical framework for developing valid, reliable, feasible, fair, and verifiable CS assessment. Given the concerns for COVID-19 and future pandemics, we established that CS can largely be assessed remotely, and what could not, can be assessed locally (school/regional consortia level) as part of a USMLE-regulated/supervised assessment regimen with established national standards, thus maintaining USMLE's fiduciary responsibilities. We have suggested a national/regional program for faculty development in CS curriculum development, and assessment, including standard setting skills. This pool of expert faculty will form the nucleus of our proposed USMLE-regulated External Peer Review Initiative (EPRI). Finally, we suggest that CS evolves into an academic discipline/department of its own, rooted in scholarship.

After the Discontinuation of Step 2 CS: A Collaborative Statement from the Directors of Clinical Skills Education (DOCS)

Issue: The United States Medical Licensing Exam (USMLE) Step 2 Clinical Skills Examination (Step 2 CS), the only clinical skills competency testing required for licensure in the United States, has been discontinued. Evidence: This exam, though controversial, propelled a movement emphasizing the value of clinical skills instruction and assessment in undergraduate medical education. While disappointed by the loss of this national driver that facilitated standardization of clinical skills education, the Directors of Clinical Skills Education (DOCS) see prospects for educational innovation and growth. DOCS is a national organization and inclusive community of clinical skills education leaders. This statement from DOCS regarding the discontinuation of USMLE Step 2 CS has been informed by DOCS meetings, listserv discussions, an internal survey, and a review of recent literature. Implications: Rigorous clinical skills assessment remains central to effective and patient-centered healthcare. DOCS shares specific concerns as well as potential solutions. Now free from the external pressure to prepare students for success on Step 2 CS, clinical skills educators can reprioritize content and restructure clinical skills programs to best meet the needs of learners and the ever-evolving healthcare landscape. DOCS, as an organization of clinical skills leaders, makes the following recommendations: 1) Collaboration amongst institutions must be prioritized; clinical skills assessment consortia should be expanded. 2) Governing, accrediting, and licensing organizations should leverage their influence to support and require high quality clinical skills assessments. 3) UME clinical skills leaders should develop ways to identify students who perform with exceptional, borderline, and poor clinical skills at their local institutions. 4) UME leadership should fully commit resources and curricular time to graduate students with excellent clinical skills.

The Florida Clinical Skills Collaborative: A New Regional Consortium for the Assessment of Clinical Skills

Discontinuation of the United States Medical Licensing Examination (USMLE) Step 2 Clinical Skills (CS) exam and Comprehensive Osteopathic Medical Licensing Examination (COMLEX) Level 2 Performance Evaluation (2-PE) raised questions about the ability of medical schools to ensure the clinical skills competence of graduating students. In February 2021, representatives from all Florida, United States, allopathic and osteopathic schools initiated a collaboration to address this critically important issue in the evolving landscape of medical education. A 5-point Likert scale survey of all members (n=18/20 individuals representing 10/10 institutions) reveals that initial interest in joining the collaboration was high among both individuals (mean 4.78, SD 0.43) and institutions (mean 4.69, SD 0.48). Most individuals (mean 4.78, SD 0.55) and institutions (mean 4.53, SD 0.72) are highly satisfied with their decision to join. Members most commonly cited a "desire to establish a shared assessment in place of Step 2 CS/2-PE" as their most important reason for joining. Experienced benefits of membership were ranked as the following: 1) Networking, 2) Shared resources for curriculum implementation, 3) Scholarship, and 4) Work towards a shared assessment in place of Step 2 CS/2-PE. Challenges of membership were ranked as the following: 1) Logistics such as scheduling and technology, 2) Agreement on common goals, 3) Total time commitment, and 4) Large group size. Members cited the "administration of a joint assessment pilot" as the highest priority for the coming year. Florida has successfully launched a regional consortium for the assessment of clinical skills competency with high levels of member satisfaction which may serve as a model for future regional consortia.

A Call to Action From the California Consortium for the Assessment of Clinical Competence: Making the Case for Regional Collaboration

The discontinuation of the United States Medical Licensing Examination Step 2 Clinical Skills Examination emphasizes the need for other reliable standardized assessments of medical student clinical skills. For 30 years, the California Consortium for the Assessment of Clinical Competence (CCACC) has collaborated in the development of clinical skills assessments and has become a valuable resource for clinicians, standardized patient educators, psychometricians, and medical educators. There are many merits to strong multi-institutional partnerships, including the integration of data across multiple schools to provide feedback to both students and curricula, pooled test performance statistics for analysis and quality assurance, shared best practices and resources, individual professional development, and opportunities for research and scholarship. The infrastructure of the CCACC allows member schools to adapt to a changing medical landscape, from emerging trends in clinical medicine to the limitations imposed by a global pandemic. In the absence of a national examination, there is now a greater need for all medical schools to develop a comprehensive, dynamic, and psychometrically sound assessment that accurately evaluates clinical skills. Medical schools working together in regional consortia have the opportunity to create and implement innovative and robust assessments that evaluate a wide range of clinical skills, ensure that medical students have met an expected level of clinical competency before graduation, and provide a framework that contributes to ongoing efforts for the development of new national clinical skills standards.

Online student culture as site for negotiating assessment in medical education

Classic studies of medical education have examined how professional socialization reproduces the prevailing professional culture, as well as how students actively negotiate their place in educational processes. However, sociological research has not re-examined student culture in light of structural transformations in medical education, such as the introduction of new assessment types and their use as modes of commensuration. In this paper, we examine data from two studies of online forums where medical trainees and applicants to medical school discuss their experiences preparing for tests of professional skills, including judgment, empathy, and communication. Examining how medical students talk about these tests on such forums allows us to understand the meaning-making processes at work as students negotiate the commensuration processes such tests enable. We examine how these negotiations take place in online forums, where participants confront common challenges, form common perspectives, and share common solutions, all hallmarks of student culture. Through qualitative analysis, we find that online communities are spaces where students grapple with these new forms of commensuration, interrogate the standards and quantifications that underlie them, and collectively negotiate how to approach these assessments. Using the case of online forum communities, our findings advance past work on student culture in medical sociology by theorizing student culture as an extra-organizational phenomenon that spans multiple career stages. In so doing, we highlight the importance of online forum data for studying social processes.

Cancel culture: exploring the unintended consequences of cancelling the Canadian national licensing clinical examination

Assessment drives learning. However, when it comes to high-stakes examinations (e.g., for licensure or certification), these assessments of learning may be seen as unnecessary hurdles by some. Licensing clinical skills assessment in particular have come under fire over the years. Recently, assessments such as the Medical Council of Canada Qualifying Examination Part II, a clinical skills objective structured clinical examination, have been permanently cancelled. The authors explore potential consequences of this cancellation including those that are inadvertent and undesirable. Future next steps for clinical skills assessment are explored.

Early Pre-clerkship Clinical Skills Assessments Predict Clerkship Performance

INTRODUCTION

Assessment of medical students' clinical skills (CS) remains a challenge. Little is known about early predictors of future CS performance. This study examines the relationship between students' pre-clerkship clinical skills (PCCS) performance and year 3 clerkship performance measures.

METHODS

The authors performed a retrospective analysis of four medical student cohorts who matriculated between 2014 and 2017 and participated in a longitudinal pre-clerkship CS curriculum. A total of 440 students were included in the analyses. Students' clinical skills were assessed through a series of PCCS exams, each consisting of a single standardized patient encounter. First-year PCCS exams assessed history taking, physical examination, professionalism, and communication skills; second-year PCCS exams also assessed clinical documentation and clinical reasoning skills. Evaluators assigned a grade of "satisfactory," "borderline," or "unsatisfactory" for each skill set. Regression analyses compared year 3 performance outcomes between students with one or more "unsatisfactory" or "borderline" PCCS skill set grades and students assessed as "satisfactory" for all PCCS skill set assessments.

RESULTS

Thirty-two percent (n = 140) of the 440 students had at least one borderline or unsatisfactory (US) PCCS skill set grade. These students performed significantly worse on year 3 National Board of Medical Examiner subject exams, workplace-based clinical performance evaluations, and overall year 3 performance compared to students who passed all PCCS exam components. In addition, a higher percentage of students with PCCS performance deficiencies failed the United States Medical Licensing Examination Step 2 CS exam on the first attempt versus students who passed all PCCS exam components.

CONCLUSIONS

PCCS exam performance at our institution aligned with future student performance on multiple year 3 clerkship outcome measures. This pre-clerkship performance data can be used to identify at-risk students who would benefit from additional resources to achieve competency in the clerkship environment and future medical training.

SUPPLEMENTARY INFORMATION

The online version contains supplementary material available at 10.1007/s40670-022-01519-8.

The Past, Present, and Future of the United States Medical Licensing Examination Step 2 Clinical Skills Examination

In January 2021, the United States Medical Licensing Examination (USMLE) announced the permanent suspension of their Step 2 Clinical Skills (CS) examination. Launched in 2004, the Step 2 CS examination was intended to ensure that physicians entering graduate medical education possess the necessary information gathering, clinical reasoning, and communication skills necessary to provide patient care. Although the requirement that doctors pass a clinical skills examination as a condition of licensure likely improved some elements of medical education and physician practice, the Step 2 CS examination was deeply unpopular among many medical students since its inception. The demise of USMLE Step 2 CS provides an opportunity to re-examine the test’s value and incorporate improvements in future iterations. However, doing so requires a clear understanding of why the test was so vigorously challenged. Here, we review the history of clinical skills examinations used for medical licensure in the United States and highlight the persistent concerns regarding Step 2 CS’s cost, value, validity, and lack of examinee feedback before proposing future improvements to address each concern.

The End of Step 2 CS Should Be the Beginning of a New Approach to Clinical Skills Assessment

The discontinuation of the United States Medical Licensing Examination Step 2 Clinical Skills (CS) in 2020 in response to the COVID-19 pandemic marked the end of a decades-long debate about the utility and value of the exam. For all its controversy, the implementation of Step 2 CS in 2004 brought about profound changes to the landscape of medical education, altering the curriculum and assessment practices of medical schools to ensure students were prepared to take and pass this licensing exam. Its elimination, while celebrated by some, is not without potential negative consequences. As the responsibility for assessing students' clinical skills shifts back to medical schools, educators must take care not to lose the ground they have gained in advancing clinical skills education. Instead, they need to innovate, collaborate, and share resources; hold themselves accountable; and ultimately rise to the challenge of ensuring that physicians have the necessary clinical skills to safely and effectively practice medicine.

Farewell to the Step 2 Clinical Skills Exam: New Opportunities, Obligations, and Next Steps

The unexpected discontinuation of the United States Medical Licensing Examination Step 2 Clinical Skills (CS) exam in January 2021 carries both risks and opportunities for medical education in the United States. Step 2 CS had far-reaching effects on medical school curricula and school-based clinical skills assessments. Absent the need to prepare students for this high-stakes exam, will the rigor of foundational clinical skills instruction and assessment remain a priority at medical schools? In this article, the authors consider the potential losses and gains from the elimination of Step 2 CS and explore opportunities to expand local summative assessments beyond the narrow bounds of Step 2 CS. The responsibility for implementing a rigorous and credible summative assessment of clinical skills that are critical for patient safety as medical students transition to residency now lies squarely with medical schools. Robust human simulation (standardized patient) programs, including regional and virtual simulation consortia, can provide infrastructure and expertise for innovative and creative local assessments to meet this need. Novel applications of human simulation and traditional formative assessment methods, such as workplace-based assessments and virtual patients, can contribute to defensible summative decisions about medical students' clinical skills. The need to establish validity evidence for decisions based on these novel assessment methods comprises a timely and relevant focus for medical education research.

Twelve tips for running an effective session with standardized patients

The incorporation of actors as standardized patients (SPs) to help students achieve learning goals across a range of topics has become widespread in medical education. SPs are integrated into formative and summative objective structured clinical examinations by medical educators and by licensing boards for assessment of competence. While SPs are useful for assessment of dynamic skills, they also have significant utility as an engaging instructional method. Few tools in teaching allow for the breadth of instruction, practice, and assessment offered by workshops involving SPs. A simulated encounter with an SP may be a trainee's only opportunity to experience working through a particular clinical scenario in an environment that carries no risk of significant harm. Thus, there is immense potential for educational innovation with SPs. The following Twelve Tips piece provides suggestions for harnessing this potential based on available literature and educational experiences of the authors.

A Qualitative Study of New York Medical Student Views on Implicit Bias Instruction: Implications for Curriculum Development

BACKGROUND

For at least the past two decades, medical educators have worked to improve patient communication and health care delivery to diverse patient populations; despite efforts, patients continue to report prejudice and bias during their clinical encounters. Targeted instruction in implicit bias recognition and management may promote the delivery of equitable care, but students at times resist this instruction. Little guidance exists to overcome this resistance and to engage students in implicit bias instruction; instruction over time could lead to eventual skill development that is necessary to mitigate the influence of implicit bias on clinical practice behaviors.

OBJECTIVE

To explore student perceptions of challenges and opportunities when participating in implicit bias instruction.

APPROACH

We conducted a qualitative study that involved 11 focus groups with medical students across each of the four class years to explore their perceptions of challenges and opportunities related to participating in such instruction. We analyzed transcripts for themes.

KEY RESULTS

Our analysis suggests a range of attitudes toward implicit bias instruction and identifies contextual factors that may influence these attitudes. The themes were (1) resistance; (2) shame; (3) the negative role of the hidden curriculum; and (4) structural barriers to student engagement. Students expressed resistance to implicit bias instruction; some of these attitudes are fueled from concerns of anticipated shame within the learning environment. Participants also indicated that student engagement in implicit bias instruction was influenced by the hidden curriculum and structural barriers.

CONCLUSIONS

These insights can inform future curriculum development efforts. Considerations related to instructional design and programmatic decision-making are highlighted. These considerations for implicit bias instruction may provide useful frameworks for educators looking for opportunities to minimize student resistance and maximize engagement in multi-session instruction in implicit bias recognition and management.

Can We Increase the Value and Decrease the Cost of Clinical Skills Assessment?

Although the authors do not agree with medical students' bid to end the United States Medical Licensing Examination Step 2 Clinical Skills or Comprehensive Osteopathic Medical Licensing Examination Level 2-Performance Evaluation tests, they concur with Ecker and colleagues that conducting further research to support the validity argument, providing greater feedback on performance, and exploring options to reduce costs are important for addressing students' concerns. Evidence to support the validity of clinical skills exam scores and associated inferences already exists. What is lacking, and would help further justify the use of these examinations, is more evidence to support the "extrapolation" argument-that is, is performance on these examinations related to actual patient care? Enhanced feedback on exam performance should also be considered. While performance data from licensing examinations should be used judiciously given the primary purpose of these tests, additional data would be helpful to learners and their institutions. Centralized testing remains the least costly design, but efficiencies of standardized patient training, case development, and scoring can be reviewed. Scoring modifications made in the past several years at substantial cost should be evaluated to determine whether they have achieved desired goals. Testing organizations can and should embrace these essential elements of transparency and accountability to address concerns about the value of clinical skills examinations.