Investigation of answer changes on the USMLE® Step 2 Clinical Knowledge examination

An experimental comparison of multiple-choice and short-answer questions on a high-stakes test for medical students

Recent advances in automated scoring technology have made it practical to replace multiple-choice questions (MCQs) with short-answer questions (SAQs) in large-scale, high-stakes assessments. However, most previous research comparing these formats has used small examinee samples testing under low-stakes conditions. Additionally, previous studies have not reported on the time required to respond to the two item types. This study compares the difficulty, discrimination, and time requirements for the two formats when examinees responded as part of a large-scale, high-stakes assessment. Seventy-one MCQs were converted to SAQs. These matched items were randomly assigned to examinees completing a high-stakes assessment of internal medicine. No examinee saw the same item in both formats. Items administered in the SAQ format were generally more difficult than items in the MCQ format. The discrimination index for SAQs was modestly higher than that for MCQs and response times were substantially higher for SAQs. These results support the interchangeability of MCQs and SAQs. When it is important that the examinee generate the response rather than selecting it, SAQs may be preferred. The results relating to difficulty and discrimination reported in this paper are consistent with those of previous studies. The results on the relative time requirements for the two formats suggest that with a fixed testing time fewer SAQs can be administered, this limitation more than makes up for the higher discrimination that has been reported for SAQs. We additionally examine the extent to which increased difficulty may directly impact the discrimination of SAQs.

Changes in the Objective Structured Clinical Examination (OSCE) of University Schools of Medicine during COVID-19. Experience with a computer-based case simulation OSCE (CCS-OSCE)

Background and objectives

The COVID-19 pandemic has forced universities to move the completion of university studies online. Spain’s National Conference of Medical School Deans coordinates an objective, structured clinical competency assessment called the Objective Structured Clinical Examination (OSCE), which consists of 20 face-to-face test sections for students in their sixth year of study. As a result of the pandemic, a computer-based case simulation OSCE (CCS-OSCE) has been designed. The objective of this article is to describe the creation, administration, and development of the test.

Materials and methods

This work is a descriptive study of the CCS-OSCE from its planning stages in April 2020 to its administration in June 2020.

Results

The CCS-OSCE evaluated the competences of anamnesis, exploration, clinical judgment, ethical aspects, interprofessional relations, prevention, and health promotion. No technical or communication skills were evaluated. The CCS-OSCE consisted of ten test sections, each of which had a 12-min time limit and ranged from six to 21 questions (mean: 1.1 min/question). The CCS-OSCE used the virtual campus platform of each of the 16 participating medical schools, which had a total of 2829 students in their sixth year of study. It was jointly held on two dates in June 2020.

Conclusions

The CCS-OSCE made it possible to bring together the various medical schools and carry out interdisciplinary work. The CCS-OSCE conducted may be similar to Step 3 of the United States Medical Licensing Examination.

Twelve tips on guiding preparation for both high-stakes exams and long-term learning

High-stakes exams including admissions, licensing, and maintenance of certification examinations are commonplace in health professions education. Although exam scores and performance can often serve gate-keeping purposes, the broader goal of health professions education is to foster deep, self-directed, meaningful, motivated learning. Establishing strong support structures that emphasize deep learning and understanding rather than exam scores can be beneficial to preparing learners who have the knowledge base to be excellent practitioners. This article offers guidance that can be used by academic support centres, medical educators, learning specialists, and faculty advisors, or even test-takers, to help learners to balance score achievement and knowledge development, while simultaneously cultivating more efficient and motivated studying and increasingly self-regulated learning. This series of tips details considerations for building academic success supports, fostering a growth mindset, planning efficient and effective studying efforts, utilizing test-enhanced learning strategies, exam-taking skills practice, and other support structures that can help strengthen learning experiences overall.

[Changes in the Objective Structured Clinical Examination (OSCE) of University Schools of Medicine during COVID-19. Experience with a computer-based case simulation OSCE (CCS-OSCE)◊]

BACKGROUND AND OBJECTIVES

The COVID-19 pandemic has forced universities to move the completion of university studies online. Spain's National Conference of Medical School Deans coordinates an objective, structured clinical competency assessment called the Objective Structured Clinical Examination (OSCE), which consists of 20 face-to-face test sections for students in their sixth year of study. As a result of the pandemic, a computer-based case simulation OSCE (CCS-OSCE) has been designed. The objective of this article is to describe the creation, administration, and development of the test.

MATERIALS AND METHODS

This work is a descriptive study of the CCS-OSCE from its planning stages in April 2020 to its administration in June 2020.

RESULTS

The CCS-OSCE evaluated the competences of anamnesis, exploration, clinical judgment, ethical aspects, interprofessional relations, prevention, and health promotion. No technical or communication skills were evaluated. The CCS-OSCE consisted of ten test sections, each of which had a 12-minutes time limit and ranged from six to 21 questions (mean: 1.1 minutes/question). The CCS-OSCE used the virtual campus platform of each of the 16 participating medical schools, which had a total of 2,829 students in their sixth year of study. It was jointly held on two dates in June 2020.

CONCLUSIONS

The CCS-OSCE made it possible to bring together the various medical schools and carry out interdisciplinary work. The CCS-OSCE conducted may be similar to Step 3 of the United States Medical Licensing Examination.