1
|
Iñarrairaegui M, Fernández-Ros N, Lucena F, Landecho MF, García N, Quiroga J, Herrero JI. Evaluation of the quality of multiple-choice questions according to the students' academic level. BMC MEDICAL EDUCATION 2022; 22:779. [PMID: 36369070 PMCID: PMC9652897 DOI: 10.1186/s12909-022-03844-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Accepted: 10/30/2022] [Indexed: 06/16/2023]
Abstract
BACKGROUND One of the most important challenges in medical education is the preparation of multiple-choice questions able to discriminate between students with different academic level. Average questions may be very easy for students with good performance, reducing their discriminant power in this group of students. The aim of this study was to analyze if the discriminative power of multiple-choice questions is different according to the students' academic performance. METHODS We retrospectively analyzed the difficulty and discrimination indices of 257 multiple-choice questions used for the end of course examination of pathophysiology and analyzed whether the discrimination indices were lower in students with good academic performance (group 1) than in students with moderate/poor academic performance (group 2). We also evaluated whether case-based questions maintained their discriminant power better than factual questions in both groups of students or not. Comparison of the difficulty and discrimination indices between both groups was based on the Wilcoxon test. RESULTS Difficulty index was significantly higher in group 1 (median: 0.78 versus 0.56; P < 0.001) and discrimination index was significantly higher in group 2 (median: 0.21 versus 0.28; P < 0.001). Factual questions had higher discriminative indices in group 2 than in group 1 (median: 0.28 versus 0.20; P < 0.001), but discriminative indices of case-based questions did not differ significantly between groups (median: 0.30 versus 0.24; P = 0.296). CONCLUSIONS Multiple-choice question exams have lower discriminative power in the group of students with high scores. The use of clinical vignettes may allow to maintain the discriminative power of multiple-choice questions.
Collapse
Affiliation(s)
- Mercedes Iñarrairaegui
- Liver Unit, Clínica Universidad de Navarra, Av. Pio XII, 36, 31008, Pamplona, Spain
- Instituto de Investigación Sanitaria de Navarra (IdiSNA), Pamplona, Spain
- Centro de Investigación Sanitaria en Red de Enfermedades Hepáticas y Digestivas, Madrid, Spain
- Department of Internal Medicine, Clínica Universidad de Navarra, Pamplona, Spain
| | - Nerea Fernández-Ros
- Instituto de Investigación Sanitaria de Navarra (IdiSNA), Pamplona, Spain
- Department of Internal Medicine, Clínica Universidad de Navarra, Pamplona, Spain
| | - Felipe Lucena
- Instituto de Investigación Sanitaria de Navarra (IdiSNA), Pamplona, Spain
- Department of Internal Medicine, Clínica Universidad de Navarra, Pamplona, Spain
| | - Manuel F Landecho
- Instituto de Investigación Sanitaria de Navarra (IdiSNA), Pamplona, Spain
- Department of Internal Medicine, Clínica Universidad de Navarra, Pamplona, Spain
| | - Nicolás García
- Instituto de Investigación Sanitaria de Navarra (IdiSNA), Pamplona, Spain
- Department of Internal Medicine, Clínica Universidad de Navarra, Pamplona, Spain
| | - Jorge Quiroga
- Liver Unit, Clínica Universidad de Navarra, Av. Pio XII, 36, 31008, Pamplona, Spain
- Instituto de Investigación Sanitaria de Navarra (IdiSNA), Pamplona, Spain
- Centro de Investigación Sanitaria en Red de Enfermedades Hepáticas y Digestivas, Madrid, Spain
- Department of Internal Medicine, Clínica Universidad de Navarra, Pamplona, Spain
| | - Jose Ignacio Herrero
- Liver Unit, Clínica Universidad de Navarra, Av. Pio XII, 36, 31008, Pamplona, Spain.
- Instituto de Investigación Sanitaria de Navarra (IdiSNA), Pamplona, Spain.
- Centro de Investigación Sanitaria en Red de Enfermedades Hepáticas y Digestivas, Madrid, Spain.
- Department of Internal Medicine, Clínica Universidad de Navarra, Pamplona, Spain.
| |
Collapse
|
2
|
Hernandez T, Fallar R, Polydorides AD. Outcomes of Remote Pathology Instruction in Student Performance and Course Evaluation. Acad Pathol 2021; 8:23742895211061822. [PMID: 34926798 PMCID: PMC8679015 DOI: 10.1177/23742895211061822] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/23/2021] [Revised: 10/22/2021] [Accepted: 10/31/2021] [Indexed: 11/17/2022] Open
Abstract
The coronavirus disease 2019 (COVID-19) pandemic has disrupted undergraduate
medical education, including preclinical class-based courses, by requiring
social distancing and essentially eliminating in-person teaching. The aim of
this study was to compare student performance and satisfaction before and after
implementation of remote instruction in a first-year introductory pathology
course. Assessments (3 quizzes, 1 practical exam, and 1 final) were compared
between courses given before (January 2020) and during (January 2021) the
COVID-19 pandemic, in terms of mean scores, degree of difficulty, and item
discrimination, both overall and across different question types. Students’
evaluations of the course (Likert scale-based) were also compared between the 2
years. Significantly higher mean scores were observed during remote instruction
(compared to the prior, in-person year) on verbatim-repeated questions (94.9 ±
8.8 vs 89.4 ± 12.2; P = .002) and on questions incorporating a
gross specimen image (88.4 ± 7.5 vs 84.4 ± 10.3; P = .007). The
percentage of questions that were determined to be moderate/hard in degree of
difficulty and good/very good in item discrimination remained similar between
the 2 time periods. In the practical examination, students performed
significantly better during remote instruction on questions without specimen
images (96.5 ± 7.0 vs 91.2 ± 15.2; P = .004). Finally, course
evaluation metrics improved, with students giving a higher mean rating value in
each measured end point of course quality during the year of remote instruction.
In conclusion, student performance and course satisfaction generally improved
with remote instruction, suggesting that the changes implemented, and their
consequences, should perhaps inform future curriculum improvements.
Collapse
Affiliation(s)
- Tahyna Hernandez
- Department of Pathology, Molecular and Cell Based Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Robert Fallar
- Department of Medical Education, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| | - Alexandros D. Polydorides
- Department of Pathology, Molecular and Cell Based Medicine, Icahn School of Medicine at Mount Sinai, New York, NY, USA
| |
Collapse
|