[Analysis of the discrimination of the final marks after the first computerized national ranking exam in Medicine in June 2016 in France]

[Association between students' clinical performance and their success in the computerized national ranking tests: A single-center retrospective cohort study]

INTRODUCTION

Before attending residency, 6th-year French medical students must validate a final examination including a practical clinical test in their faculty. However, the national ranking that determines their future specialty and region solely relies on a computerized knowledge test. Our goal was to investigate the association between the final faculty test and the national ranking test.

METHODS

In our faculty, the final examination includes a computerized theoretical test (similar to the national one) and a practical test: a standardized evaluation of semiology skills at the bedside and a standardized assessment of relational skills with role plays. The agreements between the national test and faculty computerized and practical tests were analyzed by intraclass correlation coefficients (ICC).

RESULTS

Data from 1806 students who underwent the three examinations from 2017 to 2021 were analyzed. There was a good agreement between the ranks in the faculty and national computerized tests: ICC 0.83 (95% CI 0.81-0.85). By contrast, the agreement between the ranks in the faculty practical test and the national computerized test was poor: ICC 0.13 (95% CI 0.08-0.17). Results were stable over the years.

CONCLUSION

The agreement between the ranking of the current national test and the clinical skills assessed by a specific faculty test is poor. This could relate to a true independence or to different levels of motivation to perform well. Indeed, the result of the national test is the most important one as it determines their career. Incorporating a clinical assessment into the national ranking test will motivate students to acquire clinical skills and value those who perform well this practical dimension.

Impact of integrating objective structured clinical examination into academic student assessment: Large-scale experience in a French medical school

Purpose

Objective structured clinical examinations (OSCE) evaluate clinical reasoning, communication skills, and interpersonal behavior during medical education. In France, clinical training has long relied on bedside clinical practice in academic hospitals. The need for a simulated teaching environment has recently emerged, due to the increasing number of students admitted to medical schools, and the necessity of objectively evaluating practical skills. This study aimed at investigating the relationships between OSCE grades and current evaluation modalities.

Methods

Three-hundred seventy-nine 4^th-year students of University-of-Paris Medical School participated to the first large-scale OSCE at this institution, consisting in three OSCE stations (OSCE#1–3). OSCE#1 and #2 focused on cardiovascular clinical skills and competence, whereas OSCE#3 focused on relational skills while providing explanations before planned cholecystectomy. We investigated correlations of OSCE grades with multiple choice (MCQ)-based written examinations and evaluations of clinical skills and behavior (during hospital traineeships); OSCE grade distribution; and the impact of integrating OSCE grades into the current evaluation in terms of student ranking.

Results

The competence-oriented OSCE#1 and OSCE#2 grades correlated only with MCQ grades (r = 0.19, P<0.001) or traineeship skill grades (r = 0.17, P = 0.001), respectively, and not with traineeship behavior grades (P>0.75). Conversely, the behavior-oriented OSCE#3 grades correlated with traineeship skill and behavior grades (r = 0.19, P<0.001, and r = 0.12, P = 0.032), but not with MCQ grades (P = 0.09). The dispersion of OSCE grades was wider than for MCQ examinations (P<0.001). When OSCE grades were integrated to the final fourth-year grade with an incremental 10%, 20% or 40% coefficient, an increasing proportion of the 379 students had a ranking variation by ±50 ranks (P<0.001). This ranking change mainly affected students among the mid-50% of ranking.

Conclusion

This large-scale French experience showed that OSCE designed to assess a combination of clinical competence and behavioral skills, increases the discriminatory capacity of current evaluations modalities in French medical schools.