[Assessment of clinical observation skills of last year medical students]

Objective structured clinical examination versus traditional written examinations: a prospective observational study

BACKGROUND

Recently, Objective Structured Clinical Examinations (OSCE) became an official evaluation modality for 6-year medical students in France. Before, standard examination modalities were: written progressive clinical cases (PCC), written critical reading of scientific articles (CRA), and internship evaluation (IE). The aim of this study was to assess the performances of 6-year medical students in their final faculty tests by comparing OSCE-exams with standard examination modalities.

METHODS

This was a prospective observational study. We included all 6-year medical students in our university from 2020 to 2021. The endpoints were the scores obtained at the following final faculty tests during the 6^th year of medical studies: OSCE-training, OSCE-exams, written PCC, written CRA, and IE. All scores were compared in a paired-analysis.

RESULTS

A total of 400 students were included in the study. No student was excluded in the final analysis. The mean scores obtained at the OSCE-exams were significantly different from those obtained at OSCE-training, PCC, CRA, and IE (12.6 ± 1.7, 11.7 ± 1.7, 13.4 ± 1.4, 13.2 ± 1.5, 14.7 ± 0.9, respectively; p < 0.001). OSCE-exams scores were moderately and significantly correlated with OSCE-training and PCC (Spearman rho coefficient = 0.4, p < 0.001); OSCE examination scores were lowly but significantly correlated with CRA and IE (Spearman rho coefficient = 0.3, p < 0.001). OSCE-scores significantly increased after an OSCE training session.

CONCLUSION

In our faculty, 6-year medical students obtained lower scores at OSCE exams compared to other standard evaluation modalities. The correlation was weak to moderate but significant. These results suggest that OSCE are not redundant with the other evaluation modalities. Interestingly, a single OSCE training session led to an improvement in OSCE scores underlining the importance of a specific training.

[Association between students' clinical performance and their success in the computerized national ranking tests: A single-center retrospective cohort study]

INTRODUCTION

Before attending residency, 6th-year French medical students must validate a final examination including a practical clinical test in their faculty. However, the national ranking that determines their future specialty and region solely relies on a computerized knowledge test. Our goal was to investigate the association between the final faculty test and the national ranking test.

METHODS

In our faculty, the final examination includes a computerized theoretical test (similar to the national one) and a practical test: a standardized evaluation of semiology skills at the bedside and a standardized assessment of relational skills with role plays. The agreements between the national test and faculty computerized and practical tests were analyzed by intraclass correlation coefficients (ICC).

RESULTS

Data from 1806 students who underwent the three examinations from 2017 to 2021 were analyzed. There was a good agreement between the ranks in the faculty and national computerized tests: ICC 0.83 (95% CI 0.81-0.85). By contrast, the agreement between the ranks in the faculty practical test and the national computerized test was poor: ICC 0.13 (95% CI 0.08-0.17). Results were stable over the years.

CONCLUSION

The agreement between the ranking of the current national test and the clinical skills assessed by a specific faculty test is poor. This could relate to a true independence or to different levels of motivation to perform well. Indeed, the result of the national test is the most important one as it determines their career. Incorporating a clinical assessment into the national ranking test will motivate students to acquire clinical skills and value those who perform well this practical dimension.

Benefits of semiology taught using near-peer tutoring are sustainable

BACKGROUND

Near-peer tutoring appears to be an efficient approach for teaching clinical skills. However, the clinical experience gained in the form of student medical internships may offset any interest in such tutoring programme. We then investigated the long-term benefits of this programme.

METHODS

This study was conducted in a medical school that experimented in near-peer tutoring for semiology intended for undergraduate medical students. Objective Structured Clinical Examinations and a written semiology test were used to assess students' clinical skills immediately on its conclusion and repeated one and 2 years after the tutoring was completed.

RESULTS

116 students were evaluated initially (80 tutored and 36 untutored), 38 at 1 year (16 tutored and 22 untutored), 42 at 2 years (21 tutored and 21 untutored). In the global score for Objective Structured Clinical Examinations: at 1 year, the tutored group scored 14.0 ± 1.05 and the untutored group scored 11.3 ± 2.3 (p < 0.001), at 2 years, the tutored group scored 15.1 ± 1.5 and the untutored group scored 12.4 ± 2.2 (p < 0.001). We found a similar but smaller difference for the written semiology test. The difference for Objective Structured Clinical Examinations between tutored and untutored students vanished over time for cross-cutting skills.

CONCLUSIONS

Near-peer tutoring in semiology for undergraduate medical students led to better results that remained with the passing of time. Though internships do allow an improvement in the clinical skills of untutored students, they did not reach the level of tutored students.

Impact of integrating objective structured clinical examination into academic student assessment: Large-scale experience in a French medical school

Purpose

Objective structured clinical examinations (OSCE) evaluate clinical reasoning, communication skills, and interpersonal behavior during medical education. In France, clinical training has long relied on bedside clinical practice in academic hospitals. The need for a simulated teaching environment has recently emerged, due to the increasing number of students admitted to medical schools, and the necessity of objectively evaluating practical skills. This study aimed at investigating the relationships between OSCE grades and current evaluation modalities.

Methods

Three-hundred seventy-nine 4^th-year students of University-of-Paris Medical School participated to the first large-scale OSCE at this institution, consisting in three OSCE stations (OSCE#1–3). OSCE#1 and #2 focused on cardiovascular clinical skills and competence, whereas OSCE#3 focused on relational skills while providing explanations before planned cholecystectomy. We investigated correlations of OSCE grades with multiple choice (MCQ)-based written examinations and evaluations of clinical skills and behavior (during hospital traineeships); OSCE grade distribution; and the impact of integrating OSCE grades into the current evaluation in terms of student ranking.

Results

The competence-oriented OSCE#1 and OSCE#2 grades correlated only with MCQ grades (r = 0.19, P<0.001) or traineeship skill grades (r = 0.17, P = 0.001), respectively, and not with traineeship behavior grades (P>0.75). Conversely, the behavior-oriented OSCE#3 grades correlated with traineeship skill and behavior grades (r = 0.19, P<0.001, and r = 0.12, P = 0.032), but not with MCQ grades (P = 0.09). The dispersion of OSCE grades was wider than for MCQ examinations (P<0.001). When OSCE grades were integrated to the final fourth-year grade with an incremental 10%, 20% or 40% coefficient, an increasing proportion of the 379 students had a ranking variation by ±50 ranks (P<0.001). This ranking change mainly affected students among the mid-50% of ranking.

Conclusion

This large-scale French experience showed that OSCE designed to assess a combination of clinical competence and behavioral skills, increases the discriminatory capacity of current evaluations modalities in French medical schools.

Brief Ultrasound-Aided Teaching to Improve the Accuracy and Confidence of Resident Musculoskeletal Palpation

BACKGROUND

Multiple studies demonstrate poor musculoskeletal palpation accuracy of physiatry residents. With the growing use of ultrasound clinically, it could serve as a powerful educational tool. However, there are no published studies examining the efficacy of ultrasound-aided education at the bedside, where much learning takes place during postgraduate training.

OBJECTIVE

To determine if brief ultrasound-aided teaching improves residents' accuracy and confidence of musculoskeletal palpation.

DESIGN

Before/After Trial.

SETTING

Academic Institution.

PARTICIPANTS

Ten physical medicine and rehabilitation residents were voluntary participants.

INTERVENTION

Each resident was given a survey to assess confidence in palpating the hook of the hamate and medial calcaneal tubercle. They then attempted to palpate and place a marker over these two structures in a model patient. Marker placement accuracy was verified by ultrasound. Faculty spent less than 1 minute per landmark, sonographically teaching its correct localization. The resident, after moving to a new model patient, was then re-tested on accurate marker placement for both landmarks. A repeat survey to assess confidence was administered.

MAIN OUTCOME MEASUREMENTS

Preintervention and postintervention accuracy and confidence of correct localization of both landmarks.

RESULTS

We used McNemar's and Wilcoxon's tests to determine whether the intervention improved the location accuracy and confidence of successful palpation, respectively. Confidence level of correct localization of two landmarks improved on average from 4.3 to 8.15 on a scale of 10 after intervention. Of 20 palpation attempts, 11 correctly placed the marker postintervention after an incorrect placement, 8 made no change, and one changed from correct to incorrect placement. The odds of improvement in accuracy was 11 (95% confidence interval [CI] 1.60-473.47, P < .01). Both tests showed significant improvements after the intervention at the significance level of .05.

CONCLUSIONS

Brief, ultrasound-aided teaching increases accuracy and confidence of residents' musculoskeletal palpation.

Quantifying efficacy of investigation during a simulated psychiatric interview

INTRODUCTION

Simulation is a very promising educational tool. It provides medical students with clinical experience without jeopardizing patients' health. This tool can be implemented in psychiatry training in order to develop interview and specific assessment skills which is why the Université de Lorraine relies increasingly on simulation in teaching psychiatry to its residents. However, there are few objective indicators in that field that make it possible to assess learners. This study aims to assess the relevance of a quantitative measure for objectivising the investigation efficiency by psychiatry residents of a suicidal risk during a simulated psychiatric interview.

METHODS

A simulation exercise was offered to 20 psychiatric trainees from the Université de Lorraine. Their mission was to identify relevant information for assessing and guiding a standardized patient with suicidal risk during a psychiatric interview. The key judgemental criterion was the I²/Q ratio which corresponds to the squared number of relevant items collected by the interviewer divided by the number of questions. When dealing with emergency cases it is indeed important to collect as many key items in as little time as possible. This score was assessed by two examiners based on the analysis of filmed interviews.

RESULTS

The mean I²/Q ratio was 1.13 (SEM±0.2) and ranged between 0.25 and 2.86. Statistical analysis showed a positive correlation between I²/Q ratio and students' clinical experience (R=0.46, P=0.04). The interclass correlation coefficient (ICC) of the I²/Q ratio between the two examiners was 0.94.

CONCLUSION

I²/Q ratio may provide a consistent quantitative measure of a student's efficiency in assessing suicidal risk. Further research is now needed to evaluate its consistency in other clinical cases.

[Acquisition of clinical observation skills by medical students improves with discipline-based teaching. The example of neurological examination]

INTRODUCTION

Since 2014-2015, practical teaching of clinical observation skills for 2^nd year medical students at our faculty has been discipline-based; previously, each clinical lecturer had to cover all medical fields. We assessed the impact of this teaching reform on the neurological examination skills of medical students in a before-and-after study.

METHODS

Pre-reform 3^rd and post-reform 2^nd and 3^rd year medical students (n=62, n=71 and 52, respectively) had to perform 7 neurological examination items, for which performance criteria had been pre-defined. Subsequently, we assessed whether the mean grade in neurological examination skills during the test at the end of the 2^nd year was different between students who had received neurological teaching from a neurologist (n=29) or another specialist (n=102).

RESULTS

The median [interquartile range] number of items acquired by post-reform 3^rd year students (4 [2-5]) was higher than that of pre-reform 3^rd year students (2 [1-3]; P<0.001), but lower than that of post-reform 2^nd year students (5 [4-6]; P=0.01). The mean grade obtained during the practical test was not different in students trained by a neurologist or another specialist.

CONCLUSION

Acquisition of neurological examination skills improved after the teaching reform which consisted of: (1) a discipline-based practical teaching of clinical observation skills; (2) a training of clinical lecturers to teach a limited list of educational objectives; and (3) the introduction of a practical test at the end of the 2^nd year of medical studies. However, there was a decline in clinical observation skills between 2^nd and 3^rd year medical students.

[Simulation-based learning and internal medicine: Opportunities and current perspectives for a national harmonized program]

Simulation-based learning (SBL) is developing rapidly in France and the question of its use in the teaching of internal medicine (IM) is essential. While HAS encourages its integration into medical education, French Young Internists (AJI) set up a working group to reflect on the added-value of this tool in our specialty. Different sorts of SBL exist: human, synthetic and electronic. It enables student to acquire and evaluate technical skills (strengths, invasive procedures, etc.) and non-technical skills (relational, reasoning…). The debriefing that follows the simulation session is an essential time in pedagogical terms. It enables the acquisition of knowledge by encouraging the students' reflection to reshape their reasoning patterns by self-correcting. IM interns are supportive of its use. The simulation would allow young internists to acquire skills specific to our specialty such as certain gestures, complex consulting management, the synthesis of difficult clinical cases. SBL remains confronted with human and financial cost issues. The budgets allocated to the development and maintenance of simulation centres are uneven, making the supply of training unequal on the territory. Simulation sessions are time-consuming and require teacher training. Are faculties ready to train and invest their time in simulation, even though the studies do not allow us to conclude on its pedagogical validity?

[Inventory of training of internal medicine in France: Results of a national survey]

OBJECTIVE

To make an inventory of training of Internal Medicine in France.

METHOD

This study was conducted between May and September 2015 with coordinators (interviews of 45minutes) of local Internal Medicine training and fellows (online questionnaire).

RESULTS

All coordinators (n=28) responded to the interviews. Local training of Internal Medicine exists in 86% of regions (3.1±3.1hours/month) and an interregional training in all interregions (34.7±13.9hours/year). When excluding Île-de-France, no correlation between the number of teachers and the amount of lessons was noted (P=0.61). Of the 550 fellows in Internal Medicine in 2014-2015, 223 (41%) responded to the online questionnaire. Mean level was 5.5±2.7 semesters. The rate of satisfaction (1=very dissatisfied and 5=very satisfied) was 3.0±1.0 and 3.8±0.8 for regional and interregional teaching, respectively (P<0.0001). Regional teaching satisfaction was correlated with the perceived expanse of diseases covered into the program (P<0.0001). In addition, 89% of fellows wish to evaluate themselves online, 66% wish to have a practical evaluation at the bedside and 70% in simulation centers. Finally, 91% of fellows support the establishment of a national program for the training of Internal Medicine.

CONCLUSION

This survey states for the first time an inventory of training of Internal Medicine dedicated to fellows in France. This report highlights that fellows wish to have a national program, be further evaluated and have access to more interactive approach of teaching.

[Computerized ranking test in three French universities: Staff experience and students' feedback]

BACKGROUND

The year 2016 will be pivotal for the evaluation of French medical students with the introduction of the first computerized National Ranking Test (ECNi). The SIDES, online electronic system for medical student evaluation, was created for this purpose. All the universities have already organized faculty exams but few a joint computerized ranking test at several universities simultaneously. We report our experience on the organization of a mock ECNi by universities Paris Descartes, Paris Diderot and Paris 13.

METHODS

Docimological, administrative and technical working groups were created to organize this ECNi. Students in their fifth year of medical studies, who will be the first students to sit for the official ECNi in 2016, were invited to attend this mock exam that represented more than 50% of what will be proposed in 2016. A final electronic questionnaire allowed a docimological and organizational evaluation by students. An analysis of ratings and rankings and their distribution on a 1000-point scale were performed.

RESULTS

Sixty-four percent of enrolled students (i.e., 654) attended the three half-day exams. No difference in total score and ranking between the three universities was observed. Students' feedback was extremely positive. Normalized over 1000 points, 99% of students were scored on 300 points only. Progressive clinical cases were the most discriminating test.

CONCLUSION

The organization of a mock ECNi involving multiple universities was a docimological and technical success but required an important administrative, technical and teaching investment.