Clinical Performance Evaluations of Third-Year Medical Students and Association With Student and Evaluator Gender

PURPOSE

Clinical performance evaluations are major components of medical school clerkship grades. But are they sufficiently objective? This study aimed to determine whether student and evaluator gender is associated with assessment of overall clinical performance.

METHOD

This was a retrospective analysis of 4,272 core clerkship clinical performance evaluations by 829 evaluators of 155 third-year students, within the Alpert Medical School grading database for the 2013-2014 academic year. Overall clinical performance, assessed on a three-point scale (meets expectations, above expectations, exceptional), was extracted from each evaluation, as well as evaluator gender, age, training level, department, student gender and age, and length of observation time. Hierarchical ordinal regression modeling was conducted to account for clustering of evaluations.

RESULTS

Female students were more likely to receive a better grade than males (adjusted odds ratio [AOR] 1.30, 95% confidence interval [CI] 1.13-1.50), and female evaluators awarded lower grades than males (AOR 0.72, 95% CI 0.55-0.93), adjusting for department, observation time, and student and evaluator age. The interaction between student and evaluator gender was significant (P = .03), with female evaluators assigning higher grades to female students, while male evaluators' grading did not differ by student gender. Students who spent a short time with evaluators were also more likely to get a lower grade.

CONCLUSIONS

A one-year examination of all third-year clerkship clinical performance evaluations at a single institution revealed that male and female evaluators rated male and female students differently, even when accounting for other measured variables.

Feedback

The emergency department provides a rich environment for diverse patient encounters, rapid clinical decision making, and opportunities to hone procedural skills. Well-prepared faculty can utilize this environment to teach residents and medical students and gain institutional recognition for their incomparable role and teamwork. Giving effective feedback is an essential skill for all teaching faculty. Feedback is ongoing appraisal of performance based on direct observation aimed at changing or sustaining a behavior. Tips from the literature and the author's experience are reviewed to provide formats for feedback, review of objectives, and elements of professionalism and how to deal with poorly performing students. Although the following examples pertain to medical student education, these techniques are applicable to the education of all adult learners, including residents and colleagues. Specific examples of redirection and reflection are offered, and pitfalls are reviewed. Suggestions for streamlining verbal and written feedback and obtaining feedback from others in a fast-paced environment are given. Ideas for further individual and group faculty development are presented.

Factors affecting the reliability of ratings of students' clinical skills in a medicine clerkship

OBJECTIVE

To determine the overall reliability and factors that might affect the reliability of ratings of students' clinical skills in a medicine clerkship.

DESIGN

A nine-item instrument was used to evaluate students' clinical skills. Raters were also asked to provide a grade of each student's overall clinical performance. Generalizability studies were performed to estimate the reliability of the ratings. The effects of rater experience and clerkship setting were investigated by regression analysis.

SETTING

Teaching hospitals and community-based sites in three Northwestern states.

PARTICIPANTS

All students (328) who had completed the 12-week clerkship in internal medicine at one medical school during the academic years 1987-1989. Raters included attending physicians, chief residents, and other residents.

RESULTS

Seven observations were needed to provide a reliable rating of the overall clinical grade. More observations were needed to obtain reliable ratings for individual items, ranging from seven observations needed for the rating of data gathering skills to 27 observations needed for the rating of interpersonal relationships with patients. Rater experience and clerkship setting (i.e., teaching hospitals vs. community-based clinics) were found, in general, not to affect significantly the ratings received by students.

CONCLUSIONS

Reliable ratings of students' overall clinical skills, including overall clinical grades, can be achieved by collecting a minimum of seven observations. More observations are needed to measure reliably the interpersonal aspects of clinical performance. These findings support the use of performance ratings to evaluate clinical skills and knowledge of students in clerkship settings.

The effect of nonlinear transformations on a Likert scale

The analysis of data collected from behavioral assessment instruments is typically conducted using parametric statistics, with little or no reference given to the underlying nature of the scale being used. If the nature of the distances between the scale points is not understood, the concept of normality of the distribution becomes clouded. An empirical approach to studying this problem was developed, using responses to a clinical performance evaluation instrument that uses a four-point behaviorally-anchored scale. Various combinations of nonlinear transformations were applied to the evaluation responses. The factorial structure of the fifteen items constituting the evaluation form was minimally affected by the transformations, suggesting that parametric statistics can be applied to behaviorally-anchored rating scales.