The in-training examination in internal medicine: resident perceptions and lack of correlation between resident scores and faculty predictions of resident performance

PURPOSE

We sought to survey residents' perceptions regarding the In-Training Examination in Internal Medicine and to assess the ability of faculty members to evaluate the knowledge base of internal medicine residents.

SUBJECTS AND METHODS

Residents were asked about the perceived utility of the In-Training Examination and related self-directed educational activities. Residents predicted their own performance on the examination (into upper, middle, or lower tertile). Faculty predicted housestaffs scores, and residents predicted the scores of interns.

RESULTS

Most residents (35/36; 97%) believed that the examination was useful, and 91% modified their study habits or clinical rotation schedule based on its results. Approximately half of the residents accurately predicted into which tertile they would score. Faculty predictions of resident performance on the examination were accurate 49% of the time, and resident predictions of intern scores were accurate 38% of the time. The sensitivity ofa lower-tertile prediction by faculty was 34%, with a specificity of 90%. The sensitivity of a resident prediction of a lower-tertile intern score was 15%, with a specificity of 98%. Both faculty and residents were more likely to overestimate than underestimate examination scores.

CONCLUSION

Residents believe that the In-Training Examination is useful and frequently initiate educational interventions based on results. Faculty and residents lack the ability to evaluate accurately the knowledge of trainees that they supervise. In particular, both groups may be unable to identify trainees who are deficient in this element of clinical competence.