Development of a measure of medical faculty attitudes toward clinical evaluation of students

Attitudes of affiliate faculty members toward medical student summative evaluation for clinical clerkships: a qualitative analysis

BACKGROUND

Summative evaluation of medical students is a critical component of the educational process. Despite extensive literature on evaluation, few studies have centered on affiliate faculty members' attitudes toward summative evaluation of students, though it has been suggested that these attitudes influence their effectiveness as evaluators.

PURPOSE

The objective is to examine affiliate faculty members' attitudes toward clinical clerkship evaluation using primarily qualitative research methods.

METHODS

The study used a nonexperimental research design and employed mixed methods. Data were collected through interviews, focus groups, and a questionnaire from 11 affiliate faculty members.

RESULTS

Themes emerging from the data fell into three broad categories: (a) factors that influence grading, (b) consequences of negative evaluations, and (c) disconnections in the grading process. The quantitative portion of the study revealed important discrepancies supporting the use of qualitative methods.

CONCLUSIONS

The study highlights faculty members' struggles with the evaluative process and emphasizes the need for improvements in evaluation tools and faculty development.

Perceptions of evaluation in longitudinal versus traditional clerkships

OBJECTIVES

Methods for evaluating student performance in clerkships traditionally suffer shortcomings, partly as a result of clerkship structure. The purpose of this study was to compare preceptors' and students' perceptions of student evaluation in block clerkships and longitudinal integrated clerkships (LICs).

METHODS

From 2007 to 2009, preceptors who taught on both block clerkships and an LIC were surveyed on their perceptions of clerkship evaluation. Year 3 students were surveyed on their perceptions of clerkship evaluation at the year end. Responses from preceptors who completed both block clerkship and LIC surveys were compared using paired-samples t-test; student responses were compared using independent-samples t-test.

RESULTS

Overall, 66% (67/102) of block clerkship and 75% (77/102) of LIC preceptors responded; 44% of preceptors (45/102) completed both block and LIC surveys. In total, 62% (68/110) of block clerkship and 83% (19/23) of LIC students responded. Both preceptors and students favoured evaluation in the LIC on three factors (p ≤ 0.01): validity of evaluation process, quality of clinical skill evaluation, and willingness to provide constructive feedback.

CONCLUSIONS

Preceptors and students perceived evaluation in an LIC more favourably than evaluation on block clerkships. For educators working to improve student evaluation, further examination of the LIC structure and evaluation processes that seem to enhance both formative assessment and summative evaluation may be useful to improve the quality of evaluation and feedback.

Pilot study of a 360-degree assessment instrument for physical medicine & rehabilitation residency programs

OBJECTIVE

To perform a pilot test on a new format for multidisciplinary assessment of resident physicians' professionalism and clinical performance in acute inpatient rehabilitation settings.

DESIGN

In this pilot study, a 26-item ratings instrument was developed for use by therapists, nurses, social workers, case managers, and psychologists to rate inpatient residents.

RESULTS

A total of 421 ratings forms were returned over four academic years. Alpha reliability coefficient for instrumentation sample was 0.99. chi2 and analysis of variance procedures examined item mean differences. Significant differences (P <or= 0.05) were found based on resident sex (17 items) and rotation setting (20 items). No significant differences were found based on rater profession; mean ratings by profession ranged from 6.67 (physical therapists) to 7.46 (case managers).

CONCLUSIONS

Psychometric properties of this new ratings format are encouraging. The tool was a useful way to provide formative feedback to residents regarding professionalism and performance. Residency program directors can use this approach to fulfill Accreditation Council for Graduate Medical Education mandates to use a variety of assessment methods regarding resident education. However, potential sex bias and other issues affecting performance ratings should be considered in interpreting results and warrant further study.

Development of a measure of attitude toward nutrition in patient care

BACKGROUND

Development of reliable measures of medical student and resident attitudes about nutrition in patient care is needed before the effects of educational interventions or clinical experience can be gauged. This report describes the systematic development of a measure of attitude toward nutrition in patient care. It presents evidence about scale reliability and the absence of response bias that endorses the trustworthiness of data from the measure.

METHODS

An eight-step attitude scale development procedure was used to create the Nutrition In Patient care Survey (NIPS). Data from five samples of first- and second-year medical students and first-year medical residents were subjected to factor analysis (PA2, varimax rotation), reliability analyses, and statistical analyses to test for demographic bias in the attitude data.

RESULTS

A 45-item attitude measure was developed that contains five subscales derived from the factor analysis: (1) nutrition in routine care (NRC, 8 items); (2) clinical behavior (CB, 20 items); (3) physician-patient relationship (PPR, 8 items); (4) patient behavior/motivation (PBM, 3 items); and (5) physician efficacy (PE, 6 items). Each subscale yields reliable data in terms of internal consistency (alpha coefficients) and stability (test-retest reliability). Medical student and resident demographic variables have negligible influence on attitude scores.

DISCUSSION

The NIPS subscales yield reliable data that can be used to assess outcomes in evaluation research on educational or clinical interventions or to predict patient care practices. Systematic attitude scale development increases the likelihood that the resulting measures will produce useful, trustworthy data.

Assessment of a surgery clerkship's performance evaluation system

BACKGROUND

This study was designed to examine the validity of a surgery clerkship's performance evaluation system. The study also assessed faculty members' confidence in how clerks are evaluated and promotion decisions made.

METHODS

Student files from five classes (n = 339) were analyzed. A 25-item published survey designed to study faculty perceptions of a student evaluation system was distributed. Chi-square tests of independence and descriptive statistics were used.

RESULTS

Faculty survey results showed faculty perceptions of strengths and weaknesses in the evaluation system. Significant relationships were found with prior performance indicators and clerkship performance yielding evidence of concurrent and predictive validity.

CONCLUSIONS

Findings provide guidance for enhancing how clerks are evaluated as well as specific profiles of students who may need special attention or additional challenges during the surgery clerkship. This study provides a model for other clerkships to assess their student evaluations systems.