Application of "VITALS": visual indicators of teaching and learning success in reporting student evaluations of clinical teachers

Psychometric Properties of Visual Indicators of Teaching and Learning Success "VITALS" Instrument for Evaluation of Clinical Teachers

PURPOSE

We have previously developed an instrument for students' evaluation of clinical teachers that we called Visual Indicators of Clinical Teaching and Learning Success (VITALS). This study measures the reliability of VITALS as an instrument for student evaluation of clinical tutors. Additionally, the study explores the minimum number of student raters necessary for an acceptable reliability, and provides evidence of construct validity of the evaluation scores.

MATERIALS AND METHODS

This retrospective study included 1825 evaluation forms completed by medical students evaluating clinical tutors using the VITALS instrument. Reliability was measured by applying generalizability theory (G-theory) analysis using a two-facet design (raters and items). A D-study was used to determine the minimum number of raters required to achieve a reliability ≥0.80. Face validity was tested by measuring tutors' degree of agreement with the items of the study instrument.

RESULTS

The overall G-coefficient was 0.89. The subject of measurement (clinical tutors' scores) represented 15.8% of the variance across all tutors and items. The variance due to the interaction between raters (students) and tutors contributed to 43.5%, while the variance due to items was negligible. The remaining 40% of the variance was due to unexplained sources of error. The D-study demonstrated that a minimum of 12 raters (students) are required to achieve a reliability of 0.80. Finally, most of the clinical tutors agreed that all items in the instrument were appropriate.

CONCLUSION

We demonstrate that VITALS exhibits good psychometric properties. There should be at least 12 students rating each clinical tutor to have an acceptable level of reliability for the study instrument. Face validity of the study instrument was evidenced by its high level of approval among clinical tutors.

Characteristics of effective clinical teachers identified by dental students: a qualitative study

This qualitative research study identified criteria for clinical teacher quality preferences as perceived by dental students. Third and fourth year dental students at New York University College of Dentistry were given a two question, open-ended survey asking what qualities they liked most and least in a clinical teacher. Responses were collected until data saturation was achieved. A total of 157 respondents provided a total of 995 written comments. Descriptive words within the responses were coded and grouped into key words, according to similar relationships, and further refined into 17 defined categories. Three core themes, Character, Competence and Communication, emerged from these 17 categories, which were validated according to specific references found in the existing educational literature. 'Character' comprised nine of the 17 defined categories: (caring, motivation, empathy, patience, professionalism, available, fairness, happiness, patient-centred) and yielded 59.1% of total student responses; 'Competence' consisted of five categories: knowledgeable, expertise, efficient, skilful, effective (29.2%); and 'Communication' represented the remaining three categories: feedback, approachable and interpersonal communication (11.7%). Positive and negative responses related to the defined category of caring were cited by 59.2% of all students. Motivation was the next highest category, cited by 45.9% of students. Non-cognitive attributes, especially those in the Character theme, comprised the majority of student comments. Because students' perceptions are so critical to understanding clinical teaching effectiveness in dental education, these findings can be used to develop assessments to measure clinical teaching effectiveness, to create criteria for the hiring and promotion of clinical faculty and to plan faculty development programming.

Peer group reflection helps clinical teachers to critically reflect on their teaching

BACKGROUND

Student evaluations can help clinical teachers to reflect on their teaching skills and find ways to improve their teaching. Studies have shown that the mere presentation of student evaluations is not a sufficient incentive for teachers to critically reflect on their teaching.

AIM

We evaluated and compared the effectiveness of two feedback facilitation strategies that were identical except for a peer reflection meeting.

METHOD

In this study, 54 clinical teachers were randomly assigned to two feedback strategies. In one strategy, a peer reflection was added as an additional step. All teachers completed a questionnaire evaluating the strategy that they had experienced. We analysed the reflection reports and the evaluation questionnaire.

RESULTS

Both strategies stimulated teachers to reflect on feedback and formulate alternative actions for their teaching practice. The teachers who had participated in the peer reflection meeting showed deeper critical reflection and more concrete plans to change their teaching. All feedback strategies were considered effective by the majority of the teachers.

CONCLUSIONS

Strategies with student feedback and self-assessment stimulated reflection on teaching and helped clinical teachers to formulate plans for improvement. A peer reflection meeting seemed to enhance reflection quality. Further research should establish whether it can have lasting effects on teaching quality.

Integrating applied anatomy in surgical clerkship in a problem-based learning curriculum

Little attention has been given to structured teaching of applied anatomy to senior medical students in problem-based learning (PBL) medical schools. A course named "Anatomical Principles in Surgical Practice" was introduced at the Arabian Gulf University (AGU) in 2001 for fifth- and sixth-year medical students during their surgical clerkship rotation. The course aims to emphasize, update and integrate applied anatomical concepts that are essential for surgical practice. The course consists of 15 interactive sessions held weekly to ensure topics coincide with the surgical problem-solving sessions. A questionnaire was administered to students completing their surgical rotations (n = 131) seeking their perceptions about the new course during the academic years 2001, 2002 and 2003. To measure learning outcome, 70 students were also given pre- and post-tests. Positive responses were given by 85.2% of the students for course arrangement, by 92.0% for course content, by 88.3% for clinical correlation, by 95.2% for level of teaching and by 87.2% for overall judgment. The students' mean scores in the post-test (71.7%+/-11.7) was significantly higher than their mean scores in the pre-test (42.3%+/-12.6, p < 0.001). Students liked the course and reported feeling more confident in correlating anatomy with surgery during their rotations. By extending anatomical teaching into the clerkship phase, a link between basic medical and clinical sciences has been established that further enhances vertical integration within a PBL curriculum in a spiral fashion.