Qualitative Analysis of the Impact of Changes to the Student Evaluation of Teaching Process

Teaching Through the Student Lens: Qualitative Exploration of Student Evaluations of Teaching

OBJECTIVE

The purpose of this study is to use feedback from student evaluations of teaching (SETs) to define and describe themes associated with perceived teaching effectiveness.

METHODS

We retrospectively analyzed SETs for instructors in required didactic courses from one academic year using qualitative content analysis. The analysis included student responses to the following questions: "Describe the strongest aspect of the instructor's teaching." and "What could the instructor do to improve his/her teaching effectiveness?"

RESULTS

Five themes were developed from 4683 coded segments and are described with example quotations. These themes included clarity of instruction and learning activities, alignment of learning and assessment activities, examples that offer practice or demonstrate relevance, engaging learning environment, and concern for student learning and success. Aspects of each theme are described with example coded segments.

CONCLUSION

These themes and supportive quotes offer a deeper understanding of the student perspective on similar ideas that are present throughout the teaching and learning literature and represent broad aspects of teaching, such as material design, in-class practices, and interpersonal characteristics. These themes can provide guidance for specific domains of faculty development in teaching.

Development and psychometric testing of the narrative nursing teaching effectiveness scale: A methodological study

BACKGROUND

Narrative teaching is widely used in nursing education. However, there is currently a lack of valid and reliable tool to assess the effectiveness of narrative nursing teaching from the perspective of nursing students.

OBJECTIVES

This study developed a Narrative Nursing Teaching Effectiveness (NNTE) scale and evaluated its psychometric properties.

DESIGN

A methodological design was used to develop and validate the instrument.

SETTINGS

The study took place at a university in southern China.

PARTICIPANTS

The study involved 283 nursing students enrolled at a medical university in Fujian Province, China.

METHODS

The researchers conducted literature reviews, held focus groups and individual interviews, and utilized the Delphi method to create a pool of indicators for a narrative nursing teaching evaluation system. A pretest was administered to 150 nursing students. Next, the primary study involved 283 nursing students, chosen through cluster sampling, completed a general information questionnaire along with the NNTE scale. Data analysis was performed using SPSS 26.0, Mplus 6.1, and STATA 16.0, and included general statistical analysis, critical test theory, and item response theory analyses.

RESULTS

The researchers created a 42-item scale with four subdomains, explaining 60.32 % of the total variance. Confirmatory factor analysis proved the four-factor model's validity. Cronbach's alpha values for the four subscales ranged from 0.869 to 0.949. The average content validity index of the scale was 0.923 and the item content validity index ranged from 0.872 to 1. In addition, all item characteristic curves displayed desirable shapes, and the test information functions provided precise information.

CONCLUSIONS

The NNTE scale exhibits good psychometric properties, providing a scientifically sound and effective instrument to evaluate the effectiveness of the narrative nursing teaching approach to nursing education.

Faculty Process for Reviewing and Utilizing a School's Course Evaluation Comments

OBJECTIVE

This study aimed to explore faculty engagement with qualitative comments from course evaluations.

METHODS

Course faculty from the University of North Carolina Eshelman School of Pharmacy were recruited via email to participate in a 30-minute interview session. Previous course evaluation comments were adapted to create a de-identified mock evaluation. Six interviews were conducted via Zoom, consisting of a think-aloud protocol based on the mock course evaluation followed by a cognitive interview focused on goals and current utilization of comments, and common patterns and issues sought by faculty. Interview transcripts were manually cleaned and de-identified. Transcripts were inductively coded by 1 researcher using MAXQDA.

RESULTS

Three overarching themes were identified: general faculty process for reviewing comments (ie, how faculty perceive and analyze comments), comments utilization for course change (ie, how faculty utilize comments in making course changes), and faculty analysis strategy (ie, faculty approach to locating common patterns in evaluation comments). The most common subthemes included usefulness of comments, methods for tracking comment patterns, and challenges with the large number of comments each semester.

CONCLUSION

Faculty provided useful insight and feedback regarding the current state of the course evaluation process that can be used to improve the structure, organization, and utilization of course evaluations by faculty. These findings could inform the creation of the course evaluation comment automated analysis program in the next stage of an ongoing project.

Maximizing student potential: Lessons for pharmacy programs from the patient safety movement

Higher education institutions (HEIs), including pharmacy programs, are experiencing growing pressure from the public and policy makers to develop student-centered learning experiences that meet societal needs. While HEIs may have in-house initiatives to meet such challenges, there are also opportunities for cross-domain learning and application of useful concepts from other sectors. One such sector that lends itself to cross-domain learning is the healthcare industry. Like HEIs, the healthcare industry has been experiencing pressure from its stakeholders, in this case, to address patient safety gaps. These forces intensified at the turn of the century leading to the emergence of what is now known as the patient safety movement, which enabled increased advocacy, education, and research to reduce healthcare-related harm. Despite persistent challenges, a key achievement of the patient safety movement has been application of a systems framework to understand and solve patient safety gaps. That is, patient safety gaps are often a result of system defects rather than isolated acts of individual workers operating in a complex social and technical work setting (often referred to as sociotechnical system). Commonly used systems frameworks describe a sociotechnical system through its components: 1) structure (e.g., people, tools/technology, physical workspaces); 2) processes (e.g., medication administration); and 3) outcomes (e.g., medication safety, patient satisfaction). At their core, both HEIs and healthcare organizations are complex sociotechnical systems that organize their structures to support specific processes - learning in HEIs and patient safety in healthcare - to ultimately improve outcomes for students and patients, respectively. This paper describes parallels between HEIs and the healthcare domain to illustrate how patient safety concepts and practices from healthcare can be adapted to HEIs in order to enhance educational structures, processes, and learning outcomes.