Video-based peer assessment of collaborative teamwork in a large-scale interprofessional learning activity

BACKGROUND

The assessment of team performance within large-scale Interprofessional Learning (IPL) initiatives is an important but underexplored area. It is essential for demonstrating the effectiveness of collaborative learning outcomes in preparing students for professional practice. Using Kane's validity framework, we investigated whether peer assessment of student-produced videos depicting collaborative teamwork in an IPL activity was sufficiently valid for decision-making about team performance, and where the sources of error might lie to optimise future iterations of the assessment.

METHODS

A large cohort of health professional students (n = 1218) of 8 differing professions was divided into teams containing 5-6 students. Each team collaborated on producing a short video to evidence their management of one of 12 complex patient cases. Students from two other teams, who had worked on the same case, individually rated each video using a previously developed assessment scale. A generalisability study quantified sources of error that impacted the reliability of peer assessment of collaborative teamwork. A decision study modeled the impact of differing numbers of raters. A modified Angoff determined the pass/fail mark.

RESULTS

Within a large-scale learning activity, peer assessment of collaborative teamwork was reliable (G = 0.71) based on scoring by students from two teams (n = 10-12) for each video. The main sources of variation were the stringency and subjectivity of fellow student assessors. Whilst professions marked with differing stringency, and individual student assessors had different views of the quality of a particular video, none of that individual assessor variance was attributable to the assessors' profession. Teams performed similarly across the 12 cases overall, and no particular professions marked differently on any particular case.

CONCLUSION

A peer assessment of a student-produced video depicting interprofessional collaborative teamwork around the management of complex patient cases can be valid for decision-making about student team performance. Further refining marking rubrics and student assessor training could potentially modify assessor subjectivity. The impact of professions on assessing individual peers and the case-specificity of team performances in IPL settings need further exploration. This innovative approach to assessment offers a promising avenue for enhancing the measurement of collaborative learning outcomes in large-scale Interprofessional learning initiatives.

Learner experiences of identity and global interdependence following engagement with an interprofessional education course

University core curricula and accreditation standards for healthcare profession programs can be challenging to align. Additionally, interprofessional education (IPE) requirements for healthcare professions curricula have been designed to prepare learners for future practice. This paper describes alignment of an introductory IPE course with embedded Interprofessional Education Collaborative core competencies to specific university Core Curriculum attributes. A cross-sectional, mixed methods design was used to examine 117 learners' reflections on the IPE course content and learning outcomes. Learners from seven pre-licensure health professions programs provided responses on aspects of their IPE learning experience through reflections, surveys, written examinations, and optional focus groups. Open-ended responses were interpreted thematically from a constructivist lens. Results revealed positive perceptions of the course with feedback for areas of consideration for future course activities. Learners reported more engagement with the Core Curriculum attribute of Identities in Context than that of Global Interdependence. Additionally, responses indicated a perception of task work versus teamwork within the interprofessional team activities. Outcomes provided data that enabled continuous quality improvement of the course. Educators seeking to align IPE courses with institutional core curricula and accreditation standards may use this work to inform structure, assessment, and delineation of teamwork as compared to task work.

Development and Validation of the Interprofessional Collaboration Practice Competency Scale (IPCPCS) for Clinical Nurses

Interprofessional collaborative practice is a core competency and is the key to strengthening health practice systems in order to deliver safe and high-quality nursing practice. However, there is no Interprofessional Collaboration Practice Competency Scale (IPCPCS) for clinical nurses in Taiwan. Therefore, the purposes of this study were to develop an IPCPCS and to verify its reliability and validity. This was a psychometric study with a cross-sectional survey using convenience sampling to recruit nurses from the seven hospitals of a medical foundation. A self-designed structured IPCPCS was rolled out via a Google survey. The data were analyzed using descriptive statistics, principal-axis factoring (PAF) with Promax rotation, Pearson correlation, reliability analysis, and one-way ANOVA. PAF analysis found that three factors could explain 77.76% of cumulative variance. These were collaborative leadership and interprofessional conflict resolution, interprofessional communication and team functioning, and role clarification and client-centered care. The internal consistency of the three factors (Cronbach's α) was between 0.970 to 0.978, and the Pearson correlation coefficients were between 0.814 to 0.883. Significant differences were presented in the IPCPCS score by age, education level, total years of work experience, position on the nursing clinical ladder, and participation in interprofessional education. In conclusion, the three factors used in the IPCPCS have good reliability and construct validity. This scale can be used as an evaluation tool of in-service interprofessional education courses for clinical nurses.

Building trusting relationships to support implementation: A proposed theoretical model

This paper presents a theory of change that articulates (a) proposed strategies for building trust among implementation stakeholders and (b) the theoretical linkages between trusting relationships and implementation outcomes. The theory of change describes how trusting relationships cultivate increases in motivation, capability, and opportunity for supporting implementation among implementation stakeholders, with implications for commitment and resilience for sustained implementation, and ultimately, positive implementation outcomes. Recommendations related to the measurement of key constructs in the theory of change are provided. The paper highlights how the development of a testable causal model on trusting relationships and implementation outcomes can provide a bridge between implementation research and implementation practice.

An exploratory, cross-sectional and multi-institutional study using three instruments to examine student perceptions of interprofessional education

Interprofessional education (IPE) research needs to expand beyond single site, single event inquiry. Multi-institutional studies increase methodologic rigor and generalizability, advancing the pedagogical science of IPE. Four U.S. institutions used three different validated measures to examine early learner interprofessional outcomes. The three assessment tools included the Communication and Teamwork subscale of the University of West England Entry Level Interprofessional Questionnaire (UWE-ELIQ), the Self-Assessed Collaboration Skills (SACS), and the Interprofessional Teamwork and Team-based Practice factor of the Student Perceptions of Interprofessional Clinical Education-Revised, version 2 (SPICE-R2). Across the four institutions, 659 eligible participants, representing 19 programs completed the pre-survey, and 385 completed the post-survey. The UWE-ELIQ showed a statistically significant difference between the pre- and post-survey overall, but the effect size was small. One institution demonstrated a positive change in scores on the UWE-ELIQ with a small effect size, while the other institutions saw no significant change. Two institutions observed lower post-survey scores on the SPICE-R2. Cumulative results from the study indicated no statistically significant change from pre- to post- in total SACS or SPICE-R2 scores. Additional multi-site longitudinal research is needed to investigate use of validated instruments, as well as the impact of curricula and learning environment on educational outcomes.

An interaction analysis model to study knowledge construction in interprofessional education: proof of concept

A goal of interprofessional clinical learning experiences is to facilitate learning through co-construction of knowledge in support of patient care. Yet, little is known about knowledge construction processes among health professions students working together to care for patients. Understanding knowledge construction processes can guide health professions educators in the design of interventions to support knowledge construction and high-quality learning in clinical placements. In this article, we describe findings from a proof of concept study that explores the feasibility and utility of using Gunawardena's Interaction Analysis Model (IAM) to evaluate health professions students' knowledge construction processes in clinical placements. The IAM has been used to study knowledge construction processes in computer-supported collaborative learning environments, but not in interprofessional education. The IAM describes five phases of knowledge construction - sharing/comparing; exploring dissonance; co-constructing meaning; testing; coming to agreement/applying co-constructed knowledge - each representing a progressively higher-level learning process. Application of the IAM to learner dialogue proved labor-intensive but feasible and useful as a research tool to characterize learners' knowledge construction behaviors. Our findings suggest that the IAM warrants further study and may offer a framework to guide the design of clinical placements and analysis of interprofessional learning behaviors.