A scoping review of approaches for measuring 'interdependent' collaborative performances

A critical interpretive synthesis of interprofessional education interventions

Interprofessional practice can look quite different depending on a number of dynamics. Interprofessional education interventions may or may not orient toward this range of practice possibilities. This literature review explores: (1) how interprofessional education interventions relate to different kinds of interprofessional practice and (2) the range of interprofessional practices assumed by interprofessional education interventions. Four databases were searched for articles published between 2011-2021 describing pre-licensure level interprofessional education interventions, resulting in a dataset of 110 articles. Our analysis involved (1) descriptive summaries of the articles, and (2) content analysis of the rationale and description of the intervention. Of the articles, 93% (102/110) of interprofessional education interventions were designed and/or evaluated using the concept of interprofessional education competencies. "Teamwork" was the most relied upon competency. Most articles were not explicit about the different kinds of interprofessional practices that these competencies might be oriented toward. Our study substantiates earlier claims that interprofessional education literature tends to focus on competencies and orient toward undifferentiated understandings of "teamwork." This analysis is particularly important as interprofessional teams are engaging in increasingly complex, fluid, and distributed forms of interprofessional practice that may not be captured in an undifferentiated approach to "teamwork."

A qualitative study to explore student learning and development of interprofessional collaboration during an online interprofessional education intervention

Interprofessional education (IPE) during undergraduate education and training has been found to improve collaboration between health care students. This supports interprofessional working in clinical practice to enhance patient safety and care delivery.Undergraduate students from pharmacy and medical programmes worked online in pairs to review notes of hospital patients due to be discharged. Students were tasked to complete a discharge letter and undertake an online consultation with a simulated patient prior to discharge. Online interactions were recorded and assessed using a validated tool to measure interprofessional professionalism. Students undertook this intervention in different pairings with different patient cases for three iterations after receiving feedback and undertaking a reflective exercise.The aim was to investigate the student learning and development that could be used to inform intervention optimisation and scale-up.Qualitative data were collected from different sources. Method triangulation was employed to develop a comprehensive understanding of the student learning and development. Data was collected from written feedback provided by the assessment team, student reflections on their performance, and from semi-structured interviews conducted with the student pairs and one to one with the assessment team. Content and thematic analysis was used to analyse these data and the Kirkpatrick/Barr evaluation model provided a framework to organise the themes.Eighteen students (nine from each professional programme) completed the study and a total of 27 IPE sessions were conducted. The assessment team completed 54 assessment tools and 31 student reflections were received (from a maximum of 36). Students were interviewed in their interprofessional pairs to yield nine interview transcripts and one interview was conducted with the assessment team.Students reported and were observed to improve in interprofessional collaboration over the three iterations following feedback and rehearsal opportunities. Longitudinal observation and assessment of student interprofessional working in changing teams provided the opportunity to capture the influence of interdependence on student performance and assessment of competence.

Supportive and collaborative interdependence: Distinguishing residents' contributions within health care teams

INTRODUCTION

Individual assessments disregard team contributions, while team assessments disregard an individual's contributions. Interdependence has been put forth as a conceptual bridge between our educational traditions of assessing individual performance and our imminent challenge of assessing team-based performance without losing sight of the individual. The purpose of this study was to develop a more refined conceptualisation of interdependence to inform the creation of measures that can assess the interdependence of residents within health care teams.

METHODS

Following a constructivist grounded theory approach, we conducted 49 semi-structured interviews with various members of health care teams (e.g. physicians, nurses, pharmacists, social workers and patients) across two different clinical specialties-Emergency Medicine and Paediatrics-at two separate sites. Data collection and analysis occurred iteratively. Constant comparative inductive analysis was used, and coding consisted of three stages: initial, focused and theoretical.

RESULTS

We asked participants to reflect upon interdependence and describe how it exists in their clinical setting. All participants acknowledged the existence of interdependence, but they did not view it as part of a linear spectrum where interdependence becomes independence. Our analysis refined the conceptualisation of interdependence to include two types: supportive and collaborative. Supportive interdependence occurs within health care teams when one member demonstrates insufficient expertise to perform within their scope of practice. Collaborative interdependence, on the other hand, was not triggered by lack of experience/expertise within an individual's scope of practice, but rather recognition that patient care requires contributions from other team members.

CONCLUSION

In order to assess a team's collective performance without losing sight of the individual, we need to capture interdependent performances and characterise the nature of such interdependence. Moving away from a linear trajectory where independence is seen as the end goal can also help support efforts to measure an individual's competence as an interdependent member of a health care team.

Negotiating legitimacy and belonging: Disabled students' and practitioners' experience

INTRODUCTION

People with disabilities are underrepresented in health professions education and practice. Barriers for inclusion include stigma, disabling discourses, discriminatory programme design and oppressive interactions. Current understandings of this topic remain descriptive and fragmented. Existing research often includes only one profession, excludes particular types of disability and focuses on one aspect of the career journey. To expand understanding, we examined the recurrent forms of social relations that underlie the participation of disabled individuals in learning and practice contexts across five health professions.

METHOD

We analysed 124 interviews with 56 disabled health practitioners and students. Participants were interviewed up to three times over 1.5 years. Using constructivist grounded theory, authors used a staged analytic approach that resulted in higher level conceptual categories that advance interpretations of social processes. Finally, the authors compared and integrated findings among students and practitioners.

RESULTS

Participants experience challenges to their sense of legitimacy and belonging as health providers. They describe tensions within the health education and practice between the commitment to inclusion and the day-to-day realities experienced by disabled participants. We identified six distinct, but related, conditions underlying these tensions: (i) validity and transparency of competencies' evaluation; (ii) the social and physical contexts; (iii) integration of inclusive practices; (iv) boundaries between personal and professional identities; (v) vulnerability to authority figures; and (vi) dynamic person-level factors.

DISCUSSION

If we are to commit to health practitioners and students with disabilities experiencing an overall sense of legitimacy and belonging, priority needs to be given to system-level practices and policies to support inclusion. Attention to the day-to-day marginalisation of students and practitioners with disabilities in the health professions is also needed. Additionally, inclusive and transparent delineation of competency requirements is needed. Finally, educational actions are needed to increase understanding of disability in the health professions, with particular attention to promoting social relations that foster collective responsibility for supporting inclusion.

TRainee Attributable & Automatable Care Evaluations in Real-time (TRACERs): A Scalable Approach for Linking Education to Patient Care

Competency-based medical education (CBME) is an outcomes-based approach to education and assessment that focuses on what competencies trainees need to learn in order to provide effective patient care. Despite this goal of providing quality patient care, trainees rarely receive measures of their clinical performance. This is problematic because defining a trainee's learning progression requires measuring their clinical performance. Traditional clinical performance measures (CPMs) are often met with skepticism from trainees given their poor individual-level attribution. Resident-sensitive quality measures (RSQMs) are attributable to individuals, but lack the expeditiousness needed to deliver timely feedback and can be difficult to automate at scale across programs. In this eye opener, the authors present a conceptual framework for a new type of measure - TRainee Attributable & Automatable Care Evaluations in Real-time (TRACERs) - attuned to both automation and trainee attribution as the next evolutionary step in linking education to patient care. TRACERs have five defining characteristics: meaningful (for patient care and trainees), attributable (sufficiently to the trainee of interest), automatable (minimal human input once fully implemented), scalable (across electronic health records [EHRs] and training environments), and real-time (amenable to formative educational feedback loops). Ideally, TRACERs optimize all five characteristics to the greatest degree possible. TRACERs are uniquely focused on measures of clinical performance that are captured in the EHR, whether routinely collected or generated using sophisticated analytics, and are intended to complement (not replace) other sources of assessment data. TRACERs have the potential to contribute to a national system of high-density, trainee-attributable, patient-centered outcome measures.

Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents' Performance in Pediatric Emergency Medicine

PURPOSE

Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs).

METHOD

EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children's Hospital Medical Center, a quaternary children's hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents' rank after case-mix adjustments were also identified.

RESULTS

274 residents and 1,891 individual encounters of bronchiolitis patients aged 0-1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2-21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations ( r = 0.90-0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents' absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay.

CONCLUSIONS

For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents' clinical performance and identify opportunities for quality improvement.

Thought Leader Perspectives on the Benefits, Barriers, and Enablers for Routinely Collected Electronic Health Data to Support Professional Development: Qualitative Study

BACKGROUND

Hospitals routinely collect large amounts of administrative data such as length of stay, 28-day readmissions, and hospital-acquired complications; yet, these data are underused for continuing professional development (CPD). First, these clinical indicators are rarely reviewed outside of existing quality and safety reporting. Second, many medical specialists view their CPD requirements as time-consuming, having minimal impact on practice change and improving patient outcomes. There is an opportunity to build new user interfaces based on these data, designed to support individual and group reflection. Data-informed reflective practice has the potential to generate new insights about performance, bridging the gap between CPD and clinical practice.

OBJECTIVE

This study aims to understand why routinely collected administrative data have not yet become widely used to support reflective practice and lifelong learning.

METHODS

We conducted semistructured interviews (N=19) with thought leaders from a range of backgrounds, including clinicians, surgeons, chief medical officers, information and communications technology professionals, informaticians, researchers, and leaders from related industries. Interviews were thematically analyzed by 2 independent coders.

RESULTS

Respondents identified visibility of outcomes, peer comparison, group reflective discussions, and practice change as potential benefits. The key barriers included legacy technology, distrust with data quality, privacy, data misinterpretation, and team culture. Respondents suggested recruiting local champions for co-design, presenting data for understanding rather than information, coaching by specialty group leaders, and timely reflection linked to CPD as enablers to successful implementation.

CONCLUSIONS

Overall, there was consensus among thought leaders, bringing together insights from diverse backgrounds and medical jurisdictions. We found that clinicians are interested in repurposing administrative data for professional development despite concerns with underlying data quality, privacy, legacy technology, and visual presentation. They prefer group reflection led by supportive specialty group leaders, rather than individual reflection. Our findings provide novel insights into the specific benefits, barriers, and benefits of potential reflective practice interfaces based on these data sets. They can inform the design of new models of in-hospital reflection linked to the annual CPD planning-recording-reflection cycle.

Filling the Core EPA 10 assessment void: A framework for individual assessment of Core Entrustable professional activity 10 competencies in medical students

Objectives

The goal of this study was to develop and evaluate a novel curriculum and assessment tool for Core Entrustable Professional Activity (EPA) 10 competencies and entrustment scoring in a cohort of medical students in their emergency medicine (EM) clerkship using a framework of individualized, ad hoc, formative assessment. Core EPA 10 is an observable workplace-based activity for graduating medical students to recognize a patient requiring urgent or emergent care and initiate evaluation and management.

Methods

This is a prospective, pretest-posttest study of medical students during their EM clerkship. Using the Thomas and Kern framework, we created a curriculum of simulation cases about chest pain/cardiac arrest and respiratory distress, which included novel assessment checklists, and instructional videos about recognizing and managing emergencies. Students were individually pretested on EPA 10 competencies using the simulation cases. Two raters scored students using standardized checklists. Students then watched instructional videos, underwent a posttest with the simulation cases, and were scored again by the two raters using the checklists. Differences between pretest and posttest scores were analyzed using paired t-tests and Wilcoxon signed-rank tests.

Results

Seventy-three out of 85 (86%) students completed the curriculum. Mean scores from pretest to final posttest in the chest pain/cardiac arrest and respiratory distress cases significantly improved from 14.8/19 (SD 1.91), to 17.1/19 (SD = 1.00), t(68) = 10.56, p < 0.001, and 8.5/13 (SD 1.79), to 11.1/13(SD 0.89), t(67) = 11.15, p < 0.001, respectively. The kappa coefficients were 0.909 (n = 2698, p < 0.001) and 0.931 (n = 1872, p < 0.001). Median modified Chen entrustment scores improved from 1b (i.e., "Watch me do this") to 2b (i.e., "I'll watch you") for the chest pain/cardiac arrest case (p < 0.001) and 1b/2a (i.e., "Watch me do this"/ "Let's do this together") to 3a (i.e. "You go ahead, and I'll double-check all of your findings") for the respiratory distress case (p < 0.001).

Conclusion

A new directed curriculum of standardized simulation cases and asynchronous instructional videos improved medical student performance in EPA 10 competencies and entrustment scores. This study provides a curricular framework to support formative individualized assessments for EPA 10.

The story behind the synthesis: writing an effective introduction to your scoping review

NA

Who's on your team? Specialty identity and inter-physician conflict during admissions

PURPOSE

Despite the implementation of professionalism curricula and standardised communication tools, inter-physician conflict persists. In particular, the interface between emergency medicine (EM) and internal medicine (IM) has long been recognised as a source of conflict. The social nuances of this conflict remain underexplored, limiting educators' ability to comprehensively address these issues in the clinical learning environment. Thus, the authors explored EM and IM physicians' experiences with negotiating hospital admissions to better understand the social dynamics that contribute to inter-physician conflict and provide foundational guidance for communication best practices.

METHODS

Using a constructivist grounded theory (CGT) approach, the authors conducted 18 semi-structured interviews between June and October 2020 with EM and IM physicians involved in conversations regarding admissions (CRAs). They asked participants to describe the social exchanges that influenced these conversations and to reflect on their experiences with inter-physician conflict. Data collection and analysis occurred iteratively. The relationships between the codes were discussed by the research team with the goal of developing conceptual connections between the emergent themes.

RESULTS

Participants described how their approaches to CRAs were shaped by their specialty identity, and how allegiance to members of their group contributed to interpersonal conflict. This conflict was further promoted by a mutual sense of disempowerment within the organisation, misaligned expectations, and a desire to promote their group's prerogatives. Conflict was mitigated when patient care experiences fostered cross-specialty team formation and collaboration that dissolved traditional group boundaries.

CONCLUSIONS

Conflict between EM and IM physicians during CRAs was primed by participants' specialty identities, their power struggles within the broader organisation, and their sense of duty to their own specialty. However, formation of collaborative inter-specialty physician teams and expansion of identity to include colleagues from other specialties can mitigate inter-physician conflict.

Optimising expert dyad performance in acute care settings: a scoping review protocol

INTRODUCTION

When there is miscommunication and poor coordination between experienced clinician dyads, teamwork suffers. Research on expert learning practices for the smallest team, and arguably the most important team, the healthcare dyad, is limited. The objective of this study is to map the extent and range of evidence available on learning practices which experienced dyads use, to achieve excellent performance, and to identify the gaps in effective practice. This will guide future research, policy and practice.

METHODS AND ANALYSIS

We are using the JBI methodology for scoping reviews and the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for Scoping Reviews Extension Fillable Checklist, searching for literature that meets the inclusion criteria. The searches will be conducted using Maastricht University's Libsearch, which includes MEDLINE, Education Resources Information Center and PsycINFO and a second search on Web of Science online databases. We will search grey literature and references of selected sources. Search limits include sources from 2016 to 2021, using English language only. A data extraction tool was developed, and charting will use a thematic analysis approach.

IMPLICATIONS AND DISSEMINATION

This review will be the first to examine the learning practices that experienced dyads use, which ensures excellent performance in acute care settings. The findings will be used to develop best-practices and shared with New York City hospital system. Dissemination will occur through peer-reviewed publications and at healthcare conferences.