Crowdsourcing a diagnosis? Exploring the accuracy of the size and type of group diagnosis: an experimental study

BACKGROUND

The consultation process, where a clinician seeks an opinion from another clinician, is foundational in medicine. However, the effectiveness of group diagnosis has not been studied.

OBJECTIVE

To compare individual diagnosis to group diagnosis on two dimensions: group size (n=3 or 6) and group process (interactive or artificial groups).

METHODOLOGY

Thirty-six internal or emergency medicine residents participated in the study. Initially, each resident worked through four written cases on their own, providing a primary diagnosis and a differential diagnosis. Next, participants formed into groups of three. Using a videoconferencing platform, they worked through four additional cases, collectively providing a single primary diagnosis and differential diagnosis. The process was repeated using a group of six with four new cases. Cases were all counterbalanced. Retrospectively, nominal (ie, artificial) groups were formed by aggregating individual participant data into subgroups of three and six and analytically computing scores. Presence of the correct diagnosis as primary diagnosis or included in the differential diagnosis, as well as the number of diagnoses mentioned, was calculated for all conditions. Means were compared using analysis of variance.

RESULTS

For both authentic and nominal groups, the diagnostic accuracy of group diagnosis was superior to individual for both the primary diagnosis and differential diagnosis. However, there was no improvement in diagnostic accuracy when comparing a group of three to a group of six. Interactive and nominal groups were equivalent; however, this may be an artefact of the method used to combine data.

CONCLUSIONS

Group diagnosis improves diagnostic accuracy. However, a larger group is not necessarily superior to a smaller group. In this study, interactive group discussion does not result in improved diagnostic accuracy.

Framing asynchronous interprofessional education: a qualitative study on medical, physiotherapy and nursing students

Objectives

To explore how virtual, asynchronous modules can be used in interprofessional health education curricula and to identify any advantages and shortcomings of asynchronous interprofessional education.

Methods

A sample of 27 health professional students who attended in-person interprofessional education workshops at the McMaster Centre for Simulation-Based Learning from 2019-2020 were recruited through email discourse. Participants were asked to complete an asynchronous interprofessional education module and take part in a semi-structured interview that was recorded and transcribed verbatim. Techniques of direct content analysis were used to analyze the qualitative data from recorded transcripts.

Results

The following emergent themes from participants' responses were identified: 1) the modules, as well as the features interspersed throughout, taught strategies for conflict resolution and interprofessional communication, 2) the modules have utility in preparing students for future interprofessional learning, 3) the convenience of virtual asynchronous modules introduces a sense of learner safety, and 4) a sense of isolation and fatigue was identified as a consequence of the lack of face-to-face interaction in these modules.

Conclusion

Asynchronous interprofessional education modules may be best suited to prepare students for future interprofessional learning in a synchronous setting. Asynchronous modules effectively provide an introduction to interprofessional objectives such as conflict resolution and role clarification, yet the competency of team functioning is more difficult to achieve in an asynchronous environment. Future studies may focus on establishing a sequence of completing asynchronous modules for ideal development of interprofessional competencies in health professions learners.

Measuring group function in problem-based learning: development of a reflection tool

BACKGROUND

Problem-based learning (PBL) is a pedagogy involving self-directed learning in small groups around case problems. Group function is important to PBL outcomes, but there is currently poor scaffolding around key self-reflective practices that necessarily precedes students' and tutors' attempts to improve group function. This study aims to create a structured, literature-based and stakeholder-informed tool to help anchor reflective practices on group function. This article reports on the development process and perceived utility of this tool.

METHODS

Tool development unfolded in four steps: 1) a literature review was conducted to identify existent evaluation tools for group function in PBL, 2) literature findings informed the development of this new tool, 3) a group of PBL experts were consulted for engagement with and feedback of the tool, 4) four focus groups of stakeholders (medical students and tutors with lived PBL experiences) commented on the tool's constructs, language, and perceived utility. The tool underwent two rounds of revisions, informed by the feedback from experts and stakeholders.

RESULTS

Nineteen scales relating to group function assessment were identified in the literature, lending 18 constructs that mapped into four dimensions: Learning Climate, Facilitation and Process, Engagement and Interactivity, and Evaluation and Group Improvement. Feedback from experts informed the addition of missing items. Focus group discussions allowed further fine-tuning of the organization and language of the tool. The final tool contains 17 descriptive items under the four dimensions. Users are asked to rate each dimension holistically on a 7-point Likert scale and provide open comments. Researchers, faculty, and students highlighted three functions the tool could perform: (1) create space, structure, and language for feedback processes, (2) act as a reference, resource, or memory aid, and (3) serve as a written record for longitudinal benchmarking. They commented that the tool may be particularly helpful for inexperienced and poor-functioning groups, and indicated some practical implementation considerations.

CONCLUSION

A four-dimension tool to assist group function reflection in PBL was produced. Its constructs were well supported by literature and experts. Faculty and student stakeholders acknowledged the utility of this tool in addressing an acknowledged gap in group function reflection in PBL.

Challenges facing standardised patients representing equity-deserving groups: Insights from health care educators

INTRODUCTION

Health professions training programmes increasingly rely on standardised patient (SP) programmes to integrate equity-deserving groups into learning and assessment opportunities. However, little is known about the optimal approach, and many SP programmes struggle to meet these growing needs. This study explored insights from health care educators working with SP programmes to deliver curricular content around equity-deserving groups.

METHODS

We interviewed 14 key informants in 2021 who were involved in creating or managing SP-based education. Verbatim transcripts were analysed in an iterative coding process, anchored by qualitative content analysis methodology and informed by two theoretical frameworks: sociologic translation and simulation design. Repeated cycles of data collection and analyses continued until themes could be constructed, aligned with existing theories and grounded in empirical data, with sufficient relevance and robustness to inform educators and curricular leads.

RESULTS

Three themes were constructed: (i) creating safety for SPs paid to be vulnerable, (ii) fidelity as an issue broader than who plays the role and (iii) engaging equity-deserving groups. SP work involving traditionally marginalised groups risk re-traumatization, highlighting the importance of (i) informed consent in recruiting SPs, (ii) separating role portrayal from lived experiences, (iii) adequately preparing learners and facilitators, (iv) creating time-outs and escapes for SPs and (v) building opportunity for de-roling with community support.

CONCLUSIONS

SP programmes are well positioned to be allies and advocates to equity-deserving groups and to collaborate and share governance of the educational development process from its outset. SP programmes can support the delivery of curricular content around equity-deserving groups by advocating with curricular leadership, building relationships with community partners, facilitating co-creation and co-delivery of educational content and building safety into simulation.

Does your group matter? How group function impacts educational outcomes in problem-based learning: a scoping review

BACKGROUND

Problem-based learning (PBL) is a common instructional method in undergraduate health professions training. Group interactions with and within PBL curricula may influence learning outcomes, yet few studies have synthesized the existing evidence. This scoping review summarized the literature examining the influence of group function on individual student PBL outcomes. Following Kirkpatrick's framework, experiential, academic, and behavioral outcomes were considered. The impacts of three aspects of group function were explored: (1) Group Composition (identities and diversity), (2) Group Processes (conduct and climate, motivation and confidence, and facilitation), and (3) PBL Processes (tutorial activities).

METHODS

A literature search was conducted using Medline, CINAHL, and APA PsychInfo from 1980-2021, with the help of a librarian. English-language empirical studies and reviews that related group function to learning outcome, as defined, in undergraduate health professions PBL curricula were included. Relevant references from included articles were also added if eligibility criteria were met. The methods, results, discussions, and limitations of the sample were summarized narratively.

RESULTS

The final sample (n = 48) varied greatly in context, design, and results. Most studies examined junior medical students (n = 32), used questionnaires for data collection (n = 29), and reported immediate cross-sectional outcomes (n = 34). Group Processes was the most frequently examined aspect of group function (n = 29), followed by Group Composition (n = 26) and PBL Processes (n = 12). The relationships between group function and outcomes were not consistent across studies. PBL experiences were generally highly rated, but favorable student experiences were not reliable indicators of better academic or behavioral outcomes. Conversely, problematic group behaviors were not predictors of poorer grades. Common confounders of outcome measurements included exam pressure and self-study.

CONCLUSIONS

The main findings of the review suggested that (1) group function is more predictive of experiential than academic or behavioral PBL outcomes, and (2) different Kirkpatrick levels of outcomes are not highly correlated to each other. More research is needed to understand the complexity of group function in PBL tutorials under variable study contexts and better inform curricular training and design. Standardized tools for measuring PBL group function may be required for more conclusive findings.

Understanding Feedback for Learners in Interprofessional Settings: A Scoping Review

BACKGROUND

Interprofessional feedback is becoming increasingly emphasized within health professions' training programs. The objective of this scoping review is to determine what is known about how learners perceive and interact with feedback in an interprofessional context for learning.

METHODS

A search strategy was developed and conducted in Ovid MEDLINE. Title and abstract screening were performed by two reviewers independently. Next, full texts of selected articles were reviewed by one reviewer to determine the articles included in the review. Data extraction was performed to determine the articles' study population, methodologies and outcomes relevant to the research objective.

RESULTS

Our analysis of the relevant outcomes yielded four key concepts: (1) issues with the feedback process and the need for training; (2) the perception of feedback providers, affecting how the feedback is utilized; (3) professions of the feedback providers, affecting the feedback process; and (4) learners' own attitude toward feedback, affecting the feedback process.

CONCLUSIONS

The learner's perception of interprofessional feedback can be an obstacle in the feedback process. Training around interprofessional feedback should be included as part of interprofessional programs. Research is needed to explore how to address barriers in feedback interaction that stem from misguided perceptions of feedback providers' professions.

Intrinsic or Invisible? An Audit of CanMEDS Roles in Entrustable Professional Activities

PURPOSE

Postgraduate medical education in Canada has quickly transformed to a competency-based model featuring new entrustable professional activities (EPAs) and associated milestones. It remains unclear, however, how these milestones are distributed between the central medical expert role and 6 intrinsic roles of the larger CanMEDS competency framework. A document review was thus conducted to measure how many EPA milestones are classified under each CanMEDS role, focusing on the overall balance between representation of intrinsic roles and that of medical expert.

METHOD

Data were extracted from the EPA guides of 40 Canadian specialties in 2021 to measure the percentage of milestones formally linked to each role. Subsequent analyses explored for differences when milestones were separated by stage of postgraduate training, weighted by an EPA's minimum number of observations, or sorted by surgical and medical specialties.

RESULTS

Approximately half of all EPA milestones (mean = 48.6%; 95% confidence interval [CI] = 45.9, 51.3) were classified under intrinsic roles overall. However, representation of the health advocate role was consistently low (mean = 2.95%; 95% CI = 2.49, 3.41), and some intrinsic roles-mainly leader, scholar, and professional-were more heavily concentrated in the final stage of postgraduate training. These findings held true under all conditions examined.

CONCLUSIONS

The observed distribution of roles in EPA milestones fits with high-level descriptions of CanMEDS in that intrinsic roles are viewed as inextricably linked to medical expertise, implying both are equally important to cultivate through curricula. Yet a fine-grained analysis suggests that a low prevalence or late emphasis of some intrinsic roles may hinder how they are taught or assessed. Future work must explore whether the quantity or timing of milestones shapes the perceived value of each role, and other factors determining the optimal distribution of roles throughout training.

Electronic diagnostic support in emergency physician triage: a qualitative study (Preprint)

Background

Not thinking of a diagnosis is a leading cause of diagnostic error in the emergency department, resulting in delayed treatment, morbidity, and excess mortality. Electronic differential diagnostic support (EDS) results in small but significant reductions in diagnostic error. However, the uptake of EDS by clinicians is limited.

Objective

We sought to understand physician perceptions and barriers to the uptake of EDS within the emergency department triage process.

Methods

We conducted a qualitative study using a research associate to rapidly prototype an embedded EDS into the emergency department triage process. Physicians involved in the triage assessment of a busy emergency department were provided the output of an EDS based on the triage complaint by an embedded researcher to simulate an automated system that would draw from the electronic medical record. Physicians were interviewed immediately after their experience. Verbatim transcripts were analyzed by a team using open and axial coding, informed by direct content analysis.

Results

In all, 4 themes emerged from 14 interviews: (1) the quality of the EDS was inferred from the scope and prioritization of the diagnoses present in the EDS differential; (2) the trust of the EDS was linked to varied beliefs around the diagnostic process and potential for bias; (3) clinicians foresaw more benefit to EDS use for colleagues and trainees rather than themselves; and (4) clinicians felt strongly that EDS output should not be included in the patient record.

Conclusions

The adoption of an EDS into an emergency department triage process will require a system that provides diagnostic suggestions appropriate for the scope and context of the emergency department triage process, transparency of system design, and affordances for clinician beliefs about the diagnostic process and addresses clinician concern around including EDS output in the patient record.