Clinical Reasoning as a Core Competency

Generative artificial intelligence in physiotherapy education: great potential amidst challenges- a qualitative interview study

BACKGROUND

Generative Artificial Intelligence (GAI) has significantly impacted education at all levels, including health professional education. Understanding students' experiences is essential to enhancing AI literacy, adapting education to GAI, and implementing GAI technology. Therefore, the aim was to explore physiotherapy students' experiences of and thoughts on GAI in their education, and its potential implications for their future careers in healthcare.

METHODS

Qualitative descriptive design. Focus groups were conducted, using a semi-structured interview guide, at the Physiotherapy program at Linköping University, Sweden, from March to April 2024. The 15 students were organized into three focus groups, one for each education year. The data was analyzed using inductive content analysis.

RESULTS

An overarching theme "GAI-Great potential if navigating the challenges" emerged from three categories: 1) "Areas of GAI use in the learning process": Students viewed GAI as a tool for introduction and inspiration, assimilating course content and enhancing clinical reasoning and problem-solving; 2) "Optimizing GAI use in education": Students found GAI to be timesaving, tailored, and as a virtual study partner and teacher. They discussed the pros and cons of learning, concerns on permitted GAI usage, the need for a critical approach, and how individual experiences and interests influenced their interactions with GAI; 3) "Future with GAI in education and profession": Students believed future GAI would be more reliable, use subject-specific GAI models and enhance health care delivery, but also pose risks related to profit motives and knowledge gaps.

CONCLUSION

Physiotherapy students found GAI beneficial for learning and clinical reasoning but expressed concerns about its impact on learning quality. They emphasized the importance of a critical approach when using GAI and the need for organizational support, including supporting permitted GAI use. Students believed that future advanced GAI models could provide accurate and reliable educational tools and healthcare tools supporting documentation and evidence-based decision-making. However, potential risks include business profit motives and knowledge gaps. Navigating these challenges is essential to fully leveraging GAI's benefits in education and physiotherapy practice. Therefore, fostering a critical approach and ensuring robust organizational support is crucial for maximizing the positive impact of GAI in physiotherapy.

Artificial intelligence based assessment of clinical reasoning documentation: an observational study of the impact of the clinical learning environment on resident documentation quality

BACKGROUND

Objective measures and large datasets are needed to determine aspects of the Clinical Learning Environment (CLE) impacting the essential skill of clinical reasoning documentation. Artificial Intelligence (AI) offers a solution. Here, the authors sought to determine what aspects of the CLE might be impacting resident clinical reasoning documentation quality assessed by AI.

METHODS

In this observational, retrospective cross-sectional analysis of hospital admission notes from the Electronic Health Record (EHR), all categorical internal medicine (IM) residents who wrote at least one admission note during the study period July 1, 2018- June 30, 2023 at two sites of NYU Grossman School of Medicine's IM residency program were included. Clinical reasoning documentation quality of admission notes was determined to be low or high-quality using a supervised machine learning model. From note-level data, the shift (day or night) and note index within shift (if a note was first, second, etc. within shift) were calculated. These aspects of the CLE were included as potential markers of workload, which have been shown to have a strong relationship with resident performance. Patient data was also captured, including age, sex, Charlson Comorbidity Index, and primary diagnosis. The relationship between these variables and clinical reasoning documentation quality was analyzed using generalized estimating equations accounting for resident-level clustering.

RESULTS

Across 37,750 notes authored by 474 residents, patients who were older, had more pre-existing comorbidities, and presented with certain primary diagnoses (e.g., infectious and pulmonary conditions) were associated with higher clinical reasoning documentation quality. When controlling for these and other patient factors, variables associated with clinical reasoning documentation quality included academic year (adjusted odds ratio, aOR, for high-quality: 1.10; 95% CI 1.06-1.15; P <.001), night shift (aOR 1.21; 95% CI 1.13-1.30; P <.001), and note index (aOR 0.93; 95% CI 0.90-0.95; P <.001).

CONCLUSIONS

AI can be used to assess complex skills such as clinical reasoning in authentic clinical notes that can help elucidate the potential impact of the CLE on resident clinical reasoning documentation quality. Future work should explore residency program and systems interventions to optimize the CLE.

Quantify difference between physicians and medical students in clinical reasoning: evidence from eye-tracking

BACKGROUND

The assessment of clinical reasoning in health trainees is vital yet poses challenges. We tracked the eye movements of participants while they were reviewing a neurological case with the goal of finding behavioral evidence to improve health education.

METHODS

Eleven medical students and seventeen expert physicians were required to read a neurological case within a 150-second timeframe. The case included descriptive text, a brain CT scan, and an electrocardiogram (ECG). Participants completed a multiple-choice questions (MCQs) test after reading the case. Eye movements of participants in case reading on eleven patient-related information areas (PRIAs) were compared between experts and novices, contrasted with the remaining areas.

RESULTS

Experts spent significantly more time fixating on PRIAs during case reading than novices (42.1% vs. 29.2%, adjusted p = 0.010). Experts demonstrated significantly fewer gaze shifts between Text and CT images (2.0 times) and between CT and ECG images (2.4 times) compared to novices (6.2 and 5.4 times), with adjusted p-values of 0.002 and 0.019, respectively. A positive correlation was found between the fixation rate on PRIAs and MCQs outcome (r = 0.402, p = 0.034).

CONCLUSION

Eye-tracking provides rich and reliable data reflecting physicians' ability to gather patient-relevant information during patient assessment.

Does management reasoning display context specificity? An exploration of sleep loss and other distracting situational (contextual) factors in clinical reasoning

OBJECTIVES

Context specificity occurs when a health professional sees two patients with identical signs and symptoms yet arrives at two different diagnoses due to other existing factors. For example, one patient speaks English as a first language, while the other patient has limited English proficiency. It is not known if context specificity extends beyond diagnosis and also affects management reasoning. Our study explored whether reduced sleep and other distracting contextual factors (e.g., limited English proficiency) lead to context specificity, resulting in suboptimal management reasoning.

METHODS

Seventeen medical residents participated in a two-month study (consisting of one outpatient and one inpatient rotation), in which their sleep was tracked. After each rotation, participants watched two clinical encounter videos-one with and one without distracting contextual factors-and completed think-aloud interviews for each video discussing their management plans. Interviews were transcribed and assessed for management reasoning themes.

RESULTS

Residents (n=17) on outpatient rotations received more sleep than those on inpatient rotations (450.5 min ± 7.13 vs. 425.6 min ± 10.78, p=0.023). Five management reasoning themes were identified: organized knowledge, disorganized knowledge, uncertainty, addressing non-pharmacologic interventions, and addressing patient needs and concerns. There was essentially no difference in the prevalence of utterances of organized knowledge themes between residents with more or less sleep (25 vs. 27 times, p=0.78) or those exposed to contextual factors vs. not exposed (24 vs. 28 times, p=0.58). However, disorganized knowledge themes were observed significantly more frequently in participants exposed to contextual factors (33 vs. 18 times, p=0.036).

CONCLUSIONS

Residents slept more during outpatient rotations. While sleep alone was not associated with the prevalence of management reasoning themes, residents exposed to videos with distracting contextual factors displayed significantly more instances of disorganized knowledge, supporting the phenomenon of context specificity in management reasoning.

Minding the gap: towards a shared clinical reasoning lexicon across the pre-clerkship/clerkship transition

Teaching and learning of clinical reasoning are core principles of medical education. However, little guidance exists for faculty leaders to navigate curricular transitions between pre-clerkship and clerkship curricular phases. This study compares how educational leaders in these two phases understand clinical reasoning instruction. Previously reported cross-sectional surveys of pre-clerkship clinical skills course directors, and clerkship leaders were compared. Comparisons focused on perceived importance of a number of core clinical reasoning concepts, barriers to clinical reasoning instruction, level of familiarity across the undergraduate medical curriculum, and inclusion of clinical reasoning instruction in each area of the curriculum. Analyses were performed using the Mann Whitney U test. Both sets of leaders rated lack of curricular time as the largest barrier to teaching clinical reasoning. Clerkship leaders also noted a lack of faculty with skills to teach clinical reasoning concepts as a significant barrier (p < 0.02), while pre-clerkship leaders were more likely to perceive that these concepts were too advanced for their students (p < 0.001). Pre-clerkship leaders reported a higher level of familiarity with the clerkship curriculum than clerkship leaders reported of the pre-clerkship curriculum (p < 0.001). As faculty transition students from the pre-clerkship to the clerkship phase, a shared understanding of what is taught and when, accompanied by successful faculty development, may aid the development of longitudinal, milestone-based clinical reasoning instruction.

"A safe, non-judgmental space where I can really challenge myself:" learner experiences in a virtual, case-based diagnostic reasoning conference for students

Case-based diagnostic reasoning conferences, like morning reports, allow undergraduate medical trainees to practice diagnostic reasoning alongside senior clinicians. However, trainees have reported discomfort doing so. Peer-assisted learning offers an alternative approach. We describe the design, implementation, and evaluation of a virtual, student-only diagnostic reasoning conference that leverages peer-assisted learning. Student virtual morning report's (VMR) design was informed by social and cognitive congruence and experience-based learning. We evaluated participant experiences using a survey focused on participant perceptions of Student VMR's value, their methods for participation, and their preferences for Student VMR compared with VMR with more senior clinicians. 110 participants (28.9%) completed the survey. 90 participants (81.2%) reported that Student VMR was educational. Compared to VMR, participants reported being more likely to participate in Student VMR by turning on their video (50.0%), presenting a case (43.6%), verbally participating (44.5%), or participating in the chat (70.0%). Strengths included a safe learning environment to practice DR and the opportunity to engage with an international learning community. When asked whether they preferred Student VMR or non-Student VMR, most respondents (64.5%, 71/110) identified that they did not have a preference between the two. A student-focused DR conference may offer a valuable complement to, but not a replacement of, apprenticeship-based DR case conferences.

Exploring Differences in Clinical Decisions Between Medical Students and Expert Clinicians

Background

Numerous challenges exist in effectively bridging theory and practice in the teaching and assessment of clinical reasoning, despite an abundance of theoretical models. This study compares clinical reasoning practices and decisions between medical students and expert clinicians using a problem-solving framework from the learning sciences, which identifies clinical reasoning as distinct, observable actions in clinical case solving. We examined students at various training stages against expert clinicians to address the research question: How do expert clinicians and medical students differ in their practices and decisions during the diagnostic process?.

Methods

We developed a questionnaire about a pediatric infectious disease case based on the problem-solving framework from the learning sciences to probe clinical reasoning decisions. The questionnaire had four sections: medical history, physical examination, medical tests, and working diagnosis. The questionnaire was administered at Stanford University between January 2019 and June 2023 to collect data from 10 experts and 74 medical students. We recruited participants through maximum variation sampling. We applied deductive content analysis to systematically code responses to identify patterns in the execution of the practices and decisions across the questionnaire.

Results

This research introduces a highly detailed, empirically developed framework that holds potential to bridge theory and practice, offering practical insights for medical instructors in teaching clinical reasoning to students across various stages of their training. This framework involves nine practices, with a total of twenty-nine decisions that need to be made when carrying out these practices. Differences between experts and students centered on ten decisions across the practices: Differential diagnosis formulation, Diagnostic plan and execution, Clinical data reassessment, and Clinical solution review.

Conclusion

We were able to identify nuanced differences in clinical reasoning between students and expert physicians under one comprehensive problem-solving framework from the learning sciences. Identifying key clinical reasoning practices and decision differences could help develop targeted instructional materials and assessment tools, aiding instructors in fostering clinical reasoning in students.

Concept mapping to promote clinical reasoning in multimorbidity: a mixed methods study in undergraduate family medicine

BACKGROUND

Clinical reasoning significantly impacts physicians' performance and patient care quality. Research into learning transfer within clinical reasoning education, especially in managing multimorbidity in Family Medicine, is crucial. This study evaluates the impact of concept maps (CMs) on promoting clinical reasoning skills among undergraduate students, compared to traditional teaching methods (TM).

METHODS

A mixed methods approach was used in a controlled, non-randomized study with fifth-year Family Medicine undergraduates allocated to sessions using either CMs or TM. Quantitative data included a feedback questionnaire and evaluation of an individual task. Qualitative data comprised responses to an open-ended question and analysis of problem representation in the individual task.

RESULTS

Among 313 eligible students, 112 participated (CM: 60, TM: 52). Both groups reported high satisfaction with their teaching methods. The CM group valued the holistic view and organization for managing multimorbidity cases, showing higher odds of positive scores on individual tasks (differences not statistically significant). Additionally, the CM group had a more homogeneous code matrix for problem representation in two clinical vignettes.

CONCLUSIONS

While no definitive evidence supports the superiority of CMs over traditional methods, promising trends were noted. The CM group showed improved performance in individual tasks and better organization in managing multimorbidity cases. Further investigation is recommended to explore varying levels of CM usage and modifications to pre-class workloads.

CLINICAL TRIAL NUMBER

Not applicable.

Bridging theory and practice: a scoping review protocol on gamification's impact in clinical reasoning education

INTRODUCTION

In the rapidly evolving field of medical education, gamification has emerged as a promising strategy to enhance clinical reasoning skills among healthcare professionals. By incorporating game-like elements into the learning environment, gamification strives to enhance engagement, motivation and knowledge retention. Given the importance of clinical reasoning in medical decision-making and patient care, this scoping review protocol aims to systematically explore developments, implementations and outcomes of gamification in clinical reasoning education.

METHODS AND ANALYSIS

The scoping review will follow the Arksey and O'Malley methodological framework, enhanced by guidelines from the Joanna Briggs Institute. We will search four major databases: OVID Medline, Scopus and Web of Science using key terms such as "gamification," "clinical reasoning," and "medical education". Studies will be selected based on the participants, concepts and contexts (PCC) framework, focusing on literature published in English. Two independent reviewers will screen studies and extract data on gamification elements used in clinical reasoning education. Any disagreement between the reviewers will be resolved by consulting a third person. We will provide a narrative synthesis of the findings, highlighting the variety of gamified strategies and their effects on clinical reasoning skills. This review will also map out gaps in the current literature and provide direction for future research.

ETHICS AND DISSEMINATION

The scoping review, which aggregates and synthesises publicly available studies, does not require ethics approval due to its nature as a compilation of existing research. The reporting of findings will adhere to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) checklist, promoting both thoroughness and transparency in our analysis. Our dissemination plan encompasses publication in a peer-reviewed journal and presentations at academic conferences focused on medical education. This strategy is designed to engage educators, curriculum designers and policymakers within the sector, ensuring our insights reach those who can apply them most effectively.

Clinical Reasoning in Medical Education: A Primer for Medical Students

Issue: Proper application of clinical reasoning is a prerequisite toward safe practice. Formal instruction on clinical reasoning remains lacking in medical curricula, especially in preparation for the transition from pre-clinical to clinical years. Evidence: Although medical educators have published abundantly on clinical reasoning and acknowledge this to be an essential part of medical education, there remains a global curricular deficiency in developing this skillset. Here we introduce the reader to clinical reasoning frameworks with an emphasis upon practical application. Our focus is upon medical students transitioning from pre-clinical to clinical years of medical school who tend to be overwhelmed with facts but have limited sense of diagnostic approaches due to lack of instruction. Implications: In understanding systematic approaches to clinical reasoning of relevance to medical diagnosis, students will be able to process knowledge in a clinically relevant and discriminatory manner to facilitate problem solving. Upon internship and residency, they will be better prepared for self-learning and reflection as they understand how to hone their capability for diagnosis and management. Medical educators need to acknowledge that clinical reasoning is a practical academic discipline requiring greater curricular emphasis.

Effects of SNAPPS in clinical reasoning teaching: a systematic review with meta-analysis of randomized controlled trials

INTRODUCTION

Clinical reasoning is crucial in medical practice, yet its teaching faces challenges due to varied clinical experiences, limited time, and absence from competency frameworks. Despite efforts, effective teaching methodologies remain elusive. Strategies like the One Minute Preceptor (OMP) and SNAPPS are proposed as solutions, particularly in workplace settings. SNAPPS, introduced in 2003, offers a structured approach but lacks comprehensive evidence of its effectiveness. Methodological shortcomings hinder discerning its specific effects. Therefore, a systematic review is proposed to evaluate SNAPPS' impact on clinical reasoning teaching.

CONTENT

We searched PubMed, EMBASE, and CINAHL for randomized controlled trials (RCTs) comparing SNAPPS against other methods. Data selection and extraction were performed in duplicate. Bias and certainty of evidence were evaluated using Cochrane RoB-2 and GRADE approach.

SUMMARY

We identified five RCTs performed on medical students and residents. Two compared SNAPPS with an active control such as One Minute Preceptor or training with feedback. None reported the effects of SNAPPS in workplace settings (Kirkpatrick Level 3) or patients (Kirkpatrick Level 4). Low to moderate certainty of evidence suggests that SNAPPS increases the total presentation length by increasing discussion length. Low to moderate certainty of evidence may increase the number of differential diagnoses and the expression of uncertainties. Low certainty of evidence suggests that SNAPPS may increase the odds of trainees initiating a management plan and seeking clarification.

OUTLOOK

Evidence from this systematic review suggests that SNAPPS has some advantages in terms of clinical reasoning, self-directed learning outcomes, and cost-effectiveness. Furthermore, it appears more beneficial when used by residents than medical students. However, future research should explore outcomes outside SNAPPS-related outcomes, such as workplace or patient-related outcomes.

Exploring sleep duration and clinical reasoning process in resident physicians: a thematic analysis

STUDY OBJECTIVES

Connecting resident physician work hours and sleep deprivation to adverse outcomes has been difficult. Our study explores clinical reasoning rather than outcomes. Diagnostic errors are a leading cause of medical error and may result from deficits in clinical reasoning. We used simulated cases to explore relationships between sleep duration and diagnostic reasoning.

METHODS

Residents were recruited for a 2-month study (inpatient/outpatient). Each participant's sleep was tracked (sleep diary/actigraphy). At the end of each month, residents watched 2 brief simulated clinical encounters and performed "think alouds" of their clinical reasoning. In each session, 1 video was straightforward and the other video contained distracting contextual factors. Sessions were recorded, transcribed, and interpreted. We conducted a thematic analysis using a constant comparative approach. Themes were compared based on sleep duration and contextual factors.

RESULTS

Residents (n = 17) slept more during outpatient compared with inpatient months (450.5 ± 7.13 vs 425.6 ± 10.78 hours, P = .02). We found the following diagnostic reasoning themes: uncertainty, disorganized knowledge, error, semantic incompetence, emotional content, and organized knowledge. Themes reflecting suboptimal clinical reasoning (disorganized knowledge, error, semantic incompetence, uncertainty) were observed more in cases with contextual factors (distractors). "Think alouds" from cases with contextual factors following sleep restriction had a greater number of themes concerning for problematic diagnostic reasoning.

CONCLUSIONS

Residents obtained significantly more sleep during outpatient compared with inpatient months. Several negative clinical reasoning themes emerged with less sleep combined with cases containing contextual distractors. Our findings reinforce the importance of adequate sleep and supervision in house officers, particularly in cases with distracting elements.

CITATION

Collen J, Durning S, Berk J, Mang J, Alcover K, Jung E. Exploring sleep duration and clinical reasoning process in resident physicians: a thematic analysis. J Clin Sleep Med. 2024;20(8):1279-1289.

Identifying, Understanding, and Minimizing Unconscious Cognitive Biases in Perioperative Crisis Management: A Narrative Review

Rapid clinical decision-making behavior is often based on pattern recognition and other mental shortcuts. Although such behavior is often faster than deliberative thinking, it can also lead to errors due to unconscious cognitive biases (UCBs). UCBs may contribute to inaccurate diagnoses, hamper interpersonal communication, trigger inappropriate clinical interventions, or result in management delays. The authors review the literature on UCBs and discuss their potential impact on perioperative crisis management. Using the Scale for the Assessment of Narrative Review Articles (SANRA), publications with the most relevance to UCBs in perioperative crisis management were selected for inclusion. Of the 19 UCBs that have been most investigated in the medical literature, the authors identified 9 that were judged to be clinically relevant or most frequently occurring during perioperative crisis management. Formal didactic training on concepts of deliberative thinking has had limited success in reducing the presence of UCBs during clinical decision-making. The evolution of clinical decision support tools (CDSTs) has demonstrated efficacy in improving deliberative clinical decision-making, possibly by reducing the intrusion of maladaptive UCBs and forcing reflective thinking. Anesthesiology remains a leader in perioperative crisis simulation and CDST implementation, but spearheading innovations to reduce the adverse impact of UCBs will further improve diagnostic precision and patient safety during perioperative crisis management.

Current status and ongoing needs for the teaching and assessment of clinical reasoning - an international mixed-methods study from the students` and teachers` perspective

BACKGROUND

Clinical reasoning (CR) is a crucial ability that can prevent errors in patient care. Despite its important role, CR is often not taught explicitly and, even when it is taught, typically not all aspects of this ability are addressed in health professions education. Recent research has shown the need for explicit teaching of CR for both students and teachers. To further develop the teaching and learning of CR we need to improve the understanding of students' and teachers' needs regarding content as well as teaching and assessment methods for a student and trainer CR curriculum.

METHODS

Parallel mixed-methods design that used web-surveys and semi-structured interviews to gather data from both students (nsurvey = 100; ninterviews = 13) and teachers (nsurvey = 112; ninterviews = 28). The interviews and surveys contained similar questions to allow for triangulation of the results. This study was conducted as part of the EU-funded project DID-ACT ( https://did-act.eu ).

RESULTS

Both the surveys and interview data emphasized the need for content in a clinical reasoning (CR) curriculum such as "gathering, interpreting and synthesizing patient information", "generating differential diagnoses", "developing a diagnostic and a treatment plan" and "collaborative and interprofessional aspects of CR". There was high agreement that case-based learning and simulations are most useful for teaching CR. Clinical and oral examinations were favored for the assessment of CR. The preferred format for a train-the-trainer (TTT)-course was blended learning. There was also some agreement between the survey and interview participants regarding contents of a TTT-course (e.g. teaching and assessment methods for CR). The interviewees placed special importance on interprofessional aspects also for the TTT-course.

CONCLUSIONS

We found some consensus on needed content, teaching and assessment methods for a student and TTT-course in CR. Future research could investigate the effects of CR curricula on desired outcomes, such as patient care.

Learning Clinical Reasoning: The Experience of Postgraduate Psychiatry Trainee Doctors in Qatar

Phenomenon: As a core competency in medical education, clinical reasoning is a pillar for reducing medical errors and promoting patient safety. Clinical reasoning is a complex phenomenon studied through the lens of multiple theories. Although cognitive psychology theories transformed our understanding of clinical reasoning, the theories fell short of explaining the variations in clinical reasoning influenced by contextual factors. Social cognitive theories propose a dynamic relationship between learners' cognitive process and their social and physical environments. This dynamic relationship highlights the essential role of formal and informal learning environments for learning clinical reasoning. Approach: My research aimed to explore the personal experience of learning clinical reasoning in a sample of postgraduate psychiatry trainee doctors using cognitive psychology and social cognitive theories. A stratified convenience sample of seven psychiatry trainee doctors working in the Mental Health Services in Qatar completed semi-structured interviews in 2020. I analyzed the data manually using theoretical thematic analysis. Findings: I identified three overarching themes with multiple subthemes. The first theme was the hierarchical cultural impact on perceived learning opportunities and learning behavior. The first theme had two subthemes that explored the relationship with team members and the expected hierarchy roles. The second theme was the impact of emotions on the learning and execution of clinical reasoning.The second theme had three subthemes that explored the personal approach to managing emotions related to perceived self-efficacy and professional image. The third theme was characteristics of learning environments and their role in learning clinical reasoning. The last theme included three subthemes that explored stressful, autonomous, and interactive environments. Insights: The results accentuate the complexity of clinical reasoning. Trainees' experience of learning clinical reasoning was influenced by factors not controlled for in the curricula. These factors constitute a hidden curriculum with a significant influence on learning. Our local postgraduate training programmes will benefit from addressing the points raised in this study for effective and culturally sensitive clinical reasoning learning.

Teaching clinical reasoning: principles from the literature to help improve instruction from the classroom to the bedside

Clinical reasoning has been characterized as being an essential aspect of being a physician. Despite this, clinical reasoning has a variety of definitions and medical error, which is often attributed to clinical reasoning, has been reported to be a leading cause of death in the United States and abroad. Further, instructors struggle with teaching this essential ability which often does not play a significant role in the curriculum. In this article, we begin with defining clinical reasoning and then discuss four principles from the literature as well as a variety of techniques for teaching these principles to help ground an instructors' understanding in clinical reasoning. We also tackle contemporary challenges in teaching clinical reasoning such as the integration of artificial intelligence and strategies to help with transitions in instruction (e.g., from the classroom to the clinic or from medical school to residency/registrar training) and suggest next steps for research and innovation in clinical reasoning.

Programmatic Assessment of Clinical Reasoning: New Opportunities to Meet an Ongoing Challenge

Issue: Clinical reasoning is essential to physicians' competence, yet assessment of clinical reasoning remains a significant challenge. Clinical reasoning is a complex, evolving, non-linear, context-driven, and content-specific construct which arguably cannot be assessed at one point in time or with a single method. This has posed challenges for educators for many decades, despite significant development of individual assessment methods. Evidence: Programmatic assessment is a systematic assessment approach that is gaining momentum across health professions education. Programmatic assessment, and in particular assessment for learning, is well-suited to address the challenges with clinical reasoning assessment. Several key principles of programmatic assessment are particularly well-aligned with developing a system to assess clinical reasoning: longitudinality, triangulation, use of a mix of assessment methods, proportionality, implementation of intermediate evaluations/reviews with faculty coaches, use of assessment for feedback, and increase in learners' agency. Repeated exposure and measurement are critical to develop a clinical reasoning assessment narrative, thus the assessment approach should optimally be longitudinal, providing multiple opportunities for growth and development. Triangulation provides a lens to assess the multidimensionality and contextuality of clinical reasoning and that of its different, yet related components, using a mix of different assessment methods. Proportionality ensures the richness of information on which to draw conclusions is commensurate with the stakes of the decision. Coaching facilitates the development of a feedback culture and allows to assess growth over time, while enhancing learners' agency. Implications: A programmatic assessment model of clinical reasoning that is developmentally oriented, optimizes learning though feedback and coaching, uses multiple assessment methods, and provides opportunity for meaningful triangulation of data can help address some of the challenges of clinical reasoning assessment.

Using ChatGPT in Psychiatry to Design Script Concordance Tests in Undergraduate Medical Education: Mixed Methods Study

Background

Undergraduate medical studies represent a wide range of learning opportunities served in the form of various teaching-learning modalities for medical learners. A clinical scenario is frequently used as a modality, followed by multiple-choice and open-ended questions among other learning and teaching methods. As such, script concordance tests (SCTs) can be used to promote a higher level of clinical reasoning. Recent technological developments have made generative artificial intelligence (AI)-based systems such as ChatGPT (OpenAI) available to assist clinician-educators in creating instructional materials.

Objective

The main objective of this project is to explore how SCTs generated by ChatGPT compared to SCTs produced by clinical experts on 3 major elements: the scenario (stem), clinical questions, and expert opinion.

Methods

This mixed method study evaluated 3 ChatGPT-generated SCTs with 3 expert-created SCTs using a predefined framework. Clinician-educators as well as resident doctors in psychiatry involved in undergraduate medical education in Quebec, Canada, evaluated via a web-based survey the 6 SCTs on 3 criteria: the scenario, clinical questions, and expert opinion. They were also asked to describe the strengths and weaknesses of the SCTs.

Results

A total of 102 respondents assessed the SCTs. There were no significant distinctions between the 2 types of SCTs concerning the scenario (P=.84), clinical questions (P=.99), and expert opinion (P=.07), as interpretated by the respondents. Indeed, respondents struggled to differentiate between ChatGPT- and expert-generated SCTs. ChatGPT showcased promise in expediting SCT design, aligning well with Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition criteria, albeit with a tendency toward caricatured scenarios and simplistic content.

Conclusions

This study is the first to concentrate on the design of SCTs supported by AI in a period where medicine is changing swiftly and where technologies generated from AI are expanding much faster. This study suggests that ChatGPT can be a valuable tool in creating educational materials, and further validation is essential to ensure educational efficacy and accuracy.

Student Pharmacists Provide Similar Quality Clinical Reasoning Feedback as Resident Teaching Assistants

OBJECTIVE

Pharmacists utilize clinical reasoning (CR) to improve patient outcomes via medication optimization. It is critical to develop these skills in student pharmacists, yet optimal pedagogies to teach and assess CR are unknown. Peer feedback may be used to develop CR in student pharmacists, but a certain feedback quality must be reached to be effective. This study sought to evaluate if student pharmacists could provide similar quality peer feedback compared to pharmacy resident teaching assistant (TA) feedback.

METHODS

This was a retrospective, mixed-methods pedagogical analysis comparing the quality of first-year student pharmacist peer feedback to resident TA CR feedback. The CR comments were defined using the intellectual standards of CR. Quality was assessed for task specification, gap identification, actionability, and process orientation by 2 independent investigators. Student performance and perceptions were also assessed. Mann-Whitney U, t tests, and descriptive statistics were used to analyze data where appropriate.

RESULTS

Clinical reasoning feedback from peers (N = 805) and TAs (N = 206) were analyzed. Interrater reliability for feedback quality was moderate to substantial. Overall, peer CR feedback was of higher quality regarding task specification and process orientation while TA CR feedback was of higher quality regarding gap identification and actionability. Students receiving peer feedback performed better on a final patient case than those receiving TA feedback (95.2% vs 92.3%). Overall, the peer feedback process was well received by students.

CONCLUSION

Student pharmacists can provide similar quality feedback as resident TAs. Peer feedback offers an alternative to resident TA feedback and has the potential to contribute to improved CR skills.

CRISP: An Inpatient Pediatric Curriculum for Family Medicine Residents Using Clinical Reasoning and Illness Scripts

Introduction

Clinical reasoning enables safe patient care and is an important competency in medical education but can be challenging to teach. Illness scripts facilitate clinical reasoning but have not been used to create pediatric curricula.

Methods

We created CRISP (Clinical Reasoning with Illness Scripts in Pediatrics), a curriculum comprising four 1-hour learning sessions that deliberately incorporated clinical reasoning concepts and illness scripts to organize how four common chief complaints were taught to family medicine residents on inpatient pediatric rotations. We performed a multisite curriculum evaluation project over 6 months with family medicine residents at four institutions to assess whether the use of clinical reasoning concepts to structure CRISP was feasible and acceptable for learners and instructors and whether the use of illness scripts increased knowledge of four common pediatric chief complaints.

Results

For all learning sessions, family medicine residents and pediatric hospitalists agreed that CRISP's format was preferable to traditional didactic lectures. Pre-/posttest scores showed statistically significant increases in family medicine resident knowledge (respiratory distress [n = 42]: pretest, 72%, posttest, 92%; abdominal pain [n = 44]: pretest, 82%, posttest, 96%; acute febrile limp [n = 44]: pretest, 68%, posttest, 81%; well-appearing febrile infant [n = 42]: pretest, 58%, posttest, 73%; ps < .05).

Discussion

By using clinical reasoning concepts and illness script comparison to structure a pediatric curriculum, CRISP represents a novel instructional approach that can be used by pediatric hospitalists to increase family medicine resident knowledge about diagnoses associated with common pediatric chief complaints.

Collaborative clinical reasoning: a scoping review

Background

Collaborative clinical reasoning (CCR) among healthcare professionals is crucial for maximizing clinical outcomes and patient safety. This scoping review explores CCR to address the gap in understanding its definition, structure, and implications.

Methods

A scoping review was undertaken to examine CCR related studies in healthcare. Medline, PsychInfo, SciVerse Scopus, and Web of Science were searched. Inclusion criteria included full-text articles published between 2011 to 2020. Search terms included cooperative, collaborative, shared, team, collective, reasoning, problem solving, decision making, combined with clinical or medicine or medical, but excluded shared decision making.

Results

A total of 24 articles were identified in the review. The review reveals a growing interest in CCR, with 14 articles emphasizing the decision-making process, five using Multidisciplinary Team-Metric for the Observation of Decision Making (MDTs-MODe), three exploring CCR theory, and two focusing on the problem-solving process. Communication, trust, and team dynamics emerge as key influencers in healthcare decision-making. Notably, only two articles provide specific CCR definitions.

Conclusions

While decision-making processes dominate CCR studies, a notable gap exists in defining and structuring CCR. Explicit theoretical frameworks, such as those proposed by Blondon et al. and Kiesewetter et al., are crucial for advancing research and understanding CCR dynamics within collaborative teams. This scoping review provides a comprehensive overview of CCR research, revealing a growing interest and diversity in the field. The review emphasizes the need for explicit theoretical frameworks, citing Blondon et al. and Kiesewetter et al. The broader landscape of interprofessional collaboration and clinical reasoning requires exploration.

Using an experiential learning model to teach clinical reasoning theory and cognitive bias: an evaluation of a first-year medical student curriculum

BACKGROUND

Most medical students entering clerkships have limited understanding of clinical reasoning concepts. The value of teaching theories of clinical reasoning and cognitive biases to first-year medical students is unknown. This study aimed to evaluate the value of explicitly teaching clinical reasoning theory and cognitive bias to first-year medical students.

METHODS

Using Kolb's experiential learning model, we introduced dual process theory, script theory, and cognitive biases in teaching clinical reasoning to first-year medical students at an academic medical center in New York City between January and June 2020. Due to the COVID-19 pandemic, instruction was transitioned to a distance learning format in March 2020. The curriculum included a series of written clinical reasoning examinations with facilitated small group discussions. Written self-assessments prompted each student to reflect on the experience, draw conclusions about their clinical reasoning, and plan for future encounters involving clinical reasoning. We evaluated the value of the curriculum using mixed-methods to analyze faculty assessments, student self-assessment questionnaires, and an end-of-curriculum anonymous questionnaire eliciting student feedback.

RESULTS

Among 318 total examinations of 106 students, 254 (80%) had a complete problem representation, while 199 (63%) of problem representations were considered concise. The most common cognitive biases described by students in their clinical reasoning were anchoring bias, availability bias, and premature closure. Four major themes emerged as valuable outcomes of the CREs as identified by students: (1) synthesis of medical knowledge; (2) enhanced ability to generate differential diagnoses; (3) development of self-efficacy related to clinical reasoning; (4) raised awareness of personal cognitive biases.

CONCLUSIONS

We found that explicitly teaching clinical reasoning theory and cognitive biases using an experiential learning model provides first-year medical students with valuable opportunities for developing knowledge, skills, and self-efficacy related to clinical reasoning.

Reasoning on Rounds: a Framework for Teaching Diagnostic Reasoning in the Inpatient Setting

Internal medicine trainees learn a variety of clinical skills from resident clinical teachers in the inpatient setting. While diagnostic reasoning (DR) is increasingly emphasized as a core competency, trainees may not feel entirely comfortable teaching it. In this perspective article, we provide a framework for teaching DR during inpatient rounds, which includes focusing on the one-liner, structuring a reasoning-focused A&P, and performing a day of discharge reflection.

Deliberate practice of diagnostic clinical reasoning reveals low performance and improvement of diagnostic justification in pre-clerkship students

PURPOSE

Diagnostic errors are a large burden on patient safety and improving clinical reasoning (CR) education could contribute to reducing these errors. To this end, calls have been made to implement CR training as early as the first year of medical school. However, much is still unknown about pre-clerkship students' reasoning processes. The current study aimed to observe how pre-clerkship students use clinical information during the diagnostic process.

METHODS

In a prospective observational study, pre-clerkship medical students completed 10-11 self-directed online simulated CR diagnostic cases. CR skills assessed included: creation of the differential diagnosis (Ddx), diagnostic justification (DxJ), ordering investigations, and identifying the most probable diagnosis. Student performances were compared to expert-created scorecards and students received detailed individualized formative feedback for every case.

RESULTS

121 of 133 (91%) first- and second-year medical students consented to the research project. Students scored much lower for DxJ compared to scores obtained for creation of the Ddx, ordering tests, and identifying the correct diagnosis, (30-48% lower, p < 0.001). Specifically, students underutilized physical exam data (p < 0.001) and underutilized data that decreased the probability of incorrect diagnoses (p < 0.001). We observed that DxJ scores increased 40% after 10-11 practice cases (p < 0.001).

CONCLUSIONS

We implemented deliberate practice with formative feedback for CR starting in the first year of medical school. Students underperformed in DxJ, particularly with analyzing the physical exam data and pertinent negative data. We observed significant improvement in DxJ performance with increased practice.

Clinical Reasoning Curricula in Preclinical Undergraduate Medical Education: A Scoping Review

PURPOSE

Clinical reasoning is the process of observing, collecting, analyzing, and interpreting patient information to arrive at a diagnosis and management plan. Although clinical reasoning is foundational in undergraduate medical education (UME), the current literature lacks a clear picture of the clinical reasoning curriculum in preclinical phase of UME. This scoping review explores the mechanisms of clinical reasoning education in preclinical UME.

METHOD

A scoping review was performed in accordance with the Arksey and O'Malley framework methodology for scoping reviews and is reported using the Preferred Reporting Items for Systematic Reviews and Meta-Analysis for Scoping Reviews.

RESULTS

The initial database search identified 3,062 articles. Of these, 241 articles were selected for a full-text review. Twenty-one articles, each reporting a single clinical reasoning curriculum, were selected for inclusion. Six of the reports included a definition of clinical reasoning, and 7 explicitly reported the theory underlying the curriculum. Reports varied in the identification of clinical reasoning content domains and teaching strategies. Only 4 curricula reported assessment validity evidence.

CONCLUSIONS

Based on this scoping review, we recommend 5 key principles for educators to consider when reporting clinical reasoning curricula in preclinical UME: (1) explicitly define clinical reasoning within the report, (2) report clinical reasoning theory(ies) used in the development of the curriculum, (3) clearly identify which clinical reasoning domains are addressed in the curriculum, (4) report validity evidence for assessments when available, and (5) describe how the reported curriculum fits into the larger clinical reasoning education at the institution.

Identifying the response process validity of clinical vignette-type multiple choice questions: An eye-tracking study

INTRODUCTION

Clinical vignette-type multiple choice questions (CV-MCQs) are widely used in assessment and identifying the response process validity (RPV) of questions with low and high integration of knowledge is essential. Answering CV-MCQs of different levels of knowledge application and integration can be understood from a cognitive workload perspective and this can be identified by using eye-tracking. The aim of the pilot study was to identify the cognitive workload and RPV of CV-MCQs of different levels of knowledge application and integration by the use eye-tracking.

METHODS

Fourteen fourth-year medical students answered a test with 40 CV-MCQs, which were equally divided into low-level and high-level complexity (knowledge application and integration). Cognitive workload was measured using screen-based eye tracking, with the number of fixations and revisitations for each area of interest.

RESULTS

We found a higher cognitive workload for high-level complexity (M = 121.74) compared with lower-level complexity questions (M = 51.94) and also for participants who answered questions incorrectly (M = 94.31) compared with correctly (M = 79.36).

CONCLUSION

Eye-tracking has the potential to become a useful and practical approach for helping to identify the RPV of CV-MCQs. This approach can be used for improving the design and development of CV-MCQs, and to provide feedback to inform teaching and learning.[Box: see text].

Promoting clinical reasoning with meta-memory techniques to teach broad differential diagnosis generation in a pediatric core clerkship

OBJECTIVES

Clinical reasoning with generation and prioritization of differential diagnoses (DDx) is a key skill for medical students, but no consensus exists on the best method to teach these skills. Meta-memory techniques (MMTs) may be useful, but the efficacy of individual MMTs is unclear.

METHODS

We designed a 3-part curriculum for pediatric clerkship students to teach one of 3 MMTs and provide practice in DDx generation through case-based sessions. Students submitted DDx lists during two sessions and completed pre- and post-curriculum surveys assessing self-reported confidence and perceived helpfulness of the curriculum. Results were analyzed using ANOVA with multiple linear regression.

RESULTS

A total of 130 students participated in the curriculum, with 96 % (125/130) completing at least one DDx session, and 44 % (57/130) completing the post-curriculum survey. On average, 66 % of students rated all three sessions as "quite helpful" (4/5 on 5-point Likert scale) or "extremely helpful" (5/5) without difference between MMT groups. Students generated an average of 8.8, 7.1 and 6.4 diagnoses using the VINDICATES, Mental CT, and Constellations methods, respectively. When controlling for case, case order, and number of prior rotations, students using VINDICATES produced 2.8 more diagnoses than those using Constellations (95 % CI [1.1,4.5], p<0.001). There was no significant difference between VINDICATES and Mental CT (Δ=1.6, 95 % CI [-0.2,3.4], p=0.11) or Mental CT and Constellations (Δ=1.2, 95 % CI [-0.7,3.1], p=0.36).

CONCLUSIONS

Medical education should include curricula focused on enhancing DDx development. Although VINDICATES helped students produce the most DDx, further research is needed to identify which MMT generates more accurate DDx.

Therapeutic Reasoning: A Conceptual Challenge for Medical Students' Surgical Clerkship

Issue: Medical students' transition between pre-clinical and clinical environments can be jarring. Unclear expectations, disconnect between the pre-clinical and clinical environments, and feelings of exclusion from the team all negatively influence clerkship experiences. These difficulties are magnified on interventional-based clerkships due to the central role of therapeutic reasoning. Therapeutic reasoning is the process by which clinicians review and select treatments for their patients. Surgical therapeutic reasoning is challenging and infrequently addressed explicitly in the pre-clinical curriculum. This contributes to confusion among medical students as they transition from pre-clinical to clinical learning. Evidence: Multiple studies have identified challenges to medical students' transition to clinical clerkships. Prior authors have outlined numerous barriers to successful medical student integration to clinical teams, including surgical teams. Studies that have assessed students' perceptions of their transition to a clinical setting noted poor understanding of treatment decisions as a source of anxiety. Additionally, of particular importance to medical student team learning is their assimilation into the "communities of clinical practice," groups of healthcare professionals who form teams with which students rotate. Surgeons have distinct approaches to the clinical process compared with non-interventionalists, particularly related to post-diagnostic involvement in care and performing diagnostic workup and treatment concurrently. Knowledge of these approaches is important for integration into surgical teams. Implications: An understanding of interventional therapeutic reasoning is critical to allow for medical students' team integration on surgical and interventional clerkships. By incorporating education involving these concepts during the orientation period prior to students' transition to surgical clerkships, we can help students modify traditional mental frameworks to effectively evaluate, present, and select appropriate interventions for patients. Such education should include a didactic introduction to therapeutic reasoning, interactive discussion with surgeons, and exercises for students to practice skills. This will maximize learning opportunities and clerkship experience.

Standards for Scaffolding in Health Sciences Programmes: A Delphi Consensus Study

Introduction

Health sciences programmes operate in complex, unpredictable contexts, underscoring the need for comprehensive scaffolding of the learning processes. Yet, the scaffolding approaches remain fragmented, and lack a shared approach to how programmes could integrate scaffolding across the curricula. The literature argues that standards result in the comprehensive implementation of educational practices. There are no reported standards related to scaffolding practices in these programmes.

OBJECTIVES

To develop standards for scaffolding in health sciences programmes utilising a consensus approach through a modified Delphi Technique.

METHODS

Following the recommendations on Conducting and REporting of DElphi Studies (CREDES), an online modified Delphi technique was applied. Evidence on the application of scaffolding in health sciences programmes, obtained through an integrative review, was synthesised to draft standards. Using purposive and snowball sampling, an international panel from diverse geographical and professional backgrounds refined and validated the standards. Descriptive statistics was utilised to analyse demographic data and consensus agreements to include standards and criteria. Qualitative analysis of textual comments ensured the synthesis and inclusion of critical divergent views and additions.

RESULTS

A total of 22 experts from around the globe agreed to participate in the study and one did not complete Delphi surveys. Most experts (n = 18) held a PhD; and an average of 19 years of teaching in health sciences programmes. Four standards and 27 criteria were included after achieving consensus during the two Delphi surveys rounds. The included standards focused on four areas: structuring and sequencing educational activities, resources/tools for scaffolding, structuring the programme and instructional strategies to support learning.

CONCLUSION

The principle-based standards developed in this study could direct and support scaffolding practices in health sciences programmes. The standards' emphases on macro-, meso- and micro-scaffolding present numerous opportunities for designing and applying contextually sensitive scaffolding strategies at every level of curriculum implementation.

Longitudinal clinical reasoning theme embedded across four years of a medical school curriculum

OBJECTIVES

The acquisition of clinical reasoning (CR) skills is essential for future healthcare providers as they advance through their education. There is growing consensus that CR skills should be longitudinally integrated into undergraduate curriculum for acquisition/application of these skills. However, only a minority of schools reported having CR focused teaching sessions, citing a lack of curricular time and faculty expertise as the largest barriers. We describe the design and implementation of this theme and report the effects of its early introduction in Phase One as measured by the Diagnostic Thinking Inventory (DTI).

METHODS

The Carver College of Medicine developed and implemented a longitudinal four-year clinical reasoning theme (CRT) with a special emphasis on introducing concepts in the preclinical years (Phase One). Educational strategies used to implement the theme relied on following principles: 1) new skills are best acquired in context of application; 2) contextual learning stimulates transfer of knowledge; and 3) knowledge of pathophysiology is necessary but alone is not sufficient to develop CR skills. A patient-centered CR schema served as the framework for developing the theme. Specific focus areas, pedagogies and assessment strategies were established for each of the three phases.

RESULTS

The cohort with CRT demonstrated a significant increase in total DTI score after theme implementation compared to the cohort without.

CONCLUSIONS

A formal 4-year longitudinal CR theme is feasible, allowing for integration of pathophysiology, social determinants of health, and clinical skills. Early introduction of CR concepts as assessed by DTI showed improvement in student reasoning skills post-intervention.

Development and Pilot Testing of a Data-Rich Clinical Reasoning Training and Assessment Tool

PROBLEM

Clinical reasoning is a core competency for physicians and also a common source of errors, driving high rates of misdiagnoses and patient harm. Efforts to provide training in and assessment of clinical reasoning skills have proven challenging because they are either labor- and resource-prohibitive or lack important data relevant to clinical reasoning. The authors report on the creation and use of online simulation cases to train and assess clinical reasoning skills among medical students.

APPROACH

Using an online library of simulation cases, they collected data relevant to the creation of the differential diagnosis, analysis of the history and physical exam, diagnostic justification, ordering tests; interpreting tests, and ranking of the most probable diagnosis. These data were compared with an expert-created scorecard, and detailed quantitative and qualitative feedback were generated and provided to the learners and instructors.

OUTCOMES

Following an initial pilot study to troubleshoot the software, the authors conducted a second pilot study in which 2 instructors developed and provided 6 cases to 75 second-year medical students. The students completed 376 cases (average 5.0 cases per student), generating more than 40,200 data points that the software analyzed to inform individual learner formative feedback relevant to clinical reasoning skills. The instructors reported that the workload was acceptable and sustainable.

NEXT STEPS

The authors are actively expanding the library of clinical cases and providing more students and schools with formative feedback in clinical reasoning using our tool. Further, they have upgraded the software to identify and provide feedback on behaviors consistent with premature closure, anchoring, and confirmation biases. They are currently collecting and analyzing additional data using the same software to inform validation and psychometric outcomes for future publications.

The Student-Generated Reasoning Tool (SGRT): Linking medical knowledge and clinical reasoning in preclinical education

INTRODUCTION

The simultaneous integration of knowledge acquisition and development of clinical reasoning in preclinical medical education remains a challenge. To help address this challenge, the authors developed and implemented the Student-Generated Reasoning Tool (SGRT)-a tool asking students to propose and justify pathophysiological hypotheses, generate findings, and critically appraise information.

METHODS

In 2019, students in a first-year preclinical course (n = 171; SGRT group) were assigned to one of 20 teams. Students used the SGRT individually, then in teams, and faculty provided feedback. The control group (n = 168) consisted of students from 2018 who did not use SGRT. Outcomes included academic performance, effectiveness of collaborative environments using the SGRT, and student feedback.

RESULTS

Students were five times more likely to get questions correct if they were in the SGRT group versus control group. Accuracy of pathophysiological hypotheses was significantly lower for individuals than teams. Qualitative analysis indicated students benefited from generating their own data, justifying their reasoning, and working individually as well as in teams.

CONCLUSIONS

This study introduces the SGRT as a potentially engaging, case-based, and collaborative learning method that may help preclinical medical students become aware of their knowledge gaps and integrate their knowledge in basic and clinical sciences in the context of clinical reasoning.

Developing Clinical Reasoning in a Physician Assistant Curriculum: The University of Sheffield approach

ABSTRACT

Clinical reasoning remains a key area for development amongst clinical students across the world. Physician assistant (PA) roles (physician associate in the United Kingdom) are rapidly expanding, and there is a need to ensure that this core skill is reflected in course curricula. This article presents how the University of Sheffield has integrated clinical reasoning into the curriculum for its PA course. This includes recognizing the need to consider different approaches to reasoning and how they are taught, the assessment of reasoning across Miller's pyramid, and other considerations that contribute to embedding reasoning within the course. We discuss the implications of our approach and comment on issues that we may need to consider in the future.

Morning report for all: a qualitative study of disseminating case conferences via podcasting

BACKGROUND

Despite its long-established importance, diagnostic reasoning (DR) education has suffered uneven implementation in medical education. The Clinical Problem Solvers (CPSolvers) podcast has emerged as a novel strategy to help teach DR through case conferences with expert diagnosticians and trainees. CPSolvers has 25,000 listeners in 147 countries. The aim of this study was to evaluate the podcast by eliciting the developers' goals of the podcast, then determining to what extent they aligned with the listeners' actual usage habits, features they valued, and perceptions of the podcast.

METHODS

We conducted semi-structured interviews with 3 developers and 8 listeners from April-May 2020, followed by qualitative thematic analysis.

RESULTS

Three major developer goals with sub-goals resulted: 1. To teach diagnostic reasoning in a case-based format by (1a) teaching schemas, (1b) modeling expert diagnostic reasoning, (1c) teaching clinical knowledge, and (1d) teaching diagnostic reasoning terminology. 2. To change the culture of medicine by (2a) promoting diversity, (2b) modeling humility and promoting psychological safety, and (2c) creating a fun, casual way to learn. 3. To democratize the teaching of diagnostic reasoning by leveraging technology. Listeners' usage habits, valued features, and perceptions overall strongly aligned with all these aspects, except for (1c) clinical knowledge, and (1d) diagnostic reasoning terminology. Listeners identified (1a) schemas, and (2c) promotion of psychological safety as the most valuable features of the podcast.

CONCLUSION

CPSolvers has been perceived as a highly effective and novel way to disseminate DR education in the form of case conferences, serving as an alternative to traditional in-person case conferences suspended during COVID-19. CPSolvers combines many known benefits of in-person case conferences with a compassionate and entertaining teaching style, plus advantages of the podcasting medium - democratizing morning report for listeners around the world.

Machine Scoring of Medical Students' Written Clinical Reasoning: Initial Validity Evidence

PURPOSE

Developing medical students' clinical reasoning requires a structured longitudinal curriculum with frequent targeted assessment and feedback. Performance-based assessments, which have the strongest validity evidence, are currently not feasible for this purpose because they are time-intensive to score. This study explored the potential of using machine learning technologies to score one such assessment-the diagnostic justification essay.

METHOD

From May to September 2018, machine scoring algorithms were trained to score a sample of 700 diagnostic justification essays written by 414 third-year medical students from the Southern Illinois University School of Medicine classes of 2012-2017. The algorithms applied semantically based natural language processing metrics (e.g., coherence, readability) to assess essay quality on 4 criteria (differential diagnosis, recognition and use of findings, workup, and thought process); the scores for these criteria were summed to create overall scores. Three sources of validity evidence (response process, internal structure, and association with other variables) were examined.

RESULTS

Machine scores correlated more strongly with faculty ratings than faculty ratings did with each other (machine: .28-.53, faculty: .13-.33) and were less case-specific. Machine scores and faculty ratings were similarly correlated with medical knowledge, clinical cognition, and prior diagnostic justification. Machine scores were more strongly associated with clinical communication than were faculty ratings (.43 vs .31).

CONCLUSIONS

Machine learning technologies may be useful for assessing medical students' long-form written clinical reasoning. Semantically based machine scoring may capture the communicative aspects of clinical reasoning better than faculty ratings, offering the potential for automated assessment that generalizes to the workplace. These results underscore the potential of machine scoring to capture an aspect of clinical reasoning performance that is difficult to assess with traditional analytic scoring methods. Additional research should investigate machine scoring generalizability and examine its acceptability to trainees and educators.

A Cross-Sectional Study: What Contributes to Nursing Students' Clinical Reasoning Competence?

Clinical reasoning is a vital competence for nursing students, as it is required for solving problems arising in complex clinical situations. Identifying the factors that influence nursing students' clinical reasoning competence in the social context can help their implicit educational needs. Therefore, this study aimed to determine the factors associated with developing clinical reasoning competency among undergraduate nursing students. In total, 206 senior nursing students were included in this study. Self-reported measures were used to obtain data on participants' clinical reasoning competence, problem-solving abilities, academic self-efficacy, and level of clinical practicum stress. Relationships among continuous variables were analyzed using Pearson's correlation coefficients. A multiple linear regression analysis was conducted to identify factors related to clinical reasoning competence. Our findings show that participants with better problem-solving abilities and academic self-efficacy perceived themselves as having higher levels of clinical reasoning competence. Nursing students with lower clinical practicum stress reported higher clinical reasoning competence. Significant factors identified were younger age and subcategories of problem-solving ability such as problem clarification, alternative solution development, planning/implementation, and self-regulated efficacy. Our findings highlight essential factors necessary for developing a nursing curriculum that contributes to professional nurses' clinical reasoning competence.

Virtual Interactive Case-Based Education (VICE): A Conference for Deliberate Practice of Diagnostic Reasoning

INTRODUCTION

Current approaches to teaching diagnostic reasoning minimally address the need for deliberate practice. We developed an educational conference for internal medicine residents to practice diagnostic reasoning and examine how biases affect their differential diagnoses through cognitive autopsies.

METHODS

We formatted the Virtual Interactive Case-Based Education (VICE) conference as a clinical problem-solving exercise, in which a facilitator presents a case to a single discussant selected from the audience. We delivered VICE on an internet-based conferencing platform with screen-sharing capability over approximately 30 minutes. To maximize learners' psychological safety, we employed an active facilitation model that normalized uncertainty and prioritized the diagnostic process over arriving at the correct diagnosis.

RESULTS

Resident attitudes toward VICE were assessed by utilizing a postconference survey and gathering descriptive data for 11 sessions. Ninety-seven percent of respondents (n = 35) felt that VICE was a novel and valuable addition to their curriculum. Qualitative data suggested that positive features of the conference included the opportunity to practice diagnostic reasoning, the single-discussant format, and the supportive learning environment. Discussants reported that holding the conference in person would have negatively impacted their experience.

DISCUSSION

Internal medicine residents universally valued the opportunity to engage in deliberate practice of case-based reasoning in a psychologically safe environment during the VICE conference. The virtual nature of the conference contributed significantly to discussants' positive experience. This resource includes all materials necessary to implement VICE, as well as an instructional video on facilitation.

Examining response process validity of script concordance testing: a think-aloud approach

OBJECTIVES

This study investigated whether medical student responses to Script Concordance Testing (SCT) items represent valid clinical reasoning. Using a think-aloud approach students provided written explanations of the reasoning that underpinned their responses, and these were reviewed for concordance with an expert reference panel.

METHODS

A set of 12, 11 and 15 SCT items were administered online to Year 3 (2018), Year 4 (2018) and Year 3 (2019) medical students respectively. Students' free-text descriptions of the reasoning supporting each item response were analysed, and compared with those of the expert panel. Response process validity was quantified as the rate of true positives (percentage of full and partial credit responses derived through correct clinical reasoning); and true negatives (percentage of responses with no credit derived through faulty clinical reasoning).

RESULTS

Two hundred and nine students completed the online tests (response rate = 68.3%). The majority of students who had chosen the response which attracted full or partial credit also provided justifications which were concordant with the experts (true positive rate of 99.6% for full credit; 99.4% for partial credit responses). Most responses that attracted no credit were based on faulty clinical reasoning (true negative of 99.0%).

CONCLUSIONS

The findings provide support for the response process validity of SCT scores in the setting of undergraduate medicine. The additional written think-aloud component, to assess clinical reasoning, provided useful information to inform student learning. However, SCT scores should be validated on each testing occasion, and in other contexts.

Teaching Clinical Reasoning and Critical Thinking: From Cognitive Theory to Practical Application

Teaching clinical reasoning is challenging, particularly in the time-pressured and complicated environment of the ICU. Clinical reasoning is a complex process in which one identifies and prioritizes pertinent clinical data to develop a hypothesis and a plan to confirm or refute that hypothesis. Clinical reasoning is related to and dependent on critical thinking skills, which are defined as one's capacity to engage in higher cognitive skills such as analysis, synthesis, and self-reflection. This article reviews how an understanding of the cognitive psychological principles that contribute to effective clinical reasoning has led to strategies for teaching clinical reasoning in the ICU. With familiarity with System 1 and System 2 thinking, which represent intuitive vs analytical cognitive processing pathways, respectively, the clinical teacher can use this framework to identify cognitive patterns in clinical reasoning. In addition, the article describes how internal and external factors in the clinical environment can affect students' and trainees' clinical reasoning abilities, as well as their capacity to understand and incorporate strategies for effective critical thinking into their practice. Utilizing applicable cognitive psychological theory, the relevant literature on teaching clinical reasoning is reviewed, and specific strategies to effectively teach clinical reasoning and critical thinking in the ICU and other clinical settings are provided. Definitions, operational descriptions, and justifications for a variety of teaching interventions are discussed, including the "one-minute preceptor" model, the use of concept or mechanism maps, and cognitive de-biasing strategies.